@SofieVL Thank you for the answer!
I stopped arround 300 because after that I was getting similar problems to what I reported in a previous post back in april. I seems to be working way longer/better now, but after 320 it started to just select the whole sentence and highlight that as an entity, which is why I tried to update the model at that point.
I will try out what you suggested and keep you updated! I still have access to my original training and evaluation data so I will try mixing that in too