How to improve individual label in NER

I'm trying to improve just the EVENT and WORK_OF_ART label in the default spacy model. What is a good workflow to follow? Am i in risk of encountering "catastrophic forgetting" if I do not provide labels for the half a dozen other labels? Is there a way to train an NER to improve just the individual NER label?

In short - yes, you're in risk of degrading the performance of the other labels if you don't provide training data for them. Behind the scenes, all labels are being predicted by one and the same neural network. This network will learn to tune itself to the (new) training data, so if you feed it only EVENT and WORK_OF_ART labels, you'll eventually get a model that does well on those, but might do worse on others, as it's not getting penalized for that during training (anymore).

There's no easy way around this, as far as I know.

By the way - these more generic "best practices" ML/spaCy questions would also be well suited for the forum here: Discussions · explosion/spaCy · GitHub, where it may attract the attention of the larger community.

To add to this, one approach that doesn't technically "solve" the problem but can help prevent it (and doesn't require more manual annotations) is to use the existing model to label data for the other labels for you, by running it over the text and using doc.ents in your training data. So you're not just updating the model with examples of EVENT and WORK_OF_ART, but you're also including whatever the model already predicted for other labels like PERSON etc. Essentially, you're telling the model "predict what you did before but slightly different for these two labels".

1 Like