Off-track use of Prodigy/Spacy - Custom Regex Pattern Matching and Modeling

[ines (Ines Montani)]

Yeah, something like that – it's most likely a side-effect from the weights of the existing pre-trained model. If you search for WORK_OF_ART, you'll see that this is a pretty common problem. That's likely due to the fact that WORK_OF_ART is the rarest category. See Matt's comment here:

@honnibal

One difficult question is always whether to recommend training on top of an existing NER model (such as en_core_web_lg ), or whether to recommend starting from a blank one. The existing model might know useful things, but on the other hand it can also be stubborn about the existing entity definitions, and the training data might not correct them. For instance, I think this is why you had that problem with a rare category like WORK_OF_ART. If you never label examples with that label, the model never sees any negative examples of it, so it’s hard for it to learn not to predict it.

v2.1.0 will already ship with various improvements to the entity recognizer that should hopefully prevent some of this. We're also working on bringing more built-in solutions to prevent catastrophic forgetting into the core library. For example, when training a model, you'd then be able to "remind" it of what it previously predicted.