EntityRuler and ner.match - different behavior

[ines (Ines Montani)]

@Anji.Vaidyula It really sounds like the problem is the capitalisation then!

@Anji.Vaidyula

I pretty much know all the names of the ISSUERS.

Are you sure you want to train a statistical model for this, then? If you haven't done it already, it might be worth running a quick evaluation to see what your baseline is. For instance, if you're getting to a 95% accuracy using only your rules, training a model may be kind of a waste of time.

@Anji.Vaidyula

If I chose to take advantage of existing prediction ORG, can I specify the patterns? I pretty much know all the names of the ISSUERS. And reject all other ORG entities?

Oh, I meant you could do something like: extract all ORG entities and save the result in Prodigy's JSONL. Then change the label from ORG to ISSUER and load it into Prodigy and annotate the examples. Even if only 25% of those orgs are issuers, that's still 25% less manual work for you.

@jsnleong

Should the patterns in EntityRuler be constructed GENERALLY to capture a wide coverage of text spans (eg. {"IS_ALPHA": True}, {"IS_PUNCT": True} ), or should it be constructed in a specific manner (eg. {"LOWER": "hello"}, {"LOWER": "world"} )?

This depends on what you're trying to do. The very abstract patterns are unlikely to accurately capture the spans you're looking for. So it really comes down to what gives you the best results.

@jsnleong

Should EntityRuler be placed before, or after ‘ner’ in the pipeline?

Both is possible and has different implications. See the documentation here: Rule-based matching · spaCy Usage Documentation

The entity ruler is designed to integrate with spaCy’s existing statistical models and enhance the named entity recognizer. If it’s added before the "ner" component , the entity recognizer will respect the existing entity spans and adjust its predictions around it. This can significantly improve accuracy in some cases. If it’s added after the "ner" component , the entity ruler will only add spans to the doc.ents if they don’t overlap with existing entities predicted by the model. To overwrite overlapping entities, you can set overwrite_ents=True on initialization.

@jsnleong

In Prodigy’s recipe ner.make-gold , when I make changes to the model’s prediction, does it edit the ‘ner’ pipeline, and has no interference to the EntityRuler?

If you just annotate, the model won't be changed at all. If you do use that data to train from later on, the statistical model will be updated, not the entity ruler (which is really just a collection of static rules). My comment here explains some of this in more detail:

NER Training for Corporate Names

Ah, I think that might be a small misunderstanding: The entity ruler itself doesn’t learn anything, just like the statistical entity recognizer doesn’t “learn” anything from the entity ruler. But if entities are already set in a previous pipeline step and the statistical entity recognizer encounters them, it will “predict around them” and use them as constraints for its predictions. This means that a pre-trained statistical NER model may produce better results and make fewer mistakes, because s…