Is there any way I can merge spacy entity linking model and a customized ner model I trained using ner batch train together?
Reason is that spacy entity linking model is providing amazing results for NER although not that influencing results for Entity Linking, So I was thinking to merge my custom NER model and spacy entitly linking model for NER?
Hi @Zainpann, I assume you have built and trained your own Entity Linking component, perhaps with the scripts in https://github.com/explosion/spaCy/tree/master/bin/wiki_entity_linking? When you do so, the entity linking results will actually be built on top of an existing NER component. Which nlp pipeline did you use at the time to initiate the EL training? Maybe a pre-built model such as en_core_web_lg? If that's the case, the NER results you're seeing are actually from that pre-built model. The only thing the Entity Linker does on top of that, is to add the ent.kb_id_ fields for the named entities.
To address the issue of NER: are you running ner.batch-train with a pre-trained spaCy model? You could specify it with spacy_model. If you haven't tried that before, this could definitely be interesting for you.
Then, you could create and train an entity linking component with spaCy, using an nlp model that has your newly trained ner component as part of its pipeline. This way, both algorithms should be compatible.
As an FYI - we're also currently looking into providing custom recipes for entity linking in spaCy. Let me know if you'd be interested in that!
- Oh yes the results of Entity linking NER must be from pre-built en_core_web_lg.
2)Yes I am running ner.batch-train with a pre-trained spacy model(en_core_web_sm) and I never tried the method you told.I will try it for sure.
3)Yes I am highly interested in some custom recipes. but the thing is Entity linking model is not actually providing satisfactory results for entities with same name. for example:
spacy_entities("Amir Khan is a great boxer")
Out[13]:
'{"Amir Khan": "PERSON", "Q9557": "Indian film actor"}'
now this "Amir Khan" is a boxer but spacy EL as what I have seen is assigning just one kb_id for entity with the same name, whatever the context is used in the sentence. But yes I am looking forward for some custom recipes for Entity Linking.
And I also wanted to ask that can we add our own data in spacy entity linking for some local entities?