how to use my own model to mark more data

1

At present, I have marked more than 1000 texts with the following commands in prodigy,

prodigy ner.manual ner-d2 ./ner-d2.txt --label MD,GD,DD,RD,PHON

Then i used the following commands in prodigy to train the ner model.

prodigy train ./ --ner ner-d2 --eval-split 0.2

Now I want to use the trained model to automatically label another dataset. I see that prodigy provides the ner.teach command as follows:

prodigy ner.teach ner_news en_core_web_sm ./news_headlines.jsonl --label PERSON,EVENT

I tries to change spacy-model part to the directory where the ner model I trained is located, but it doesn't seem very successful. So I want to ask if I can use the ner model I trained above to label more data?

prodigy ner.teach ner-d1 ./d3/model-best/ner/model ./merge-pure.txt

thank you.

2

Hi there!

When you train a NER model via this line:

prodigy train ./ --ner ner-d2 --eval-split 0.2

Then it will generate two model folders in your local directory, one called model-best and another called model-last. The former contains the model with the best scores while the other one contains the model state at the end of the training procedure.

If you want to use this model, you will need to refer to it. In the case of ner.teach the call would look something like:

prodigy ner.teach ner-d2 ./model-best/ more-examples.jsonl

This call is different from yours. In your example you're pointing the recipe to the ./d3/model-best/ner/model path, which seems like it's a path too deep and not pointing to the model that you've trained. Does this help?

A final tip; you can also use the ner.correct recipe to correct the output of the NER model. Give you're dealing with multiple entity types this might be an interface worth considering too.