Hi Matthew and Jashmi,
When I run ner.teach, Prodigy attempts to serve examples which the model is most uncertain about. This is excellent when it’s applied to just a few labels.
However, my model currently has a total of 12 labels. In the scenario that the model labels ALL entities in an example correctly, I’d click accept. If all my labels are wrong - I’d click reject.
Now, if my model’s labels are only partially correct, should I accept or reject the example ? And if I do reject this example, I’m concerned that the model wouldn’t be able to pinpoint exactly which label is wrong since I’m using multiple labels. I don’t wish for my model’s accuracy to suffer due to this.
Just to give some context on what I’ve done thus far. I’ve been following your suggestion on this link: Understanding ner.batch-train stats and best practices
I’ve gathered about ~10,000 examples from ner.manual and ner.make-gold. Almost all of these examples were ‘accepted’. I’d like to proceed with ner.teach soon to introduce ‘rejected’ examples into my model, but due to the problem mentioned above, I am unsure if ner.teach will be beneficial for my model.
Could you kindly suggest what would be the best way forward? Would love to hear from you soon. Thanks!