It does take a while for the model to learn the category, so at first it’s expected that the model will suggest a lot of unlikely entities. A little bit of background on how this is working:
The model takes the outputs of the pattern recognizer and the NER model, and interleaves the two outputs. This means it’s trying to show you roughly one suggestion from the model for each suggestion from the pattern matcher. The true matches from the pattern matcher are added as training examples for the model, and the model also learns when you click yes or no to the suggestions.
In the screenshot you’ve attached, the model has only assigned a score of 0.06 to that suggestion — so, pretty low. But if all of the predictions are low, the model still asks you some of those as questions. If we didn’t, we wouldn’t be able to escape from situations where the model is assigning low scores, but is miscalibrated.
So, you do need to click through the examples for a while to bootstrap the model this way. Sometimes the pattern-based bootstrapping really works well, especially if the entity category is easy to learn, and it’s common in your dataset. In other situations, it’s better to use one of the other modes to get initial annotations, e.g.
ner.make-gold, and once you have a dataset of correct examples, then train a model with
ner.batch-train. Sometimes that’s a better way to get started.
In summary, active learning suffers from a cold-start problem. The initial model doesn’t know enough to make good suggestions, so it’s hard to make the initial progress. The patterns file helps a lot with this, but sometimes the model still struggles, so sometimes another approach is better to get started.