Prediction or probability score for prediction results using ner model developed by ner.teach

Hey @ines @honnibal.

First of all, thanks for the amazing tool.
I have created a ner model using ner.teach recipe. The next step I do is take this model and apply it over incoming records to predict each record with a label based on the entities present in it along with its score.

The highlighted words are the ones i have tagged as entities. while training the model in prodigy.

I have 2 questions.

  1. How to I return a single label for each record and not a label every time a entity appears
  2. How do I associate each predicted label with a score or probability which would be another column named score.?

Since ner.teach associates a score while it is being annotated. Is there a way to give a probability for each prediction?

I have just started using spacy. Any help would be much appreciated.

Hi and thanks! :slightly_smiling_face:

If these are your requirements, are you sure that framing it as a named entity recognition task actually makes sense? It sounds like text classification would be a much better fit? This will give you one or more labels and their scores per label. And if you don't care that much about the exact boundaries, making your model predict them can be counterproductive, because it makes the whole problem harder to learn, for no reason.

The screenshot contains sample tags that I custom wrote as I cannot share the data. I will be classifying tags which makes no sense to english dictionary like LMT for limit, ANLZ for analyser.
So NER would be better to find entities and we would like to pick on those entities alone too.

That is why I am trying to get a score or a probability of some sort.

Is that possible in the case of NER.

It's possible, but it's a bit more involved in the current version of spaCy. See these threads for details and background:

Thank you. I will look into it