Getting warning while using ner.correct

The model you're using isn't setting sentence boundaries (e.g. via the parser
or sentencizer). This means that incoming examples won't be split into
after training the model I was trying to use ner.correct in order to increase my accuracy and it is not showing any tokens(words) that are annotated in the document was this due to the above warning ???

This warning just lets you know that sentences are not segmented, which is otherwise the default behaviour. So if that's surprising to you, you should use a different base model. One reason why sentence segmentation is more relevant for workflows with models in the loop is that it protects you against unexpectedly long texts crashing the server because you run out of memory. It can also matter if all your other data and evaluation examples are sentences, because it'd mean you end up with inconsistent data.

But I don't think any of this is a problem in your case. ner.correct will show you whatever the model predicted for the given text (equivalent to what spaCy outputs in the doc.ents). If you don't see any suggestions here, check that the labels you're setting on the command line cover everything you want to annotate, and that you're using the correct base model.

Maybe the model just didn't predict anything for a given text – that doesn't always mean there's a deep problem. To really evaluate that, you typically want to look at a larger sample.

Thank you Ines. its working now :slightly_smiling_face:

1 Like