Understanding BILUO error

[ines (Ines Montani)]

Hi! It's likely that this is related to the update to spaCy v2.1, which is stricter about gold standard data and constraints for the parser and named entity recognizer. See my reply from this thread:

@ines

Do any of the entity spans you’ve annotated start or end on whitespace characters? In spaCy v2.1, it’s now “illegal” for the named entity recognizer to predict entities that start or end with whitespace, or consist of only whitespace. For example, "\n" , but also "hello\n" . This should be a really helpful change, because those entities are pretty much always wrong, and making them “illegal” limits the options and moves the entity recognizer towards correct predictions. But it also means that if you data contains training examples like this, you probably want to remove or fix them.

So you might want to double-check the data and see if you have any "illegal" spans in there. It's usually pretty rare and removing them should be no problem, because in most cases, they'd be rejected suggestions anyway.

@Rajat

Also could you please help us understand what the numbers 584/1373 mean in the following progress bar?
(Because if we count the total number annotations in the merged dataset, there are more than 1373 annotations, assuming 1373 is the number of annotations)

This should be the total number of examples used for annotation after the examples were merged before training and all spans on the same input text were combined.