Prodigy annotations from older from to newer version

[ines (Ines Montani)]

Hi! The main difference between spaCy v2.0 and v2.1 is the handling of whitespace in entities. This used to be an issue and could cause significantly worse results, because the entity recognizer was allowed to predict newlines and whitespace tokens as entities.

So if your data contains spans starting with / consisting of only whitespace, that's the one thing you need to fix and filter out. See these threads for details:

ner.batch-train after ner.maual results error (Value error : [E024])

Hi! Did you use the same Prodigy version for annotation and training, or did you collect the annotations in a previous version? It’s likely that this is related to the update to spaCy v2.1, which is stricter about gold standard data and constraints for the parser and named entity recognizer. See my reply from this thread: So you might want to double-check the data and see if you have any “illegal” spans in there. It’s usually pretty rare and removing them should be no problem, because in most…

Recipe ner.batch-train results in ValueError: [E030]

The --unsegmented flag only means that Prodigy won’t apply the sentence segmenter to split your texts into sentences. If your examples are already pre-segmented, this is fine – but if your data contains lots of really long texts, you probably want to split them, because otherwise training may be slow and the long texts may throw off the model. So it should be fine in your case. Ahh, I meant examples that consist of only one token. So basically, where "text" has only one word. Do you find any …