Port from old to new version

[ines (Ines Montani)]

Yes, see my comment here for details on what makes training from yes/no annotations special and how it works:

@ines

In a typical training scenario, you're updating a model with examples and the correct answer – e.g. with a text and the entities in it. In some cases you may also have partial annotations: you know some entities but not all.

Prodigy's active learning recipes like ner.teach also let you collect binary yes/no decisions. The data you create here is different again: for some spans, you know that they are entities, because you accepted them. For the ones you rejected, you know that they're not of type X – but they could potentially be something else. This requires a different way of updating the model: you want to update with the positive examples where you know the answer, and proportionally with the "negative" example where you only know that a certain label doesn't apply. That's the type of training the --binary flag enables.

@jsnleong

Also, if I mix binary annotations together with ner.correct recipe in the same Prodigy database, would it be an issue in training?

Yes, you shouldn't mix those in the same dataset because you want to update differently depending on the type of annotation. For the binary annotations, you want to set the --binary flag to take advantage of the yes and no decisions and to treat all other tokens as missing values. If you've collected data with ner.correct and the annotations are complete (all entities in the text are labelled), you want to take advantage of that and let the model treat all unlabelled tokens as outside of an entity and not missing values. This gives you better results.