Training a model on both gold and binary data

[ines (Ines Montani)]

@einarbmag

I'm actually quite confused by the --binary flag... isn't all the training data technically "binary", i.e you either accept or reject the annotation?

The --binary flag uses a more complex mechanism to update the model that lets you take advantage of both the accepted and rejected suggestions of single entity spans – basically, like the default behaviour of ner.batch-train. I've written some more about it on this thread:

Prodigy 1.90 train recipe --ner-missing argument

In a typical training scenario, you're updating a model with examples and the correct answer – e.g. with a text and the entities in it. In some cases you may also have partial annotations: you know some entities but not all. Prodigy's active learning recipes like ner.teach also let you collect binary yes/no decisions. The data you create here is different again: for some spans, you know that they are entities, because you accepted them. For the ones you rejected, you know that they're not of t…

The --ner-missing flag lets you to specify that unannotated tokens should be treated as missing values (and not as explicitly as "not an entity"). This allows you to still train from incomplete annotations, e.g. if you only have annotations for one or two labels.

The new train command was designed to harmonise training between Prodigy and spaCy, and use spaCy's regular update mechanism to train from gold-standard data (with and without missing values). This also makes it easier to ensure that results are consistent and reproducible.