Textcat.teach with Multiple Choice & Update Model

Hi! To update the annotation model, you can call model.update with a batch of examples in Prodigy's format. Since your recipe is creating predict/update functions depending on whether you're using patterns or not, you probably want to be using that update function. For example:

if patterns is None:
    # etc.
    predict = model
    update = model.update
else:
    # etc.
    predict, update = combine_models(model, matcher)

def review_answers(answers):
    update(answers)

Alternatively, you could also create a spaCy text classifier directly and then update that, just like you'd update it during training (by calling nlp.update with texts and annotations). This removes one layer of abstraction introduced by Prodigy's annotation model, and makes it easier to see what's going on.

(The annotation model helps dealing with binary accept/reject annotations, converting data etc., but in your case, the annotation part is pretty straightforward: you have multiple choice options, and it's pretty clear what the annotations "mean" and how you want to use them to update your model.)

Note that any modifications you make to the answers in the update callback won't be reflected in the database – the annotations in the dataset will always match whatever the annotator saw and created. This is by design, because it'd otherwise be too easy to accidentally overwrite your annotations in place when performing the update and potentially destroying datapoints.

1 Like