Best practise for correcting annotations

1

I have annotated 1600 examples for training a spancat_singlelabel component. In production some spans are missed/labled wrongly, probably because i annotated some examples wrong.

Whats the best practise to update my model? Should i just add those incorrectly labled examples to prodigy, annotate them and add them to my training data? I guess this would confuse the model if i dont have enough of those new examples.

Can i revisit certain examples and correct the annotations there? (In my case a certain string gets wrongly labled so i would only want to reannotate data thats containing that string)

2

Hi @Eric,

If you already have a trained spancat pipeline, the most convenient way to correct annotations with Prodigy would be with spans.correct recipe.
This recipe will preannotate the dataset with the current model and you'll be able to correct each example.
You can use the --update flag to update the model in the loop, so that you have fewer and fewer corrections to make as you progress, but make sure retrain on the complete corrected dataset once you're done to give the model the chance to learn from the entire dataset.