In this release we have added a new family of "model as annotators" recipes which facilitate pre-annotating datasets with spaCy models, including the new spacy-llm pipelines!
With these recipes we would like to suggest a workflow whereby you could prioritize the examples based on whether the models disagree about them!
To achieve that you would first pre-annotate dataset with one of our new recipes: ner.model-annotate, spans.model-annotate or textcat.model-annotate and then use the review recipe to qualitatively compare models' predictions:
This way you not only prioritize the examples that requires human attention the most, but you can also gain valuable insights for improving the prompt or making a better model choice for your task.
You can learn more about this workflow in our guide to using models as annotators.
Since the review recipe is such an important component of this workflow (and a recommended component of any workflow that involves more than one annotator or model) we have added a dedicated guide to reviewing annotations in Prodigy.
Looking forward to hear what you think about our new recipes!
To download Prodigy v1.13.1 run:
pip install --upgrade prodigy -f https://XXXX-XXXX-XXXX-XXXX@download.prodi.gy