prodigy use case for annotation having pre-annotated text

[ines (Ines Montani)]

I think the main issues around loading your data should be resolved by fixing your JSON, as described here:

Loading annotations

ouch! I’m so sorry I overlooked … it was written in big characters in the exception

@lfoppiano

The part where how to load annotations is explained, is not clear, because it is assumed that there is a model in spacy that we can use (Prodigy 101 – everything you need to know · Prodigy · An annotation tool for AI, Machine Learning & NLP), this is not my case

The example commands in this section show two of the most commonly used built-in recipes: ner.teach and textcat.teach. Both of those use a spaCy model in the loop because that's part of that particular annotation workflow. None of this has anything to do with loading data.

Loading data into Prodigy should be as straightforward as passing the path to the JSONL file to a given recipe as the source argument. For example, the mark recipe if all you want to do is collect feedback on every example that comes in:

python -m prodigy mark your_dataset /path/to/file.jsonl

This will load the examples from file.jsonl, ask you to accept/reject each example and save the resulting annotations to the dataset your_dataset.

If you want to manually correct the examples (e.g. delete or add spans), you can use the ner.manual recipe, which also respects pre-defined spans in your data. Note that in this case, you do need a spaCy model for tokenization only to allow token-based highlighting.

python -m prodigy ner.manual your_dataset en_core_web_sm /path/to/file.jsonl --label supercon

You can find more details on this in the docs, specifically in the PRODIGY_README.html, available for download with Prodigy.

@lfoppiano

how can I relate each text with their respective batch of annotations? Should I use the ‘meta’ / ‘source’? if so, how? with a recipe?

Yes, using the meta is a good option. You can do this when you create your JSONL files – how you do that is up to you. You can do it by hand, write a Python script, or use any other programming language or tool you're comfortable with. In the end, you should have a file containing your data.

@lfoppiano

I’d like to create a recipe and add it to the ‘registered’ ones but again, this doesn’t seems to be the way it has been designed, so I can use a recipe only for annotation, correct?

I'm not 100% sure I understand your question. Recipes in Prodigy are Python scripts that let you set up annotation workflows. They can be executed from the command line, and if they return a dictionary of components, they start the annotation server. If you've written your own recipe, you can use it in Prodigy just like the other recipes – you just need to set -F path/to/recipe.py at the end of the command to point Prodigy to the file with your code, so it can load it.