While it's a bit of a hack, you could include the annotator information in the task_hash
. This way, each annotator would be part of the task definition. This comes with some downsides, because you cannot identify each task on it's own anymore without the annotator info, but theoretically it'd do what you're asking for.
This task_hash
is documented on our docs here. In case it's of interest, the difference between the input and task hash is explained in detail here:
If you're working on a custom recipe you should be able to use the set_hashes function get the behavior. In your case, I imagine it would look something like:
from prodigy import set_hashes
stream = (add_annotator_info(eg, annotator_name) for eg in stream)
stream = (set_hashes(eg, input_keys=("text",), task_keys=("label", "options", "annotator")) for eg in stream)
Just be aware
That said, I do want to stress that this is a bit of a hack. If you're going to be working with multiple annotators you'd also want to have a system that can regulate the annotator overlap and this sort of functionality is planned for the Prodigy Teams product. A thread on this product, which is still in development, can be found here: