How to keep count of annotations done by a person ?

If you care about who did the annotations, you ideally want to be starting multiple sessions of Prodigy, e.g. on separate ports, one for each annotator. You can do this manually, or programmatically in Python, a Bash script, etc. Each annotator can then receive their own dedicated dataset, and you'll be able to compare their annotations to others, and keep track of who did what.

If you search for "multiple annotators", you'll find various strategies for implementing a workflow multiple consumers and one annotation task queue. Streams are just Python generators, so you could implement a check that looks at the _task_hash(the unique identifier of an annotation question), and only yields out the question if it's not already present in any of the other datasets.

You might also be interested in Prodigy Scale, an extension we've been working on that will let you manage annotation teams and build large datasets:

In the meantime, you should also check out this open-source multi-user extension, developed by a fellow Prodigy user:

1 Like