How to keep count of annotations done by a person ?

8

I never fully documented the multiuser_prodigy code and it'll be superseded by Prodigy Scale, but I can quickly walk through what it's doing.

  1. it draws the tasks to be annotated from a MongoDB that gets filled up from a JSONL loaded in with this code
  2. There's some logic that governs how examples are pulled from the Mongo (for example, I wanted three annotators to see each document).
  3. The coder identity is hackily stored in the Prodigy output here and here.
  4. Information about what each annotator has done is also written back to the Mongo, both to prevent an annotator from seeing the same example twice, and also so summaries of what each coder has done can be written out and displayed as a report.
2 Likes