I using Doccano and considering moving to Prodigy. Is there a way to allow a team of 10-20 people to annotate but only have 3 people at most annotate a text? Also is there a way to assign one of the of person as a "approver" or verifier of annotations?
When we were laying out the design for Prodigy, there are some features that we decided didn't belong so naturally within the design of a command-line oriented, interactive tool. Intricate annotator management and task-allocation policies is one of those features, because once you're working with a lot of annotators like that, the balance of considerations shifts in a different direction.
We're building Prodigy Teams as a separate product to provide features like annotator accounts, task allocations, starting and stopping annotation tasks from a GUI, review and conflict resolution, etc. The primary difference is that current Prodigy (which I guess we can call "Prodigy Standalone" in contrast) doesn't ask to be wrapped in a container, hosted on a server, and connected to a hosted database. You're often running it yourself with Python scripts and a local SQLite database.
All that said, there are some features that let you do what you want with Prodigy. You can set up a different feed for different annotators, and have each annotator visit a different URL, differentiated via query string. You could even have a web server in front that authenticated the users separately.
Prodigy also features review recipes designed for the conflict resolution (and these are the same recipes we'll be using ourselves in Prodigy Teams --- so they'll stay updated and improved). By default, the recipe expects to be run in a batch-wise fashion, I don't think it constantly pulls new updates as they're pushed to the individual datasets. So if you wanted to leave the task up persistently, you'd need to modify the recipe a bit. Alternatively you can start and stop the service at discrete intervals.
1 Like