Help - Multiple Annotators setup

, , ,
1

Hello,
We seem to be stuck. We understand 1 personal license allows unlimited annotators. Need is to have multiple (10+) annotators, each annotating a separate document simultaneously. However, we are unable to get more than one annotator using Prodigy at the same time. The way it was setup each annotator has to remote-access log into the server. But basic Microsoft server set-up in AWS permits 2 users. Even with this setup 2 annotators cannot work at the same time.
How should it be setup so that 10 annotators can work simultaneously on a project setup on the 1 person license?
Can the annotators access prodigy via their web browser without remote desktop login to the server?
Thank you

2

Hi! Annotators who only label the data typically only need access to the Prodigy web app – so you wouldn't have to give them any remote server access? You'd just need to start the annotation tasks for them, serve the annotation UI on separate hosts/ports and give the annotators access to the URL of the respective instance.

3

Thank you for the very helpful information.
So, this is 1 project. Each annotator is tasked with 100 documents each (total approximately 1,000+ documents to be annotated).

What is considered an annotation task... that will require a separate host or port?

10 (1 for each annotator)?
1,000 (for each individual document)? or
1 for the one project?

4

Ah, sorry if the terminology I used was confusing! What I meant was the session/instance/process. There are generally two ways you could set this up and it depends on your project, annotators and how you want to set up the workflow:

  1. Start one Prodigy process one a given host/port per annotator and save the annotations to separate datasets. For example, on port 8080, you start the Prodigy server with 100 documents and save to annotations_andrew, on port 8081 you start Prodigy with the next 100 documents and save to annotations_ines, and so on.
    • Pros: very straightforward, easy to keep a separation between the work, no problem to update a model in the loop because every annotator has their own model instance and there are no conflicts
    • Cons: you have to run multiple processes (could be automated with a script but there is more going on), it's harder to share state between the sessions because process 1 doesn't know what process 2 has queued up
  2. Start Prodigy once and have multiple annotators access the same process using named sessions. For example, you would be accessing the app with ?session=andrew.
    • Pros: you only have to start one process, it's easier to send out examples to a different session based on shared state
    • Cons: can be harder to reason about because there's more state and more things influencing each other, harder to share a model in the loop because you have multiple people potentially updating it in different directions
3 Likes
5

Thank you!