For two different annotators working on annotating a single file, on a single running instance of prodigy. I see that in both the sessions, the same text is appearing.
Sessions r1 and r2 are started by two users, the first example they see is the same.
In the window for session r1, the following example is tagged
,
In the session r1, six examples are tagged and saved,
Now in session r2, the example already tagged by r1(shown in screenshot above) reappears as following:
The named sessions support in prodigy has a few quirks. Specifically, when using force_stream_order=True and feed_overlap=False, if users annotate at the same time they can see duplicate questions. The server filters out duplicate answers before storing them in the database, so you end up with no overlap there.
The reason for this behavior is that when prodigy sends out questions, it doesn't know which ones have been asked but not answered by the client. So if multiple users ask for questions at the same time, that have to appear in the same order, there's not much we can do but show the same questions.
If you'd like to avoid all duplication for asked questions, you can set force_stream_order=False.
I have two users and feed_overlap and force_stream_order set to false. We were annotating at the same time and saw overlapping documents. I checked the output using db-out and also checked prodigy progress, and I confirmed the dupes. You mentioned that the server filters out duplicates before storing them? Are you saying that User 1 and User 2 cannot annotate at the same time? Thanks!
