Option feed_overlap=false doesn't show expected behaviour

Hi @rohit1

The named sessions support in prodigy has a few quirks. Specifically, when using force_stream_order=True and feed_overlap=False, if users annotate at the same time they can see duplicate questions. The server filters out duplicate answers before storing them in the database, so you end up with no overlap there.

The reason for this behavior is that when prodigy sends out questions, it doesn't know which ones have been asked but not answered by the client. So if multiple users ask for questions at the same time, that have to appear in the same order, there's not much we can do but show the same questions.

If you'd like to avoid all duplication for asked questions, you can set force_stream_order=False.