Thanks for the report, that’s very interesting I’ll need to experiment and reproduce this, but my guess is that for some reason, the app doesn’t seem to correctly request the next batch when the queue is empty (i.e. always with batch size 1) if the task was updated before submission…
I actually get the error with larger batch sizes too (2 or 3), as soon as I’m through the batch and need to refresh. As with @plusepsilon, it only happens if I actually annotate text. It works fine if I just accept/reject.
Here’s a reproducible example (with the batch_size in prodigy.json set at 1-3):
prodigy mark assault_highlight_test highlight_quiz.jsonl --label ACTOR,EVENT,TARGET --view-id ner_manual
@andy Oh, sorry, I didn’t realise it was blocking issue (I had it on my list as “weird quirk to fix before next release”.) I hope I can get to this by early next week! (I’ll be working on the annotation manager anyways, so it makes sense to investigate the potential issue here at the same time.)
It’s mostly because of the multi-annotator setup we’re using. I want to update the DB of tasks as soon as a coder has completed each one to prevent double-coding, and we were losing annotations from coders forgetting to save their work before stopping. The latter is fixable with some reminders, but the first would require some reworking of our annotation setup. But you’re right, it could be the better of the two approaches.
So sorry, everything else ended up taking longer than expected But I finally had a look and tried out a possible patch for this issue – I’ve sent you an email with an updated bundle to test. If this works as expected and doesn’t have any bad side-effects, we can push the fix to v1.5.2!
@mikeross Sure, do you just want to send me an email to email@example.com? Sorry we haven’t shipped the fix yet – there’s a bunch of other stuff we wanted to push into that release but everything was taking longer than expected.