Thanks for the report, that’s very interesting I’ll need to experiment and reproduce this, but my guess is that for some reason, the app doesn’t seem to correctly request the next batch when the queue is empty (i.e. always with batch size 1) if the task was updated before submission…
I actually get the error with larger batch sizes too (2 or 3), as soon as I’m through the batch and need to refresh. As with @plusepsilon, it only happens if I actually annotate text. It works fine if I just accept/reject.
Here’s a reproducible example (with the batch_size in prodigy.json set at 1-3):
prodigy mark assault_highlight_test highlight_quiz.jsonl --label ACTOR,EVENT,TARGET --view-id ner_manual
@ines Any progress on figuring out the bug? Our annotators are getting slowed down by having to refresh the page. I’m happy to help with reproducing the bug in any way I can!
@andy Oh, sorry, I didn’t realise it was blocking issue (I had it on my list as “weird quirk to fix before next release”.) I hope I can get to this by early next week! (I’ll be working on the annotation manager anyways, so it makes sense to investigate the potential issue here at the same time.)
Sorry to bug you again on this, but any suggestions for how to work or address the issue? I wouldn’t be asking if there weren’t annotators I feel bad for.
It’s mostly because of the multi-annotator setup we’re using. I want to update the DB of tasks as soon as a coder has completed each one to prevent double-coding, and we were losing annotations from coders forgetting to save their work before stopping. The latter is fixable with some reminders, but the first would require some reworking of our annotation setup. But you’re right, it could be the better of the two approaches.
So sorry, everything else ended up taking longer than expected But I finally had a look and tried out a possible patch for this issue – I’ve sent you an email with an updated bundle to test. If this works as expected and doesn’t have any bad side-effects, we can push the fix to v1.5.2!
@mikeross Sure, do you just want to send me an email to ines@explosion.ai? Sorry we haven’t shipped the fix yet – there’s a bunch of other stuff we wanted to push into that release but everything was taking longer than expected.