duplicate images when annotating

Thanks for clarifying about the jsonl file, and sharing your DB timings, it was helpful!

I was able to reproduce seeing duplicates by holding down the shortcut key to accept images as quickly as possible until all 100 were done. :sweat_smile: When I did that, prodigy would show me 101 or 102 images, but only store the expected 100 in the database.

I think the trouble is that when you answer questions at the same time as the app asks for more, it can miss reporting one or two items that you've answered but not yet sent back to the server, which causes the server to send them again.

I came up with a fix that filters received duplicates from the server. It fixes the duplicates in all my tests. Would you be willing to try a preview build with my fix? If so, please send me an email at justin@explosion.ai.

Thanks,
-Justin