# Duplicate annotations in output **URL:** \ **Category:** Getting Started\ **Tags:** bug, to-be-released, streams\ **Created:** [November 16, 2021, 4:59pm UTC](https://support.prodi.gy/t/duplicate-annotations-in-output/4961 "2021-11-16T16:59:27Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![cheyanneb](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/cheyanneb/32/2409_2.png) [@cheyanneb](https://support.prodi.gy/u/cheyanneb)\ **Post date:** [November 16, 2021, 4:59pm UTC](https://support.prodi.gy/t/duplicate-annotations-in-output/4961/1 "2021-11-16T16:59:27Z")
Hi, We have an input `.jsonl` file with 1000 pre-annotated documents. We have multiple users (3) working on the same dataset. The output contained 1025 annotations for all users, so the users annotated 25 of the documents twice. Input and task hashes are the same in the output for the dupes. We are all on the latest version of Prodigy (1.11.5). We did not use `force_stream_order`. User 1: - used a dataset with 1000 lines with no duplicates - saw 25 duplicates when running `db-out`. - saw 1025 completed tasks in the UI - was on 1.11.5 the entire time - dupes differed from other users - used textcat.manual recipe and a different dataset name than other two users User 2: - used a dataset with 1000 lines with no duplicates - saw only 4 duplicates when running `db-out`. - saw 1025 completed tasks in the UI - upgraded from 1.11.2 to 1.11.5 during annotation - dupes differed from other users - used mark recipe, same dataset name as User 3 User 3: - used a dataset with 1000 lines with no duplicates - saw 1004 completed tasks in the UI - upgraded from 1.11.3 to 1.11.5 during annotation - used mark recipe, same dataset name as User 2 - `db-out` shows 2008 lines, 1004 from User 2 and User 3 - db-out showed 4 dupes (did not match dupes from other users) We had set `feed_overlap` to true so that we all saw the same documents in the dataset. We are using the mark recipe. ```python PRODIGY_ALLOWED_SESSIONS=jane,john prodigy mark october_dataset .jsonl --view-id classification ``` Thanks, Cheyanne ---
**Author:** ![kab](https://avatars.discourse-cdn.com/v4/letter/k/9dc877/32.png) [@kab](https://support.prodi.gy/u/kab)\ **Post date:** [November 24, 2021, 8:00pm UTC](https://support.prodi.gy/t/duplicate-annotations-in-output/4961/2 "2021-11-24T20:00:47Z")
Hi! Seems you definitely found an issue here. I'll be taking a look today. It would be helpful to know around what point in your input data you start to see duplicates if that's something you can share (towards the beginning, somewhere in the middle, only at the end). ---
**Author:** ![cheyanneb](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/cheyanneb/32/2409_2.png) [@cheyanneb](https://support.prodi.gy/u/cheyanneb)\ **Post date:** [November 29, 2021, 4:19pm UTC](https://support.prodi.gy/t/duplicate-annotations-in-output/4961/3 "2021-11-29T16:19:43Z")
Thank you for looking into this! We all noticed some dupes before all tasks were complete (documents seemed familiar, and `db-out` confirmed the dupes), but Users 2 and 3 upgraded Prodigy to the latest version at that time and thought that might solve the issue. It didn't solve the issue. The significant dupes were discovered at the end when each user completed their tasks. We all had the same input (1000 documents), but the UI showed additional tasks and `db-out` confirmed we each had dupes in our final set. ---
**Author:** ![cheyanneb](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/cheyanneb/32/2409_2.png) [@cheyanneb](https://support.prodi.gy/u/cheyanneb)\ **Post date:** [December 7, 2021, 7:15pm UTC](https://support.prodi.gy/t/duplicate-annotations-in-output/4961/4 "2021-12-07T19:15:35Z")
Had another instance of duplicates in the output in a recent annotation exercise, this time with a custom recipe. This was a single user. Original file had 828 documents, and the user received 840 tasks. Any update on this @kab? ---
**Author:** ![marc](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/marc/32/2500_2.png) [@marc](https://support.prodi.gy/u/marc)\ **Post date:** [December 8, 2021, 3:49am UTC](https://support.prodi.gy/t/duplicate-annotations-in-output/4961/5 "2021-12-08T03:49:38Z")
Hello, I am seeing a very similar bug. I was, at first, using named sessions. Thinking it could be the problem, I stopped using them and the problem still occurs. Prodigy asks to annotate doc that have already been annotated. I am on version 1.11.6 and I did not upgrade during the process. Looks like `prodigy progress` is able to spot that there are duplicates : ```python New Unique Total Unique -------- --- ------ ----- ------ Dec 2021 122 109 122 109 ``` Start command : ```python prodigy ner.correct testdataset ./model_7500/model-best ./shuffled_data.json --label NAME,PHONE ``` And my prodigy.json file : ```python { "host": "0.0.0.0", "port": 8081, "show_stats": true, "show_flag": true, "ui_lang": "fr", "feed_overlap": false, "custom_theme": { "labels": { "NAME": "#fabed4", "PHONE": "#aaffc3" } }, "keymap_by_label": {"NAME": "q", "PHONE": "e"}, "keymap": {"accept":["d"]} } ``` Thanks ---
**Author:** ![kab](https://avatars.discourse-cdn.com/v4/letter/k/9dc877/32.png) [@kab](https://support.prodi.gy/u/kab)\ **Post date:** [December 8, 2021, 3:48pm UTC](https://support.prodi.gy/t/duplicate-annotations-in-output/4961/6 "2021-12-08T15:48:01Z")
Hi sorry for the lack of response, I have been looking into this a bit but it's been tough to debug. I'm dedicating the rest of my week to it and I'll update this thread when I make some headway. ---
**Author:** ![kab](https://avatars.discourse-cdn.com/v4/letter/k/9dc877/32.png) [@kab](https://support.prodi.gy/u/kab)\ **Post date:** [December 9, 2021, 6:17pm UTC](https://support.prodi.gy/t/duplicate-annotations-in-output/4961/7 "2021-12-09T18:17:01Z")
Quick question for you both because I was able to identify a specific case where duplicates were being shown. How quickly are you answering questions? If I answer questions very quickly (basically just immediately hitting accept) I can run into a state where duplicates are queued. I've fixed the underlying issue for this and this will be released in version 1.11.7 however I'm not convinced this is the only issue. Thanks for bearing with us on this! ---
**Author:** ![cheyanneb](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/cheyanneb/32/2409_2.png) [@cheyanneb](https://support.prodi.gy/u/cheyanneb)\ **Post date:** [December 9, 2021, 6:32pm UTC](https://support.prodi.gy/t/duplicate-annotations-in-output/4961/8 "2021-12-09T18:32:06Z")
We had some really simple tasks (true or false) where this happened, so often 5-10 seconds. We had more complex tasks, though, where it took a bit longer and this occurred, but still most likely less than a minute per doc. ---
**Author:** ![marc](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/marc/32/2500_2.png) [@marc](https://support.prodi.gy/u/marc)\ **Post date:** [December 10, 2021, 8:40pm UTC](https://support.prodi.gy/t/duplicate-annotations-in-output/4961/9 "2021-12-10T20:40:23Z")
We are using ner.correct and when the model is 100% right, we see it in less then a second and accept really quickly! ---
**Author:** ![kab](https://avatars.discourse-cdn.com/v4/letter/k/9dc877/32.png) [@kab](https://support.prodi.gy/u/kab)\ **Post date:** [December 10, 2021, 10:54pm UTC](https://support.prodi.gy/t/duplicate-annotations-in-output/4961/10 "2021-12-10T22:54:44Z")
Great thank you both for the replies. I think we'll release a fix for this particular issue early next week, sounds like you both might be running into it. ---
**Author:** ![cheyanneb](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/cheyanneb/32/2409_2.png) [@cheyanneb](https://support.prodi.gy/u/cheyanneb)\ **Post date:** [December 13, 2021, 1:33pm UTC](https://support.prodi.gy/t/duplicate-annotations-in-output/4961/11 "2021-12-13T13:33:06Z")
Thanks @kab! How can we access the fix when it becomes available? Via an emailed link? ---
**Author:** ![kab](https://avatars.discourse-cdn.com/v4/letter/k/9dc877/32.png) [@kab](https://support.prodi.gy/u/kab)\ **Post date:** [December 14, 2021, 10:53pm UTC](https://support.prodi.gy/t/duplicate-annotations-in-output/4961/12 "2021-12-14T22:53:20Z")
We just published an alpha version (1.11.7a0). We're still trying to track down another potential issue with duplicates but please try this out and report any issues back on this thread when you get the chance. You can install from our PyPI server using your Prodigy License Key: ```python pip install prodigy==1.11.7a0 -f https://XXXX-XXXX-XXXX-XXXX@download.prodi.gy ``` ---
**Author:** ![valentinoli](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/valentinoli/32/2642_2.png) [@valentinoli](https://support.prodi.gy/u/valentinoli)\ **Post date:** [December 15, 2021, 9:39am UTC](https://support.prodi.gy/t/duplicate-annotations-in-output/4961/13 "2021-12-15T09:39:32Z")
Linking to my reply to another thread which might be related to this one. > [@Prodigy looping over again on the same texts to annotate](https://support.prodi.gy/t/prodigy-looping-over-again-on-the-same-texts-to-annotate/4719/14): > > My initial report stated we were not experiencing any more problems after updating to v1.11.6. However, I confirm now that we are still seeing duplicated tasks. Not as many as before, and none of our annotators have reported any problems like they did before. So my guess is that they don't appear in a loop anymore but only pop up subtly so the annotator doesn't even notice. So it might be that one part of the problem has been fixed, i.e. the "looping tasks" problem. I suspect the remaining part … ---
**Author:** ![cheyanneb](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/cheyanneb/32/2409_2.png) [@cheyanneb](https://support.prodi.gy/u/cheyanneb)\ **Post date:** [December 15, 2021, 5:12pm UTC](https://support.prodi.gy/t/duplicate-annotations-in-output/4961/14 "2021-12-15T17:12:09Z")
I was unable to install `1.11.7a0` using the license key. Is there a wheel available? ---
**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\ **Post date:** [December 15, 2021, 6:54pm UTC](https://support.prodi.gy/t/duplicate-annotations-in-output/4961/15 "2021-12-15T18:54:20Z")
> [@cheyanneb](#): > > I was unable to install `1.11.7a0` using the license key What was the exact problem here? The PyPi server is really the easiest and most convenient way for us to distribute alpha releases. If there's no other way we could mail you a wheel, but it'd be better to get the PyPi download working. ---
**Author:** ![cheyanneb](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/cheyanneb/32/2409_2.png) [@cheyanneb](https://support.prodi.gy/u/cheyanneb)\ **Post date:** [December 15, 2021, 6:58pm UTC](https://support.prodi.gy/t/duplicate-annotations-in-output/4961/16 "2021-12-15T18:58:08Z")
```python ERROR: Could not find a version that satisfies the requirement prodigy==1.11.7a0 (from versions: none) ERROR: No matching distribution found for prodigy==1.11.7a0 ``` ---
**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\ **Post date:** [December 15, 2021, 7:01pm UTC](https://support.prodi.gy/t/duplicate-annotations-in-output/4961/17 "2021-12-15T19:01:49Z")
I just tested it and it works fine for me 🤔 Are you sure your license key is active and correct? ---
**Author:** ![cheyanneb](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/cheyanneb/32/2409_2.png) [@cheyanneb](https://support.prodi.gy/u/cheyanneb)\ **Post date:** [December 15, 2021, 7:38pm UTC](https://support.prodi.gy/t/duplicate-annotations-in-output/4961/18 "2021-12-15T19:38:56Z")
Yes, we received the license keys on September 13, 2021. ---
**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\ **Post date:** [December 16, 2021, 12:05pm UTC](https://support.prodi.gy/t/duplicate-annotations-in-output/4961/19 "2021-12-16T12:05:18Z")
> [@cheyanneb](#): > > Yes, we received the license keys on September 13, 2021. Can you install other versions via the PyPi server? I definitely want to investigate whether there's a problem with your license key so maybe you could send us an email with your order ID and/or license key so I can have a look? ---
**Author:** ![cheyanneb](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/cheyanneb/32/2409_2.png) [@cheyanneb](https://support.prodi.gy/u/cheyanneb)\ **Post date:** [December 16, 2021, 2:30pm UTC](https://support.prodi.gy/t/duplicate-annotations-in-output/4961/20 "2021-12-16T14:30:37Z")
I tried this again today with a different key (we have company list of 5) and didn't get the error, so I have successfully upgraded. [Next page](https://support.prodi.gy/t/duplicate-annotations-in-output/4961.md?page=2)