# Duplicate examples shown after restarting server

**URL:** <https://support.prodi.gy/t/duplicate-examples-shown-after-restarting-server/424>\
**Category:** Uncategorized\
**Tags:** done\
**Created:** [March 21, 2018, 7:23am UTC](https://support.prodi.gy/t/duplicate-examples-shown-after-restarting-server/424 "2018-03-21T07:23:06Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![mikaell](https://avatars.discourse-cdn.com/v4/letter/m/e9c0ed/32.png) [@mikaell](https://support.prodi.gy/u/mikaell)\
**Post date:** [March 21, 2018, 7:23am UTC](https://support.prodi.gy/t/duplicate-examples-shown-after-restarting-server/424/1 "2018-03-21T07:23:06Z")

</div>

I noticed that if I rerun ner.make-gold with the same dataset and the same data file, I will (at least sometimes) be asked to label the same example twice. In fact, after exporting the annotated data, I see that two annotated examples have the same input\_hash and task\_hash.

I’m assuming this isn’t the intended behavior? I can manually remove already-annotated examples from the data file every time I restart ner.make-gold, but it would be nice if the recipe would automatically realize to not show the same example twice.

---

<div class="post-metadata">

**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\
**Post date:** [March 21, 2018, 12:40pm UTC](https://support.prodi.gy/t/duplicate-examples-shown-after-restarting-server/424/2 "2018-03-21T12:40:32Z")

</div>

Thanks for the report. This is strange and definitely not intended and one set shouldn’t contain examples for the same hashes. Did you spot the duplicate examples towards the end of the dataset or last? This would indicate that maybe there’s an unintended overlap somewhere.

I suspect that the cause of this problem might be related to this issue: [Old examples are automatically added to new dataset](https://support.prodi.gy/t/old-examples-are-automatically-added-to-new-dataset/222/11). I’m pretty sure we tracked this down to a problem with how the database handles the input and task hashes, and we’ve already fixed that for the upcoming v1.4.1 (just running a few test and will release the update asap, so you can check if it fixes the problem).

---

<div class="post-metadata">

**Author:** ![mikaell](https://avatars.discourse-cdn.com/v4/letter/m/e9c0ed/32.png) [@mikaell](https://support.prodi.gy/u/mikaell)\
**Post date:** [March 21, 2018, 6:31pm UTC](https://support.prodi.gy/t/duplicate-examples-shown-after-restarting-server/424/3 "2018-03-21T18:31:35Z")

</div>

It looked like after restarting the command I got the same examples again from the beginning of the file. But thanks for the quick reply, I’ll see if this got fixed in the new version already.

---

<div class="post-metadata">

**Author:** ![baeumer](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/baeumer/32/260_2.png) [@baeumer](https://support.prodi.gy/u/baeumer)\
**Post date:** [January 14, 2022, 11:00am UTC](https://support.prodi.gy/t/duplicate-examples-shown-after-restarting-server/424/4 "2022-01-14T11:00:31Z")

</div>

Sorry for resurrecting such an old post. We are currently seeing the same behavior. After restarting the application, prodigy starts presenting the data from the beginning again. We are working with a MySQL connection. The version we are using is 1.11.7 Any tips?

---

<div class="post-metadata">

**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\
**Post date:** [January 17, 2022, 10:37am UTC](https://support.prodi.gy/t/duplicate-examples-shown-after-restarting-server/424/5 "2022-01-17T10:37:48Z")

</div>

@baeumer Are the hashes in the incoming examples the same as the hashes of the duplicate examples already in the dataset?