# Duplicated examples in db-out for ner.train

**URL:** <https://support.prodi.gy/t/duplicated-examples-in-db-out-for-ner-train/5896>\
**Category:** Uncategorized\
**Tags:** usage, ner, database\
**Created:** [August 30, 2022, 7:18am UTC](https://support.prodi.gy/t/duplicated-examples-in-db-out-for-ner-train/5896 "2022-08-30T07:18:21Z")\
**Posts on this page:** 1\
**Showing post:** 3

<div class="post-metadata">

**Author:** ![vsocrates](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/vsocrates/32/3218_2.png) [@vsocrates](https://support.prodi.gy/u/vsocrates)\
**Post date:** [August 30, 2022, 4:29pm UTC](https://support.prodi.gy/t/duplicated-examples-in-db-out-for-ner-train/5896/3 "2022-08-30T16:29:45Z")

</div>

Thanks so much for the fast and detailed answer!!

> [@ryanwesslen](#):
>
> Said differently, are you asking how are duplicates handled in `prodigy train`? That's a great question because by default, `prodigy train` will drop duplicates based on the `input_hash`.

Yes, that is what I was asking. I see, so in essence, `prodigy train` arbitrarily chooses one of the annotations to go with? The [link](https://support.prodi.gy/t/question-about-example-data-during-ner-batch-train/1833/2) you provided seems to suggest that the span annotations would be merged instead though, and not dropped. Please let me know which one is the case and how we can control the behavior?

> [@ryanwesslen](#):
>
> Just curious, did each annotator annotate all of the examples or were all records annotated only once?

I believe it was a mix, as not all examples were annotated by all annotators (mostly due to time/cost limitations), but there are definitely duplicate annotations.

We did use multi-user sessions, and have not modified `feed_overlap`, so it looks like that lines up!

---

_[View the full topic](https://support.prodi.gy/t/duplicated-examples-in-db-out-for-ner-train/5896)._