# Proper way to add new samples to an existing database **URL:** \ **Category:** Uncategorized\ **Tags:** usage, ner, solved, multi-user\ **Created:** [June 30, 2022, 1:52pm UTC](https://support.prodi.gy/t/proper-way-to-add-new-samples-to-an-existing-database/5762 "2022-06-30T13:52:44Z")\ **Posts on this page:** 5\ **Page:** 1

**Author:** ![klopez](https://avatars.discourse-cdn.com/v4/letter/k/67e7ee/32.png) [@klopez](https://support.prodi.gy/u/klopez)\ **Post date:** [June 30, 2022, 1:52pm UTC](https://support.prodi.gy/t/proper-way-to-add-new-samples-to-an-existing-database/5762/1 "2022-06-30T13:52:44Z")

Hi I am currently using ner.manual to annotate samples using a text file as input. I recently trained a model on these annotations and found that the model is performing poorly and I believe it needs more examples (using \< 500 samples on en\_core\_web\_trf model). I have two questions: 1. What is the proper procedure to add new samples to the db without losing the current annotations. 2. If I had a reviewer already review annotations from the aforementioned db, would updating the first database influence the reviews task (i.e adding the new samples after the annotator annotates the newly added samples) My assumption is that we would just add/append text to the input text file and prodigy will figure out (using hashes) which text has been annotated and which ones are new and will add it as so to the annotators task. If its a different procedure could you describe it? ---

**Author:** ![ryanwesslen](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ryanwesslen/32/2969_2.png) [@ryanwesslen](https://support.prodi.gy/u/ryanwesslen)\ **Post date:** [June 30, 2022, 8:25pm UTC](https://support.prodi.gy/t/proper-way-to-add-new-samples-to-an-existing-database/5762/2 "2022-06-30T20:25:55Z")

hi @klopez! > [@klopez](#): > > What is the proper procedure to add new samples to the db without losing the current annotations. Yes, your assumption is correct - you can just add/append the text to the same database. You're right - this is where the [hashes come in](https://prodi.gy/docs/api-loaders/#hashing) to detect deduplications (either on the input or task level). The one exception may be if you had created the labels somewhere else and then uploaded them (`db-in`) they may not have the correct hashes. Also it may be important to note the `exclude_by` setting in the [configuration file](https://prodi.gy/docs/install#config). This will set which [hash](https://prodi.gy/docs/api-loaders#hashing) to use (`"task"` or `"input"` ) to determine whether two examples are the same and an incoming example should be excluded if it’s already in the dataset. By default the value is set to `"task"`. > [@klopez](#): > > If I had a reviewer already review annotations from the aforementioned db, would updating the first database influence the reviews task (i.e adding the new samples after the annotator annotates the newly added samples) What recipe/interface would that reviewer use? I ask because the [`review` recipe](https://prodi.gy/docs/recipes#review) requires multiple datasets (e.g., `in-sets`). It is really meant to review and merge across different datasets by reviewer, so it may not be the one you were thinking of using since it sounded like you would have only 1 dataset. You could create a [custom recipe](https://prodi.gy/docs/custom-recipes) where you could have control. You could use some of the custom components like `validate_answer`, `before_db`, `get_session_id`, or `exclude` to create your own logic. Since it seems like you may have multiple annotators, have you seen how to appropriately set up [multi-user sessions](https://prodi.gy/docs/api-web-app#multi-user-sessions) by setting unique `session_ids`? By default Prodigy will create timestamped `session_ids`. Setting custom `session_ids` is helpful to name and keep track of different users (especially if you're saving them into the same dataset). If so, here's a good post on some helpful tips: > [@Resume Multi-Session Annotation](https://support.prodi.gy/t/resume-multi-session-annotation/4153): > > Hi, I am trying to establish inter-annotator agreement and running text-classification annotations in multiple sessions (with feed\_overlap = true and force\_stream\_order = true, with ?session=session\_name). There are two issues which I have been running into, which I hope you can help me out with: Resuming annotation The annotation process was killed at some point - I have my dataset and annotations stored savely in the database, however, am looking for a way how I could "resume" annotations… Thanks for your question! ---

**Author:** ![klopez](https://avatars.discourse-cdn.com/v4/letter/k/67e7ee/32.png) [@klopez](https://support.prodi.gy/u/klopez)\ **Post date:** [July 1, 2022, 3:53pm UTC](https://support.prodi.gy/t/proper-way-to-add-new-samples-to-an-existing-database/5762/3 "2022-07-01T15:53:43Z")

Thank you for your response @ryanwesslen This is excellent. Just to clarify, I assume that if I set it to input for the `exclude_by` setting this would exclude samples by the hash of the text. So I would just append the new text and it will add new "tasks" for that annotator. And in theory if I were to add the same text again it should not show up on the annotators task list. Reviewer background: For the reviewer, this review recipe can take in 1 in-set. It will show the difference between the two annotators and allow the reviewer to adjudicate between the two. I have set the two annotators with their own session for ease of identification. We use this review task to get a gold standard dataset to train a model on later. Currently our annotators have annotated each sample giving 2 annotators per sample and our reviewer has reviewed and adjudicated all the samples and generated a gold dataset. Question: The crux of this question is if the annotators now have new samples to work on, and they complete the task, will the reviewer (after the annotators finish of course) get new samples to review? or will we have to restart the entire review task again? ---

**Author:** ![klopez](https://avatars.discourse-cdn.com/v4/letter/k/67e7ee/32.png) [@klopez](https://support.prodi.gy/u/klopez)\ **Post date:** [July 1, 2022, 7:08pm UTC](https://support.prodi.gy/t/proper-way-to-add-new-samples-to-an-existing-database/5762/4 "2022-07-01T19:08:37Z")

Okay, I have actually tested this on a dummy db, I did the following: 1. I set the exclude\_by flag to input, and set feed\_overlap to true 2. I created a new text file with some sentences, and started a ner.manual task with labels 3. I then annotated it with 2 annotators 4. I then started a review task to adjudicate the annotators 5. I then added/appended text to the original text file 6. The annotators only saw the new text in their tasks! 7. I then ran the review on the newly updated db to see if it would make me review all again. It only loaded the new samples 8. I then took the input file, copied its contents into a new file, added some new sentences, and used this as input to the task. (same command just different input name/file with same data) 9. This only showed the annotators new samples 10. Same with the review task, only new samples. 11. I then tried adding duplicate samples in the text file to see if it would add the duplicated samples into the task for the annotators, it did not! Thank you @ryanwesslen for all the help! ---

**Author:** ![ryanwesslen](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ryanwesslen/32/2969_2.png) [@ryanwesslen](https://support.prodi.gy/u/ryanwesslen)\ **Post date:** [July 1, 2022, 7:29pm UTC](https://support.prodi.gy/t/proper-way-to-add-new-samples-to-an-existing-database/5762/5 "2022-07-01T19:29:23Z")

Excellent! I really like your approach. If I find time, I may try to create a [mermaid](https://mermaid-js.github.io/mermaid/#/) diagram of the workflow and repost. This would be an excellent workflow to document -- especially if you have [inter-annotator agreement too](https://support.prodi.gy/t/proper-way-to-calculate-inter-annotator-agreement-for-spans-ner/5760/4)😃 ! Thank you!