# Change some annotations for existing dataset

**URL:** <https://support.prodi.gy/t/change-some-annotations-for-existing-dataset/3432>\
**Category:** Uncategorized\
**Tags:** usage, ner, database, review\
**Created:** [September 22, 2020, 12:45pm UTC](https://support.prodi.gy/t/change-some-annotations-for-existing-dataset/3432 "2020-09-22T12:45:49Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![jsnfly](https://avatars.discourse-cdn.com/v4/letter/j/f1d935/32.png) [@jsnfly](https://support.prodi.gy/u/jsnfly)\
**Post date:** [September 22, 2020, 12:45pm UTC](https://support.prodi.gy/t/change-some-annotations-for-existing-dataset/3432/1 "2020-09-22T12:45:49Z")

</div>

Hi!

I already have a dataset with ner-annotations, which I imported using the `db-in` command.  
Now I want to change some of these annotations, which I do by exporting the original dataset with the `db-out` command, filtering for the examples that I want to reannotate and feed them into the `ner.manual` recipe.  
The resulting examples have the correct (revised) spans, but the same `_task_hash` as the original examples. This I fixed by including the following in the `ner.manual` recipe:

```python
    def before_db(answers):
        if highlight_chars:
            answers = remove_tokens(answers)
        return [set_hashes(eg, overwrite=True) for eg in answers]

```

But what is the best way to combine these new annotations with the existing dataset? If I use the `review` recipe, it shows all of the examples but I'd like to review only those for which I changed the annotations. Also I don't really understand the `review` interface. What does accept/reject mean in this context and how do I create a "regular" dataset from the `review` dataset?  
And `db-merge` would only concatenate the two datasets, correct?

So is this really the best workflow for this problem or is there something more straightforward? And what is the best way to do this merge? Do I need to customize the `review` recipe?

Thanks!

---

<div class="post-metadata">

**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\
**Post date:** [September 23, 2020, 5:49pm UTC](https://support.prodi.gy/t/change-some-annotations-for-existing-dataset/3432/2 "2020-09-23T17:49:12Z")

</div>

> [@jsnfly](#):
>
> If I use the `review` recipe, it shows all of the examples but I'd like to review only those for which I changed the annotations. Also I don't really understand the `review` interface. What does accept/reject mean in this context and how do I create a "regular" dataset from the `review` dataset?

Hi! The idea of the `review` interface and workflow is basically to let you walk through all annotations again and create a "master annotation session" where you have the final say on every example (while being able to view all other answers – e.g. the different versions created in the manual interfaces or the accept/reject decisions made by different annotators). The data you create will have the same format as a regular Prodigy task, so you can export and train from it. It will also preserve the versions, so you can always reconstruct what lead to the decision.

The accept/reject mapping works just like it does in the regular annotation interfaces – for binary decisions, it's the answer, for manual decisions you can use "reject" to mark examples that are otherwise wrong (tokenization, broken markup etc).

> [@jsnfly](#):
>
> So is this really the best workflow for this problem or is there something more straightforward? And what is the best way to do this merge? Do I need to customize the `review` recipe?

One thing that maybe makes your workflow a bit special is that you already know what you want to re-annotate (which isn't always the case). This also means that you do need one extra process to filter out the examples you want to use, so Prodigy knows what to queue up.

A straightforward workflow for this could be:

- Run your script and create two segments of the data: one to reannotate, one to keep.
- Create a new dataset and import the annotations you want to keep.
- Start up the server with the recipe you want to use (e.g. `review` if you want to review multiple annotations on the same data, or a recipe like `ner.manual` if you just want to stream in the data again). Save the annotations you collect to the previously created set.