# Editing datasets

**URL:** <https://support.prodi.gy/t/editing-datasets/66>\
**Category:** Uncategorized\
**Tags:** usage, database, solved\
**Created:** [October 25, 2017, 11:37am UTC](https://support.prodi.gy/t/editing-datasets/66 "2017-10-25T11:37:39Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Mede](https://avatars.discourse-cdn.com/v4/letter/m/a3d4f5/32.png) [@Mede](https://support.prodi.gy/u/Mede)\
**Post date:** [October 25, 2017, 11:37am UTC](https://support.prodi.gy/t/editing-datasets/66/1 "2017-10-25T11:37:39Z")

</div>

I may have missed it in the documentation, but is there some functionality for editing datasets, i.e. renaming, copying or deleting them, or editing the comments to them?

---

<div class="post-metadata">

**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\
**Post date:** [October 25, 2017, 1:05pm UTC](https://support.prodi.gy/t/editing-datasets/66/2 "2017-10-25T13:05:18Z")

</div>

There are currently the following commands to interact with the datasets in the database:

- `prodigy db-in`: Import annotations to a dataset.
- `prodigy db-out`: Export annotations from a dataset or session.
- `prodigy drop`: Remove a dataset or session from the database.

To view the datasets and sessions (each individual annotation session, named after the timestamp), you can use the `prodigy stats` command:

```bash
prodigy stats -l # view stats and list all datasets
prodigy stats -ls # view stats and list all datasets and sessions

```

We didn’t want to add too many arbitrary, Prodigy-specific commands to interact with the database at this point, because it easily gets messy and we weren’t sure how much the users would actually _really_ need. So for now, if you want to rename an existing dataset, or change the description, you’d have to export and re-add it:

```bash
prodigy db-out my_set /tmp
prodigy db-in my_new_set /tmp/my_set.jsonl "Some description"
prodigy drop my_set # optional: delete dataset

```

You can also preview an existing dataset on the command line using `ner.print-dataset` ([example output](https://prodi.gy/docs/recipes#ner-print-dataset)) and `textcat.print-dataset` ([example output](https://prodi.gy/docs/recipes#textcat-print-dataset)). If the dataset is large, I’d recommend using `less` so you can navigate through them (with the `-r` flag to make sure the colors are displayed correctly):

```bash
prodigy ner.print-dataset news_headlines | less -r

```

---

<div class="post-metadata">

**Author:** ![Mede](https://avatars.discourse-cdn.com/v4/letter/m/a3d4f5/32.png) [@Mede](https://support.prodi.gy/u/Mede)\
**Post date:** [October 25, 2017, 1:19pm UTC](https://support.prodi.gy/t/editing-datasets/66/3 "2017-10-25T13:19:24Z")

</div>

OK, great. This should be all I need for now. Thanks!

---

<div class="post-metadata">

**Author:** ![koaning](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/koaning/32/230_2.png) [@koaning](https://support.prodi.gy/u/koaning)\
**Post date:** [March 19, 2018, 3:34pm UTC](https://support.prodi.gy/t/editing-datasets/66/4 "2018-03-19T15:34:24Z")

</div>

It allmost feels like this should be part of the docs. Is there a place for everything you can do with the command line?

---

<div class="post-metadata">

**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\
**Post date:** [March 19, 2018, 3:58pm UTC](https://support.prodi.gy/t/editing-datasets/66/5 "2018-03-19T15:58:26Z")

</div>

> [@koaning](#):
>
> It allmost feels like this should be part of the docs. Is there a place for everything you can do with the command line?

The `PRODIGY_README.html` which is available for download with Prodigy has all those commands grouped into "Other recipes and commands" in the "Recipes" section of the table of contents. There's also an overview online [on the recipes page](https://prodi.gy/docs/recipes) – the examples focus more on the annotation recipes, but the database commands are also listed towards the end.

The thing is, there's not always a perfectly clear distinction between "commands" and "recipes" – technically, all Prodigy commands are also recipes, since they're wrapped in the `@recipe` decorator and use the same CLI style. You can also easily add your own Prodigy commands via custom recipes. (If a recipe doesn't return a dictionary of components, Prodigy will just execute the function and not start the server – so you could easily write your own database commands if there's extra functionality you need.)

---

<div class="post-metadata">

**Author:** ![yw2903](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/yw2903/32/1614_2.png) [@yw2903](https://support.prodi.gy/u/yw2903)\
**Post date:** [June 1, 2021, 6:53pm UTC](https://support.prodi.gy/t/editing-datasets/66/6 "2021-06-01T18:53:05Z")

</div>

Is there a way to drop certain annotations from a database, but not dropping the entire dataset?

---

<div class="post-metadata">

**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\
**Post date:** [June 2, 2021, 3:00am UTC](https://support.prodi.gy/t/editing-datasets/66/7 "2021-06-02T03:00:29Z")

</div>

> [@yw2903](#):
>
> Is there a way to drop certain annotations from a database, but not dropping the entire dataset?

How are you identifying the examples? Do you have their hashes, or are you using some custom logic for filtering?

There's no direct command for this because specifying task IDs on the CLI typically isn't that useful. But you could do this by filtering the examples to remove the ones you want to drop, saving them to a new dataset, and dropping the old one. (If you want to use the same name for both datasets, don't forget to back up your previous annotations – otherwise, a small bug in your code between dropping the old set and re-adding the examples can cause you to lose the data.)
