# Share datasets between prodi gy users **URL:** \ **Category:** Uncategorized\ **Tags:** usage, database\ **Created:** [May 5, 2020, 12:10pm UTC](https://support.prodi.gy/t/share-datasets-between-prodi-gy-users/2870 "2020-05-05T12:10:08Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![Arnault](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/arnault/32/4257_2.png) [@Arnault](https://support.prodi.gy/u/Arnault)\ **Post date:** [May 5, 2020, 12:10pm UTC](https://support.prodi.gy/t/share-datasets-between-prodi-gy-users/2870/1 "2020-05-05T12:10:08Z")
Hi, What are the best ways to share an annotation database between multiple data scientists ? I guess one way would be to share a common external database. But it might raise security, connectivity and quality concerns (trial and errors during recipe iterations). Is the best option to use `db-in` & `db-out` to share the data as a jsonl ? Is there a way to have multiple databases : a local one and a remote one ? Thanks a lot for your help and prodi gy. ---
**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\ **Post date:** [May 5, 2020, 10:02pm UTC](https://support.prodi.gy/t/share-datasets-between-prodi-gy-users/2870/2 "2020-05-05T22:02:13Z")
Hi! Having a single shared remote database is probably the most straightforward solution. It doesn't have to be in the cloud โ€“ if everyone has access to the same shared drive, you could set the `PRODIGY_HOME` environment variable so everyone uses the same config and writes to the same SQLite database file. Is your main motivation for sharing datasets to make sure that examples aren't annotated multiple times, and so you can exclude examples if they're already in the dataset? > [@Arnault](#): > > Is the best option to use `db-in` & `db-out` to share the data as a jsonl ? > Is there a way to have multiple databases : a local one and a remote one ? That's definitely an option, yes. If passing around files is too messy, a more elegant solution would be to use the [database API in Python](https://prodi.gy/docs/api-database#database) and write a script that syncs your annotations. By default, Prodigy expects to write the examples to one database. However, you could write your own custom [`Database` class](https://prodi.gy/docs/api-database#database) or implement your own logic to save to a second locations. You could even have a custom recipe that uses the `update` callback to send completed annotations (or just their [task hashes](https://prodi.gy/docs/api-loaders#hashing)!) to a remote database. If you're just storing the hashes, you'll likely won't have any data privacy issues โ€“ but you can still use them to filter examples and detect whether something is a duplicate or not. ---
**Author:** ![Arnault](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/arnault/32/4257_2.png) [@Arnault](https://support.prodi.gy/u/Arnault)\ **Post date:** [May 6, 2020, 10:00pm UTC](https://support.prodi.gy/t/share-datasets-between-prodi-gy-users/2870/3 "2020-05-06T22:00:01Z")
Hi Ines ! My main motivation would be to have a common annotation base : a way for the team to share their datasets and exploit them via prodigy. At the same time a personal database (from one ds) can contain imperfect annotations or undocumented datasets. I was wondering if you had any advice on a flow, a format and any good practice to update and download annotations from a shared annotation base. Thanks for the various tips on the Database api. ---
**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\ **Post date:** [May 7, 2020, 11:48am UTC](https://support.prodi.gy/t/share-datasets-between-prodi-gy-users/2870/4 "2020-05-07T11:48:20Z")
Yes, that makes a lot of sense and is a good way to approach datasets IMO. Maybe you could just implement a simple command/recipe that each developer can run to add their annotations to the "master database". Under the hood, that would just use `db.get_dataset` (personal DB) and then `db.add_examples` (master DB). Since it's just a Python script, you can do pretty much anything in there โ€“ for example, you could even make it send you a Slack notification like "Arnault just added 345 examples to master dataset xyz and said: 'Annotation done! Let me know what you think!'" ๐Ÿ˜… If you have overlapping annotations (e.g. same data annotated by different people, potentially with conflicts), you might also want to look at the `review` recipe as a way to create "merged master datasets": [Built-in Recipes ยท Prodigy ยท An annotation tool for AI, Machine Learning & NLP](https://prodi.gy/docs/recipes#review) You might also find this comment helpful โ€“ it's more about general strategies for annotating and developing together as a team: > [@How to use Prodigy for NER labelling as a team?](https://support.prodi.gy/t/how-to-use-prodigy-for-ner-labelling-as-a-team/2487/2): > > Hi! There's no one correct answer and how you structure your projects depends on your specific use case. When you're starting a new project that involves training a model and collecting new data, you'll usually have two distinct phases: Development phase. During this phase, you'd validate ideas for the model and it typically requires the data scientists/developers working very closely with the annotators, and often even annotating small samples themselves to test what works best. Even if the โ€ฆ