# \#database **URL:** https://support.prodi.gy/tag/database/10.md [Latest](https://support.prodi.gy/latest.md) · [Categories](https://support.prodi.gy/categories.md) · [Tags](https://support.prodi.gy/tags.md) --- ## [A landing page for annotation tasks available across multiple Prodigy instances?](https://support.prodi.gy/t/a-landing-page-for-annotation-tasks-available-across-multiple-prodigy-instances/7540)
**Author:** [@tuomo\_h](https://support.prodi.gy/u/tuomo_h)\ **Replies:** 4\ **Last updated:** [May 6, 2025, 8:50am UTC](https://support.prodi.gy/t/a-landing-page-for-annotation-tasks-available-across-multiple-prodigy-instances/7540 "2025-05-06T08:50:55Z")
Hey, we're currently developing fairly complex annotation pipelines using Prodigy with @helmiina, in which the tasks flow from one recipe to another. We will have multiple annotators working on various tasks in the pip… --- ## [Run prodigy with changing Postgres DB credentials](https://support.prodi.gy/t/run-prodigy-with-changing-postgres-db-credentials/7545)
**Author:** [@sara.vallejo](https://support.prodi.gy/u/sara.vallejo)\ **Replies:** 1\ **Last updated:** [April 25, 2025, 11:30am UTC](https://support.prodi.gy/t/run-prodigy-with-changing-postgres-db-credentials/7545 "2025-04-25T11:30:40Z")
I have a running Prodigy instance that connects to a Postgres DB. I am currently storing the credentials in prodigy.json and the connection works well. However, the credentials for this DB change every day at random time… --- ## [Run prodigy without a database](https://support.prodi.gy/t/run-prodigy-without-a-database/7541)
**Author:** [@lukasa](https://support.prodi.gy/u/lukasa)\ **Replies:** 1\ **Last updated:** [April 21, 2025, 10:32am UTC](https://support.prodi.gy/t/run-prodigy-without-a-database/7541 "2025-04-21T10:32:04Z")
Hi, I am trying to run prodigy without a database backend. The documentation ( Database · Prodigy · An annotation tool for AI, Machine Learning & NLP ) says " The database plugged in via a custom recipe can also be Fal… --- ## [No more tasks in ordered DB stream](https://support.prodi.gy/t/no-more-tasks-in-ordered-db-stream/7412)
**Author:** [@nicolaiberk](https://support.prodi.gy/u/nicolaiberk)\ **Replies:** 7\ **Last updated:** [December 17, 2024, 2:11pm UTC](https://support.prodi.gy/t/no-more-tasks-in-ordered-db-stream/7412 "2024-12-17T14:11:27Z")
Hi! I am running an interface streaming from a postgres database. This database gets a fixed size of weekly additional examples. Now my annotators do not always manage to keep up with this pace and this generates backlo… --- ## [Custom database connection being reset to prodigy.json paramters](https://support.prodi.gy/t/custom-database-connection-being-reset-to-prodigy-json-paramters/7458)
**Author:** [@FourthPartyAI](https://support.prodi.gy/u/FourthPartyAI)\ **Replies:** 2\ **Last updated:** [December 17, 2024, 10:30am UTC](https://support.prodi.gy/t/custom-database-connection-being-reset-to-prodigy-json-paramters/7458 "2024-12-17T10:30:37Z")
I have a postgres instance with two databases in it: my\_annotated\_db and my\_input\_db and I have custom stream manager that will watch my\_input\_db for new input, perform some custom operations, and then feed the stream ov… --- ## [Dataset download from web interface](https://support.prodi.gy/t/dataset-download-from-web-interface/7444)
**Author:** [@FourthPartyAI](https://support.prodi.gy/u/FourthPartyAI)\ **Replies:** 1\ **Last updated:** [November 27, 2024, 1:32pm UTC](https://support.prodi.gy/t/dataset-download-from-web-interface/7444 "2024-11-27T13:32:04Z")
Hi I'm relatively new to prodigy. Someone at my company is doing annotations for me, but they are non-technical and cannot really interact with Python. I've dockerized the spans.manual recipe to provide a simple web int… --- ## [Unable to delete dataset](https://support.prodi.gy/t/unable-to-delete-dataset/7397)
**Author:** [@sunnielou](https://support.prodi.gy/u/sunnielou)\ **Replies:** 12\ **Last updated:** [October 30, 2024, 4:28pm UTC](https://support.prodi.gy/t/unable-to-delete-dataset/7397 "2024-10-30T16:28:38Z")
Hi, I am trying to delete a dataset (and session associated) to create a new one. The dataset has 36269 annotations and I get this error: $ prodigy drop -n 500 uc-specific-reviewed ✘ Unable to delete dataset 'uc-spe… --- ## [Error when 'prodigy drop'](https://support.prodi.gy/t/error-when-prodigy-drop/928)
**Author:** [@Arul](https://support.prodi.gy/u/Arul)\ **Replies:** 10\ **Last updated:** [December 26, 2018, 7:00pm UTC](https://support.prodi.gy/t/error-when-prodigy-drop/928 "2018-12-26T19:00:06Z")
I get this error when i try to ‘drop’ the db. “prodigy drop equinor\_split\_20p” Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/peewee.py", line 3… --- ## [Adding new data to be annotated without re-starting the server](https://support.prodi.gy/t/adding-new-data-to-be-annotated-without-re-starting-the-server/6876)
**Author:** [@aCampello](https://support.prodi.gy/u/aCampello)\ **Replies:** 10\ **Last updated:** [November 3, 2023, 9:56am UTC](https://support.prodi.gy/t/adding-new-data-to-be-annotated-without-re-starting-the-server/6876 "2023-11-03T09:56:35Z")
Please let me know if my understanding is completely flawed. At the moment, you can add a jsonl files to be annotated for (say) NER by running: prodigy ner.manual my\_dataset en\_core\_web\_sm my\_file.jsonl --label my\_label… --- ## [Issues Setting Custom Database](https://support.prodi.gy/t/issues-setting-custom-database/6829)
**Author:** [@lpreuett](https://support.prodi.gy/u/lpreuett)\ **Replies:** 2\ **Last updated:** [October 17, 2023, 1:08pm UTC](https://support.prodi.gy/t/issues-setting-custom-database/6829 "2023-10-17T13:08:40Z")
I have been struggling to get prodigy to use my custom database. I have followed the examples available in support, the Mondigy code, and documentation all to no avail. I wrote a custom recipe which I debugged using the… --- ## [Querying Relations Layer Annotations (for Rhetoric Text Structure Analysis)](https://support.prodi.gy/t/querying-relations-layer-annotations-for-rhetoric-text-structure-analysis/6759)
**Author:** [@yanirmr](https://support.prodi.gy/u/yanirmr)\ **Replies:** 3\ **Last updated:** [September 6, 2023, 10:57am UTC](https://support.prodi.gy/t/querying-relations-layer-annotations-for-rhetoric-text-structure-analysis/6759 "2023-09-06T10:57:57Z")
Hello, I'm embarking on a project focused on tagging relationships between entities to annotate the structure of rhetorical texts. The goal is to identify and analyze structures such as claims, rhetorical questions, fac… --- ## [Configure a non-sqlite database (e.g. postgres) without storing the password in prodigy.json](https://support.prodi.gy/t/configure-a-non-sqlite-database-e-g-postgres-without-storing-the-password-in-prodigy-json/108)
**Author:** [@hunan-rostomyan](https://support.prodi.gy/u/hunan-rostomyan)\ **Replies:** 9\ **Last updated:** [August 29, 2023, 1:15am UTC](https://support.prodi.gy/t/configure-a-non-sqlite-database-e-g-postgres-without-storing-the-password-in-prodigy-json/108 "2023-08-29T01:15:39Z")
Is there a way to configure non-sqlite db settings without using the prodigy.json file? Currently, we do: "db\_settings": { "postgresql": { "host": "$PRODIGY\_DB\_HOST", "dbname": "$PRODIGY\_DB\_NAME", … --- ## [Integrations with the Knowledge graph](https://support.prodi.gy/t/integrations-with-the-knowledge-graph/6719)
**Author:** [@Harsha](https://support.prodi.gy/u/Harsha)\ **Replies:** 1\ **Last updated:** [August 7, 2023, 3:22pm UTC](https://support.prodi.gy/t/integrations-with-the-knowledge-graph/6719 "2023-08-07T15:22:37Z")
Hi Team, Is there any way to save the annotated data to knowledge graph directly, I have gone through the Database · Prodigy · An annotation tool for AI, Machine Learning & NLP page and couldn't find any. Thanks. --- ## [Use AWS RDS PostGres to store annotations](https://support.prodi.gy/t/use-aws-rds-postgres-to-store-annotations/6713)
**Author:** [@vivek\_iiitm1](https://support.prodi.gy/u/vivek_iiitm1)\ **Replies:** 1\ **Last updated:** [August 6, 2023, 7:43pm UTC](https://support.prodi.gy/t/use-aws-rds-postgres-to-store-annotations/6713 "2023-08-06T19:43:21Z")
Hello, We would like to use AWS managed RDS Postgres service to store the annotations. We have modified below config but we are not able to make the connection. Any idea if this configuration is supported. { "port": 8… --- ## [Where is the annotation saved when prodigy.json is empty?](https://support.prodi.gy/t/where-is-the-annotation-saved-when-prodigy-json-is-empty/5798)
**Author:** [@merleyc](https://support.prodi.gy/u/merleyc)\ **Replies:** 1\ **Last updated:** [July 15, 2022, 6:06pm UTC](https://support.prodi.gy/t/where-is-the-annotation-saved-when-prodigy-json-is-empty/5798 "2022-07-15T18:06:32Z")
I am using this command: python -m prodigy textcat.manual mykeyword myjsonfile.jsonl" --label labl1,label2,label3 The following file is empty: /home/myuser/.prodigy But after the annotation was completed, I click… --- ## [get\_hashes method in v1.12](https://support.prodi.gy/t/get-hashes-method-in-v1-12/6677)
**Author:** [@Michelle-Ming96](https://support.prodi.gy/u/Michelle-Ming96)\ **Replies:** 8\ **Last updated:** [July 18, 2023, 6:36pm UTC](https://support.prodi.gy/t/get-hashes-method-in-v1-12/6677 "2023-07-18T18:36:15Z")
Hello, I am using a custom database to retrieve text and store annotations. I've set the configuration "allow\_work\_stealing" to "false" in the Prodigy config file. However, I encountered an error: AttributeError: 'Annota… --- ## [custom task routing](https://support.prodi.gy/t/custom-task-routing/6590)
**Author:** [@ngawangtrinley](https://support.prodi.gy/u/ngawangtrinley)\ **Replies:** 13\ **Last updated:** [June 26, 2023, 11:23am UTC](https://support.prodi.gy/t/custom-task-routing/6590 "2023-06-26T11:23:17Z")
so we are trying to use the latest prodigy==1.12a3 and we are using it for task routing. we have many annotators so we want to use task router to route specific image to a specific annotator using the stream id and sessi… --- ## [Database performance/connection issues with a remote postgresql DB](https://support.prodi.gy/t/database-performance-connection-issues-with-a-remote-postgresql-db/6437)
**Author:** [@kmahelona](https://support.prodi.gy/u/kmahelona)\ **Replies:** 9\ **Last updated:** [June 9, 2023, 5:24pm UTC](https://support.prodi.gy/t/database-performance-connection-issues-with-a-remote-postgresql-db/6437 "2023-06-09T17:24:59Z")
I'm having issues with a remote postgresql database. The issues appear to be connection related. While I can connect and export data via python, when I run prodigy it just hangs. I'm really not sure what's going on or h… --- ## [ner.correct not showing suggestions](https://support.prodi.gy/t/ner-correct-not-showing-suggestions/6528)
**Author:** [@mattdr](https://support.prodi.gy/u/mattdr)\ **Replies:** 3\ **Last updated:** [May 4, 2023, 6:03pm UTC](https://support.prodi.gy/t/ner-correct-not-showing-suggestions/6528 "2023-05-04T18:03:18Z")
Hi there, I am new to Prodigy and I am using it to annotate domain-specific data. I first annotated 8.000 instances using ner.manual and then I went on and trained the model using the Prodigy recipe. I then tried to us… --- ## [Reduce the number of categories in textcat project](https://support.prodi.gy/t/reduce-the-number-of-categories-in-textcat-project/6523)
**Author:** [@a-meneghini](https://support.prodi.gy/u/a-meneghini)\ **Replies:** 5\ **Last updated:** [May 4, 2023, 12:51pm UTC](https://support.prodi.gy/t/reduce-the-number-of-categories-in-textcat-project/6523 "2023-05-04T12:51:01Z")
Hi there, i'm working on a textcat project with three categories. I almost reached 2000 annotations in my training set and realized that: it's quite difficult to distinguish between categories 1 and 3 Category n°3 … --- ## [db-out is giving a empty .jsonl file](https://support.prodi.gy/t/db-out-is-giving-a-empty-jsonl-file/6454)
**Author:** [@ngawangtrinley](https://support.prodi.gy/u/ngawangtrinley)\ **Replies:** 7\ **Last updated:** [March 30, 2023, 9:55am UTC](https://support.prodi.gy/t/db-out-is-giving-a-empty-jsonl-file/6454 "2023-03-30T09:55:15Z")
Below command is how I use to start the prodigy recipe to annotate page detection on my server /usr/bin/python3.9 -m prodigy bdrc-crop-images-recipe bdrc\_crop '/usr/local/prodigy/prodigy-tools/data/page\_cropping.csv' -F… --- ## [Implications of "answer": "accept" being automatically added to all imported examples with db-in](https://support.prodi.gy/t/implications-of-answer-accept-being-automatically-added-to-all-imported-examples-with-db-in/6447)
**Author:** [@Gruff](https://support.prodi.gy/u/Gruff)\ **Replies:** 4\ **Last updated:** [March 21, 2023, 1:53pm UTC](https://support.prodi.gy/t/implications-of-answer-accept-being-automatically-added-to-all-imported-examples-with-db-in/6447 "2023-03-21T13:53:26Z")
Edit: I've now determined that the issues I initially reported are not down to the data, so I've removed those references. I'll ask about those particular issues on the spaCy discussion board. Hi, I've been trying to m… --- ## [Large number of database connections.](https://support.prodi.gy/t/large-number-of-database-connections/3743)
**Author:** [@defined-user-poz](https://support.prodi.gy/u/defined-user-poz)\ **Replies:** 7\ **Last updated:** [March 11, 2023, 9:57am UTC](https://support.prodi.gy/t/large-number-of-database-connections/3743 "2023-03-11T09:57:07Z")
Hey! Recently we started using Prodigy for sentences annotation. We use a default ner.manual recipe for that purpose and have four people working on the same dataset at the same time (web). A few days ago our other ser… --- ## [Slow database operations on PostgreSQL](https://support.prodi.gy/t/slow-database-operations-on-postgresql/5209)
**Author:** [@psimm](https://support.prodi.gy/u/psimm)\ **Replies:** 1\ **Last updated:** [March 11, 2023, 9:23am UTC](https://support.prodi.gy/t/slow-database-operations-on-postgresql/5209 "2023-03-11T09:23:51Z")
Hi, I've noticed that the db-in, db-merge and drop operations are running slowly for me. Importing a data set of 20000 annotations (25mb) takes more than 20 minutes. In contrast, db-out takes just seconds. I'm using Pos… --- ## [How to download the dataset I annotated using the prodigy tool in json format？](https://support.prodi.gy/t/how-to-download-the-dataset-i-annotated-using-the-prodigy-tool-in-json-format/6400)
**Author:** [@Zanejins](https://support.prodi.gy/u/Zanejins)\ **Replies:** 3\ **Last updated:** [March 6, 2023, 4:07pm UTC](https://support.prodi.gy/t/how-to-download-the-dataset-i-annotated-using-the-prodigy-tool-in-json-format/6400 "2023-03-06T16:07:51Z")
From the prodigy documentation, I can use the following command to download the annotated dataset to the local computer, but the data set is in the format of spacy. I want to get the data set in json format. How can I as… --- ## [Feature Request: Bulk Dataset Drop](https://support.prodi.gy/t/feature-request-bulk-dataset-drop/301)
**Author:** [@wpm](https://support.prodi.gy/u/wpm)\ **Replies:** 11\ **Last updated:** [February 3, 2023, 3:20pm UTC](https://support.prodi.gy/t/feature-request-bulk-dataset-drop/301 "2023-02-03T15:20:37Z")
While experimenting with Prodigy I find myself creating and dropping lots of datasets. Every once in a while I go through and clean up every dataset. This is a little tedious because prodigy drop only works on one datase… --- ## [Missing data: TOTAL and SOURCE are inconsistent](https://support.prodi.gy/t/missing-data-total-and-source-are-inconsistent/6234)
**Author:** [@NNN](https://support.prodi.gy/u/NNN)\ **Replies:** 1\ **Last updated:** [January 13, 2023, 5:50pm UTC](https://support.prodi.gy/t/missing-data-total-and-source-are-inconsistent/6234 "2023-01-13T17:50:16Z")
Dear Prodigy team, I'm experiencing a problem with the platform. It seems as if a portion of the source is always left out. Namely, the sum in the TOTAL in the GUI shows less than the actual total number of json lines i… --- ## [empty spans and spans with no 'text' attribute](https://support.prodi.gy/t/empty-spans-and-spans-with-no-text-attribute/6224)
**Author:** [@stefan.bartell](https://support.prodi.gy/u/stefan.bartell)\ **Replies:** 10\ **Last updated:** [January 11, 2023, 9:52pm UTC](https://support.prodi.gy/t/empty-spans-and-spans-with-no-text-attribute/6224 "2023-01-11T21:52:03Z")
Hi. I'm having two issues. The output of db-out when I save annotations produces empty spans and spans with no 'text' attribute, while when I used db-out before, spans were an empty list (\[ \]) or a list of dictionaries … --- ## [How to reuse the prodigy.db to retrain the older (spacy v2) ner custom model](https://support.prodi.gy/t/how-to-reuse-the-prodigy-db-to-retrain-the-older-spacy-v2-ner-custom-model/5861)
**Author:** [@rahul1](https://support.prodi.gy/u/rahul1)\ **Replies:** 8\ **Last updated:** [December 5, 2022, 5:38am UTC](https://support.prodi.gy/t/how-to-reuse-the-prodigy-db-to-retrain-the-older-spacy-v2-ner-custom-model/5861 "2022-12-05T05:38:52Z")
Hi, Last year in June 2021 I created a ner custom model with Prodi.gy (and spacy 2.x.x.) on my windows laptop: python -m prodigy train ner dataset,dataset\_correct,dataset\_correct1,dataset\_correct3 en\_vectors\_web\_lg — o… --- ## [How does db-merge work?](https://support.prodi.gy/t/how-does-db-merge-work/6114)
**Author:** [@vsocrates](https://support.prodi.gy/u/vsocrates)\ **Replies:** 2\ **Last updated:** [November 22, 2022, 11:45pm UTC](https://support.prodi.gy/t/how-does-db-merge-work/6114 "2022-11-22T23:45:13Z")
How does the db-merge method work? Several other built-in recipes are in the public repo but I don't db-merge in there. Specifically, I haven't been able to find anywhere if input\_hashes or task\_hashes are used to merge… [Next page](https://support.prodi.gy/tag/database/10.md?match_all_tags=true&page=1&tags%5B%5D=database)