# Issues Setting Custom Database

**URL:** <https://support.prodi.gy/t/issues-setting-custom-database/6829>\
**Category:** Uncategorized\
**Tags:** database, custom\
**Created:** [October 6, 2023, 5:30pm UTC](https://support.prodi.gy/t/issues-setting-custom-database/6829 "2023-10-06T17:30:25Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![lpreuett](https://avatars.discourse-cdn.com/v4/letter/l/eada6e/32.png) [@lpreuett](https://support.prodi.gy/u/lpreuett)\
**Post date:** [October 6, 2023, 5:30pm UTC](https://support.prodi.gy/t/issues-setting-custom-database/6829/1 "2023-10-06T17:30:26Z")

</div>

I have been struggling to get prodigy to use my custom database. I have followed the examples available in support, the [Mondigy code](https://github.com/jdagdelen/mondigy), and documentation all to no avail.

I wrote a custom recipe which I debugged using the default SQLite database adapter. All worked well. I would like to store my results in MongoDB, so I began working on a custom adapter. In my custom recipe I tried returning the following dictionary

 ![image](https://us1.discourse-cdn.com/flex020/uploads/prodigy/original/2X/0/02a29e61916c92aeca28763d8d107a31c69b85e1.png)  
Initially, everything seems to work fine (i.e., new datasets and sessions are created in MongoDB). However, when I try to save annotations via the Prodigy UI (note using the default save button), the annotations are saved to SQLite. A little hacking revealed sometime after the controller is instantiated, a call to Controller.set\_db with the default SQLite database is made.

To address this issue, I subclassed Controller to prevent modification of the Database after construction

 ![image](https://us1.discourse-cdn.com/flex020/uploads/prodigy/original/2X/c/ca1788071bb25a810e2c5330c0bdfb44e91ae527.png)  
My custom recipe returns an instance of the subclassed Controller  
 ![image](https://us1.discourse-cdn.com/flex020/uploads/prodigy/original/2X/2/25b726979dcec1a08137682dd0b8fa90e8ccc879.png)  
Note while the return statement is not shown, I do return the controller.

This is very hacky, but I can now store annotations in my MongoDB collection  
 ![image](https://us1.discourse-cdn.com/flex020/uploads/prodigy/original/2X/e/e0ff4584747a684d2aba8d5d5a83b568e8a2dbe7.png)  
Seen above are the three MongoDB documents written as the result of a single annotation save. The first entry is the debug\_dataset. Second is some default session I don't know how to disable. The final item is the session for one of my predefined users (i.e., user '1'). Note the annotations are correctly saved in both the debug\_dataset and the session for user '1'.

The issue at this point is the session names created for my users at time of connect do not match the prepopulated sessions when specifying the `PRODIGY_ALLOWED_SESSIONS` environment variable. The session names should be `<dataset_name>-<session_name>`. As seen in the image above, the session names created at time of connect are `no_dataset-<session_name>`. Note if I call `Controller.get_session_name('1')` I get the desired session name `debug_dataset-1`. See the following debug output for evidence

 ![image](https://us1.discourse-cdn.com/flex020/uploads/prodigy/original/2X/1/1c22f59bbd5e68edf884a3d0cb971986d584bccb.png)  
The first line is printed in Controller.confirm\_session  
The second line is printed from my session factory. The list is Controller.session\_ids and `no_dataset-1` is the session\_id being created.  
The third line is printed from Controller.add\_session  
The fourth line is printed from my task router. The list is Controller.session\_ids and `no_dataset-1` is the session\_id we're routing for.

TL;DR  
I believe there is a bug in Controller preventing the use of custom databases and possibly a bug creating session names using a custom database.

Any support would be greatly appreciated.

Prodigy Stats  
Version: 1.13.3  
Platform macOS-13.5.2-arm64-arm-64bit  
Python Version 3.10.11  
Spacy Version: 3.6.1  
Database Name: SQLite  
Database Id: sqlite  
Total Datasets: 1  
Total Sessions: 301

---

<div class="post-metadata">

**Author:** ![lpreuett](https://avatars.discourse-cdn.com/v4/letter/l/eada6e/32.png) [@lpreuett](https://support.prodi.gy/u/lpreuett)\
**Post date:** [October 6, 2023, 8:58pm UTC](https://support.prodi.gy/t/issues-setting-custom-database/6829/2 "2023-10-06T20:58:35Z")

</div>

After a lot of debugging, I discovered we need `dataset` to be specified in the Controller config like so

 ![image](https://us1.discourse-cdn.com/flex020/uploads/prodigy/original/2X/9/96658ed71e528f16444c0734b959d6cfeb5d7c80.png)

Additionally, it seems Prodigy sets the database according to the config at least once during startup. This explains why my custom DB was getting replaced. But I don't think this behavior is desirable.

It took me days to resolve these issues. Updated documentation and a clear tutorial of how to add in your own custom database would have saved me a lot of time.

---

<div class="post-metadata">

**Author:** ![koaning](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/koaning/32/230_2.png) [@koaning](https://support.prodi.gy/u/koaning)\
**Post date:** [October 17, 2023, 1:08pm UTC](https://support.prodi.gy/t/issues-setting-custom-database/6829/3 "2023-10-17T13:08:40Z")

</div>

> [@lpreuett](#):
>
> I have been struggling to get prodigy to use my custom database. I have followed the examples available in support, the [Mondigy code](https://github.com/jdagdelen/mondigy), and documentation all to no avail.

I fear that [mondigy](https://github.com/jdagdelen/mondigy) hasn't been maintained in two years and that it's no longer compatible with the Prodigy codebase. Many methods have been added to the database class in version 1.12 to accommodate task routers, which is one of the reasons why it won't work for recent versions.

> [@lpreuett](#):
>
> However, when I try to save annotations via the Prodigy UI (note using the default save button), the annotations are saved to SQLite.

I'm curious, do you have anything specific configured in your `prodigy.json` file with regards to databases?

> [@lpreuett](#):
>
> Updated documentation and a clear tutorial of how to add in your own custom database would have saved me a lot of time.

I understand the frustration. On the longer term we are planning to replace the current ORM with SqlAlchemy and we can't consider integrations/documentation improvements on this topic until this refactory has taken place. That said, there are currently no plans to support MongoDB directly.
