Issue with Prodigy 1.11.8a2 with "experimental_feed"

4

Hi @nomiizz,
We have just releasedv1.11.8a4of the alpha that includes thedb-migratecommand ( as well as all improvements that come with the main Prodigy releasev1.11.8)
You can downloadv1.11.8a4 from our PyPi index like so:

pip install prodigy==1.11.8a4 --extra-index-url https://{YOUR_LICENSE_KEY}@download.prodi.gy

In order to use thedb-migratecommand, you need to specify 1) the legacy DB (v1) and 2) the target DB (v2) in your prodigy.json:

{
// prodigy.json
    "experimental_feed": true,
    "feed_overlap": false,
    "legacy_db": "postgresql",
    "legacy_db_settings": {
        "postgresql":{
            "dbname": "prodigy",
            "user": "username",
            "password": "xxx"
        }
    },
    "db": "postgresql",
    "db_settings": {
        "postgresql": {
            "url": "postgresql://username:xxx@localhost:5432/prodigy_v2"
         }
     }
}

The legacy DB settings are processed with peewee so the configuration style is the same as in v1. The target DB, however, should usesqlalchemy style which in the case of postgresql is slightly different as explained here. You can also specify the sqlite or mysql as the target.

With that configuration in place, you should be able to run:

prodigy db-migrate

That should migrate all the datasets from the v1 legacy database to the v2 target database. Also, check
prodigy db-migrate --help for more options (migrating a selected dataset, excluding selected datasets, performing a dry run etc.)

1 Like