Problem connecting to a custom database using entry_points

I built my own custom database connector (very simple, just using the peewee PostgresSQL adapter but loading config from the environment), and installed to my environment using the entry_points. Following is my setup.py, if you need it:

setup.py
from setuptools import setup

setup(
    name='prodigy_utils',
    entry_points={
        'prodigy_db': [
            'pg_from_env = prodigy_utils.db:pg_from_env'
        ]
    },
    requirements=[
        'prodigy>=1.8.3,<1.9.0'
    ]
)

It installs just fine, but when I run prodigy stats, I get the following output:

15:22:05 - APP: Using Hug endpoints (deprecated)
15:22:05 - DB: Initialising database PostgreSQL From Env
15:22:05 - DB: Added 1 connector(s) via entry points
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/local/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/opt/venv/lib/python3.7/site-packages/prodigy/__main__.py", line 372, in <module>
    plac.call(commands[command], arglist=args, eager=False)
  File "/opt/venv/lib/python3.7/site-packages/plac_core.py", line 328, in call
    cmd, result = parser.consume(arglist)
  File "/opt/venv/lib/python3.7/site-packages/plac_core.py", line 207, in consume
    return cmd, self.func(*(args + varargs + extraopts), **kwargs)
  File "/opt/venv/lib/python3.7/site-packages/prodigy/__main__.py", line 105, in stats
    DB = connect()
  File "/opt/venv/lib/python3.7/site-packages/prodigy/components/db.py", line 73, in connect
    raise ValueError("Invalid database id: {}".format(db_id))
ValueError: Invalid database id: pg_from_env

As a potential lead, when I look into prodigy/components/db.py, I notice that in the connect function the db_id always comes in as None, and loading it from the config occurs after checking the user_dbs object. I changed the function db.py to the following:

modified prodigy/components/db.py
def connect(db_id=None, db_settings=None):
    """Connect to the database.

    db_id (unicode): 'sqlite' (default), 'postgresql' or 'mysql'.
    db_settings (dict): Optional database connection parameters.
    RETURNS (prodigy.components.db.Database): The initialized database.
    """
    global _DB
    if _DB is not None:
        return _DB
    connectors = {
        "sqlite": connect_sqlite,
        "postgresql": connect_postgresql,
        "mysql": connect_mysql,
    }
    user_dbs = get_entry_points("prodigy_db")
    if user_dbs:
        log("DB: Added {} connector(s) via entry points".format(len(user_dbs)))
    config = get_config()
    if db_id in (True, False, None):
        db_id = config.get("db", "sqlite")
    if db_settings in (True, False, None):
        config_db_settings = config.setdefault("db_settings", {})
        db_settings = config_db_settings.get(db_id, {})
    if db_id in user_dbs:  # this block was originally before config = get_config()
        _DB = user_dbs[db_id]
        return _DB
    if db_id not in connectors:
        raise ValueError("Invalid database id: {}".format(db_id))
    db_name, db = connectors[db_id](**db_settings)
    _DB = Database(db, db_id, db_name)
    log("DB: Connecting to database {}".format(db_name))
    return _DB

And it appears to have resolved the issue, as I get the following output:

15:24:25 - APP: Using Hug endpoints (deprecated)
15:24:25 - DB: Initialising database PostgreSQL From Env
15:24:25 - DB: Added 1 connector(s) via entry points

  ✨  Prodigy stats

Version          1.8.3
Location         /opt/venv/lib/python3.7/site-packages/prodigy
Prodigy Home     /prodigy
Platform         Linux-4.9.184-linuxkit-x86_64-with-debian-10.0
Python Version   3.7.4
Database Name    PostgreSQL From Env
Database Id      postgresql
Total Datasets   2
Total Sessions   44

Thanks for the very detailed report and for looking into this! I think you're right – sorry about that. (I think what happened here was that in my tests, I'd typically call into connect directly with the connection parameters, so the problem didn't come up :woman_facepalming:) I've already fixed it internally and we'll ship the fix with the next release.

We've all made mistakes like that, not a problem - happy to help.