Problems exposing a custom Database class via an entry point?

,
2

Hi! This looks like a cool project :slightly_smiling_face:

Which version of Prodigy are you using? I've been trying to find where this problem could be happening and I don't see anything that could explain it :thinking: The Database class shouldn't have to be iterable.

One thing to note maybe: If a custom database is provided, Prodigy currently expects an already instantiated object (because it can't make any assumptions about how your database wants to be initialized – the built-in one for instance just holds a peewee object). So maybe that's the problem? If you need access to any user settings, you can call Prodigy's get_config() to access the prodigy.json.

Here's the section we previously had on entry points in the README – maybe we should add the example back! I think I ended up replacing that with a reference to the sense2vec package because that actually showed a full working example.

Plugging in your own loaders or database connectors usually required writing a custom recipe – even if you don't want to change anything about the built-in
recipes themselves. Prodigy v1.5.0 introduces a new way of making your own
functions available to built-in recipes and CLI commands, without having to
modify the source. All you need is a simple Python package that exposes your
components via entry_points, and is installed in the same environment as
Prodigy. For a quick introduction to entry points in Python, we recommend
this blog post.

Consider the following structure of your prodigy_utils package:

└── prodigy_utils       # package directory
    ├── recipes.py      # recipe functions
    ├── loaders.py      # loader functions
    ├── db.py           # database classes
    └── setup.py        # package setup

In the setup.py, you can then define a dictionary of entry_points for one
or more of the available categories. Each category is mapped to a list of
strings in the format [name] = module:function. For example,
'custom_json = loaders:CustomJSON' will make the function or class
CustomJSON in the file loaders.py available via the name custom_json.

from setuptools import setup

setup(
    name='prodigy_utils',
    entry_points={
        'prodigy_recipes': [
            'custom_recipe = recipes:custom_recipe'
        ],
        'prodigy_loaders': [
            'database = loaders:DatabaseLoader',
            'custom_json = loaders:CustomJSON'
        ],
        'prodigy_db': [
            'mongodb = db:MongoDBLoader'
        ]
    },
    requirements=[
        'prodigy>=1.4.3,<1.5.0'
    ]
)

Prodigy checks the following entry point categories:

Name Description
prodigy_recipes Custom recipe functions, one entry per recipe.
prodigy_loaders File or API loaders.
prodigy_db Database connectors that follow the same API as Prodigy's Database class.

To install your package and expose the entry points, navigate to the package
directory and run the setup – for example, in development mode:

cd prodigy_utils
python setup.py develop

If your package is installed in the same environment, you won't have to do
anything else. Prodigy will automatically find and load your entry points. To
verify that your entry points were read in correctly, you can set the
PRODIGY_LOGGING=basic environment variable. On startup, Prodigy will log how
many components were added via entry points.