prodigy.serve(<any command>) throws an error "AttributeError: 'NoneType' object has no attribute 'config'"

,
1

Hi. I am trying to run any recipe in a python script using prodigy.serve function and every time I get an error that looks like (in case of simple prodigy.serve("stats"))

Traceback (most recent call last):
  File "d:/Sources/Python/SpacyTest/main.py", line 34, in <module>
    prodigy.serve("stats")
  File "D:\Sources\Python\SpacyTest\.venv\lib\site-packages\prodigy\__init__.py", line 40, in serve
    print("controller.config:", controller.config)
AttributeError: 'NoneType' object has no attribute 'config'

If to dig into a file where the error is thrown lib\site-packages\prodigy\__init__.py we could find that execution of loaded_recipe function returns nothing (None)

            loaded_recipe = get_recipe(recipe_name)
            if not loaded_recipe:
                msg.fail(f"Can't find recipe '{recipe_name}'", exits=1)
            controller = loaded_recipe(*cli_args, use_plac=True)
            controller.config.update(config)    # Exception is thrown here because controller is None
            server(controller, controller.config)

I am using prodigy 1.9.10 version (Windows). All recipes that are run from command line work fine, but if to call them using prodige.serve() you will end up with the error I described (but recipe logic will be executed and a correct output will be get. Error is thrown at the end)

2

Hi! The prodigy.serve function is really only intended to run recipes that rely on the server being started. This depends on additional state, you couldn't just call the recipe function itself from Python. prodigy.serve solves this.

For all other commands, there's nothing to serve and you can just call them from Python as regular functions.

from prodigy.recipes.commands import stats
stats()

We should raise a better error if the loaded controller is None, because that typically indicates that the recipe doesn't return any components that can be served, and that the function can just be called from Python.

3

Thank you for a quick response.
If to use another commandprodigy.serve("train ner test_dataset en_vectors_web_lg"), training will be run but in the end I will still get a very similar error message.

Traceback (most recent call last):
      File "d:/Sources/Python/SpacyTest/main.py", line 35, in <module>
        prodigy.serve("train ner test_dataset en_vectors_web_lg")
      File "D:\Sources\Python\SpacyTest\.venv\lib\site-packages\prodigy\__init__.py", line 37, in serve
        controller.config.update(config)
    AttributeError: 'tuple' object has no attribute 'config'

What I'm trying to do is to create a script that will train a model using ML Azure computing power (script is executed on ml azure side). We have a prodigy dataset in a cloud and in the script I want to call atrain recipe using prodigy.serve command.

I'll be very thankful if you help me to fix this problem or, maybe, to suggest a better way how to achieve my goals. Thank you

4

I was able to achieve my goals using next commands

    train_recipe = get_recipe("train", path=None)
    params = ["ner", "test_dataset", "blank:en", "--output=trained_model"]
    train_recipe(*params, use_plac=True)

It does training and doesn't throw any exceptions. But is it a good approach? Why do we need a prodigy.serve command than?

5

Yes, the train command is another function that doesn't rerturn a dictionary of components and doesn't require the annotation server to be started. It just runs a function, so you can call it directly from Python (which is also what I'd recommend doing on your remote machine):

from prodigy.recipes.train import train
train(...)  # arguments here

The prodigy.serve helper is intended for recipes that need to start the annotation server and serve the web app – for example, ner.manual or any other annotation recipe. Here, you have functions that return a dictionary of components: the dataset, the stream, various callbacks, config, etc. Calling that function won't start a server – it will just give you the components. So you'll need additional logic that takes these components, puts them together, and uses this configuration to start the web server that you can interact with from the web app. That's what prodigy.serve does.