using prodigy.serve for textcat.batch-train

,
1

I get the following error when using prodigy.serve to call textcat.batch-train recipe:

File “/usr/local/lib/python3.6/site-packages/prodigy/init.py”, line 23, in serve
controller.config.update(config)
AttributeError: ‘float’ object has no attribute ‘config’

Should I change what the recipe is returning and include a config?

Thanks,
Ati

1 Like
2

Hi! This is a strange error – could you share the code you’re running? It looks like that for some reason, the controller actually ends up being a float :thinking:

3

Hi Ines,

Thanks for quick reply.
Here is how I use prodigy.serve():

prodigy.serve(‘textcat.batch-train’, ‘my_dataset’, ‘en_core_web_sm’, ‘/tmp’)

4

I should say the recipe works: the batch-train is done, and the model is saved in the right directory. But after saving the model the run is exited with this error.

5

Ahhh, thanks for the clarification. That makes more sense now. For some reason (not sure why), running the recipes via prodigy.serve doesn’t seem to handle “non-annotation recipes” properly that don’t return a dictionary of components. The textcat.batch-train recipe function only returns a float (the accuracy number) – this allows it to be called from within other recipes like textcat.train-curve and report the training results.

I’ll look into this! In the meantime, you could probably add a try/except around your function call and check for the attribute error to suppress it.

1 Like
6

Thanks Ines. I did the try/except.

7

hello Ines

Any news on this error ?

I use this function:

import logging
from pathlib import Path

import prodigy
from prodigy.recipes.ner import batch_train

@prodigy.recipe('sentinel.ner.batch.train',
                dataset=prodigy.recipe_args['dataset'],
                spacy_model=prodigy.recipe_args['spacy_model'],
                label=prodigy.recipe_args['label']
                )
def batch_train_custom(dataset, spacy_model, label):
    print("Train {} to {}".format(dataset, label))
    output_path = Path('models/'+dataset)

    return batch_train(dataset=dataset,
                       input_model=spacy_model,
                       output_model=output_path,
                       label=label,
                       factor=1,
                       dropout=0.2,
                       n_iter=10,
                       batch_size=-1,
                       beam_width=16,
                       eval_id=None,
                       eval_split=0.2,
                       unsegmented=False,
                       no_missing=False,
                       silent=False)

It works and the model is created. But I have this error:

Model: /tmp/models/temporary_model
Training data: /tmp/models/temporary_model/training.jsonl
Evaluation data: /tmp/models/temporary_model/evaluation.jsonl
Traceback (most recent call last):
  File "/Users/iero/miniconda/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/Users/iero/miniconda/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/iero/code/sentinel/sorcerer.py", line 300, in <module>
    prodigy.serve('sentinel.ner.batch.train', training_name, nlp_models[option.lang], training_labels)
  File "/Users/iero/miniconda/lib/python3.6/site-packages/prodigy/__init__.py", line 23, in serve
    controller.config.update(config)
AttributeError: 'dict' object has no attribute 'config'

Any idea ?

8

Maybe I do need to replace my return call by something as:

    return {
        'view_id': 'ner',  
        'dataset': dataset,
        'output_model': output_path,
        'config': {
            'label': label
        }

???

9

This is how I solved it and Ines suggested the same as a quick fix:

    try:
        prodigy.serve('recipe', arg1, arg2)
    except AttributeError: 
        pass
1 Like
10

This is the same problem described earlier in the thread – it’ll be fixed in the next release! The thing here is that the batch training recipes don’t return a dictionary of components and other objects instead – like, the accuracy numbers. (The error raised is an AttributeError when it’s trying to access the controller’s config – not a KeyError, so your initial idea of adding a config to the recipe components won’t make a difference.)

And yes, in the meantime, wrapping it in a try/except should make the error go away :slightly_smiling_face:

1 Like
11

Thanks @Ati and @ines , i’ll do that !

12

Btw, no idea why I didn’t think of this earlier but… prodigy.serve is really mostly intended for serving the Prodigy application. The batch_train recipe is just a regular function, so the easiest way to use it is to import it and call it as a function.

from prodigy.recipes.textcat import batch_train

batch_train(...)  # etc.
2 Likes
13

Thanks again, it’s working now !

1 Like
14

Okay, great! I think I’ll actually remove the “bug” label from this – the recommended way of calling a recipe from Python that doesn’t need to serve the app should be to just import the function and call it. Much more straightforward and also more logical.

prodigy.serve is the helper function for executing recipes that need to serve something – like the annotation server and web app.