Prodigy Attempts to Iterate on Database Object Returned by Custom Recipe

, ,
1

Hello Prodigy,

When I wrote a custom recipe that returned a "stream" value and a "db" value, the Prodigy kernel attempted to call the __iter__ method of the "db" value. This behavior isn't documented in the "Database" documentation page, so I would like to know what it's doing and what objects I should supply as the iterator. Should this iterator behave like the "stream" generator that I returned?

Thank you!

2

Hi @cxsmith!

Thanks for your question!

The database class shouldn't have to be iterable. Did you see this post?

If this doesn't help, can you provide a small working example of your recipe, especially your custom class?

3

Hi Ryan,

Thanks for the reply. I looked over that response and decided to take another approach. Now, prodigy is listening to the generator I return as the "stream" value but it throws a TypeError: e is undefined on the webpage. My generator yields dictionaries with text and meta entries. The value of the text entry is a string, and the value of meta is a dictionary with metadata. I've tested this with the stock textcat.manual recipe which uses JSONL to load a json file that's a list of those dictionaries, and it worked fine. My recipe looks roughly like this:

@prodigy.recipe('name', dataset=('desc', 'positional', None, str), param1=('description', 'positional', None, str), param2=...)
def recipe(dataset, param1, param2...):
    db = CustomDBClass(params)
    return {
        "view_id": "choice",
        "dataset": dataset,
        "db": db,
        "stream": generator(),
        "config": {
            "choice_style": "multiple",
            "exclude_by": "task"
        }
    }

Am I overlooking something that I need to get the web app working? Thanks.

4

hi @cxsmith!

At quick glance I don't see any problems with your code.

Can you provide any of the syntax within your recipe (e.g., function for your stream/generator?)?

Given the error TypeError: e is undefined, I suspect you may have used e incorrectly (e.g., say within a list comprehension for examples)?

Alternatively, if you don't think that would be the problem, can you provide details of your custom DB class?

5

Hi Ryan,

Thanks for the reply. I did a bit more debugging and found out what was going wrong, so I'll post it here for future reference. Even though my loader was producing dicts identical to what was in the source json, I was using a custom recipe and I forgot to add the "options" key to the dicts like how textcat.manual does. Once I changed that, I started working. Hopefully this is helpful. My problem is solved.

6

hi @cxsmith!

Happy that you worked out your problem! Thanks for re-posting your answer too to help the community. Let us know if you have any other questions.