When I wrote a custom recipe that returned a "stream" value and a "db" value, the Prodigy kernel attempted to call the __iter__ method of the "db" value. This behavior isn't documented in the "Database" documentation page, so I would like to know what it's doing and what objects I should supply as the iterator. Should this iterator behave like the "stream" generator that I returned?
Thanks for the reply. I looked over that response and decided to take another approach. Now, prodigy is listening to the generator I return as the "stream" value but it throws a TypeError: e is undefined on the webpage. My generator yields dictionaries with text and meta entries. The value of the text entry is a string, and the value of meta is a dictionary with metadata. I've tested this with the stock textcat.manual recipe which uses JSONL to load a json file that's a list of those dictionaries, and it worked fine. My recipe looks roughly like this:
Thanks for the reply. I did a bit more debugging and found out what was going wrong, so I'll post it here for future reference. Even though my loader was producing dicts identical to what was in the source json, I was using a custom recipe and I forgot to add the "options" key to the dicts like how textcat.manual does. Once I changed that, I started working. Hopefully this is helpful. My problem is solved.