No Tasks Available for Non-Active Learning Classification


I'm using a custom recipe for annotating multi-classification problems. Occasionally, I get "No tasks available" even though there are over 13,000 tasks loaded into the UI. Right now, there's no model in the loop, so I know the "task relevance" answer given in other prodigy support posts doesn't apply. My custom recipe has a "choice" view_id, and uses an "add_options" helper function.

I've been blocked by this for a few days now, so any help you can provide would be greatly appreciated! Thank you!

Hi! How are you loading in the 13k examples to annotate (from a file or from somewhere else?) and what batch size are you using? And when you see "No tasks available" and reload the app, what happens? Also, if you run it with PRODIGY_LOGGING=basic, is there anything in the logs that looks suspicious?

The "No tasks available" message is displayed when the server responds with no new questions and the stream doesn't yield out new examples anymore. If your stream is loading from an API or another source and isn't just a list of dicts loaded into memory, this could be one source of the problem.

Thanks for your quick response! I used the db.add_examples() command to add data to a Postgres instance, which is where the UI is pulling from.
The batch size is set to 5.
Reloading the app yields the same message.

I will try add the logging and report back about the output.

So just to make sure I understand the setup correctly: You're loading the raw data into a database and then loading it back in with your custom recipe? And if so, how are you doing that? Are you calling get_examples and loading the whole 13k set into memory? (This wouldn't be a problem and at least this lets you verify there are indeed 13k records in there.)

Other things you could try: Use a higher batch size, i.e. 10 and above. It shouldn't happen, but maybe there is a race condition somewhere that causes the server to respond with an empty batch if you annotate very fast. Maybe also log the hashes and make sure you definitely have 13k unique "_task_hash" values (and didn't end up with duplicates, which would cause Prodigy to filter them out).