This relates to my post on "depth-first" workflows ("Depth-first" workflows?)
Hi all! I’m playing around with batch size 1 to see whether I can create a variable workflow in which the next task depends on the current task’s result.
I’m running into some strange behavior.
Here's a paste of my test setup, infinite_stream.py: infinite_stream.py - Pastebin.com
I'm trying to simply set up an infinite stream of trivial
text tasks and serve those up.
I’m running it via
prodigy test_infinite_stream -F infinite_stream.py
I’m using only the port, instant_submit, batch_size, and history_size options in my Prodigy config. As an example, in one scenario I’m using the following prodigy.json:
Scenario 1: batch size 1, history size 0, instant_submit true
Once I accept (or reject) the first example, Prodigy tells me there are no tasks available. I've attached a screenshot of the HTTP requests I've observed in this scenario. Note how it does not try to fetch another task.
Scenario 2: batch size 1, history size 0, instant_submit false
I’ve done two tests with these settings:
Scenario 2a: I hit save after each task.
Scenario 2b: I don’t hit save after each task.
In scenario 2a, I seem to get an infinite stream of tasks. This is the network trace after hitting accept (and saving) a couple of times:
In scenario 2b, I just keep accepting tasks without saving in between them. It allows me to annotate three tasks, then tells me there are no more tasks left. This is the network trace:
Note that in this case,
give_answersis called but the interface still indicates that I need to save.
If add an
updatemethod to the custom recipe which simply prints a) that it was called and b) each received answer, I can see that the automatic call to
give_answerssaves the task with the ID 0. If I save manually, the tasks with ID 1 and 2 are saved. Here's a paste of the output: output after saving manually - Pastebin.com (Note that in this paste, I had changed the port but kept everything else the same)
I couldn't find anything on this. Is this a known bug, or am I doing something wrong? Did I misunderstand task generation?