Custom end-screen (“no tasks available”) fires prematurely in multi-recipe workflow

, , ,
2

Welcome to the forum @Hijacqued !

Thank you for sharing all the details and apologies for the late reply!

Is there anything that has changed between Prodigy version 1.15 and 1.18 that generates this problem?

In 1.18 we have modfied the UI update cycle a bit to address the issue of the custom js code not being available in some conditions. I can't be 100% sure, but I suspect that this might have made a race condition more likely between the get_questions request and the empty queue detection which is what triggers the prodigyend event.
This interpretation would be in line with the fact that that the problem occurs only occasionally and can't be consistently reproduced. We'll definitely look into it, but in the meantime let's try to work around your problem.

Am I missing a more reliable client-side approach than MutationObservers + timeouts?

I think we can simplify it by relying on Prodigy events and just making sure the prodigyend can only be triggered once at least one task has been loaded. This of course assumes that all of your servers are expected to have non empty source.
We could inspect the DOM like you tried, but we can also listen to prodigyanswer event and access the task structure from there. It could be enough just to detect prodigyanswer event, but you could also store some indicator in the meta dictionary of the task (you'd need to modify your input JSONL file) just make it more tractable. What I mean is:

let firstTaskAnswered = false;

document.addEventListener('prodigyend', () => {
    console.log("Prodigy end event")
    if (!firstTaskAnswered) {
        console.warn("Blocked premature prodigyend event");
        return;
    }
    showYourCustomEndScreen(); // whatever you normally do
});

document.addEventListener('prodigyanswer', event => {

    const { answer, task } = event.detail;
    if (task && task.meta && task.meta.first) {
        console.log('First task was answered');
        firstTaskAnswered = true;
    }
});

This should result in "Loading..." screen until the first task is loaded.

I hope that also answers your first question:

Is there a built-in Prodigy hook or event that fires only once the first annotation is actually rendered and accepted?

There's not such event currently, but you could work around as described above.

Any other best practice for chaining multiple Prodigy ports/recipes with a single session, where you only want the final “no tasks” screen after all annotations are done?

I think your strategy of customizing prodigyend in certain conditions is pretty good - it's just unfortunate that that this race condition kicks in. Hopefully the suggested workaround is enough to unblock you.
In general, we recommend having a separate server per annotation tasks, but if sharing the state is not a concern you might take a look at this alternative approach here.
I do think, though that using separate servers should be more reliable.

1 Like