Auto save when no more tasks left

[provRaminHamediZavie (Hamedi)]

we would like to automatically save the stream when it is exhausted (no more tasks left) or when closing a tab. Wondering if this can be enabled or if there are any examples of it.

For example when I run:

prodigy textcat.manual news_topics ./new_headlines_short.jsonl --label Technology,Politics,Economy,Entertainment

and my dataset is this:

{"text":"Uber\u2019s Lesson: Silicon Valley\u2019s Start-Up Machine Needs Fixing","meta":{"source":"The New York Times"}} {"text":"Pearl Automation, Founded by Apple Veterans, Shuts Down","meta":{"source":"The New York Times"}} {"text":"How Silicon Valley Pushed Coding Into American Classrooms","meta":{"source":"The New York Times"}} {"text":"Women in Tech Speak Frankly on Culture of Harassment","meta":{"source":"The New York Times"}}

It seems like prodigy does not auto send the example when the stream is exhausted -> (the give_answers endpoint)

And to invoke give_answers I have to hit save.

Wondering if there could be such feature to give control over annotated examples that are smaller than batch size.

Thanks so much

[koaning (Vincent D Warmerdam)]

I just loaded an examples.jsonl with a single example and was able to confirm that when you're presented with this interface after a single annotation.

When I shut down Prodigy after this, indeed no examples are saved.

Quick Fix

I'd need to check with the Prodigy team about this though. Part of me feels like it could be a feature to automatically save the examples when you hit the end of the queue, but it also feels a bit like an anti-pattern because the interface, as you mention, does provide a solution.

I can imagine it can be un-intuitive to some users, but it does seem like a valid quick fix for now, mainly because it's also the general mechanic in Prodigy to ensure annotations actually get stored. I'll pick this up internally to see what other devs think about this though.