auto-save plus history/go back functionality

1

I implemented instant_submit to auto-save documents in order to avoid losing key annotations when the annotators forgot to submit in the event that their browser was idle / server stopped for some reason.

While the annotators love the auto-save functionality, they also want to see the history (at least 10 would be nice) that you see with manual saving and have the go back functionality. The go back functionality doesn't seem to work for us.

Use cases:

  • annotators get clarity on a guideline and want to go back to a previous document

  • annotators actually choose the wrong option

  • annotators see some additional examples and want to go back and change their answer based on how they interpreted the guidelines

Is there a way to have auto-save + history of 10 documents or so + go back functionality?

Thanks!

2

At the moment this isn't possible.

Prodigy is designed to be append-only. You can add an item to the database, but you can't go back and change the annotation or remove an annotation once it has been submitted. This is an opinionated design decision where the thinking is that it becomes a lot harder to reason about your annotations once you allow them to change at any point in time.

That said, there may be a middle ground to consider. What if you don't turn on instant submit but keep a history of 5 items? That way the amount of data that might be lost due to inactivity remains small, but you'd still be able to undo something that happened recently. Would that be a sufficient improvement?

1 Like