Span Categorization Removing Span Callback

Hi Prodigy Team,

We want to measure how often annotators are correcting their span annotations. I wonder if there is a callback that we can use when an annotator removes a span?

Thanks :slight_smile:

Hi! By "correcting their span annotations", do you mean correcting pre-annotated spans that are already present in the data? In this case, the most elegant solution might be to include another key in the data, e.g. "orig_spans" that contain the original spans, in addition to the regular "spans". The annotator then makes changes to the "spans", but the "orig_spans" stay the same in the data, so you can easily calculate which spans were changed, how often a span was removed vs. added, and so on.

If you want to log everything that happens in the UI, you could also add custom JavaScript with a listener on prodigyupdate and store every version of the annotations with the data every time it's updated. Check out this thread for a simple code example for how to add a "history" to tasks using JavaScript: