I am working on a active learning setup (very similar to intent classifier video), on my annotation interface I am looking for two features:
How many total annotations will be shown in this round?
Total number of Accept / Reject and Ignore across all rounds - that means for the complete dataset as opposed to session based Accept/Reject/Ignore statistics that is currently being populated.
Can anyone help with how to set this up?
Thanks,
Vaishnavi
You can get stats about a given dataset, including the breakdown of accept/reject/ignore answers by running the prodigy stats command with the name of a dataset: Built-in Recipes · Prodigy · An annotation tool for AI, Machine Learning & NLP The widget in the UI will only show the progress and breakdown of the current annotation session.
Do you mean the total number of examples that are still queued up? The progress in the UI will indicate the percentage that's already completed, but this is only available for streams and recipes where the stream can be counted. (The stream is a generator that can potentially be infinite and some recipes will dynamically modify the stream, e.g. to select the best possible examples to annotate in active learning workflows. So in these cases, you wouldn't know upfront how many examples there are to annotate.)