Display statistics per label in NER

, ,
1

Hi,

What is a good way to display basic statistics (e.g. label count and/or ratio in %) per label in the web app left progress section? My current recipe is ner.manual, but potentially applied to other recipes.

2

Hi! There's not currently a built-in feature to add arbitrary stats to the UI and keep them in sync, especially if the data has to come from the server. If you're mostly interested in seeing the breakdown of labels during annotation, one option would be to print them periodically to the terminal by adding an update callback to the recipe that counts the labels:

from collections import Counter

# in your recipe
counter = Counter()

def update(answers):
    nonlocal counter
    for eg in answers:
        for eg in eg.get("spans", []):
            counter[eg["label"]] += 1
    print(counter)

If you do want it all to happen in the browser, you could do the equivalent in JavaScript and listen for the prodigyanswer event. You can print the result to your console, or even add it to the stream if you want. Untested – but something like this should work. One downside is that you'd have to add logic to compare the task hashes to account for "undo" actions (which you don't need if you're using the answers sent back to the server, which only gets the final decisions).

const counts = {}

document.addEventListener('prodigyanswer', event => {
    const { task } = event.detail
    const spans = task.spans || []
    for (let { label } of spans) {
        counts[label] = (counts[label] || 0) + 1
    }
    console.log(counts)
})
1 Like