Adding a little descriptive reference in ner.manual page

Hi,

This is something I want to have in my prodigy interface rather than the default behaviors.
I am using ner.manual, and there are some codes and texts for performing annotations, but I want to display a short reference to the codes in the annotation-ui so that it can be easy for the annotator.

I searched through the prodigy support, but could not really get something.
Can this be achieved here, if yes, how?

Thank you!

Hi! You could use the annotation instructions, which let you load in an HTML file with instructions. Just add the following to your prodigy.json or the recipe config:

{
    "instructions": "/path/to/instructions.html"
}

The instructions will then be available as a help modal that looks like this and is available via the button or by pressing H:

prodigy_instructions_modal

Because the modal supports HTML, you should be able to format things pretty flexibly, depending on what you want to explain. You could add a table, or explain the codes in a list, and even add external links if you like.

1 Like

But, this is something static. In this page I would have to load all instructions.
But, what if I have to show according to the code chosen from dropdown?

That’s currently not easily possible, because the app doesn’t expose the label selection event.

I’m also not so sure I would recommend interactions like this. The interface is only really powerful if you can move through the tasks very quickly. If the label scheme is so complex that the annotator can’t easily learn it and needs to be reminded for each label, you’ll often end up with worse results overall: it introduces a high potential for human error and the annotator won’t be able to focus as efficiently, because the label scheme definitions get in the way of actually making the annotation decisions.

Instead, I would try to break down the task into smaller pieces – for example, start with a smaller label set of 1-3 labels that are easy to remember so that the annotators can get into a good flow and move through the examples quickly. Then repeat with more labels and merge the data laber on. Even better, maybe you can pre-select some examples programmatically (e.g. like the ner.make-gold recipe) or solve some annotation decisions with a fully binary interface.

Will go with the help & instructions thing for now then. :slight_smile:

1 Like