Hi, I think it would great if we can put some instructions for labeling (say in the sidebar) so it would be a constant reminder on how to label (some labeling tasks can get a little confusing).
Sure, that’s a nice idea!
If the instructions are not too long, you could use the dataset’s description, which can be set via --description on the command line when creating a new set. The text you enter here will be displayed in the sidebar, underneath the dataset name etc. The only problem is that it’s not so easy to edit or change the description afterwards and once the set is created.
1 Like
An NER-specific version of this that I’ve wished for is to display the definition for the named entity type that’s currently being asked about. This would be especially nice for LOC vs. GPE, which annotators mess up all the time, or the more obscure ones like EVENT. Not sure if other people would find that useful or worth the clutter, and it’s only really useful when people are starting out.
1 Like
That's an interesting idea – maybe we do need some option to display a "help" modal or some other way of showing more information (like the annotation manual) without cluttering the UI.
In the meantime, a simple workaround could be to add the label that occurs in the task to its meta, so it's displayed underneath the card. But of course, this only really works in ner.teach when there's only one label per task:
labels_desc = {'GPE': 'geopolitical entity - everything with a governing body!'}
for eg in stream: # assuming a stream with one span per example
label = eg['spans'][0]['label']
if label in labels_desc:
eg['meta'][label] = labels_desc[label]
1 Like
Quick update: Just implemented this feature and it will be available in the next release 
You can then specify a config parameter instructions that takes a path to a text file (both plain text and HTML are supported, so it should be very flexible in terms of presentation). The instructions will then be displayed in a modal, accessible via the little help icon at the top, or by pressing h.
Because it’s a regular config setting, you can either have global or project-based instructions, or add them within the recipe config.
{
"instructions": "/path/to/instructions.html"
}
3 Likes
See my comment above or I’ve really been liking the instructions feature. Thanks!
I’m running into a strange bug, though. When I run Prodigy on my local Mac machine, the instructions work just fine. When I run it on a server, clicking the instructions blanks out the entire page. The only differences seem to be that it’s on Linux vs. Mac and running on 0.0.0.0 instead of on localhost. Any idea what could be causing it? I can share the page where it’s running (privately) if you’d like to take a look.
@andy Thanks, and hmm, that’s strange and shouldn’t be happening! The blank page happens because an error on the front-end isn’t caught correctly (sorry about that – likely forgot to add the boundaries around the modal correctly, since it’s happening outside of the regular annotation area).
Could you check the browser console for the error? (It’s a compiled app, so likely not as meaningful – but it’d still be good to verify.)
Also, can you double-check that the instructions markup that goes out is read in correctly? This happens in the server function in the app.py (which is included with Prodigy), so you can simply add print(config['instructions']) at the end. A possible explanation could be that your two machines produce different results here and the front-end can’t handle one of those and dies.
If none of this gets us any further, then yes, feel free to email me with more details!
@ines Printing out in app.py gives the same instructions locally and on the server. Here’s the error on the server from the console, but it’s unfortunately not that meaningful to me:
bundle.js:1 ReferenceError: translate is not defined
at t.value (bundle.js:1)
at u (bundle.js:1)
at beginWork (bundle.js:1)
at r (bundle.js:1)
at i (bundle.js:1)
at a (bundle.js:1)
at S (bundle.js:1)
at x (bundle.js:1)
at batchedUpdates (bundle.js:1)
at lt (bundle.js:1)
u @ bundle.js:1
a @ bundle.js:1
S @ bundle.js:1
x @ bundle.js:1
batchedUpdates @ bundle.js:1
lt @ bundle.js:1
$t @ bundle.js:1
Thanks – looks like there was possibly a bug in the front-end that caused this. Is it possible that your machines are running different versions of Prodigy? Already fixed the bug and it will be included in the next release. (If you’re blocking on this, let me know and I can send you an updated JS bundle that you can replace!)
Yes, it turns out they’re different versions. The version with the bug is 1.4.2. My local computer without the error is running 1.4.0. If you wouldn’t mind sending the JS, that would be great! We have a few annotators starting a new task today and it would be great to have the instructions up to remind them.
1 Like
Hi all,
I am in the same situation as Andy. I got the translate is not defined error with version 1.4.2 and I would like to give clear instructions to the annotators that are gonna start next week.
It would be very helpful to have the temporary bundle.js to use or, if it is not feasible, an ETA on this fix.
Many thanks for your group amazing work.
@raffaele Sorry about that bug – I managed to track down the issue and it will be included in the next release (should be coming this week and will include a few other improvements and features). In the meantime, feel free to email me at ines@explosion.ai and I’m happy to send you the patched version.
Thank you for the quick answer @ines. This is really amazing. I look forward for the the next release.
Many thanks.