Can you give the buttons of the labels (above the text) a static order in the interface (e.g. ner_manual in ner.make-gold)? Currently, when I run the server, each time there is a random order for the labels, which is not very intuitive.
run 1, buttons: PER | ORG | LOC
run 2, buttons: ORG | PER | LOC
Thanks for the report – this is strange and shouldn’t be happening. The labels setting was actually designed to make no assumptions about the user labels and forward them to the app exactly how they were entered (to give the user control over the ordering).
I’ll look into this!
Edit: Okay, turns out that the problem is that Prodigy calls set() on the list of labels, which doesn’t necessarily preserve the order. We’ll fix this in the next release and either use an OrderedDict, or not use a set if the labels are input manually (I guess it’s okay to expect the user to provide a label scheme with no duplicates).
As a quick workaround, you could change the following line in recipes.ner, e.g. in ner.manual:
labels = get_labels(label, nlp)
labels = [l.strip() for l in label.split(',')]
Note that this will now expect the --label argument to be a string of comma-separated labels (and won’t perform more complex checks, like loading labels from a file etc).