Currently, if there are too many labels in the interface (ner_manual in ner.make-gold) , the layout falls back to a dropdown with all labels. A dropdown with labels requires lot of clicks which is inconvenient.
May I propose to make the interface such that the buttons are always presented above the text, maybe in multiple rows instead of one row?
I also widened the cardMinWidth/cardMaxWidth custom_theme properties, which also help to showmore label-buttons. But there seems to be a maximum of 9 or 10 buttons (even if there is still space enough for more)
Well…There’s always going to be a limit to the number of buttons, so at some point we would have to back off to a dropdown menu. We can debate exactly when that limit should be, but I think 10 is a good number. With 10 buttons you can give them numeric keys. At more than 10, you’ll have an inconsistency: how do you hotkey the extra buttons? Or do some of the buttons get hotkeys and not others?
I would also suggest that annotation with so many entity types probably isn’t going to be efficient. You might want to think of alternative workflows. If you have an entity scheme with lots of entity types that are rare, you might ant to apply only a generic entity type “ENTITY” while doing the ner.manual mode. You can the go back and mark the types of the entities, after you’ve identified them. I would suggest sorting the annotation tasks so that you’re asked all the questions for one name in a row. For instance, you might have a lot of examples of the names “Donald Trump” and “Barack Obama”. You can click through these very quickly if you’re asked them together, because you only have to think about the first couple of examples.
Remembering a complicated entity annotation scheme is quite difficult, so you’ll see a higher rate of annotation errors if you try to fully label each sentence before moving onto the next one. The annotation work will also be much slower. If you restructure it so that the annotator only has to think about one category at a time, the task will feel much easier and less frustrating, and it should also be easier to stay self-consistent.
Quick update: I’ve been refactoring some parts of the front-end for the upcoming release, so the "list" labels view will now be able to support larger label sets (and won’t fall back to a dropdown anymore). Just like in the "choice" interface, you’ll be able to access the labels 1-10 via the keys 1 to 0. Labels 11-20 will be accessible via shift+1 to shift + 0 and labels 21-30 via shift+alt+1 to shift+alt+0.
(As @honnibal mentioned above, we’d still recommend working with smaller label sets in general and there probably aren’t many cases where using the full 30 labels and keyboard shortcuts – or even more than 10 to 15 labels – would be efficient.)
Hi thanks for the change. I totally agree with @honnibal for sticking to smaller label sets, I also experienced this as a quicker exercise. The use case here is that I also want to improve an existing annotated set, that already consists of multiple labels and that generate a completely annotated sentence (incl POS and dependencies). So for creating this, it is great to have all labels as an option.