Well…There’s always going to be a limit to the number of buttons, so at some point we would have to back off to a dropdown menu. We can debate exactly when that limit should be, but I think 10 is a good number. With 10 buttons you can give them numeric keys. At more than 10, you’ll have an inconsistency: how do you hotkey the extra buttons? Or do some of the buttons get hotkeys and not others?
I would also suggest that annotation with so many entity types probably isn’t going to be efficient. You might want to think of alternative workflows. If you have an entity scheme with lots of entity types that are rare, you might ant to apply only a generic entity type “ENTITY” while doing the ner.manual mode. You can the go back and mark the types of the entities, after you’ve identified them. I would suggest sorting the annotation tasks so that you’re asked all the questions for one name in a row. For instance, you might have a lot of examples of the names “Donald Trump” and “Barack Obama”. You can click through these very quickly if you’re asked them together, because you only have to think about the first couple of examples.
Remembering a complicated entity annotation scheme is quite difficult, so you’ll see a higher rate of annotation errors if you try to fully label each sentence before moving onto the next one. The annotation work will also be much slower. If you restructure it so that the annotator only has to think about one category at a time, the task will feel much easier and less frustrating, and it should also be easier to stay self-consistent.