NER with dozens of entities

As a more general comment/question: Is your goal to actually train an NER model? If so, are you sure you want to structure your annotation task and model this way and have that many labels? You might be making your life a lot harder this way, because you'll need significantly more data and make sure you have enough examples per label so the model can distinguish between them. If the labels are hierarchical, you'd also be encoding multiple aspects within each annotation that the model has to learn: every layer of the hierarchy, plus the token boundaries. For each token, the model has to make all of these decisions, which can make it harder to achieve accurate results.

If your label scheme is hierarchical, you could experiment with starting off with the top level, which is also typically the most crucial one (e.g. is it a product or a plant?). Once you have a model that can predict these with sufficient accuracy, you can add a second step to predict/determine the lower level categories. For example, given an entity span with the label PLANT, which type of plant is it?

For some labels, you might find that a simple rule-based approach (e.g. dictionary lookup) performs much better. For other labels, you can make a prediction given what you already know, which can be a much easier prediction problem, since there's only a small number of labels to pick from. Depending on the problem, using a text classifier to predict categories over the whole text and context can often be useful as well to add more concrete information to the entity predictions.

1 Like