Hi, I don’t understand how ner.make-gold works/ why it doesn’t work in my case:
I have a pre-annotated dataset with two named entity categories. Now I’d like to iterate over this dataset again, to check if the annotations really are correct and if not to correct them.
The explanation in the recipes section says that with ner.make-gold, I can “correct the model’s suggestions” and “can remove and correct (the annotations) if necessary”.
When I run the command as suggested, I only get the text without any annotations. Why doesn’t ner.make-gold work? Or isn’t it meant to work like this anyways?
(I use the German model “de_core_news_sm”.)
Do you have any suggestions, how I can go through my pre-annotated dataset, check the annotations and in case of wrong or missing annotations correct resp. add those?
Sorry if the description of the recipe was a little confusing in the docs!
The ner.make-gold recipe runs an existing pre-trained model over a (raw) dataset, gets the “best” predictions (i.e. doc.ents) and highlights them in the app and makes them editable. The main idea here is that it can potentially save you a lot of time compared to ner.manual – if you want to label all entities in the text to create gold-standard data, and your model is already 80% correct, you only have to manually annotate 20% instead of everything.
If I understand your use case correctly, you already have pre-annotated data with entities and don’t need to ask the model to highlight them for you, right? So instead of using ner.make-gold (which will add spans from the model), you could just load your data into ner.manual. The recipe respects already set entity spans and will make them editable.
(Btw, make sure to save your new, corrected annotations to a separate dataset! This also lets you compare the differences later on, by reading in both datasets and checking which spans have changed.)
thanks a lot for your fast and detailed reply. Now it makes sense that ner.make-gold didn’t work in my use case (yes, I already have pre-annotated data with entities and don’t need to ask the model to highlight them for me). I’ll try the ner.manual instead!