I am very new to prodigy, but it appears that we have to know the labels upfront for our NER task. In my case, if I have a new dataset, I may not know the Entities I want to capture in advance, and would like to be able to add the labels as I go.
Is that possible? I apologize if this is already on the roadmap or has been asked before.
Hi! Prodigy expects you to define the label scheme when you start the annotation process, and if your goal is to collect annotations for machine learning, you typically do not want the annotator to be able to decide about your label scheme and enter labels manually.
The presence and absence of a given label is very important and will have a big impact on your entire model. Also, if a new label is introduced later, this can potentially invalidate previous annotations and you'll end up with inconsistent data and much worse results. So we wouldn't recommend a workflow like this, and it's also why Prodigy wants you to define a fixed label scheme.
That said, during model development, you could use Prodigy to iteratively develop your label scheme and click through a random sample of examples, label them yourself and add notes about labels that might be unclear or missing, so you can add them later. One option would be to have a label OTHER and a text_input block you can use for notes.