Hi! Prodigy expects you to define the label scheme when you start the annotation process, and if your goal is to collect annotations for machine learning, you typically do not want the annotator to be able to decide about your label scheme and enter labels manually.
The presence and absence of a given label is very important and will have a big impact on your entire model. Also, if a new label is introduced later, this can potentially invalidate previous annotations and you'll end up with inconsistent data and much worse results. So we wouldn't recommend a workflow like this, and it's also why Prodigy wants you to define a fixed label scheme.
That said, during model development, you could use Prodigy to iteratively develop your label scheme and click through a random sample of examples, label them yourself and add notes about labels that might be unclear or missing, so you can add them later. One option would be to have a label
OTHER and a
text_input block you can use for notes.