So I have a dataset containing a bunch of “gold standard” annotations (i.e. all relevant entities are annotated and only them, and there’s no ‘rejects’) that I used as evaluation set for training my model with another gold standard dataset (so, using the
After my model got good enough to make decent suggestion for binary annotation, I made a bunch of those as well (in a new dataset) and then tried to further train the model on that dataset (without the
My question is, can I use the same evaluation set as I used when training with the first batch of “gold standard” annotations? That evaluation set only contains
accepts, and without the
--no-missing flag, there is no way for the model to know that whatever is not marked as entity is not an entity.