Understanding ner.batch-train stats

[ines (Ines Montani)]

@Mindaugas

Does this mean that, while training, the model will assume that entities are the ones marked and the rest is not an entity?

If you run ner.batch train with the --no-missing flag, then yes. All tokens that are not part of an entity will be assumed to be not part of an entity. Otherwise, non-marked tokens will be treated as missing values.

@Mindaugas

Are there use cases when someone would like to mark something in ner.manual and reject?

The recipe / interface lets you assign labels to text – what they mean is up to you. So you could use the "reject" action to create negative examples, or to mark examples with fundamental problems, like wrong tokenization. Also see this thread:

When to reject in ner.manual or ner.make-gold?

I think I found the answer while looking for something else. In this answer @honnibal brings it right to the point: To save others the click: One should only accept. Rejection is a way to mark serious trouble like wrong tokenizations.