Improve a NER on multiple labels

[ines (Ines Montani)]

Yes, that sounds like a good workflow. And if possible, you might want to keep the merged files somewhere for reference, so you can always go back to a previous state.

You can also automate the merging process in Python btw – I've posted a little snippet here:

Merging annotations from different datasets

Each example that comes in is assigned an _input_hash – for text-based tasks, that hash is generated from the "text" property. So you can easily check whether annotations refer to the same input text by comparing their input hashes, and then merging their "spans". The NER model comes with a helper function merge_spans that should do exactly that – however, it’s currently only used internally during updating and we haven’t really tested its isolated usage yet. But you can try the following: fr…