transition from ner to spancat

, ,
1

Hello!

Sorry, if this has been already asked and answered, but i could not find it by searching the support forum.
I have annotated some data using ner.manual/correct and trained some initial models, with ok results.
I want to check if using spancat and retraining on the same dataset, would produce better results, but i don't have the time to reannotate everything from scratch. Is there an efficient way to transition from ner labels to spans for spancat?
In addition, I tried using spans.correct loading my trained ner model and i would like to see the highlighted span predictions (the same ones that shows in ner.correct). Is this supported or possible?

Thank you, as always!

2

Hi! Training a span categorizer from data annotated with ner.manual should just work out of the box, because the underlying format is the same :slightly_smiling_face: So instead of passing your dataset in as --ner when you train or export the Prodigy annotations, you can use --spancat instead.

Whether or not it improves the results really depends on the data and type of annotated spans. In case you haven't seen it, here's an overview of the advantages of NER vs. span categorization, depending on the problem type: https://prodi.gy/docs/span-categorization#ner-vs-spancat).

After training a spancat model from your data, you can then load it in and improve it using spancat.correct.

1 Like