Exclude labels when training spancat

[joebuckle]

Good day,

Is there a way to include only a subset of the labels when training spancat?

Regards,
Joe

[ryanwesslen]

hi @joebuckle!

@joebuckle

Is there a way to include only a subset of the labels when training spancat?

If so, thinking off the top of my head -- the simplest solution would be to create a new dataset that only has the labels you want. Have you tried that?

I answered a similar question yesterday where I showed a simple code to export annotations (db-out), change a few mislabeled examples, then import back in (db-in).

unannotated labels from meta.json file

hi @kushalrsharma! It seems like you have labels that were misnamed SOFTSKILL and HARDSKILL in your annotations. The simplest approach is change those labels. Let's say your annotations are in a Prodigy dataset called ner_dataset: python -m db-out ner_dataset > ner_dataset.jsonl Then run a python script that changes any span labels with SOFTSKILL and HARDSKILL to SOFTSKILLS and HARDSKILLS. This isn't the most elegant but a simple way to correct those: import srsly examples = srsly.read_js…

The only difference is you may want to remove specific spans type instead of changing the label name.

[joebuckle]

Thanks!