Bootstrapping on NER Accuracy test

We need to perform bootstrapping to test a NER model accuracy. Does there is a function to do it with a .spacy file?

Thank you,

Hi @vtorres , I'm curious as to what you mean by "bootstrapping" in this context.

  • If you wish to use a particular NER model to label data, you can pass the base_model parameter in the ner.manual recipe. You can also use ner.teach if you wish to do active learning.
  • If it's about model evaluation, you can use spacy evaluate. You only need to pass the model you've trained, and the .spacy file of your test data.

Thanks, yes, its about model evaluation. What Im looking is how to do boostraping on the evaluation data. In other words, use test data.spacy and perfome boostraping.

I though spacy had a function like evaluate_boostraping. I know pytorch have a boostrap function but I can apply it to the .spacy format.

Thank you,

Any idea?

Hi @vtorres ,

Hmm, I don't think there's native support to the functionality you've mentioned. The best scenario here would be for you to write that bootstrapping function, save that sampled batch as a .spacy file, and evaluate manually for each using spacy evaluate.

Yes, that what I ended up doing. Thanks!

1 Like