Is there a way to use spans.correct with patterns? I explain the use case further:
We have labeled a dataset with spans.manual containing 10 labels, and have trained a model with it. After measuring the performance of the model, we saw that it was weak on 2 labels, so we though to label more samples that contained these 2 underperforming labels.
It would be great if we could have model suggestions for the spans the model already know well, and then use a pattern file to match some of the spans of the underperforming labels. It would be great for the person labeling if he could receive labeling suggestions from both the model and the pattern list. Is there a way to do this in Prodigy? We have seen that spans.correct does not accept a --patterns parameter, maybe because this was not the intended use.
Also, If there is a better way to achieve this it would be great to know.
Thanks for your question and welcome to the Prodigy community
It may be helpful to know why patterns with non-manual recipes can be challenging:
But if you wanted to, you can create a custom recipe. My recommendation would be to start with the spans.manual to view how the patterns file is used and then combine it with the spans.correct.
To view the recipes, run
python -m prodigy stats
And find your Location path (see Location: path/to/python/site-packages/prodigy). Then open that folder and look for the /recipes/spans.py file. If you're able to get something, feel free to post a solution! We're always grateful for custom recipes.
Another option may be to use an entity ruler along with a span or ner model. This post has a good example:
Also, you may find this post helpful too:
I hope this helps get you started and let us know if you have questions!
