Prodigy present text with no matching pattern (ner.manual)

, ,
1

Hello,

I'm using the command

prodigy ner.manual XXX en_core_web_sm XXX.jsonl --label ORG,PERSON,DATE --patterns ./pattern.jsonl

to annotate docs that have matching patterns. However, Prodigy interface keeps giving me texts with no matching patterns at all.

image

2

Hi! This behaviour is intentional for ner.manual, because patterns are typically used to help pre-label examples. In that case, it's important to also show examples with no pattern matches, because those could either still be containing other entities you want to assign, or no entities at all, in which case you also want to include that example in your training data.

If you just want to select examples based on the matches they contain, maybe the generic binary match recipe is a better fit for your use case. Or, if you want a manual labelling workflow that only shows you examples containing matches, you could use your own version of the ner.manual recipe and set all_examples=False when initializing the PatternMatcher. This will only show examples containing pattern matches and filter out everything else.

2 Likes
3

Thanks Ines. However, I'm unable to modify the source file, nor can I replace a with a new file. So how can I use my custom recipe?

4

Check out the docs on custom recipes here: https://prodi.gy/docs/custom-recipes
You can use this example recipe of ner.manual as a template: https://github.com/explosion/prodigy-recipes/blob/master/ner/ner_manual.py

(In general, it should be no problem to modify the installed source btw. It's maybe not a best practice and not as clean as using a custom recipe, but it's possible if you just want to try something out. You can run prodigy stats to find the location of your Prodigy installation and the recipes are included in prodigy/recipes.)

1 Like
5

I try to use the command

python -m prodigy -F XXX/ner_teach.py

but got the following error.

image

I'm very confident the path to ner_teach.py is valid.

6

You need to also tell it which recipe to load after prodigy and the arguments of the recipe, if available. Otherwise, Prodigy won't know what to do and what to load from your file. See the custom recipe example here that shows the recipe vs. how it's called.

So in your case, that could look something like this (depending on what arguments your reicpe takes):

python -m prodigy [name of your recipe] [recipe arguments] -F XXX/ner_teach.py
1 Like