UserWarning: [W036] The component 'matcher' does not have any patterns defined. stream = (eg for _, eg in pattern_matcher(stream))

[popover]

Hi,

when I run

prodigy ner.manual data_model en_core_web_sm data.jsonl --label LABEL1,LABEL2 --patterns pattern.jsonl

I got the following error.
UserWarning: [W036] The component 'matcher' does not have any patterns defined. stream = (eg for _, eg in pattern_matcher(stream))

Would you plz help us to identify how to approach the issue? (FYI this is not the show stopper as it runs the web server but just wanted to make sure to do things right)

[ljvmiranda921 (Lj Miranda)]

Hi @popover , welcome to Prodigy!

Usually, this warning shows up when there's an issue with the pattern.jsonl file that you passed. Can you ensure that it's correct and follows the format here? More specifically, if your patterns are only string-based patterns, you'll see this warning saying that there are no patterns in the regular Matcher in spaCy. At this point, I'd say that this warning is safe to ignore.

[dave-espinosa (David Espinosa)]

Hello @ljvmiranda921 ,

In my case, as a quick test, I have a single sample in patterns.jsonl file:

{"label": "VEGETABLE", "pattern": "Lamb's lettuce"}

Which is valid according to this documentation (originally recommended in this case); but then I see such structure is not included in the documentation you recommend.

Could you please confirm if my pattern.jsonl file is OK to be used for the --patterns argument?

Thanks!

[ryanwesslen]

hi @dave-espinosa!

@dave-espinosa

Could you please confirm if my pattern.jsonl file is OK to be used for the --patterns argument?

Yes, it should work!

For anyone else who wants to see examples, we have a "example-patterns" folder in our prodigy-recipes folder that has five template examples of patterns.

I also like this post by Vincent (see near the bottom) where he mentions that when passing only a string, you're actually using spaCy's PhraseMatcher instead of spaCy's Matcher. His point was noting that PhraseMatcher is much faster than Matcher. The nice thing is Prodigy is set up to use either.

ner.manual training pauses indefinitely every 10 saves?

To put it very simply; the .jsonl file isn't read into memory all at once. It's more loaded like a Python generator where items are picked one-by-one. If the current item matches the pattern it's added to the batch. Once there's a batch of 10 items (or whatever you've configured) the batch is sent to the front end. I'm glossing over some details here because Prodigy also checks if the item is labelled before, but this is the gist. One thing that's not totally clear yet, am I able to run the sa…

[dave-espinosa (David Espinosa)]

Hello @ryanwesslen ,

Crystal clear now, thanks for the quick reply!