Prodigy patterns not behaving like Spacy patterns

[ines (Ines Montani)]

@jsidell

That’s why using CARDINAL never worked. Using “IS-DIGIT” does work, but still displays the problem with skipping described above.

This is one of the downsides of relying on statistical predictions in match patterns – you kinda lose the reliability of rule-based matching, because the results now depend on what the model happens to predict for a particular example. So for things like numbers, IS_DIGIT or LIKE_NUM is definitely the more reliable option. In spaCy v2.1+, you can also use the IN operator (for set membership) or REGEX to describe the token using a regular expression.

Do you have a small reproducible example that shows the "skipping" behaviour you mean?

@jsidell

I’m now getting errors using terms.teach, but I’ll start a separate thread for that.

Have you upgraded to v1.8.2? If you were seeing warnings about empty vectors that slowed down the startup, that might be the problem. See here for details:

Error when adding seed terms to terms.teach

Just implemented the fix for this, which should be released today. I’ve implemented an extra check in the recipe to avoid iterating over words which don’t have a vector, to avoid the warning spam. In the meantime, you can set the environment variable SPACY_WARNING_IGNORE="W008" to prevent the warnings from coming up. The warnings are really the only problem here: with so many warnings printed, the loop runs fairly slowly, which is why you’re not seeing anything coming up. If you suppress the wa…