# REGEX operator in the patterns file **URL:** \ **Category:** Uncategorized\ **Tags:** usage, spacy, solved\ **Created:** [December 3, 2018, 10:14am UTC](https://support.prodi.gy/t/regex-operator-in-the-patterns-file/1008 "2018-12-03T10:14:09Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![mmeasic](https://avatars.discourse-cdn.com/v4/letter/m/9fc348/32.png) [@mmeasic](https://support.prodi.gy/u/mmeasic)\ **Post date:** [December 3, 2018, 10:14am UTC](https://support.prodi.gy/t/regex-operator-in-the-patterns-file/1008/1 "2018-12-03T10:14:09Z")
Hi there! First of all, thank you for the support! In order to use the new REGEX operator in the patterns file, I would like to provide a pattern in the patterns.jsonl file. So, let’s say I have a lot of examples where I expect a token or a sequence of tokens to be labelled with a specific label, but after a specific token (that specific token actually designates where the bank transaction occurred). Therefore, a pattern is a simple one, using positive look behind and captures everything after. {“label”: “MERCHANT”,“pattern”: [{“REGEX”: “(?\<=IL\s).\*”}]} P.S: I have added an escaping backslash because of JSON decoder However, after I run the ner.match recipe, every token is labelled as a MERCHANT with the pattern ID being 0 (the one I have provided). What am I doing wrong? ---
**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\ **Post date:** [December 3, 2018, 2:26pm UTC](https://support.prodi.gy/t/regex-operator-in-the-patterns-file/1008/2 "2018-12-03T14:26:38Z")
Sorry if this was confusing – I assume you’re referring to the `REGEX` attribute proposal in this [GitHub thread](https://github.com/explosion/spaCy/issues/1971)? This thread is still only the _spec and proposal_, i.e. the planned implementation. The changes will hopefully ship with spaCy v2.1.0 (since some of the changes to the `Matcher` internals are not fully backwards compatible). But they’re not yet available in the stable release and not implemented in the current nightly build. ---
**Author:** ![mmeasic](https://avatars.discourse-cdn.com/v4/letter/m/9fc348/32.png) [@mmeasic](https://support.prodi.gy/u/mmeasic)\ **Post date:** [December 5, 2018, 2:05pm UTC](https://support.prodi.gy/t/regex-operator-in-the-patterns-file/1008/3 "2018-12-05T14:05:14Z")
Thanks! I implemented the custom recipe and adjusted it to receive the various regular expressions in order to speed-up the gathering of annotations. Have a nice day! ---
**Author:** ![dancsalo](https://avatars.discourse-cdn.com/v4/letter/d/779978/32.png) [@dancsalo](https://support.prodi.gy/u/dancsalo)\ **Post date:** [March 25, 2020, 2:32pm UTC](https://support.prodi.gy/t/regex-operator-in-the-patterns-file/1008/4 "2020-03-25T14:32:02Z")
Hi @ines, I am interested in using the REGEX attribute now that it is available in spaCy. But every token in every text is still being labeled by that pattern (as described by @mmeasic) . When can we expect the REGEX to be supported in prodigy? Or am I doing something wrong? ---
**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\ **Post date:** [March 25, 2020, 6:29pm UTC](https://support.prodi.gy/t/regex-operator-in-the-patterns-file/1008/5 "2020-03-25T18:29:44Z")
> [@dancsalo](#): > > When can we expect the REGEX to be supported in prodigy? Or am I doing something wrong? The matching is all done via spaCy so if you're using a recent version of spaCy that supports the `REGEX` operator (v2.1+), it should work as expected and described [here](https://spacy.io/usage/rule-based-matching#regex). ---
**Author:** ![dancsalo](https://avatars.discourse-cdn.com/v4/letter/d/779978/32.png) [@dancsalo](https://support.prodi.gy/u/dancsalo)\ **Post date:** [March 27, 2020, 2:52am UTC](https://support.prodi.gy/t/regex-operator-in-the-patterns-file/1008/6 "2020-03-27T02:52:40Z")
Ah great, a recent version of spaCy worked! ---
**Author:** ![cverluise](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/cverluise/32/1547_2.png) [@cverluise](https://support.prodi.gy/u/cverluise)\ **Post date:** [May 8, 2020, 5:36pm UTC](https://support.prodi.gy/t/regex-operator-in-the-patterns-file/1008/7 "2020-05-08T17:36:10Z")
Hello, first of all, thanks for the great tool. I'm facing the same issue as @dancsalo - namely the fact that all tokens are labeled by the pattern while it should not be the case. However @dancsalo fix does not apply to me since I'm using spaCy 2.2.3. I have another question related to the regex in patterns. How can I use regex which requires special characters from a JSON point of view (e.g. `{[`)? from what I see, it produces a JSON format issue. Does it mean that it is not supported or that the syntax should be adapted - I did not find anything in the doc on that point but I might have missed it. If this is not the place for this question, let me know. Cheers! ---
**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\ **Post date:** [May 8, 2020, 7:51pm UTC](https://support.prodi.gy/t/regex-operator-in-the-patterns-file/1008/8 "2020-05-08T19:51:34Z")
Thanks! From what you describe, it sounds like your pattern contains properties that aren't interpreted or that apply to all tokens, so a token pattern is either interpreted as a wildcard (`{}`), or it just always matches all tokens. > [@cverluise](#): > > How can I use regex which requires special characters from a JSON point of view (e.g. `{[` )? from what I see, it produces a JSON format issue. If you're providing them in a JSON string, you should be able to just escape them – like `"\{"`. And if you want to use `\` in your expression, that'd be `\\`. ---
**Author:** ![cverluise](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/cverluise/32/1547_2.png) [@cverluise](https://support.prodi.gy/u/cverluise)\ **Post date:** [May 8, 2020, 8:22pm UTC](https://support.prodi.gy/t/regex-operator-in-the-patterns-file/1008/9 "2020-05-08T20:22:02Z")
Hello @ines , thanks a lot for your swift reply. > From what you describe, it sounds like your pattern contains properties that aren't interpreted or that apply to all tokens, so a token pattern is either interpreted as a wildcard ( `{}` ), or it just always matches all tokens. I don't think that's the issue. I have tested with a `patterns.jsonl` restricted to the following pattern `{"label":"AC","pattern":[{"text":{"regex":"^abc$"}}]}` Then, running: `prodigy ner.manual tmp en_core_web_sm ./data.json --label AC --patterns ./patterns.jsonl` I get all tokens highlighted: ![image](https://us1.discourse-cdn.com/flex020/uploads/prodigy/original/2X/a/ae4609256ea8df3c34cc0f5e92602f00dd536439.png) which should not be the case, right? My set-up is the following: - prodigy 1.9 - spaCy 2.2.3 - python3.7 - OSX Any idea of what is happening? Thanks in advance, no rush of course. Cheers! ---
**Author:** ![cverluise](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/cverluise/32/1547_2.png) [@cverluise](https://support.prodi.gy/u/cverluise)\ **Post date:** [May 24, 2020, 1:29pm UTC](https://support.prodi.gy/t/regex-operator-in-the-patterns-file/1008/10 "2020-05-24T13:29:31Z")
Hello @ines, I'm still curious to have your thoughts on the issue above if you have any time for that. Am I making a mistake somewhere? Also, I'm very excited about the new features that the team prepared for v1.10. Thanks for making such a cool tool, that's crazy how it can boost productivity. Best, Cyril ---
**Author:** ![id8ai](https://avatars.discourse-cdn.com/v4/letter/i/ecb155/32.png) [@id8ai](https://support.prodi.gy/u/id8ai)\ **Post date:** [June 25, 2020, 10:17am UTC](https://support.prodi.gy/t/regex-operator-in-the-patterns-file/1008/11 "2020-06-25T10:17:40Z")
Any update on using complex regex patterns in Spacy ? Im keen to use them to identify @FirstName Surname patterns which require variable length + ---
**Author:** ![ford](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ford/32/1512_2.png) [@ford](https://support.prodi.gy/u/ford)\ **Post date:** [August 3, 2020, 9:33pm UTC](https://support.prodi.gy/t/regex-operator-in-the-patterns-file/1008/12 "2020-08-03T21:33:26Z")
Hi @cverluise, can you try changing your patterns file to have a capital "REGEX" key like this: `{"label":"AC","pattern":[{"text":{"REGEX":"^abc$"}}]}`