# Bootstrapping terms with pattern file

**URL:** <https://support.prodi.gy/t/bootstrapping-terms-with-pattern-file/1743>\
**Category:** Uncategorized\
**Tags:** usage\
**Created:** [July 5, 2019, 8:24am UTC](https://support.prodi.gy/t/bootstrapping-terms-with-pattern-file/1743 "2019-07-05T08:24:28Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![jsnleong](https://avatars.discourse-cdn.com/v4/letter/j/b9bd4f/32.png) [@jsnleong](https://support.prodi.gy/u/jsnleong)\
**Post date:** [July 5, 2019, 8:24am UTC](https://support.prodi.gy/t/bootstrapping-terms-with-pattern-file/1743/1 "2019-07-05T08:24:29Z")

</div>

Hi all,

I am interested in teaching Prodigy to tag all tokens (until a punctuation) following a certain “critical” word(s).

Eg.  
John is suffering from a bad stomachache after having the food at ABC restaurant.

I would like to tag the terms “having the food at ABC restaurant” to a particular entity, for example NE “Cause of Illness”.

I did the following in my pattern file:

```
{"label": "CAUSES", "pattern":[{"TEXT":{"REGEX": "(?<=after ).*$"}}]}

```

I believe this is the correct REGEX format, however when I feed it to Prodigy ner.teach, it does not seem to work that well… **It only highlights “after” in all my documents, instead of the intended terms AFTER the term “after”.**

What seems to be the problem? Thanks!

---

<div class="post-metadata">

**Author:** ![justindujardin](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/justindujardin/32/169_2.png) [@justindujardin](https://support.prodi.gy/u/justindujardin)\
**Post date:** [July 7, 2019, 4:18pm UTC](https://support.prodi.gy/t/bootstrapping-terms-with-pattern-file/1743/2 "2019-07-07T16:18:23Z")

</div>

Hi @jsnleong,

Sounds like you’re on the right path. It’s hard to say what’s wrong with your limited example setup, but a wild guess would be that prodigy might not support the Regexp you’re using. In particular, I think support for lookaround’s in regexp implementations is spotty.

Can you provide a snippet of code to run your failing example in a contrived way? Also, what version of prodigy/spacy are you running?

---

<div class="post-metadata">

**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\
**Post date:** [July 7, 2019, 6:31pm UTC](https://support.prodi.gy/t/bootstrapping-terms-with-pattern-file/1743/3 "2019-07-07T18:31:28Z")

</div>

Yes, the syntax looks okay, so there are two things to check here:

- Under the hood, the patterns are matched using spaCy’s `Matcher`. Does it work as expected if you match your pattern directly in spaCy?
- Are you using the latest version of Prodigy with spaCy v2.1+? The `REGEX` option was only introduced in that version and is not supported if you’re running spaCy v2.0.

---

<div class="post-metadata">

**Author:** ![jsnleong](https://avatars.discourse-cdn.com/v4/letter/j/b9bd4f/32.png) [@jsnleong](https://support.prodi.gy/u/jsnleong)\
**Post date:** [July 8, 2019, 7:25am UTC](https://support.prodi.gy/t/bootstrapping-terms-with-pattern-file/1743/5 "2019-07-08T07:25:00Z")

</div>

Hi @justindujardin and @ines,

Thanks for the reply.

I tried running the basic Matcher with the RegEx pattern indicated above, this is the result I received…

 ![1211](https://us1.discourse-cdn.com/flex020/uploads/prodigy/original/1X/091e1364d8d58275377d5c1d3ea99596651676d5.png)  
As expected, the match was similiar to that of Prodigy. Only matching the literal word “after”. What seems to be the problem?

In case you are wondering, this is the RegEx that matches perfectly on a RegEx Checker Tool!  
 ![3131](https://us1.discourse-cdn.com/flex020/uploads/prodigy/original/1X/72d7617e059c4364f74fea478eaf02a5cb985413.png)

Kindly advice, thanks!

---

<div class="post-metadata">

**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\
**Post date:** [July 8, 2019, 7:58am UTC](https://support.prodi.gy/t/bootstrapping-terms-with-pattern-file/1743/6 "2019-07-08T07:58:50Z")

</div>

Thanks for sharing more details – I think I understand it now. Under the hood, Prodigy uses spaCy's `Matcher`, so if the matcher doesn't match, Prodigy won't produce a match either.

spaCy's rule-based `Matcher` works on **tokens** , not the entire text. Each dictionary in the patterns represents **one token**. So the regex in your first pattern entry will look for one token that matches `(?<=after).*$`, which correctly matches the token with the text `"after"`. If you're looking for a token "after" and one or more tokens following it, you could for instance represent it like this one the token level:

```python
[{"TEXT": "after"}, {"OP": "+"}]

```

If you do want to match over the whole text using one regular expression, you could also just write your own little regex matcher that matches on the incoming text and yields examples with the matched `"spans"`. See my comment here for an example:

> [@Off-track use of Prodigy/Spacy - Custom Regex Pattern Matching and Modeling](https://support.prodi.gy/t/off-track-use-of-prodigy-spacy-custom-regex-pattern-matching-and-modeling/614/36):
>
> Sorry, I was getting a big confused by all of that nested logic and the the pattern matcher that yields examples etc. And yes, if you want to create static annotation examples (one for each span) and render them exactly as they come in to accept/reject (e.g. with the mark recipe), you’d really only need something like this? examples = [] for text in LOTS\_OF\_TEXTS: for label, regex\_patterns.items(): for match in re.finditer(expression, text): start, end = match.span() …

---

<div class="post-metadata">

**Author:** ![jsnleong](https://avatars.discourse-cdn.com/v4/letter/j/b9bd4f/32.png) [@jsnleong](https://support.prodi.gy/u/jsnleong)\
**Post date:** [July 8, 2019, 8:16am UTC](https://support.prodi.gy/t/bootstrapping-terms-with-pattern-file/1743/7 "2019-07-08T08:16:51Z")

</div>

Hi @ines,

Awesome, big thanks!! I see why things didn’t work out previously… 🙂

After fixing the pattern file, what seems to be the difference between ner.teach and ner.match? Both seems to be performing the same action of matching, isn’t it?

Also, with regards to my previous post ([Domain-specific NER project](https://support.prodi.gy/t/domain-specific-ner-project/1742)), do you have any suggestion to pre-trained healthcare models that can be imported to spaCy? How about the ones in spaCy’s universe?

Thanks once again!

---

<div class="post-metadata">

**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\
**Post date:** [July 8, 2019, 3:27pm UTC](https://support.prodi.gy/t/bootstrapping-terms-with-pattern-file/1743/8 "2019-07-08T15:27:18Z")

</div>

> [@jsnleong](#):
>
> After fixing the pattern file, what seems to be the difference between ner.teach and ner.match? Both seems to be performing the same action of matching, isn’t it?

Both recipes do matching, but in addition to that, `ner.teach` will also show you the model's predictions and update the model in the loop with your annotations. The patterns can help you get over the "cold start problem" and ensure you get to annotate enough positive examples in the beginning. Once the model has seen enough of them, it'll start suggesting entities as well, based on the previous annotations.

`ner.match` only performs the matching itself and doesn't do any active learning or use the model to suggest examples. It's useful if you want to go through a list of patterns and label their matches in context.

---

<div class="post-metadata">

**Author:** ![jsnleong](https://avatars.discourse-cdn.com/v4/letter/j/b9bd4f/32.png) [@jsnleong](https://support.prodi.gy/u/jsnleong)\
**Post date:** [July 9, 2019, 2:08am UTC](https://support.prodi.gy/t/bootstrapping-terms-with-pattern-file/1743/9 "2019-07-09T02:08:26Z")

</div>

Hi @ines, thanks!

> [@ines](#):
>
> [{"TEXT": "after"}, {"OP": "+"}]

I tried your token matching attribute, but didn't work for me. Instead, I made a small change and got it to work.  
`[{"TEXT": "after"}, {"TEXT": {"REGEX": "[a-zA-Z0-9]*$"}, "OP": "+"}]`

However, what happens now is that in Prodigy annotation server, I get matches that increment one token at a time. This is an illustration. The one in bold are the highlighted texts.

John feels sick **after eating** prawns at Restaurant ABC.  
John feels sick **after eating prawns** at Restaurant ABC.  
John feels sick **after eating prawns at** Restaurant ABC.  
John feels sick **after eating prawns at Restaurant** ABC.  
John feels sick **after eating prawns at Restaurant ABC**.

So what I did was, I rejected the first 4 matches, and only accepted the final one. Is this the right approach? Am I erroneously teaching the model by rejected so many examples?

Also, this is pretty time consuming. I wanted to just allow the machine to highlight the chunk of text "after eating prawns at Restaurant ABC" and from there I just simply rej/accept.

Thanks!