# Text classification \`accept\` showing both matcher spans and annotator labels

**URL:** https://support.prodi.gy/t/text-classification-accept-showing-both-matcher-spans-and-annotator-labels/4051
**Category:** Uncategorized
**Tags:** solved, front-end, usage, textcat
**Created:** [March 22, 2021, 3:05am UTC](https://support.prodi.gy/t/text-classification-accept-showing-both-matcher-spans-and-annotator-labels/4051 "2021-03-22T03:05:21Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![JeanneC](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/jeannec/32/2052_2.png) [@JeanneC](https://support.prodi.gy/u/JeanneC)
#### Post date: [March 22, 2021, 3:05am UTC](https://support.prodi.gy/t/text-classification-accept-showing-both-matcher-spans-and-annotator-labels/4051/1 "2021-03-22T03:05:21Z")

</div>

Hello,

I currently have a text classification recipe that uses the Spacy RulesMatcher to identify and highlight spans of text. The point of the highlighting is to flag keywords to help annotators assign labels to sentences. An example of the UI is here:

 ![Screenshot 2021-03-21 at 8.48.09 PM](https://us1.discourse-cdn.com/flex020/uploads/prodigy/original/2X/8/8f1df7f1b7d0386bf6a9220bea5b2a22bda943df.png)

After exporting the data with `db-out`, I see that the `accept` field in the JSONL seems to contain both the matcher name (eg. maturity\_date) as well as the annotator's labels (eg. Maturity Date). This means that the `accept` field will often have duplicates such as ('Maturity Date' and 'maturity\_date').

Is this expected behaviour? And if so, is there a recommended way to isolate only the annotator's label without the matcher spans?

Thanks a lot 🙏

---

<div class="post-metadata">

### Author: ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)
#### Post date: [March 23, 2021, 12:28am UTC](https://support.prodi.gy/t/text-classification-accept-showing-both-matcher-spans-and-annotator-labels/4051/2 "2021-03-23T00:28:04Z")

</div>

> [@JeanneC](#):
>
> After exporting the data with `db-out`, I see that the `accept` field in the JSONL seems to contain both the matcher name (eg. maturity\_date) as well as the annotator's labels (eg. Maturity Date). This means that the `accept` field will often have duplicates such as ('Maturity Date' and 'maturity\_date').

Hi! Just to make sure I understand your recipe correctly: does your matcher also pre-select labels by adding them to the `"accept"` key of the outgoing tasks? And if so, could you share the code? Maybe the ID it's adding doesn't map the IDs of the `"options"`, or you have the text/ID swapped in the options you provide?

Under the hood, selecting a choice option in the UI will always add the `"id"` value of the respective option in `"options"` to the `"accept"` list. For example, if your option looks like this:

```python
{"id": "MATURITY_DATE", "text": "Maturity Date"}

```

... the annotator will see "Maturity Date" and if they select it, Prodigy will add `"accept": ["MATURITY_DATE"]`. If you want to pre-populate the selected options based on matches, that's also what you would stream in. If the data that's loaded in contains an unknown ID, e.g. `"accept": ["m_date"]`, it will be ignored – so maybe that's what's happening.

---

<div class="post-metadata">

### Author: ![JeanneC](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/jeannec/32/2052_2.png) [@JeanneC](https://support.prodi.gy/u/JeanneC)
#### Post date: [March 24, 2021, 12:19pm UTC](https://support.prodi.gy/t/text-classification-accept-showing-both-matcher-spans-and-annotator-labels/4051/3 "2021-03-24T12:19:43Z")

</div>

Hello Ines,

Thanks for the explanation about how `"options"` are added to the `"accept"` list. I realised what was happening was like you mentioned - the matcher was pre-selecting labels by adding them to the `accept` key of outgoing tasks. This was not what we wanted so commenting out that line removed the preselection from the `accept` key. For reference, the code is below:

```python
def get_stream_with_matches(stream, patterns, nlp): 
    """load patterns file, match text to patterns and return text for annotation with patterns highlighted"""
    # load patterns file and convert to matcher format     
    patterns = srsly.read_jsonl(patterns) 
    patterns_by_label = defaultdict(list) 
    for pattern in patterns: 
        patterns_by_label[pattern["label"]].append(pattern["pattern"]) 
    matcher = Matcher(nlp.vocab) 
    for label, rules in patterns_by_label.items(): 
        matcher.add(label, None, *rules)
    data_tuples = ((eg["text"], eg) for eg in stream) 
    for doc, eg in nlp.pipe(data_tuples, as_tuples=True): 
        spans = [] # matched spans 
        matched_labels = set() # all labels that were matched 
        for match_id, start, end in matcher(doc): 
            span = Span(doc, start, end, label=match_id) 
            matched_labels.add(span.label_) 
            spans.append({"start": span.start_char, "end": span.end_char, "label": span.label_}) 
            eg["spans"] = spans 
            # eg["accept"] = list(matched_labels) 
            yield eg

```

Thank you for your help!
