# Textcat with customer sorter didn't exclude dataset

**URL:** <https://support.prodi.gy/t/textcat-with-customer-sorter-didnt-exclude-dataset/2683>\
**Category:** Uncategorized\
**Tags:** textcat\
**Created:** [March 19, 2020, 9:02pm UTC](https://support.prodi.gy/t/textcat-with-customer-sorter-didnt-exclude-dataset/2683 "2020-03-19T21:02:49Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![curious](https://avatars.discourse-cdn.com/v4/letter/c/6bbea6/32.png) [@curious](https://support.prodi.gy/u/curious)\
**Post date:** [March 19, 2020, 9:02pm UTC](https://support.prodi.gy/t/textcat-with-customer-sorter-didnt-exclude-dataset/2683/1 "2020-03-19T21:02:49Z")

</div>

I'm using Prodigy 1.9.8. I've updated the textcat to only return the data with score higher than 0.7. I used it to teach and exclude some dataset ID. I've found data which should be excluded data were still displayed during the teach. Is it a bug in Prodigy or something is missing in my custom sorter?

I used the following command to teach -

```
prodigy textcat.teach --label lab db_model_2 db2 /tmp/trained_model/ ~/new_dataset.jsonl -e db_pattern,db1

```

The db\_pattern dataset was created by the new match recipe. I compared the output of this teach db\_model2 and db\_pattern. The same text has the same input\_hash, but the task\_hash is different. I wonder whether that's the problem. Here is the custom\_sort method in the textcat -

```
def custom_sorter(scored_examples):
for score, example in scored_examples:
    # your own logic here to decide whether to send out an
    # example for annotation
    if score > 0.70:
        yield example
```

---

<div class="post-metadata">

**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\
**Post date:** [March 20, 2020, 10:55am UTC](https://support.prodi.gy/t/textcat-with-customer-sorter-didnt-exclude-dataset/2683/2 "2020-03-20T10:55:27Z")

</div>

> [@curious](#):
>
> The same text has the same input\_hash, but the task\_hash is different. I wonder whether that's the problem. Here is the custom\_sort method in the textcat -

Yes, the `_task_hash` value is what determines the exclusion by default – two tasks are considered to be different questions if they have differen task hashes. (If their input hashes are the same, they're considered different questions about the same input.) You can read more about this [here](https://prodi.gy/docs/api-loaders#hashing).

So I think the hashes in your dataset were created by taking the text and the matches into account (which makes sense in context of the match recipe). So there are two things you can do in your custom recipe:

- Rehash the examples with [`set_hashes`](https://prodi.gy/docs/api-components#set_hashes) and `overwrite=True`, and the keys you want to consider (probably text for input hash and label for task hash)?
- Set `"exclude_by": "input"` in the `"config"` returned by your recipe, to treat examples with the same _input_ hashes as duplicates. (Note that the exclude logic is only applied after the recipe has started, so you'd still see duplicates in your sorter etc. – they just won't be shown in the app when the server starts and puts together the stream.)