# Efficient binary annotation using textcat.teach

**URL:** <https://support.prodi.gy/t/efficient-binary-annotation-using-textcat-teach/2343>\
**Category:** Uncategorized\
**Tags:** usage, textcat\
**Created:** [December 19, 2019, 3:51pm UTC](https://support.prodi.gy/t/efficient-binary-annotation-using-textcat-teach/2343 "2019-12-19T15:51:49Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![fhamborg](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/fhamborg/32/1010_2.png) [@fhamborg](https://support.prodi.gy/u/fhamborg)\
**Post date:** [December 19, 2019, 3:51pm UTC](https://support.prodi.gy/t/efficient-binary-annotation-using-textcat-teach/2343/1 "2019-12-19T15:51:49Z")

</div>

Hi, I'm using the `textcat.teach` recipe for some straight forward annotation of only two labels: biased-language and neutral-language, like shown in the screenshot.

 ![image](https://us1.discourse-cdn.com/flex020/uploads/prodigy/original/2X/3/3dcc1ec9f9e5662e33c551e72bcf1f1f8bf60792.png)  
I started prodigy using: `prodigy textcat.teach wordchoice en_core_web_lg prod.jsonl --label NEUTRAL-LANG,BIASED-LANG`

As far as I understand the documentation, in `textcat.teach` prodigy uses the model in the loop to already choose one label, which will be shown at the top (e.g., in the screenshot, prodigy predicted `neutral-lang`). Further, human annotators have to accept the task if the shown label is correct in their opinion, and have to reject if the shown label is incorrect.

I got two questions:

1. Is that correct?
2. In terms of manual labeling, I find that considering both the predicted label as well as the text in order to determine which button to press (accept, if the label matches the text, or else reject) puts a higher cognitive load than, for instance, always pressing accept for one label (e.g., biased-language) and always reject for the other (e.g., neutral-language). How would I do that (e.g., accept=biased-language, reject=neutral-language) and still use `textcat.teach` including its active learning part?

Cheers,  
Felix

---

<div class="post-metadata">

**Author:** ![honnibal](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/honnibal/32/35_2.png) [@honnibal](https://support.prodi.gy/u/honnibal)\
**Post date:** [December 19, 2019, 5:01pm UTC](https://support.prodi.gy/t/efficient-binary-annotation-using-textcat-teach/2343/2 "2019-12-19T17:01:11Z")

</div>

Hi,

If your two classes are mutually exclusive (a text can be either biased, or it can be neutral), can you just use one label? You could even get by with positive/neutral/negative if you use ignore for neutral, although that might not be advisable (as then you don't have a way to signal ignore).

---

<div class="post-metadata">

**Author:** ![fhamborg](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/fhamborg/32/1010_2.png) [@fhamborg](https://support.prodi.gy/u/fhamborg)\
**Post date:** [December 19, 2019, 7:31pm UTC](https://support.prodi.gy/t/efficient-binary-annotation-using-textcat-teach/2343/3 "2019-12-19T19:31:36Z")

</div>

Yes, they are mutually exclusive. So, I would call prodigy using:  
`prodigy textcat.teach wordchoice en_core_web_lg prod.jsonl --label BIASED-LANG`  
... and during annotation press accept if the text is biased, and reject if it is not?

I'm just wondering, since prodigy will always predict BIASED-LANG (because there are no other labels it could choose from), how does it learn what to predict?

---

<div class="post-metadata">

**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\
**Post date:** [December 20, 2019, 5:05pm UTC](https://support.prodi.gy/t/efficient-binary-annotation-using-textcat-teach/2343/4 "2019-12-20T17:05:49Z")

</div>

> [@fhamborg](#):
>
> I'm just wondering, since prodigy will always predict BIASED-LANG (because there are no other labels it could choose from), how does it learn what to predict?

spaCy's implementation for text classification will learn to predict a score between `0.0` and `1.0` for each given label. So having only one label shouldn't be a problem. If Prodigy suggests a text for `BIASED-LANG` and it's not biased, you hit reject and the feedback the model gets is "this should be `0.0`". If you accept, the feedback is "this should be `1.0`".