# KeyError: 'label' Error with Prodigy 1.10.7

**URL:** <https://support.prodi.gy/t/keyerror-label-error-with-prodigy-1-10-7/4097>\
**Category:** Uncategorized\
**Tags:** usage, textcat, solved\
**Created:** [April 1, 2021, 8:29pm UTC](https://support.prodi.gy/t/keyerror-label-error-with-prodigy-1-10-7/4097 "2021-04-01T20:29:01Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![inceatakan](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/inceatakan/32/1741_2.png) [@inceatakan](https://support.prodi.gy/u/inceatakan)\
**Post date:** [April 1, 2021, 8:29pm UTC](https://support.prodi.gy/t/keyerror-label-error-with-prodigy-1-10-7/4097/1 "2021-04-01T20:29:01Z")

</div>

Hello,

I am trying to train a grammar tool for certain ungrammaticality patterns for English in a similar way to [Training a grammar tool - #2 by ines](https://support.prodi.gy/t/training-a-grammar-tool/259/2).

Here are a few examples from the labeled dataset in json format (after labeling in Prodigy):

{"text":"Energy Australia will do practically all the work","label":"BAD\_GRAMMAR","\_input\_hash":-1212092456,"\_task\_hash":-510335938,"\_session\_id":null,"\_view\_id":"classification","answer":"reject"}  
{"text":"And they are there for there customers","label":"BAD\_GRAMMAR","\_input\_hash":-1323887238,"\_task\_hash":1000448416,"\_session\_id":null,"\_view\_id":"classification","answer":"accept"}

Each example/line has 'label: "BAD\_GRAMMAR"' label.

Below is the code for training a model with 'prodigy train textcat':

!python -m prodigy train textcat new\_set ./Desktop/Retraining\_POS\_Tagger/tagger\_model\_3 --output ./Desktop/Grammaticality\_Classifier/grammaticality\_model --eval-id test\_dataset -TE

However, I get the following error message:

"""  
✔ Loaded model './Desktop/Retraining\_POS\_Tagger/tagger\_model\_3'  
Traceback (most recent call last):  
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/runpy.py", line 193, in \_run\_module\_as\_main  
return \_run\_code(code, main\_globals, None,  
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/runpy.py", line 86, in \_run\_code  
exec(code, run\_globals)  
File "/Users/atakanince/groupsolver\_env/lib/python3.8/site-packages/prodigy/ **main**.py", line 53, in   
controller = recipe(_args, use\_plac=True)  
File "cython\_src/prodigy/core.pyx", line 321, in prodigy.core.recipe.recipe\_decorator.recipe\_proxy  
File "/Users/atakanince/groupsolver\_env/lib/python3.8/site-packages/plac\_core.py", line 367, in call  
cmd, result = parser.consume(arglist)  
File "/Users/atakanince/groupsolver\_env/lib/python3.8/site-packages/plac\_core.py", line 232, in consume  
return cmd, self.func(_(args + varargs + extraopts), \*\*kwargs)  
File "/Users/atakanince/groupsolver\_env/lib/python3.8/site-packages/prodigy/recipes/train.py", line 103, in train  
data, labels = merge\_data(nlp, \*\*merge\_cfg)  
File "/Users/atakanince/groupsolver\_env/lib/python3.8/site-packages/prodigy/recipes/train.py", line 402, in merge\_data  
for eg in convert\_options\_to\_cats(textcat\_validated, exclusive=textcat\_exclusive):  
File "cython\_src/prodigy/components/preprocess.pyx", line 353, in prodigy.components.preprocess.convert\_options\_to\_cats  
KeyError: 'label'  
"""

I have no idea what's wrong. Help would be much appreciated.

Best,  
-Atakan

---

<div class="post-metadata">

**Author:** ![SofieVL](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/sofievl/32/915_2.png) [@SofieVL](https://support.prodi.gy/u/SofieVL)\
**Post date:** [April 1, 2021, 9:11pm UTC](https://support.prodi.gy/t/keyerror-label-error-with-prodigy-1-10-7/4097/2 "2021-04-01T21:11:42Z")

</div>

Hm, the data sample you posted seems OK to me.

To double check, I loaded those two examples in a custom db `db_4097` and ran

```python
prodigy train textcat db_4097 blank:en --eval-id db_4097

```

Which ran without issue.

Have you double checked the format of your eval set, are you 100% there's a `label` annotation in each and every example?

Or could you provide a longer data sample (both for train and test) that helps me reproduce the issue?

---

<div class="post-metadata">

**Author:** ![inceatakan](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/inceatakan/32/1741_2.png) [@inceatakan](https://support.prodi.gy/u/inceatakan)\
**Post date:** [April 1, 2021, 9:17pm UTC](https://support.prodi.gy/t/keyerror-label-error-with-prodigy-1-10-7/4097/3 "2021-04-01T21:17:09Z")

</div>

Hi Sofie,

Thank you for your prompt response. Here is the eval dataset:

"""  
{"text":"There products are good.","label":"BAD\_GRAMMAR","\_input\_hash":-1645639406,"\_task\_hash":-1410066889,"\_session\_id":null,"\_view\_id":"classification","answer":"accept"}  
{"text":"There great products are awesome.","label":"BAD\_GRAMMAR","\_input\_hash":-819965774,"\_task\_hash":-1994949002,"\_session\_id":null,"\_view\_id":"classification","answer":"accept"}  
{"text":"There is a shop around the corner.","label":"BAD\_GRAMMAR","\_input\_hash":-471097359,"\_task\_hash":-437723881,"\_session\_id":null,"\_view\_id":"classification","answer":"reject"}  
{"text":"Are there elections soon?","label":"BAD\_GRAMMAR","\_input\_hash":836169608,"\_task\_hash":1490833088,"\_session\_id":null,"\_view\_id":"classification","answer":"reject"}  
{"text":"There product is high quality.","label":"BAD\_GRAMMAR","\_input\_hash":715642260,"\_task\_hash":746544323,"\_session\_id":null,"\_view\_id":"classification","answer":"accept"}  
{"text":"Is there product high quality?","label":"BAD\_GRAMMAR","\_input\_hash":1034366084,"\_task\_hash":1326463427,"\_session\_id":null,"\_view\_id":"classification","answer":"accept"}  
{"text":"There life is better with us.","label":"BAD\_GRAMMAR","\_input\_hash":-34810624,"\_task\_hash":1149767129,"\_session\_id":null,"\_view\_id":"classification","answer":"accept"}  
{"text":"Is there life outside?","label":"BAD\_GRAMMAR","\_input\_hash":855489025,"\_task\_hash":676857513,"\_session\_id":null,"\_view\_id":"classification","answer":"reject"}

"""

---

<div class="post-metadata">

**Author:** ![inceatakan](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/inceatakan/32/1741_2.png) [@inceatakan](https://support.prodi.gy/u/inceatakan)\
**Post date:** [April 1, 2021, 9:18pm UTC](https://support.prodi.gy/t/keyerror-label-error-with-prodigy-1-10-7/4097/4 "2021-04-01T21:18:35Z")

</div>

here is some more data from train dataset:

"""  
{"text":"AOPA & EAA both cost a fraction of the annual dues at NBAA yet they are super interactive with there members and it is easy to know what issues they are working on.","label":"BAD\_GRAMMAR","\_input\_hash":-996205124,"\_task\_hash":-1132416227,"\_session\_id":null,"\_view\_id":"classification","answer":"accept"}  
{"text":"Again freedom of speech everyone has that right to speak there mind","label":"BAD\_GRAMMAR","\_input\_hash":1644417985,"\_task\_hash":-1304798905,"\_session\_id":null,"\_view\_id":"classification","answer":"accept"}  
{"text":"Alota people lost there jobs prices of food going up petrol","label":"BAD\_GRAMMAR","\_input\_hash":36461726,"\_task\_hash":-303123952,"\_session\_id":null,"\_view\_id":"classification","answer":"accept"}  
{"text":"I know you guys making alota money","label":"BAD\_GRAMMAR","\_input\_hash":-2019791434,"\_task\_hash":-227280561,"\_session\_id":null,"\_view\_id":"classification","answer":"reject"}  
{"text":"But help the poor people","label":"BAD\_GRAMMAR","\_input\_hash":-1057632978,"\_task\_hash":-410494013,"\_session\_id":null,"\_view\_id":"classification","answer":"reject"}  
{"text":"Are there precautions in place? Do you sanitized after each patient?","label":"BAD\_GRAMMAR","\_input\_hash":660409566,"\_task\_hash":-1666602451,"\_session\_id":null,"\_view\_id":"classification","answer":"reject"}  
{"text":"Australia is nice country there environment there sense of people","label":"BAD\_GRAMMAR","\_input\_hash":738950169,"\_task\_hash":572933876,"\_session\_id":null,"\_view\_id":"classification","answer":"accept"}  
{"text":"Australia such as beautiful country there many people so kindness","label":"BAD\_GRAMMAR","\_input\_hash":1370149089,"\_task\_hash":1863762462,"\_session\_id":null,"\_view\_id":"classification","answer":"accept"}  
{"text":"Bcz it is good to study in different country u can know traditional and there cultures","label":"BAD\_GRAMMAR","\_input\_hash":-2024846060,"\_task\_hash":2102297053,"\_session\_id":null,"\_view\_id":"classification","answer":"accept"}  
{"text":"And also food is delicious","label":"BAD\_GRAMMAR","\_input\_hash":-458338878,"\_task\_hash":2092736225,"\_session\_id":null,"\_view\_id":"classification","answer":"ignore"}  
{"text":"Because it is a matter of freedon of speech. Like it or not everyone should be allowed to speak there opinions through social media. Thats what it is there for","label":"BAD\_GRAMMAR","\_input\_hash":-1574744707,"\_task\_hash":1059412996,"\_session\_id":null,"\_view\_id":"classification","answer":"accept"}  
{"text":"Because there juice is sweet and really good","label":"BAD\_GRAMMAR","\_input\_hash":-1390572401,"\_task\_hash":1896111745,"\_session\_id":null,"\_view\_id":"classification","answer":"accept"}  
{"text":"Because they lie on anything that they push for there political agenda","label":"BAD\_GRAMMAR","\_input\_hash":1708305497,"\_task\_hash":-354000252,"\_session\_id":null,"\_view\_id":"classification","answer":"accept"}  
{"text":"Been there many times but prefer Tenerife.","label":"BAD\_GRAMMAR","\_input\_hash":132587129,"\_task\_hash":-144354063,"\_session\_id":null,"\_view\_id":"classification","answer":"reject"}  
{"text":"Being atound other people and they don't want to cover there face or stay 6 feet away","label":"BAD\_GRAMMAR","\_input\_hash":1781422388,"\_task\_hash":-27939727,"\_session\_id":null,"\_view\_id":"classification","answer":"accept"}  
{"text":"But can't wait to get a lunch lunch or lunch tomorrow or tomorrow I'll be there tomorrow morning and then I pick up the kids tomorrow or lunch lunch or","label":"BAD\_GRAMMAR","\_input\_hash":1880689730,"\_task\_hash":1450162884,"\_session\_id":null,"\_view\_id":"classification","answer":"reject"}  
{"text":"By training ing there employees to use technology","label":"BAD\_GRAMMAR","\_input\_hash":-1012638018,"\_task\_hash":-401584579,"\_session\_id":null,"\_view\_id":"classification","answer":"accept"}  
{"text":"Do you think there should be a rule, where everyone gets a standing ovation once in there life. You already started read on...","label":"BAD\_GRAMMAR","\_input\_hash":1013251118,"\_task\_hash":-850667332,"\_session\_id":null,"\_view\_id":"classification","answer":"accept"}  
{"text":"It is trying to get people to start reading on Amazon read","label":"BAD\_GRAMMAR","\_input\_hash":-1749264730,"\_task\_hash":1804529721,"\_session\_id":null,"\_view\_id":"classification","answer":"ignore"}  
{"text":"Easy to communicate with there people and so many peaceful university","label":"BAD\_GRAMMAR","\_input\_hash":-798600673,"\_task\_hash":-21034858,"\_session\_id":null,"\_view\_id":"classification","answer":"accept"}  
{"text":"Energy Australia are always trying to ensure there customers know they are trying to make the most affordable and clean energy","label":"BAD\_GRAMMAR","\_input\_hash":1836122933,"\_task\_hash":-589373510,"\_session\_id":null,"\_view\_id":"classification","answer":"accept"}  
{"text":"Everyone In This World Should Get A Standing Innovation. At Least Once In There Life. # Read","label":"BAD\_GRAMMAR","\_input\_hash":264735208,"\_task\_hash":2069375211,"\_session\_id":null,"\_view\_id":"classification","answer":"accept"}  
{"text":"Excitement but the question would be are there real savings?","label":"BAD\_GRAMMAR","\_input\_hash":-444669240,"\_task\_hash":-1184069775,"\_session\_id":null,"\_view\_id":"classification","answer":"reject"}  
{"text":"Fashion nova. There way of making you feel comfortable and fitted to feel pretty is amazing","label":"BAD\_GRAMMAR","\_input\_hash":-1618707928,"\_task\_hash":2120577728,"\_session\_id":null,"\_view\_id":"classification","answer":"accept"}  
{"text":"Graphics as well as the landscape/setting looks appealing. Would want to know roughly the size of the island, are there alot of dungeons to explore, are there guilds to join,","label":"BAD\_GRAMMAR","\_input\_hash":1921931521,"\_task\_hash":934089965,"\_session\_id":null,"\_view\_id":"classification","answer":"reject"}  
{"text":"Hard to find people who see things there way I dio and should read books","label":"BAD\_GRAMMAR","\_input\_hash":495779735,"\_task\_hash":1632352676,"\_session\_id":null,"\_view\_id":"classification","answer":"accept"}  
"""

---

<div class="post-metadata">

**Author:** ![inceatakan](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/inceatakan/32/1741_2.png) [@inceatakan](https://support.prodi.gy/u/inceatakan)\
**Post date:** [April 1, 2021, 9:19pm UTC](https://support.prodi.gy/t/keyerror-label-error-with-prodigy-1-10-7/4097/5 "2021-04-01T21:19:02Z")

</div>

Validation dataset format has label for each line.

---

<div class="post-metadata">

**Author:** ![inceatakan](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/inceatakan/32/1741_2.png) [@inceatakan](https://support.prodi.gy/u/inceatakan)\
**Post date:** [April 1, 2021, 9:23pm UTC](https://support.prodi.gy/t/keyerror-label-error-with-prodigy-1-10-7/4097/6 "2021-04-01T21:23:12Z")

</div>

I double-checked and all lines in the train dataset have label.

---

<div class="post-metadata">

**Author:** ![SofieVL](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/sofievl/32/915_2.png) [@SofieVL](https://support.prodi.gy/u/SofieVL)\
**Post date:** [April 1, 2021, 9:30pm UTC](https://support.prodi.gy/t/keyerror-label-error-with-prodigy-1-10-7/4097/7 "2021-04-01T21:30:26Z")

</div>

Thanks!

This is frustrating, because I can't replicate your issue which makes it really difficult to debug on my end.  
What I've done is, I've taken your two data samples, stored them in a `jsonl` file, read them into Prodigy with `db-in` and then ran

```python
prodigy train textcat train blank:en --eval-id test -TE

```

which gives me

```python
✔ Loaded model 'blank:en'
Created and merged data for 24 total examples
Created and merged data for 8 total examples
Using 24 train / 8 eval (from 'test')
Component: textcat | Batch size: compounding | Dropout: 0.2 | Iterations: 10
ℹ Baseline accuracy: 100.000

=========================== ✨ Training the model ===========================

# Loss F-Score
-- -------- --------
1 7.00 100.000
2 7.00 100.000
...
Label F-Score
----------- -------
BAD_GRAMMAR 100.000
...

```

Could you try the same - running with the limited datasets and a blank English model? Then perhaps change to your custom model and see whether it runs on the sample data? Then change the training set, and only after that change the test set, to see when the error starts occurring?

---

<div class="post-metadata">

**Author:** ![inceatakan](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/inceatakan/32/1741_2.png) [@inceatakan](https://support.prodi.gy/u/inceatakan)\
**Post date:** [April 1, 2021, 10:24pm UTC](https://support.prodi.gy/t/keyerror-label-error-with-prodigy-1-10-7/4097/8 "2021-04-01T22:24:50Z")

</div>

Thank you so much for looking into this Sofie!  
As training data, I am using only 23 examples and the same test dataset I sent to you.  
The following is giving the same error:

"""  
!python -m prodigy train textcat new\_set\_2 blank:en --eval-id test\_dataset -TE  
"""

When I split the train dataset for validation, no error message:

"""  
!python -m prodigy train textcat new\_set\_2 blank:en -es 0.2 -TE  
"""

When I try the custom model with the small dataset as above, it works fine:

"""  
!python -m prodigy train textcat new\_set\_2 ./Desktop/Retraining\_POS\_Tagger/tagger\_model\_3 -es 0.2 -TE  
"""

When I use the full train dataset with blank:en and the custom model, both fail:

"""  
!python -m prodigy train textcat new\_set blank:en -es 0.2 -TE  
"""

"""  
!python -m prodigy train textcat new\_set ./Desktop/Retraining\_POS\_Tagger/tagger\_model\_3 -es 0.2 -TE  
"""

It looks like the problem is with both the train and test dataset files. Both are json. Can I email them to you?

-Atakan

---

<div class="post-metadata">

**Author:** ![SofieVL](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/sofievl/32/915_2.png) [@SofieVL](https://support.prodi.gy/u/SofieVL)\
**Post date:** [April 2, 2021, 7:28am UTC](https://support.prodi.gy/t/keyerror-label-error-with-prodigy-1-10-7/4097/9 "2021-04-02T07:28:43Z")

</div>

Yes, you can email them to sofie **at** explosion.ai, then I can hopefully replicate and help you debug this!

---

<div class="post-metadata">

**Author:** ![inceatakan](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/inceatakan/32/1741_2.png) [@inceatakan](https://support.prodi.gy/u/inceatakan)\
**Post date:** [April 2, 2021, 4:26pm UTC](https://support.prodi.gy/t/keyerror-label-error-with-prodigy-1-10-7/4097/10 "2021-04-02T16:26:46Z")

</div>

I imported the train and test datasets with db-in using new names: new\_set\_2 and test\_dataset\_2 instead of new\_set and test\_dataset, respectively. and it worked:

✔ Loaded model './Desktop/Retraining\_POS\_Tagger/tagger\_model\_3'  
Created and merged data for 149 total examples  
Created and merged data for 8 total examples  
Using 149 train / 8 eval (from 'test\_dataset\_2')  
Component: textcat | Batch size: compounding | Dropout: 0.2 | Iterations: 10  
ℹ Baseline accuracy: 100.000

=========================== ✨ Training the model ===========================

# Loss F-Score

* * *

1 32.00 100.000  
2 32.00 100.000  
3 32.00 100.000  
4 32.00 100.000  
5 18.00 100.000  
6 4.88 100.000  
7 4.00 100.000  
8 4.88 100.000  
9 4.00 100.000  
10 4.00 100.000

============================= ✨ Results summary =============================

Label F-Score

* * *

BAD\_GRAMMAR 100.000

Best F-Score 100.000  
Baseline 100.000

---

<div class="post-metadata">

**Author:** ![SofieVL](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/sofievl/32/915_2.png) [@SofieVL](https://support.prodi.gy/u/SofieVL)\
**Post date:** [April 4, 2021, 6:08pm UTC](https://support.prodi.gy/t/keyerror-label-error-with-prodigy-1-10-7/4097/11 "2021-04-04T18:08:39Z")

</div>

Does that mean your issue is resolved then?  
Perhaps the datasets with the other names contained some older, incorrect examples from a previous experiment?

---

<div class="post-metadata">

**Author:** ![inceatakan](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/inceatakan/32/1741_2.png) [@inceatakan](https://support.prodi.gy/u/inceatakan)\
**Post date:** [April 5, 2021, 3:52pm UTC](https://support.prodi.gy/t/keyerror-label-error-with-prodigy-1-10-7/4097/12 "2021-04-05T15:52:58Z")

</div>

That's what I think too. One more question. In the report I share above, the baseline accuracy is 100, as best f-score is, after training with ~300 examples. When I try the model both on grammatical and ungrammatical sentences, the score is 1. Is that because the model does not have enough data? How much data should I have for training such a model?

Thanks in advance.

-Atakan

---

<div class="post-metadata">

**Author:** ![SofieVL](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/sofievl/32/915_2.png) [@SofieVL](https://support.prodi.gy/u/SofieVL)\
**Post date:** [April 5, 2021, 4:52pm UTC](https://support.prodi.gy/t/keyerror-label-error-with-prodigy-1-10-7/4097/13 "2021-04-05T16:52:11Z")

</div>

I'm not sure I understand what you mean? If the score is 1, that means 100%.  
Also, if you're measuring on just 8 evaluation examples, that might be a bit too few to get a reliable performance score.

---

<div class="post-metadata">

**Author:** ![inceatakan](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/inceatakan/32/1741_2.png) [@inceatakan](https://support.prodi.gy/u/inceatakan)\
**Post date:** [April 5, 2021, 5:19pm UTC](https://support.prodi.gy/t/keyerror-label-error-with-prodigy-1-10-7/4097/14 "2021-04-05T17:19:17Z")

</div>

Oh, sorry for not being clear. When I test the model with a grammatical sentence, I get 1.0 (100% ungrammatical).

grammar\_nlp("Their book is weak").cats  
{'BAD\_GRAMMAR': 1.0}

When I test it with an ungrammatical one, I get the same score:

grammar\_nlp("There book is weak.").cats  
{'BAD\_GRAMMAR': 1.0}

I would expect some score lower than 1.0 for grammatical cases. I was wondering whether I get 1.0 for both grammatical and ungrammatical cases because the model has not learned anything yet and I need to label more data.

Thanks,  
-Atakan

---

<div class="post-metadata">

**Author:** ![SofieVL](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/sofievl/32/915_2.png) [@SofieVL](https://support.prodi.gy/u/SofieVL)\
**Post date:** [April 5, 2021, 7:43pm UTC](https://support.prodi.gy/t/keyerror-label-error-with-prodigy-1-10-7/4097/15 "2021-04-05T19:43:51Z")

</div>

Right, I hadn't looked into the details of your annotation/challenge yet.

So if I understand correctly, you're training a textcat with examples `label="BAD_GRAMMAR"`, and that is the only label you're feeding the classifier. What happens then, is that the classifier will simply learn to predict that all possible input is `BAD_GRAMMAR`, because it hasn't received any counter examples. If it just always predicts bad grammar, the training loss is zero and the ML algorithm is happy, but your classifier will not be very useful.

So, what you'd need to do is make sure that you also include examples that have good grammar. Only then will it become a challenge for the ML algorithm, and will it try to actually learn that difference.

---

<div class="post-metadata">

**Author:** ![inceatakan](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/inceatakan/32/1741_2.png) [@inceatakan](https://support.prodi.gy/u/inceatakan)\
**Post date:** [April 6, 2021, 6:27pm UTC](https://support.prodi.gy/t/keyerror-label-error-with-prodigy-1-10-7/4097/16 "2021-04-06T18:27:53Z")

</div>

Thank you Sofie! Actually, the training data has 239 ungrammatical and 83 grammatical examples. I guess the dataset is not big enough.

-Atakan

---

<div class="post-metadata">

**Author:** ![SofieVL](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/sofievl/32/915_2.png) [@SofieVL](https://support.prodi.gy/u/SofieVL)\
**Post date:** [April 6, 2021, 7:55pm UTC](https://support.prodi.gy/t/keyerror-label-error-with-prodigy-1-10-7/4097/17 "2021-04-06T19:55:37Z")

</div>

Hi Atakan,

I think there's one more issue with your training command: the fact that you're using `-TE` which means the labels are interpreted as "mutually exclusive". This means that exactly 1 true label is expected per instance. This setting really is only applicable when you're training on more than one label.

In the case of training on one label, as in your use-case, this `-TE` setting has a bit of an unexpected consequence. The internal validation will only look at the set of labels that got applied, which is always "BAD\_GRAMMAR" in your case. The evaluation will, artificially, always say this is 100% correct because it saw the right label.

Instead, you need to remove the `-TE` setting and you'll see that you'll get a much more realistic training performance, that should increase as you add more data / train longer.

---

<div class="post-metadata">

**Author:** ![inceatakan](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/inceatakan/32/1741_2.png) [@inceatakan](https://support.prodi.gy/u/inceatakan)\
**Post date:** [April 6, 2021, 8:10pm UTC](https://support.prodi.gy/t/keyerror-label-error-with-prodigy-1-10-7/4097/18 "2021-04-06T20:10:17Z")

</div>

Hi Sofie,

Thank you so much! Now I get realistic results.

-Atakan

---

<div class="post-metadata">

**Author:** ![inceatakan](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/inceatakan/32/1741_2.png) [@inceatakan](https://support.prodi.gy/u/inceatakan)\
**Post date:** [April 12, 2021, 3:40pm UTC](https://support.prodi.gy/t/keyerror-label-error-with-prodigy-1-10-7/4097/19 "2021-04-12T15:40:59Z")

</div>

Sofie,

One more question,

Can I train multiple ungrammatical patterns in the same model?

Let's say pattern1 = 'There book is expensive.'  
pattern2 = 'The book are expensive.'  
pattern3 etc.

Can I label data with the patterns above and train a single model?

-Atakan

---

<div class="post-metadata">

**Author:** ![SofieVL](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/sofievl/32/915_2.png) [@SofieVL](https://support.prodi.gy/u/SofieVL)\
**Post date:** [April 13, 2021, 12:07pm UTC](https://support.prodi.gy/t/keyerror-label-error-with-prodigy-1-10-7/4097/20 "2021-04-13T12:07:54Z")

</div>

Sure, you could try that, if you systematically label your gold data with different labels!

[Next page](https://support.prodi.gy/t/keyerror-label-error-with-prodigy-1-10-7/4097.md?page=2)
