# \--eval-id in textcat.batch-train not working in 1.8

**URL:** <https://support.prodi.gy/t/eval-id-in-textcat-batch-train-not-working-in-1-8/1559>\
**Category:** Uncategorized\
**Tags:** textcat, done\
**Created:** [May 21, 2019, 4:39pm UTC](https://support.prodi.gy/t/eval-id-in-textcat-batch-train-not-working-in-1-8/1559 "2019-05-21T16:39:31Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![thejamesmarq](https://avatars.discourse-cdn.com/v4/letter/t/c5a1d2/32.png) [@thejamesmarq](https://support.prodi.gy/u/thejamesmarq)\
**Post date:** [May 21, 2019, 4:39pm UTC](https://support.prodi.gy/t/eval-id-in-textcat-batch-train-not-working-in-1-8/1559/1 "2019-05-21T16:39:31Z")

</div>

I updated prodigy to 1.8 yesterday and am no longer able to provide a dataset to --eval-id in `textcat.batch-train`. Previously the experiment would run and report out results, but now I have this error:

```python
Traceback (most recent call last):                                                                                                         
  File "/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    " __main__", mod_spec)
  File "/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/james/Personal/Prodigy/prodigy_venv/lib/python3.7/site-packages/prodigy/ __main__.py", line 380, in <module>
    controller = recipe(*args, use_plac=True)
  File "cython_src/prodigy/core.pyx", line 212, in prodigy.core.recipe.recipe_decorator.recipe_proxy
  File "/Users/james/Personal/Prodigy/prodigy_venv/lib/python3.7/site-packages/plac_core.py", line 328, in call
    cmd, result = parser.consume(arglist)
  File "/Users/james/Personal/Prodigy/prodigy_venv/lib/python3.7/site-packages/plac_core.py", line 207, in consume
    return cmd, self.func(*(args + varargs + extraopts), **kwargs)
  File "/Users/james/Personal/Prodigy/prodigy_venv/lib/python3.7/site-packages/prodigy/recipes/textcat.py", line 256, in batch_train
    acc = model.evaluate(tqdm.tqdm(evals, leave=False))
  File "cython_src/prodigy/models/textcat.pyx", line 263, in prodigy.models.textcat.TextClassifier.evaluate
KeyError: 'cats'

```

For evaluation sets I went through a bunch of examples and annotated them with `textcat.mark`. Has there been a change to the data formats I need to fix?

---

<div class="post-metadata">

**Author:** ![honnibal](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/honnibal/32/35_2.png) [@honnibal](https://support.prodi.gy/u/honnibal)\
**Post date:** [May 21, 2019, 4:47pm UTC](https://support.prodi.gy/t/eval-id-in-textcat-batch-train-not-working-in-1-8/1559/2 "2019-05-21T16:47:45Z")

</div>

Damn, thanks. I made a mistake in the recipe there. Will ship an update. In the meantime, you can apply the patch to the recipe, as the source for it is provided. If you edit the file:  
/Users/james/Personal/Prodigy/prodigy\_venv/lib/python3.7/site-packages/prodigy/recipes/textcat.py

You should be able to add the line `evals = convert_options_to_cats(evals)` under line 230. The result should look like:

```python

229 if eval_id:
230 evals = DB.get_dataset(eval_id)
231 evals = convert_options_to_cats(evals)
232 print_("Loaded {} evaluation examples from '{}'".format(len(evals), eval_id))
233 else:

```

---

<div class="post-metadata">

**Author:** ![thejamesmarq](https://avatars.discourse-cdn.com/v4/letter/t/c5a1d2/32.png) [@thejamesmarq](https://support.prodi.gy/u/thejamesmarq)\
**Post date:** [May 21, 2019, 5:12pm UTC](https://support.prodi.gy/t/eval-id-in-textcat-batch-train-not-working-in-1-8/1559/3 "2019-05-21T17:12:08Z")

</div>

Thanks! Made the change and works fine now.