# Additional metrics (recall, precision, accuracy F1) in textcat.train-curve

**URL:** <https://support.prodi.gy/t/additional-metrics-recall-precision-accuracy-f1-in-textcat-train-curve/2046>\
**Category:** Uncategorized\
**Tags:** enhancement, textcat\
**Created:** [September 26, 2019, 6:08pm UTC](https://support.prodi.gy/t/additional-metrics-recall-precision-accuracy-f1-in-textcat-train-curve/2046 "2019-09-26T18:08:28Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![cbjrobertson](https://avatars.discourse-cdn.com/v4/letter/c/87869e/32.png) [@cbjrobertson](https://support.prodi.gy/u/cbjrobertson)\
**Post date:** [September 26, 2019, 6:08pm UTC](https://support.prodi.gy/t/additional-metrics-recall-precision-accuracy-f1-in-textcat-train-curve/2046/1 "2019-09-26T18:08:28Z")

</div>

Hi--

I was wondering whether it is possible at all to include (and do you have plans to include) additional performance metrics in the output of `textcat.train-curve`. Accuracy is not always the most useful when dealing with unbalanced classes (as I am). Are additional metrics in the pipeline for `textcat.train-curve`, and do you suggest any workarounds in the meantime? (I guess apart from manually splitting up the data in various sizes and then running `textcat.batch-train` on them).

Cheers!

---

<div class="post-metadata">

**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\
**Post date:** [September 27, 2019, 9:18am UTC](https://support.prodi.gy/t/additional-metrics-recall-precision-accuracy-f1-in-textcat-train-curve/2046/2 "2019-09-27T09:18:25Z")

</div>

That's a nice idea! The `textcat.train-curve` recipe currently uses the number returned by the `textcat.batch-train` recipe function. If you take a look at the code, you'll see that this is `best_acc["accuracy"]`. The full stats returned by `model.evaluate` are the following:

```python
stats = {
    "tp": tp,
    "fp": fp,
    "fn": fn,
    "tn": tn,
    "avg_score": total_score / total,
    "precision": precision,
    "recall": recall,
    "fscore": 2 * ((precision * recall) / (precision + recall + 1e-8)),
    "loss": loss / (len(examples) + 1e-8),
    "accuracy": (tp + tn) / (tp + tn + fp + fn + 1e-8),
    "baseline": baseline,
}

```

So if you want the train curve recipe to compare the recall instead, the easiest way would be to change the `batch-train` recipe in `recipes/textcat.py` and make it return `best_acc["recall"]`.

Btw, you can run the following to find the location of your Prodigy installation:

```bash
python -c "import prodigy; print(prodigy. __file__ )"

```

---

<div class="post-metadata">

**Author:** ![sofiejb](https://avatars.discourse-cdn.com/v4/letter/s/57b2e6/32.png) [@sofiejb](https://support.prodi.gy/u/sofiejb)\
**Post date:** [January 18, 2023, 8:30pm UTC](https://support.prodi.gy/t/additional-metrics-recall-precision-accuracy-f1-in-textcat-train-curve/2046/3 "2023-01-18T20:30:34Z")

</div>

Hi, since I cannot find the batch-train recipe in texcat.py now. I assume we use handle\_scores\_per\_type to evaluate now? Where would you recommend making these changes now?

---

<div class="post-metadata">

**Author:** ![ryanwesslen](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ryanwesslen/32/2969_2.png) [@ryanwesslen](https://support.prodi.gy/u/ryanwesslen)\
**Post date:** [January 18, 2023, 9:20pm UTC](https://support.prodi.gy/t/additional-metrics-recall-precision-accuracy-f1-in-textcat-train-curve/2046/4 "2023-01-18T21:20:18Z")

</div>

> [@sofiejb](#):
>
> Hi, since I cannot find the batch-train recipe in texcat.py now. I assume we use handle\_scores\_per\_type to evaluate now? Where would you recommend making these changes now?

Yes, `batch-train` was replaced with `train` in [v1.9 (Dec 2019)](https://prodi.gy/docs/changelog#v1.9.0).

Does this answer your question?

> [@\`train-curve textcat\` - display AUC for each classification label](https://support.prodi.gy/t/train-curve-textcat-display-auc-for-each-classification-label/3036/2):
>
> Hi! The new train recipe function returns a (best\_scores, baseline) tuple – best\_scores is an instance of spaCy's [Scorer](https://spacy.io/api/scorer#properties), which includes overall accuracy scores, as well as scores per label. The train-curve mostly just runs train with different portions of the data and then outputs the best score for each training run at the end. So you could write your own version of the recipe that prints the best\_scores.textcats\_per\_cat instead of using the default results printer. (You can find the recipe i…