# Span Categorizer - Labels Prediction

**URL:** <https://support.prodi.gy/t/span-categorizer-labels-prediction/4965>\
**Category:** Uncategorized\
**Tags:** usage, training, spancat\
**Created:** [November 17, 2021, 11:17am UTC](https://support.prodi.gy/t/span-categorizer-labels-prediction/4965 "2021-11-17T11:17:33Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![hmousa961](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/hmousa961/32/2181_2.png) [@hmousa961](https://support.prodi.gy/u/hmousa961)\
**Post date:** [November 17, 2021, 11:17am UTC](https://support.prodi.gy/t/span-categorizer-labels-prediction/4965/1 "2021-11-17T11:17:33Z")

</div>

Hello,  
I'm having this issue with the spans categorizer. So I'm annotating data with 10 labels and training a model using it. Now whenever I train the model, 4 of these labels always has 0 accuracy. I thought I need to add more data on these labels, but even though I did that nothing changes. I've attached a screenshot of the results.

![Capture](https://us1.discourse-cdn.com/flex020/uploads/prodigy/original/2X/a/a596a8d86cc8e66a018b464f3d6458eb27754cbc.jpeg)

Thanks a lot for your help.

Prodigy model: 1.11.5

---

<div class="post-metadata">

**Author:** ![ljvmiranda921](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ljvmiranda921/32/3197_2.png) [@ljvmiranda921](https://support.prodi.gy/u/ljvmiranda921)\
**Post date:** [November 18, 2021, 1:15am UTC](https://support.prodi.gy/t/span-categorizer-labels-prediction/4965/2 "2021-11-18T01:15:12Z")

</div>

Hi @hmousa961 ,

To sanity-check, do you have training and evaluation data included for these four labels? If you don't have that many examples and use a random split, you'd end up with few examples to train your model and "detect" those other labels. As an illustration, if you only have one example and the model gets it wrong, you immediately get 0 accuracy.

To help you debug, you can first check for imbalanced data. Try running:

```python
python -m spacy debug data path/to/file

```

and assess if your data distributions are roughly the same.  
C.f. [https://spacy.io/api/cli#debug-data](https://spacy.io/api/cli#debug-data)

---

<div class="post-metadata">

**Author:** ![hmousa961](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/hmousa961/32/2181_2.png) [@hmousa961](https://support.prodi.gy/u/hmousa961)\
**Post date:** [November 18, 2021, 7:19am UTC](https://support.prodi.gy/t/span-categorizer-labels-prediction/4965/3 "2021-11-18T07:19:49Z")

</div>

Thank you for yor help. I'll check it and let you know if it works.

---

<div class="post-metadata">

**Author:** ![hmousa961](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/hmousa961/32/2181_2.png) [@hmousa961](https://support.prodi.gy/u/hmousa961)\
**Post date:** [November 18, 2021, 7:21am UTC](https://support.prodi.gy/t/span-categorizer-labels-prediction/4965/4 "2021-11-18T07:21:36Z")

</div>

> [@ljvmiranda921](#):
>
> To sanity-check, do you have training and evaluation data included for these four labels?

Yes I do. Its not as much data as for the other labels. But I've been annotating data for these labels. Is there a data threshold for the model to start predicting?

---

<div class="post-metadata">

**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\
**Post date:** [November 18, 2021, 9:51am UTC](https://support.prodi.gy/t/span-categorizer-labels-prediction/4965/5 "2021-11-18T09:51:57Z")

</div>

How many instances of these labels do you have in your evaluation data in total? The thing is, if you only have 2 examples and the model gets both wrong, that's an accuracy of 0. If you have 10 examples and it gets 2 wrong, that's 80%. So depending on the number of examples you have, the evaluation can be quite brittle and less representative. Similarly, if you only have very few examples to learn from, the model is more likely to struggle learning a given label.

---

<div class="post-metadata">

**Author:** ![hmousa961](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/hmousa961/32/2181_2.png) [@hmousa961](https://support.prodi.gy/u/hmousa961)\
**Post date:** [November 18, 2021, 10:23am UTC](https://support.prodi.gy/t/span-categorizer-labels-prediction/4965/6 "2021-11-18T10:23:03Z")

</div>

Okays now I understood the way it works. Thank you so much for the help.