# Hierarchical text classification - multiple passes on same dataset

**URL:** <https://support.prodi.gy/t/hierarchical-text-classification-multiple-passes-on-same-dataset/3297>\
**Category:** Uncategorized\
**Tags:** textcat, database, solved\
**Created:** [August 13, 2020, 3:32pm UTC](https://support.prodi.gy/t/hierarchical-text-classification-multiple-passes-on-same-dataset/3297 "2020-08-13T15:32:52Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![oneextrafact](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/oneextrafact/32/588_2.png) [@oneextrafact](https://support.prodi.gy/u/oneextrafact)\
**Post date:** [August 13, 2020, 3:32pm UTC](https://support.prodi.gy/t/hierarchical-text-classification-multiple-passes-on-same-dataset/3297/1 "2020-08-13T15:32:52Z")

</div>

Hi - I'm working with a hierarchical text classification problem. For the children of each hierarchy, it's possible that there could be multiple overlapping labels. If I break this into a set of one-vs-all problems, and annotate the dataset separately for each, I would have N datasets. In order to train a model for prediction, would I have to create a gold dataset that combines all of these sets? Is there a way I can do that with prodigy so that, if a text has multiple labels, they are combined? Or would I have to review the process manually?

---

<div class="post-metadata">

**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\
**Post date:** [August 14, 2020, 11:58am UTC](https://support.prodi.gy/t/hierarchical-text-classification-multiple-passes-on-same-dataset/3297/2 "2020-08-14T11:58:37Z")

</div>

> [@oneextrafact](#):
>
> Is there a way I can do that with prodigy so that, if a text has multiple labels, they are combined? Or would I have to review the process manually?

Yes, Prodigy will do this automatically if you pass multiple datasets to the built-in `train` or `data-to-spacy` recipes 😃

Alternatively, it should also be pretty easy to combine your annotations yourself if you want to do it in a separate process: each annotation will have two hashes, an `_input_hash` and a `_task_hash` (you can read more about the hashing [here](https://prodi.gy/docs/api-loaders#hashing)). The input hash lets you identify all examples with the _same input_ – so in this case, all annotations with the same text. So examples with the same input hashes will be annotations on the same text, but with different labels.