# Best labeling strategy for hierarchical concepts

**URL:** <https://support.prodi.gy/t/best-labeling-strategy-for-hierarchical-concepts/1262>\
**Category:** Uncategorized\
**Tags:** usage, textcat\
**Created:** [March 1, 2019, 3:48pm UTC](https://support.prodi.gy/t/best-labeling-strategy-for-hierarchical-concepts/1262 "2019-03-01T15:48:26Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![rickdeshon](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/rickdeshon/32/679_2.png) [@rickdeshon](https://support.prodi.gy/u/rickdeshon)\
**Post date:** [March 1, 2019, 3:48pm UTC](https://support.prodi.gy/t/best-labeling-strategy-for-hierarchical-concepts/1262/1 "2019-03-01T15:48:26Z")

</div>

Hello.

Imagine you have a hierarchically structured concept space and you want to label text with respect to this space. To be concrete, imagine 4 top level concepts each having 5 sub-concepts for a total of 24 possible labels.

What would be the best approach for labeling text for this concept space?

For instance, would it be more effective to first label text using the 4 (multi)labels at the top of the hierarchy? Or would it be a better labeling practice to perform 4 separate labeling processes each using a binary strategy?

In general, would it be better to perform 24 binary labeling tasks or 5 multiple labeling tasks?

Thanks for your thoughts!

Rick

---

<div class="post-metadata">

**Author:** ![justindujardin](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/justindujardin/32/169_2.png) [@justindujardin](https://support.prodi.gy/u/justindujardin)\
**Post date:** [March 1, 2019, 4:30pm UTC](https://support.prodi.gy/t/best-labeling-strategy-for-hierarchical-concepts/1262/2 "2019-03-01T16:30:39Z")

</div>

> [@rickdeshon](#):
>
> In general, would it be better to perform 24 binary labeling tasks or 5 multiple labeling tasks?

I would start with binary labels if possible since they're much easier to annotate quickly and accurately. With binary you have to decide "does this match the label, yes/no?" instead of "which of (n) groups does this belong to" This is especially helpful if your concepts might have some overlap or ambiguity that might make it hard to choose one label over the other.

> [@rickdeshon](#):
>
> Imagine you have a hierarchically structured concept space and you want to label text with respect to this space. To be concrete, imagine 4 top level concepts each having 5 sub-concepts for a total of 24 possible labels.

Your example isn't super clear to me, perhaps you can give a more concrete one if you want further thoughts?

---

<div class="post-metadata">

**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\
**Post date:** [March 1, 2019, 6:12pm UTC](https://support.prodi.gy/t/best-labeling-strategy-for-hierarchical-concepts/1262/3 "2019-03-01T18:12:36Z")

</div>

There was a similar question the other day and I wrote up some ideas for a workflow that starts with the top-level categories and moves on to the sub categories, using a custom recipe that creates multiple choice tasks automatically:

> [@Does Prodigy supports hierarchical annotation?](https://support.prodi.gy/t/does-prodigy-supports-hierarchical-annotation/1249/2):
>
> Prodigy itself is pretty agnostic to what your annotations “mean” so you can definitely build a workflow like this. The only thing that’s kinda built-in is a strong focus on single decisions at a time and automating as much as possible. One approach could be to use the [choice interface](https://prodi.gy/demo?view_id=textchoice) and start by annotating the top-level buckets like A, B and C, without worrying about the lower-level categories. [See here](https://prodi.gy/docs/workflow-custom-recipes#example-choice) for an example recipe code. In the next step, you can then stream in the examples again a…

But as Justin mentioned, you might find that structuring it as a binary task is actually much better and faster, especially for the top-level decisions.

---

<div class="post-metadata">

**Author:** ![rickdeshon](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/rickdeshon/32/679_2.png) [@rickdeshon](https://support.prodi.gy/u/rickdeshon)\
**Post date:** [March 1, 2019, 6:43pm UTC](https://support.prodi.gy/t/best-labeling-strategy-for-hierarchical-concepts/1262/4 "2019-03-01T18:43:10Z")

</div>

Thank you both. Very helpful ideas. If I need more, I’ll provide a concrete example.

Best!

Rick