Best labeling strategy for hierarchical concepts

[ines (Ines Montani)]

There was a similar question the other day and I wrote up some ideas for a workflow that starts with the top-level categories and moves on to the sub categories, using a custom recipe that creates multiple choice tasks automatically:

Does Prodigy supports hierarchical annotation?

Prodigy itself is pretty agnostic to what your annotations “mean” so you can definitely build a workflow like this. The only thing that’s kinda built-in is a strong focus on single decisions at a time and automating as much as possible. One approach could be to use the choice interface and start by annotating the top-level buckets like A, B and C, without worrying about the lower-level categories. See here for an example recipe code. In the next step, you can then stream in the examples again a…

But as Justin mentioned, you might find that structuring it as a binary task is actually much better and faster, especially for the top-level decisions.