textcat.teach with multiple choice interface?

I am annotating a corpus with 5 labels, bought, sold, buyer, seller, owns. Documents can have more than 1 label (sold and owns for example).

When I use textcat.teach with exclusive left set to false I am getting a pretty unbalanced set of annotations. Is it possible to use textcat.teach with the multiple choice UI used by textcat.manual so that can label more than one category when they appear. For example when something like "John sold a camera, and owns wide angle lens", I want to annotate with sold and own, but textcat.teach only asks me sold applies. I suppose at a latter stage textcat.teach might ask me if the own label applies to the doc, but as I said I am getting very unbalanced annotations. Not many accepted 'own' labels.


Hi! We don't currently have a built-in workflow for this, but you should be able to implement it with a custom recipe or by slightly modifying the textcat.teach recipe. Internally, Prodigy's annotation models for text classification should now be able to handle both the "text" + "label", as well as the "options" + "accept" format. So before you send out the examples for annotation, you can add the "options" (all labels), pre-select the suggested label ("label") and then use the choice interface in the app.

We haven't tried out this exact workflow and it can sometimes make the active learning a little trickier if you're updating all labels at the same time. But if you try it out, I'd definitely be curious to hear how well it works :slightly_smiling_face:

Thanks will hack around and report back.

1 Like

@Superscope did it work? do you mind sharing the recipe?

1 Like

@Superscope Hi, this is exactly what I need. Did you find any solution to this?

hi @JanP and @gladiator , I gave on this. TBH I find annotating one label at a time more productive. I then export the labels, and merge them outside prodigy, with the new dataset reflecting any overlapping labels. Apologies for not reporting back earlier.

1 Like