Assuming that I am using more than one labels, can I use a single multi-class classifier to learn to predict these labels and use it to annotate unseen text (inference step), in which case all labels will be assigned in one step to the unseen text?
Sure, that's no problem – how you present the data in the UI is up to you. For example, if you're using the
choice interface with multiple options, you can stream in data that pre-select all options your classifier predicted above a given threshold (e.g.
0.5 or whatever works best). So if your classifier predicted labels with the IDS
B, you can add
"accept": ["A", "B"] to pre-select those options.
You might also find the docs on text classification and custom recipes helpful: