I want my annotators to generate an evaluation dataset but each label can have multiple options, eg
Is there a way to create a dropdown box against each of the labels?
Hi! You could probably build a dropdown like this with a custom UI block and some JavaScript (let user select from dropdown 1, then use the value to display dropdown 2, then write the selected value of dropdown 2 to the task).
However, we typically recommend making the label selection more complicated for large or nested label sets and rather break up the task so annotators don't have to remember or think about the whole label scheme. This section on using large or hierarchical label schemes for text classification explains the idea in more detail: https://prodi.gy/docs/text-classification#large-label-sets
For your use case, you could for instance start by just labelling mentions of A, B, C, D etc. This also lets you validate the basic label scheme (and resolve problems, if necessary), before you spend more time going into the fine-grained details. For each label, you can then do another pass over the data and label the present vs. not present distinction. You could even set up this part as a very fast binary decision (and use the "ignore" action for uncertainty).