Is there support for Image/text side-by-side preference annotation? This is useful for training a preference ranking model used for RLHF.
Hi @wjhtomwjh ,
For text preference you could use the compare recipe which can take care of randomizing the questions to prevent the bias.
For images you could definitely achieve that with a custom recipe. I believe this thread can help you to get started: Two images at the same time
The simplest option would probably be to combine a custom HTML view as discussed in the cited thread with a choice view for selecting the preferred option via blocks.
If you want to get a smoother experience of , for example, accepting via clicking on the image you should be able to achieve that by defining a custom js event