Conditional behavior based on selections in choice interface

freepskov · September 11, 2024, 3:41pm

I have two problems where I want other parts of the UI to react to specific selections made in a multiple choice dialogue.

I have a blocks interface with a choice interface and a Notes text field. I want the Notes text field to be mandatory unless a specific selection is made in the choice dialogue, in which case the Notes field can be optionally left empty. That is to say, if the user selects option A, B, or C, it should be impossible to accept the input unless the Notes field has text in it. If the user selects option D, the Notes field can be left empty if desired.
Some questions can become irrelevant depending on selections made on previous questions about the same query. For example, if No is selected in the Clinical Harmfulness choice dialogue, the Clinical Harmfulness Level question should be skipped.

Is it possible to achieve these effects in Prodigy?

magdaaniol · September 17, 2024, 2:46pm

Hi @freepskov,

Yes, both are definitely possible with a bit of custom code.

I want the Notes text field to be mandatory unless a specific selection is made in the choice dialogue, in which case the Notes field can be optionally left empty.

You can use Prodigy validate_answer callback to implement this condition. This callback has access to the latest annotated example so you can perform any custom checks on any field. In case, the validation doesn't pass, you can define an error message that will be displayed as a browser pop up.

Some questions can become irrelevant depending on selections made on previous questions about the same query.

This one is a bit trickier as Prodigy has been designed to model the annotation as a stream of independent tasks. The easiest way to fulfill your requirement would be to conditionally display the follow up questions on the same annotation card via javascript logic. Here you can find a similar solution. Let me know if you need help adapting it to your use case!

freepskov · September 17, 2024, 3:55pm

I think our use case is a bit trickier in some ways. The example I gave for my second question is solved by your suggestion, but there is another example that would probably require slightly more involved code than what is presented in your example:

There is anther question, Comprehension. If the answer to the Comprehension question is No, the Completeness and Correctness questions are both moot. Modifying the custom JavaScript too much is going to be a bit tricky for us as none of us are fluent in JavaScript.

One idea I had, but never fleshed out regarding this problem was as follows: I wondered if it might be possible to abuse the task routing functionality to read the answers provided to previous questions and simply not route the moot questions to anyone, or to route them to a dummy bot annotator that doesn't actually do anything.

freepskov · September 17, 2024, 5:00pm

@magdaaniol My guess is that my idea doesn't work because the Task Router is probably sequential and waiting to decide whether to assign a task based on the previous answer would result in other annotators not having tasks. So I'd appreciate it if you could help with creating JavaScript for something that works as follows:

Part 1:
Comprehension choice interface with Notes text field
Correctness choice interface with corresponding notes text field then only occurs if Comprehension selection is Yes.
Completeness choice interface with corresponding notes text field then only occurs if Comprehension selection is Yes.

Part 2:
Clinical Harmfulness choice interface with no other field
Clinical Harmfulness Level choice interface with Notes text field that only occurs if Clinical Harmfulness selection is Yes

magdaaniol · September 18, 2024, 1:38pm

Hi @freepskov ,

Modifying the stream based on the previous answer in multi-annotator scenarios is not really something that Prodigy has been designed it for. Your instinct to go with the JavaScript solution is definitely what I would recommend. After all, you need to reshape the annotation task specifically to each annotator's behavior.

I assume that part_1 and part_2 questions is something you know upfront and we can use this information to modularize both the definition of custom blocks and javascript functions that handle them.
The following solution assumes that the input examples contain the following information in the meta field:

{"text": "She's a member of AAA", "meta": {"ui_block": "part_1"}}
{"text": "She's a member of BBB", "meta": {"ui_block": "part_2"}}

In your custom recipe, you can now define two block view_ids, one for part_1 type of tasks and one for part_2 type of tasks:

part_1_blocks = [
                {"view_id": "text"},
                {"view_id": "html", "html": "<h3>Comprehension</h3>"},
                {"view_id": "choice", "options":[{"id": "compr_1", "text": "Comprehension Option 1"}, {"id":"compr_2", "text": "Comprehension Option 2"}],"text": None, "html": None},
                {"view_id": "text_input", "field_id": "comprehension_user_input", "field_placeholder": "Type here...", "field_autofocus": False},
                {"view_id": "html", "html": "<h3>Correctness</h3>"},
                {"view_id": "choice","options":[{"id": "corr_1", "text": "Correctness Option 1"}, {"id":"corr_2", "text": "Correctness Option 2"}],"text": None, "html": None},
                {"view_id": "text_input", "field_id": "correctness_user_input", "field_placeholder": "Type here...", "field_autofocus": False},
                {"view_id": "html", "html": "<h3>Completeness</h3>"},
                {"view_id": "choice", "options":[{"id": "compl_1", "text": "Completeness Option 1"}, {"id":"compl_2", "text": "Completeness Option 2"}],"text": None, "html": None},
                {"view_id": "text_input", "field_id": "completeness_user_input", "field_placeholder": "Type here...", "field_autofocus": False},
            ]

part_2_blocks = [
                {"view_id": "text"},
                {"view_id": "html", "html": "<h3>Clinical Harmfulness</h3>"},
                {"view_id": "choice", "options":[{"id": "harm_1", "text": "Clinical Harmfulness Option 1"}, {"id":"harm_2", "text": "Clinical Harmfulness Option 2"}],"text": None, "html": None},
                {"view_id": "html", "html": "<h3>Clinical Harmfulness Level</h3>"},
                {"view_id": "choice","options":[{"id": "lev_1", "text": "Clinical Harmfulness Level Option 1"}, {"id":"lev_2", "text": "Clinical Harmfulness Level Option 2"}],"text": None, "html": None},
                {"view_id": "text_input", "field_id": "clinical_harmfulness_level_user_input", "field_placeholder": "Type here...", "field_autofocus": False},
            ]

You can assign which blocks definition to use based on the information in the meta field of the input task:

def assign_blocks(stream: StreamType) -> StreamType:
    for eg in stream:
        eg_copy = copy.deepcopy(eg)
        eg_copy["config"] = {}
        if eg.get("meta").get("ui_block") == "part_1":
            eg_copy["config"]["blocks"] = part_1_blocks
        elif eg.get("meta").get("ui_block") == "part_2":
            eg_copy["config"]["blocks"] = part_2_blocks
        else:
            raise RecipeError(f"Input with input hash {eg.get(INPUT_HASH_ATTR)} does not contain blocks defnition")
        yield eg_copy

stream = get_stream(source)
stream.apply(assign_blocks, stream=stream)

The only missing bit is custom javascript for toggling the visibility of these different blocks based on the updates to the UI.
Here's one way to achieve that:

// Utility function to toggle element visibility
const toggleVisibility = (element, isVisible) => {
  if (element) {
    element.style.display = isVisible ? "block" : "none";
  }
};

// Function to get elements by their selectors
const getElements = (selectors) => {
  return selectors.map(selector => document.querySelector(selector));
};

// Function to update block visibility
const updateBlockVisibility = (blockElements, showCondition) => {
  blockElements.forEach(element => toggleVisibility(element, showCondition));
};

// Function to update part 1 block
function updatePart1Block(allAccept) {
  const [correctnessHTML, correctnessOptions, correctnessInputField, 
         completenessHTML, completenessOptions, completenessInputField] = getElements([
    'div.prodigy-content:nth-child(7)',
    'div._a7-root-0-1-190:nth-child(9)',
    'div.prodigy-content:nth-child(10)',
    'div.prodigy-content:nth-child(11)',
    'div._a7-root-0-1-190:nth-child(13)',
    'div.prodigy-content:nth-child(14)'
  ]);


  const startsWithCompr = Array.isArray(allAccept) && allAccept.some(item => item.startsWith('compr'));
  const startsWithCorr = Array.isArray(allAccept) && allAccept.some(item => item.startsWith('corr'));


  updateBlockVisibility([correctnessHTML, correctnessOptions, correctnessInputField], startsWithCompr);
  updateBlockVisibility([completenessHTML, completenessOptions, completenessInputField], startsWithCorr);
}

// Function to update part 2 block
function updatePart2Block(allAccept) {
  const [harmfulnessLevelHTML, harmfulnessLevelOptions, harmfulnessLevelInputField] = getElements([
    'div.prodigy-content:nth-child(6)',
    'div._a7-root-0-1-190:nth-child(8)',
    'div.prodigy-content:nth-child(9)'
  ]);

  const startsWithHarm = Array.isArray(allAccept) && allAccept.some(item => item.startsWith('harm'));

  updateBlockVisibility([harmfulnessLevelHTML, harmfulnessLevelOptions, harmfulnessLevelInputField], startsWithHarm);
}

// Main function to update form structure
function updateFormStructure(allAccept = []) {
  const ui_block = window.prodigy.content.meta.ui_block;
  
  const updateFunctions = {
    part_1: updatePart1Block,
    part_2: updatePart2Block
  };

  const updateFunction = updateFunctions[ui_block];
  if (updateFunction) {
    updateFunction(allAccept);
  } else {
    throw new Error("Unknown UI block.");
  }
}

// Event listeners
document.addEventListener("prodigymount", () => updateFormStructure());
document.addEventListener("prodigyupdate", (event) => updateFormStructure(event.detail.task.accept));

This script listens to two Prodigy events: prodigymount and prodigyupdate (docs). Whenever any of this happens the updateFormStructure function is triggered that calls specific subfunctions depending whether we are dealing with part_1 or part_2 task.
The logic of both subfunctions is essentially the same:

Get reference to all blocks of interest for this particular type of task using CSS selectors
Check whether the condition to toggle visibility applies (in this case the Correctness block appears if any of the Comprehension options was checked - you will have to adjust it to your particular "yes" condition - I wasn't sure if it was supposed to be one of the choice options)
Toggle the visibility by modifying the display attribute of the block

I added comments to the script but let me know if you'd like some extra explanations!
This should result in the following UI:
conditional_blocks

Just wanted to reiterate that using using multiple choice blocks is actually a workaround and the only way it can work is by setting choice_style to multiple. Otherwise there will be only one answer permitted across all choice blocks.
Also, the choice of options ids is crucial as all accepted options will be stored under one accept key in the annotated example:

Here's the full minimal custom recipe for the reference:

import copy
from pathlib import Path

import prodigy
from prodigy.components.stream import get_stream
from prodigy.errors import RecipeError
from prodigy.types import StreamType
from prodigy.util import INPUT_HASH_ATTR

part_1_blocks = [
    {"view_id": "text"},
    {"view_id": "html", "html": "<h3>Comprehension</h3>"},
    {
        "view_id": "choice",
        "options": [
            {"id": "compr_1", "text": "Comprehension Option 1"},
            {"id": "compr_2", "text": "Comprehension Option 2"},
        ],
        "text": None,
        "html": None,
    },
    {
        "view_id": "text_input",
        "field_id": "comprehension_user_input",
        "field_placeholder": "Type here...",
        "field_autofocus": False,
    },
    {"view_id": "html", "html": "<h3>Correctness</h3>"},
    {
        "view_id": "choice",
        "options": [
            {"id": "corr_1", "text": "Correctness Option 1"},
            {"id": "corr_2", "text": "Correctness Option 2"},
        ],
        "text": None,
        "html": None,
    },
    {
        "view_id": "text_input",
        "field_id": "correctness_user_input",
        "field_placeholder": "Type here...",
        "field_autofocus": False,
    },
    {"view_id": "html", "html": "<h3>Completeness</h3>"},
    {
        "view_id": "choice",
        "options": [
            {"id": "compl_1", "text": "Completeness Option 1"},
            {"id": "compl_2", "text": "Completeness Option 2"},
        ],
        "text": None,
        "html": None,
    },
    {
        "view_id": "text_input",
        "field_id": "completeness_user_input",
        "field_placeholder": "Type here...",
        "field_autofocus": False,
    },
]

part_2_blocks = [
    {"view_id": "text"},
    {"view_id": "html", "html": "<h3>Clinical Harmfulness</h3>"},
    {
        "view_id": "choice",
        "options": [
            {"id": "harm_1", "text": "Clinical Harmfulness Option 1"},
            {"id": "harm_2", "text": "Clinical Harmfulness Option 2"},
        ],
        "text": None,
        "html": None,
    },
    {"view_id": "html", "html": "<h3>Clinical Harmfulness Level</h3>"},
    {
        "view_id": "choice",
        "options": [
            {"id": "lev_1", "text": "Clinical Harmfulness Level Option 1"},
            {"id": "lev_2", "text": "Clinical Harmfulness Level Option 2"},
        ],
        "text": None,
        "html": None,
    },
    {
        "view_id": "text_input",
        "field_id": "clinical_harmfulness_level_user_input",
        "field_placeholder": "Type here...",
        "field_autofocus": False,
    },
]


def assign_blocks(stream: StreamType) -> StreamType:
    for eg in stream:
        eg_copy = copy.deepcopy(eg)
        eg_copy["config"] = {}
        if eg.get("meta").get("ui_block") == "part_1":
            eg_copy["config"]["blocks"] = part_1_blocks
        elif eg.get("meta").get("ui_block") == "part_2":
            eg_copy["config"]["blocks"] = part_2_blocks
        else:
            raise RecipeError(
                f"Input with input hash {eg.get(INPUT_HASH_ATTR)} does not contain blocks defnition"
            )
        yield eg_copy


@prodigy.recipe(
    "custom-ui",
    dataset=("The dataset to use", "positional", None, str),
    source=("The source data as a JSONL file", "positional", None, str),
)
def custom_ui(
    dataset: str,
    source: str,
):
    stream = get_stream(source)
    stream.apply(assign_blocks, stream=stream)

    custom_js = Path("custom.js").read_text()

    return {
        "view_id": "blocks",
        "dataset": dataset,  # Name of dataset to save annotations
        "stream": stream,  # Incoming stream of examples
        "config": {
            "blocks": part_2_blocks,
            "choice_style": "multiple",
            "javascript": custom_js,
        },
    }

Finally, you need to decide what should happen if the user unchecks the box. For example if I check the Comprehension block and Completeness block, but then uncheck the Comprehension block should my answer to Completeness be deleted? Currently, we are only toggling visibility but only the explicitly checked and unchecked options are stored in the DB. Let me know if you need help with adding this modification.

Topic		Replies	Views
Asking multiple questions in one task (different input types) usage , custom , front-end , solved	12	5848	November 1, 2021
Compare recipe: prevent accepting blank annotation usage	1	306	September 3, 2021
Answers are missing for view_id='choice' custom	8	727	October 29, 2019
Multiple choice with free text input	4	156	April 25, 2024
Using choice interface with pre-set options usage , front-end , solved	1	829	April 24, 2018

Conditional behavior based on selections in choice interface

Related Topics