Custom alert when annotations condition is met

Hi @ines, hope I everything is doing well. I was wondering if I can set an UI alert when the number of annotations of each given dataset_id reaches a set amount. I have been looking at the custom interfaces, and I'm a bit confused on how I can apply a Custom JavaScript to my custom recipe.

Here is my custom recipe. I will have this also linked to the AFK feature to save time stamp once this alert also pops up.

def condition(stats):
    for user in stats:
        if user[1] == n_examples:
            # custom javascript alert : "You have reached set number of annotations, please exit"


def add_label_options_to_stream(stream, labels,annotations):
    options = [{"id": label, "text": label} for label in labels]
    for task in stream:
        task["options"] = options
        yield task

def add_labels_to_stream(stream, labels):
    for task in stream:
        task["label"] = label[0]
        yield task

@prodigy.recipe(
    "title_classification",
    dataset=("The dataset to use", "positional", None, str),
    source=("The source data as a JSONL file", "positional", None, str),
    label=("One or more comma-separated labels", "option", "l", split_string),
    n_examples = ("Number of examples to randomly review, -1 for all", "option", "n", int),
    exclusive=("Treat classes as mutually exclusive", "flag", "E", bool),
    exclude=("Names of datasets to exclude", "option", "e", split_string),
)

def title_classification(
    dataset: str,
    source: str,
    label: Optional[List[str]] = None,
    n_examples : int,
    exclusive: bool = False,
    exclude: Optional[List[str]] = None,
):  

    db = connect()

    stats = []
    
    # I will have to add a while loop to update stats for condition function

    for dataset_id in db.sessions:
        annotations = db.get_dataset(dataset_id)
        stats.append([str(dataset_id), len(annotations)])

    stream = JSONL(source)
    print(stream)

    #Add labels to each task in stream
    has_options = len(label) > 1
    if has_options:
        stream = add_label_options_to_stream(stream, label)
    else:
        stream = add_labels_to_stream(stream, label)

would a viable solution is to create a controller and input a config javascript alert, if the condition function is met?

Hi! You can provide custom JavaScript via the "javascript" setting returned by your custom recipe so if you want to handle it all in JavaScript, you could do something like this and check whether the target was reached every time an answer is submitted. To make sure you're only counting unique examples (and not cases wher the user undos and reannotates), you can count the _input_hash values in the answers:

const answered = Set()
const target = 1234

document.addEventListener('prodigyanswer', event => {
  const { task } = event.detail
  answered.add(task._input_hash)
  if (answered.length == target) {
      alert(`You have reached ${target} examples, please save and exit`)
  } 
})

Alternatively, if you don't want the user to keep submitting answers if the target was reached, you could also use the validate_answer callback: https://prodi.gy/docs/custom-recipes#validate_answer This would let you implement the logic in Python and the function will be called on every example that's submitted in the UI. An error raised here will be shown as an alert in the UI and the user won't be able to keep annotating:

answered = set()
target = 1234

def validate_answer(answer):
    answered.add(answer["_input_hash"])
    if len(answered) == target:
        raise ValueError(f"You have reached {target} examples, please save and exit")
1 Like