Welcome to the forum @meitt8! 
Thanks for the detailed report and the screenshots, they made this easy to track down.
Prodigy's manual interfaces are designed around a set of labels that stays the same for the whole session. This tends to work well for annotators, since they can stay focused on one label scheme throughout. But we do understand there are different use cases, like yours, and we'd like Prodigy to be flexible enough to support them.
Currently, the interface isn't rebuilt when a new task loads; it keeps its state and only updates the content. Things like the text and the spans refresh for each task, but the selected label doesn't. So when task 2 comes with a different set of labels, the interface still holds "1. あえて" as the active label. It's not in the new list, so no button is highlighted, but it's still applied to anything you highlight.
We'll look into handling changing labels better in a future version. In the meantime, there are two things you can add to your recipe. Both assume you're setting the labels per task via "config": {"labels": [...]} in each example.
1. Select the first label automatically when a new task loads
This custom JavaScript selects the first label of each new task, so the active label is always one of the options shown:
let lastHash = null
document.addEventListener('prodigyupdate', event => {
const hash = event.detail.task._task_hash
if (hash === lastHash) return
lastHash = hash
const labels = (event.detail.task.config || {}).labels || []
if (!labels.length) return
// Wait for the label buttons to re-render with the new task's label
setTimeout(() => {
const button = document.querySelector(
`.prodigy-labels label[data-prodigy-label="${CSS.escape(labels[0])}"]`
)
if (button) button.click()
}, 0)
})
You can pass it in via javascript in your recipe's config (see below).
2. Block answers that use a label not available for the task
This validate_answer check makes sure nothing with an invalid label can be saved. If they annotator accidentally hits accept without choosing a label for a given set they will get a popup prompting them to fix the error:
def validate_answer(eg):
if eg.get("answer") != "accept":
return
allowed = set(eg.get("config", {}).get("labels", []))
invalid = sorted({s["label"] for s in eg.get("spans", []) if s["label"] not in allowed})
if invalid:
raise ValueError(
f"Label(s) {', '.join(invalid)} aren't available for this ta
"Please remove those spans and select a label from the list above."
)
Putting it together, your recipe's return dict would look something like
return {
"dataset": dataset,
"stream": stream,
"view_id": "spans_manual",
"validate_answer": validate_answer,
"config": {"javascript": JAVASCRIPT}, # the JS above as a string
}
Here's more info on how to use custom JS in Prodigy recipes: Custom Interfaces · Prodigy · An annotation tool for AI, Machine Learning & NLP
And here's more info on callbacks such as validate_answer: Custom Recipes · Prodigy · An annotation tool for AI, Machine Learning & NLP
Hope this helps, and let us know how you get on!
