Is there a way to combine the choices review with `diff` and `text_input`?

Hello!
diff andtext_input are essential for the tasks I'm doing.
I want to combine them with the choices review interface like the one below. But I can't find a way to do it.

Below are the results I have tried.

  1. I set blocks in prodigy.json for the basic review recipe.
PRODIGY_CONFIG=[path_to/prodigy.json] prodigy review test_review [dataset1,dataset2] --view-id choice --show-skipped

If I set --view-id choice, I can see the choices review, but diff and text_input by blocks are not reflected. With --view-id blocks, I can see the diff, text_input, and choice, but not the choices review,.

  1. I modified review.py to include blocks.

Switching between "view_id": "choice" and "view_id": "blocks" causes the same thing as 1.
Is there a way to achieve the combination I want?

Hi! Can you share the settings you've used for the blocks? And was the data you're reviewing annotated with the multiple choice UI? That's important if you actually want to show which annotators selected which options.

If you use a slightly modified version of the review recipe, it should work if you use --view-id choice, which will render the review UI with multiple choices. Your blocks could then be review, diff and text_input.

Thank you for checking :smiley:

I will share with you the prodigy.json I have set.
The blocks that I tried to incorporate into review.py have a similar structure.

{
    "global_css": ".prodigy-content{text-align: left} .prodigy-text-input",
    "validate": "True",
    "view_id": "blocks",
    "blocks": [
        {
            "view_id": "diff",
            "text": ""
        },
        {
            "view_id": "html",
            "html_template": "<small>[校正前]<br>{{original}}<br>[校正後]<br>{{revised}}<br>[choice]<br>{{choice}}<br>[comments]{{comments}}</small>"
        },
        {
            "view_id": "choice",
            "text": ""
        },
        {
            "view_id": "text_input",
            "field_label": "comment",
            "field_id": "comment",
            "field_rows": 3
        }
    ],
    "instructions": "/data/sugino/kousei/annotation/aap/02_annotation/scr/instructions.html"
}

Yes, that's right. I will share with you the contents of the dataset to be reviewed. original,revised,choice, comments, comment key etc. are my own settings.

{
    "original": "This are a sample.",
    "revised": "This is a sample.",
    "domain": "ai",
    "text": "This are a sample.\n\nThis is a sample.",
    "added": "This are a sample.",
    "removed": "This is a sample.",
    "meta": {
        "pred_label": "error",
        "score": "0.938"
    },
    "label": null,
    "options": [
        {
            "id": 1,
            "text": "error"
        },
        {
            "id": 2,
            "text": "Probably error"
        },
        {
            "id": 3,
            "text": "not error"
        },
        {
            "id": 4,
            "text": "Probably not error"
        }
    ],
    "_input_hash": 2100084490,
    "_task_hash": -1640287291,
    "_view_id": "blocks",
    "accept": [
        3
    ],
    "config": {
        "choice_style": "single"
    },
    "comment": "Neither is wrong.",
    "answer": "accept",
    "_timestamp": 1639536500,
    "_annotator_id": "dataset-annotator1",
    "_session_id": "dataset-annotator1",
    "choice": "annotator1 : [3]\nannotator2 : [2]\nannotator3 : [1]\nannotator4 : [3]\nannotator5 : [3]\n",
    "comments": "annotator1:Neither is wrong.\n"
}

Thanks for sharing! Instead of the choice interface block, you probably want to use a review block. This will show the different choice options that were selected by the different reviewers, if you run your review recipe with --view-id choice.

Thanks for your advice!
Following your advice, I used review for the interface block and ran the review recipe with the option --view-id choice. I was able to see a review of the choices like the first image, but the diff and text_input were not added.
I share the modified prodigy.json and the review execution command.
Is there anything else I should try?

{
    "global_css": ".prodigy-content{text-align: left} .prodigy-text-input",
    "validate": "True",
    "view_id": "blocks",
    "blocks": [
        {
            "view_id": "diff",
            "text": ""
        },
        {
            "view_id": "html",
            "html_template": "<small>[校正前]<br>{{original}}<br>[校正後]<br>{{revised}}<br>[choice]<br>{{choice}}<br>[comments]{{comments}}</small>"
        },
        {
            "view_id": "review",
            "text": ""
        },
        {
            "view_id": "text_input",
            "field_label": "comment",
            "field_id": "comment",
            "field_rows": 3
        }
    ],
    "instructions": "/data/sugino/kousei/annotation/aap/02_annotation/scr/instructions.html"
}

PRODIGY_CONFIG=[path_to/prodigy.json] prodigy review test_review dataset --view-id choice

What about this? @ines
I hope there is good progress!