How to run the `choice` view?

, ,
1

Hi, I am able to tag with the view_id of text and ner. I’m currently trying to use the choice view so that we can tag categories instead of binary.

I created a recipe that returns {'view_id': 'choice'} and I added options in the source data. Each data point will have some options, e.g.:

{
    "text": "how are you?",
    "label": "oink",
...
    "options": [
        {"id": 1, "text": "Apple"},
        {"id": 2, "text": "Google"},
        {"id": 3, "text": "McDonalds"}
    ]
...
}

When I run the Prodigy server, it still looks like the text view with binary labels.

Also, examples would be great in the docs!

2

Hmm, this is strange – your example looks correct. I just tested it with a dummy recipe and it works fine for me. It displays the label, text and the three options. My recipe looks like this:

@prodigy.recipe('choice')
def choice():
    stream = [{
        'label': 'SOME_LABEL',
        'text': 'hello {}'.format(i),
        'options': [{"id": 1, "text": "Apple"}, {"id": 2, "text": "Google"}, {"id": 3, "text": "McDonalds"}]
    } for i in range(10)]

    return {
        'view_id': 'choice',
        'dataset': False,
        'stream': stream
    }

The result:

choice
choice.png768×395 11.3 KB

If you use the above recipe and run it, does it cause the same problem for you?

And you’re right, a choice example is currently missing from the docs! I might use that for the “Custom recipes” workflow I’m currently working on for the website.

1 Like
3

Thanks! This works.

1 Like
4

Is it possible to automatically “accept” when clicking one of the choices?

5

Maybe this could be an additional option in the settings, like "choice_auto_accept": true? The reason I didn't implement it this way by default was that "auto accept" could only apply to the single choice interface and not the multiple choice – so the behaviour on the UI would be inconsistent.

It can also be nice and more intuitive to have a clearer separation between the accept/reject/ignore action and the actual choice of the options. For example, in theory, you could also use the interface to create a set of "wrong" training examples, by selecting an option and clicking "reject". But I definitely see your point – if you only care about accepting choices and you're annotating a lot of these, the two clicks or key presses can easily become annoying.

6

Thanks, it would nice to have that option as most of my classifiers are multi-class. But if I understand correctly, all of the spaCy models are trained on binary labels so I can see the incompatibilities.

7

I’d second @plusepsilon’s view that it would be nice to have some sort of auto-accept behavior and I like your implementation suggestion, @ines. I’ve done about an hour of choice annotating and it does break up the flow a bit to hit the number and then accept.

8

@andy @plusepsilon Thanks for your feedback! I’ve put this on my list for the next release :+1:
This will be easy to implement and I’ll add it as an option in the settings. (Breaking UI consistency is fine, as long as it’s a conscious decision the user makes.)

2 Likes
9

@andy @plusepsilon Update: A choice_auto_accept setting is now available in the latest update of Prodigy. If set to true, the task will be accepted automatically when an answer is selected in single-choice mode.

2 Likes
10

Hi, congratulations on the launch!
Is there a way to remove the accept/reject buttons when choice_auto_accept=True? Or maybe there is some use for them that I can’t see?

11

Thanks!

I think this might be a bit weird... I do see your point, but depending on the task and annotation strategy, a user may want to accept or reject a task with no selection. We'd also still have to keep the ignore and undo buttons. So just in terms of consistency, I'm not sure hiding the action buttons would make sense.

12

After some searching around I found that the custom recipe example uses choice, I think linking to it from the docs would have been very helpful!