How to use multiple choice annotation api in prodigy for images

, ,
1

Hello Prodigy team,

I am trying to create an image mash and I need some help to get the multiple choice image annotation api (also shown in live demo on prodigy website) up and running. I am looking for a step by step tutorial of sort regarding how to use that api…or at least a description on how to use the choice api command to make it work. Any help is highly appreciated.

Thanks,
Vandana

2

If you want to use a multiple-choice UI for an image, you want to be using the choice interface and then stream in examples with a list of "options". It's actually very simiar to the example recipe described in the workflow here – only that you'll be using "image" instead of "text" in the places you want to use an image:

So if you want a text plus image options, a task you create could look something like this:

{
    "text": "Hello world",
    "options": [
        {"id": 0, "image": "https://example.com/image0.jpg"},
        {"id": 1, "image": "https://example.com/image1.jpg"},
        {"id": 2, "image": "https://example.com/image2.jpg"}
    ]
}

You could also create a task with an image as the main content and text as options – whatever you need for your annotation task. So a different version could look like this:

{
    "image": "https://example.com/image.jpg",
    "options": [
        {"id": 0, "text": "Option 0"},
        {"id": 1, "text": "Option 1"},
        {"id": 2, "text": "Option 2"}
    ]
}

One quick note on using images in the interface: If you provide your images as local file paths on your system, modern browsers will block them, because they're considered unsafe. So you either want to serve your images on localhost, upload them somewhere (e.g. an S3 bucket) or temporarily adjust your browser. See here for details:

3

Hi,

Thanks a lot for your input. I m trying to use the input jsonl file with the below content along to test with the following test recipe.

test_sample.json

}
{
    "text": "Hello world",
    "options": [
           {"id": 0, "image": "/home/vandana_p/Selfies_Hair/0147F0201.jpg"},
           {"id": 1, "image": "/home/vandana_p/Selfies_Hair/0147F0202.jpg"},
           {"id": 2, "image": "/home/vandana_p/Selfies_Hair/0147F0203.jpg"}
    ]
}
{
    "text": "Hello world",
    "options": [
           {"id": 0, "image": "/home/vandana_p/Selfies_Hair/0147F0207.jpg"},
           {"id": 1, "image": "/home/vandana_p/Selfies_Hair/0147F0208.jpg"},
           {"id": 2, "image": "/home/vandana_p/Selfies_Hair/0147F0209.jpg"}
    ]
}

test_recipe.py

import prodigy
from prodigy.components.loaders import JSONL

@prodigy.recipe('sentiment',
    dataset=prodigy.recipe_args['dataset'],
    file_path=("Path to texts", "positional", None, str))
def sentiment(dataset, file_path):
    """Annotate the sentiment of texts using different mood options."""
    stream = JSONL(file_path)     # load in the JSONL file
    #stream = add_options(stream)  # add options to each task
    return {
        'dataset': dataset,   # save annotations in this dataset
        'view_id': 'choice',  # use the choice interface
        'stream': stream,
    }

I am getting the following error.

<generator object add_options at 0x7f1a4e943fc0>

  ?  Starting the web server at http://0.0.0.0:8080 ...
  Open the app in your browser and start annotating!

Exception when serving /get_session_questions
Traceback (most recent call last):
  File "cython_src/prodigy/components/loaders.pyx", line 145, in prodigy.components.loaders.JSONL
  File "/home/vandana_p/hair_diagnostics/my_hair_selfie/lib/python3.6/site-packages/srsly/_json_api.py", line 37, in json_loads
    return ujson.loads(data)
ValueError: Expected object or value

During handling of the above exception, another exception occurred:
......
......
File "/home/vandana_p/hair_diagnostics/my_hair_selfie/lib/python3.6/site-packages/toolz/itertoolz.py", line 368, in first
    return next(iter(seq))
  File "recipe.py", line 24, in add_options
    for task in stream:
  File "cython_src/prodigy/components/loaders.pyx", line 152, in JSONL
ValueError: Failed to load task (invalid JSON).

--------------------------------------------------------------------

It is failing to load the jsonl input file to stream. Is there anything I am missing? I could test the example feedback recipe in the prodigy site but not my test case.
Looking forward to your suggestions on how to fix this.

Regards,
Vandana

4

The "Invalid JSON" error usually means exactly that: the JSON file you're loading in is invalid. Did you run your file through a JSON validator? It should be a valid JSON file with a list of objects. If the file you posted above is the full file, then it's definitely wrong: there's no outer list, and it starts with a }, which isn't valid JSON.

Also note that this is likely not going to work:

"image": "/home/vandana_p/Selfies_Hair/0147F0201.jpg"

Due to the reason I outlined in my post above:

5

Thanks a lot for the quick reply. Starting with “}” is a copy paste mistake. I passed the jsonl file through the json validator and it is saying the file is valid.

I will try the fix you mentioned below.

6

Ah okay. I think the problem is here then:

You're using the JSONL loader to load the file as JSONL (newline-delimited JSON). But you're passing in a regular JSON file, so loading fails, because it's not JSONL. So either format your data as JSONL, or use the JSON loader instead.