How to add labels to 'image_manual' view_id in custom recipe

, , ,
1

Hello,

Is there a way to specify label names when using the ‘image_manual’ view_id in a custom recipe? Everything I have tried results in a single label named “NO_LABEL”.

In trying to create a customized version of the ‘image_manual’ task, I cannot find a way to specify labels for the bounding boxes. I have tried adding various fields to both each task dictionary in the stream and the primary dictionary returned by the recipe function, but nothing I have tried has worked.

I have also tried importing image_manual and setting components = image_manual(…), but using this set up I am unable to customize the config and updater programatically.

See code below, including both a similar recipe that works and the ‘image_manual’ recipe which does not yet work (with some comments regarding what I have tried.

# THIS RECIPE WORKS
@prodigy.recipe('image.choice',
    dataset=prodigy.recipe_args['dataset'],
    file_path=("Path to texts", "positional", None, str),
    options_path=("Path to line-separated json options file",
                  "positional", None, str),
    port=("port to use", "positional", None, int)
    )
def image_choice(dataset,
               file_path,
               options_path,
               port,
               instructions="./templates/instructions_image_choice.html"
               show_flag=True,
               ):
    """choice tasks with image and text"""
    image_path = "/data/imgs"
    stream = image_generator(
                        file_path,
                        image_path,
                        )
    stream = fetch_images(stream, skip=True)
    stream = add_options(stream, options_path)

    # add html_template to each task
    path_html_template = "templates/image_text_choice.html"
    with open(path_html_template, 'r') as fh:
        html_template = fh.read()
    stream = add_html(stream, html_template)

    # Customize the update function
    updater = ImagelessUpdater(dataset)
    update = updater.update

    # Custom config
    config = {
                "custom_theme": {"cardMaxWidth": 2000,},
                "port": port,
                "show_flag": show_flag,
                "hide_true_newline_tokens": True,
                "instructions": instructions,
                "choice_auto_accept": True,
                }

    output_dict = {
        'dataset': dataset,
        'view_id': 'choice',
        'stream': stream,
        'db': False,
        'update': update,
        'config': config,
        }
    return output_dict


## THIS RECIPE DOES NOT WORK
## How do I add multiple (or any) box-labels when using the 'image_manual' view_id?
@prodigy.recipe('temp',
    dataset=prodigy.recipe_args['dataset'],
    file_path=("Path to texts", "positional", None, str),
    label=("Box Labels", "positional", None, str),
    options_path=("Path to line-separated json options file",
                  "positional", None, str),
    port=("port to use", "positional", None, int)
    )
def temp(dataset,
         file_path,
         options_path, # options that work for other tasks
         label,        # do not work for this view_id
         port,
         instructions="./templates/instructions_temp.html",
         show_flag=False,
        ):
    """image annotation task
    prodigy temp my_set $data_path 7777 -F ./src/temp.py
    """
    image_path = "/data/imgs"
    stream = image_generator(
                        file_path,
                        image_path,
                        )
    stream = fetch_images(stream, skip=True)

    ## Adding a 'label' field to each task
    ## (as I would do for the 'classification' view_id)
    ## does not add labels in the image_manual interface
    stream = add_label(stream, label)

    ## Adding an 'options' field to the task
    ## (as I would do for the 'choice' view_id)
    ## does not add labels in the image_manual interface
    stream = add_options(stream, options_path)

    # Custom Update
    updater = ImagelessUpdater(dataset)
    update = updater.update

    Custom Config
    config = {
        "custom_theme": {"cardMaxWidth": 2000,},
        "port": port,
        "show_flag": show_flag,
        "instructions": instructions,
        }

    output_dict = {
        'dataset': dataset,   # save annotations in this dataset
        'view_id': 'image_manual',
        ## adding a label field here does not add labels
        ## in the image_manual interface
        # 'label': 'thing1,thing2',
        'stream': stream,
        'components': components,
        'db': False,
        'update': update,
        'config': config,
        }

    ## If I return 'components' how can I
    ## programatically set 'config' 'db' and 'update'
    # components = image_manual(dataset=dataset,
    #                     source=stream,
    #                     label=label,
    #                     )

    return output_dict #components
2

Yes, that setting is called "labels" and it takes a list of label names. So in your config, you should be able to do something like this:

config = {
    "labels": ["CAR", "PERSON", "ANIMAL"],
    "custom_theme": {"cardMaxWidth": 2000,},
     "port": port,
     "show_flag": show_flag,
     "instructions": instructions,
}

(Sorry about the slightly unideal naming here – the "label" key in the task is the single label that's displayed above, just like in classification view. The "labels" returned by the recipe config is the available label set. If I could go back now, I would have chosen a different name for this!)

You could also do this programmatically using the label argument of your custom recipe. This will become available on the command line as the third argument. So if you pass in CAR,PERSON,ANIMAL, the value of the label argument your function receives will be a string "CAR,PERSON,ANIMAL". So you could also do this:

label_set = [l.strip() for l in label.split(",")]
3

Thank you! This is exactly what I was looking for.

1 Like