What's a recipe for (dead) simple binary (or multiclass) image classification?

Hello, I would like to use Prodigy for simple labeling of images into 2 (or optionally more) categories streaming from a local images_dir directory.
Think about cats vs dogs, or MNIST digits classification. I also don't care about integrating an active learning component as of now. In the binary case I'd like an image to appear with a single proposed label like

I tried to look for this in the forum and found



but to no avail... :face_with_head_bandage:

I came up with a recipe.py along the lines of

# coding: utf8
from __future__ import unicode_literals

import prodigy
from prodigy.components.loaders import Images
from prodigy.util import split_string

@prodigy.recipe('image-classification',
    dataset=("The dataset to use", "positional", None, str),
    source=("Path to a directory of images", "positional", None, str),
    label=("One or more comma-separated labels", "option", "l", split_string)
)
def image_class(dataset, source, label=None):

    stream = Images(source)

    return {
        'dataset': dataset,        # Name of dataset to save annotations
        'stream': stream,          # Incoming stream of examples
        'view_id': 'classification',
        'config': {'label': label}
    } 

which I invoke with something along the lines of

prodigy image-classification my_dataset  images_dir -l spam,eggs -F recipe.py

This fails in the UI with

ERROR: Can't fetch tasks.

and in the console with

Exception when serving /get_questions

Can someone amend my recipe above, or point me to an analog? I suspect the problem lies in the view_id and config syntax. I tried variations but with no luck..

Hi! The recipe you put together is a good start :+1: When you saw that error in the UI, what did it say in your terminal? This is typicallly where the real problem happens – for instance, maybe it failed to load the images from images_dir or something like that.

One small thing that's still missing from your recipe is adding the label(s) to the data. The classification interface expects every example to have a "label" property, which is then displayed at the top. So if your task is a simple binary classification task and you're labelling whether it's SPAM or not, you could do something like this:

def add_label_to_stream(stream, label):
    for eg in stream:
        # The 'label' you get from the command line is a list
        # so let's just assume it's always one and take the first
        eg["label"] = label[0]
        yield eg

stream = add_label_to_stream(stream, label)

If you have more than one label, e.g. SPAM and EGGS, there are two main options for structuring the task. One would be to duplicate the example for each label and ask two questions for each image:

import copy

def add_label_to_stream(stream, label):
    for eg in stream:
        for la in label:
            task = copy.deepcopy(eg)
            task["label"] = la
            yield task

This can work well for small label sets and still lets you annotate very fast. But once you have more labels, it does become a bit tedious because you have to answer len(examples) * len(labels) questions. In that case, it probably makes more sense to use the choice interface, which lets you add multiple-choice options for the labels. Here's an example.

The choice interface expects a list of "options" (each with a "text" that's displayed in the UI and an "id" that's used internally). So you could set up your stream like this and add an option for each label:

def add_options_to_stream(stream, label):
    options = [{"id": la, "text": la} for la in label]
    for eg in stream:
        eg["options"] = options
        yield eg

In your recipe, you'd then change "view_id": "classification" to "view_id": "choice".

1 Like

Thanks (as usual) @ines . I have put a working recipe (following your suggestions) for the easiest case in this gist: https://gist.github.com/davidefiocco/7bbb3e3f1203c2c7bd11960b83eca982 for future reference.

P.S.: While this is an easy exercise, IMHO this example problem would be a nice addition for https://github.com/explosion/prodigy-recipes/blob/master/image/ for newbies like me. My main obstacle for completion was understanding how to get the classification interface working.