What's a recipe for (dead) simple binary (or multiclass) image classification?

[davidefiocco]

Hello, I would like to use Prodigy for simple labeling of images into 2 (or optionally more) categories streaming from a local images_dir directory.
Think about cats vs dogs, or MNIST digits classification. I also don't care about integrating an active learning component as of now. In the binary case I'd like an image to appear with a single proposed label like

I tried to look for this in the forum and found

Image classification

Hello! I’m new to AI and have been playing with prodigy for some days now. It seems like an awesome product! Congratz on the good work. I have been trying to do a simple image classification. I want to have a model to get a set of images and classify them as related or not (related can mean train a model for given a set of images tell me if they are the same color, for example). I was trying to do this with Prodigy but couldn’t find a way, so thought I could simplify the task by having the set …

Image Classification Example?

Hello, Looking at the example recipes, and I see there is a documented recipe for creating annotations within images as bounding box annotations. Is there a similar recipe for regular image labeling? E.g. This is a photo of X, where you are classifying the entire image, rather than bounding boxes within the image? It isn't clear from the documentation (at least at this point) how we would be able to build a custom recipe to use something like MobileNet on TF to learn from our annotations on Pr…

but to no avail...

I came up with a recipe.py along the lines of

# coding: utf8
from __future__ import unicode_literals

import prodigy
from prodigy.components.loaders import Images
from prodigy.util import split_string

@prodigy.recipe('image-classification',
    dataset=("The dataset to use", "positional", None, str),
    source=("Path to a directory of images", "positional", None, str),
    label=("One or more comma-separated labels", "option", "l", split_string)
)
def image_class(dataset, source, label=None):

    stream = Images(source)

    return {
        'dataset': dataset,        # Name of dataset to save annotations
        'stream': stream,          # Incoming stream of examples
        'view_id': 'classification',
        'config': {'label': label}
    } 

which I invoke with something along the lines of

prodigy image-classification my_dataset  images_dir -l spam,eggs -F recipe.py

This fails in the UI with

ERROR: Can't fetch tasks.

and in the console with

Exception when serving /get_questions

Can someone amend my recipe above, or point me to an analog? I suspect the problem lies in the view_id and config syntax. I tried variations but with no luck..

[ines (Ines Montani)]

Hi! The recipe you put together is a good start When you saw that error in the UI, what did it say in your terminal? This is typicallly where the real problem happens – for instance, maybe it failed to load the images from images_dir or something like that.

One small thing that's still missing from your recipe is adding the label(s) to the data. The classification interface expects every example to have a "label" property, which is then displayed at the top. So if your task is a simple binary classification task and you're labelling whether it's SPAM or not, you could do something like this:

def add_label_to_stream(stream, label):
    for eg in stream:
        # The 'label' you get from the command line is a list
        # so let's just assume it's always one and take the first
        eg["label"] = label[0]
        yield eg

stream = add_label_to_stream(stream, label)

If you have more than one label, e.g. SPAM and EGGS, there are two main options for structuring the task. One would be to duplicate the example for each label and ask two questions for each image:

import copy

def add_label_to_stream(stream, label):
    for eg in stream:
        for la in label:
            task = copy.deepcopy(eg)
            task["label"] = la
            yield task

This can work well for small label sets and still lets you annotate very fast. But once you have more labels, it does become a bit tedious because you have to answer len(examples) * len(labels) questions. In that case, it probably makes more sense to use the choice interface, which lets you add multiple-choice options for the labels. Here's an example.

The choice interface expects a list of "options" (each with a "text" that's displayed in the UI and an "id" that's used internally). So you could set up your stream like this and add an option for each label:

def add_options_to_stream(stream, label):
    options = [{"id": la, "text": la} for la in label]
    for eg in stream:
        eg["options"] = options
        yield eg

In your recipe, you'd then change "view_id": "classification" to "view_id": "choice".

[davidefiocco]

Thanks (as usual) @ines . I have put a working recipe (following your suggestions) for the easiest case in this gist: https://gist.github.com/davidefiocco/7bbb3e3f1203c2c7bd11960b83eca982 for future reference.

P.S.: While this is an easy exercise, IMHO this example problem would be a nice addition for https://github.com/explosion/prodigy-recipes/blob/master/image/ for newbies like me. My main obstacle for completion was understanding how to get the classification interface working.