Model prediction for image classification

, , ,
1

I have a classifier model that predicts one of two labels for the input images. To perfect the model, I need to more tagged images to train further. For the ease of further labeling, I want to create a custom recipe that pre-annotates the image stream. This is the recipe I wrote,

@prodigy.recipe("tag-from-model")
def tag_from_model(dataset, images_path):
    model = load_model()
    blocks = [
        {"view_id": "image"},
        {"view_id": "text_input", "field_id": "caption", "field_autofocus": True},
    ]

    def get_stream():
        stream = Images(images_path)
        for eg in stream:
            caption = generate_caption(eg["image"], model)
            eg["caption"] = caption
            yield eg

    return {
        "dataset": dataset,
        "stream": get_stream(),
        "view_id": "blocks",
        "config": {"blocks": blocks},
    }

What I would like here is to display the model output as choice (exclusive) and not as a text input.
This is what I tried..

@prodigy.recipe("tag-from-model")
def tag_from_model(dataset, images_path):
    model = load_model()
    blocks = [
        {"view_id": "image"},
        {"view_id": "choice"},
    ]
    options = [
        {'id': 'VENT', 'text':'cat'}, 
        {'id':'CARD', 'text':'dog'}
    ]

    def get_stream():
        stream = Images(images_path)
        for eg in stream:
            caption = generate_caption(eg["image"], model)
            eg["options"] = caption
            yield eg

    return {
        "dataset": dataset,
        "stream": get_stream(),
        "view_id": "blocks",
        "config": {"blocks": blocks},
    }

But this is obviously not going to work as there is no connection between the model output and the options block. I'm not able to figure out how..

I searched the support/documentation for similar questions, could not find any. Could you please help.

Thank you much
DU

2

Hi! I think you're almost there – but I don't think you want to have the model define the options, right? If you know the labels, you know which options you need and you can just use the model to pre-populate the answers.

You can check out the docs for the choice interface and its format here: https://prodi.gy/docs/api-interfaces#choice When you select an option, it will be added to a list under the key "accept". So if your model predicts label CARD, you can add eg["accept"] = ["CARD"] to your outgoing task to pre-select that label.

3

Works perfectly.. thank you very much @ines

1 Like