Horizontal View for Image Classification

I'd like to have the image displayed on the left or right of the browser window, and the list of categories/tags on the left or right of this image. This is because my list of categories is rather large (40 distinct categories) and it takes a long time to scroll all the way down.

How can I implement this instead of the default (image and tags displayed stacked on top of one another vertically)?

Hi! One solution would be to use the global_css setting and define two columns: one for the image container and one for the container of the choice options. This example here shows how to align the options themselves in two columns but the idea is the same for the .prodigy-container block with the image and choice options.

If your labels are very short, you could even do both: display the labels in two columns, in a column next to the image :slightly_smiling_face:

Hi, curious has anyone successfully implemented two columns using image/image_manual and choice?

This is in my prodigy.json:

"global_css": ".prodigy-content, .c01185 {width: 100%;display: grid;grid-template-columns: 1fr 1fr;grid-gap: 20px;}",

This is in my recipe:

import prodigy
from prodigy.components.loaders import Images
from prodigy.util import split_string
from typing import List, Optional


# Recipe decorator with argument annotations: (description, argument type,
# shortcut, type / converter function called on value before it's passed to
# the function). Descriptions are also shown when typing --help.
@prodigy.recipe("image2.manual",
    dataset=("The dataset to use", "positional", None, str),
    source=("Path to a directory of images", "positional", None, str),
    loader=("Loader if source is not directory of images", "option", "lo", str),
    label=("One or more comma-separated labels", "option", "l", split_string),
    exclude=("Names of datasets to exclude", "option", "e", split_string),
    darken=("Darken image to make boxes stand out more", "flag", "D", bool),
    remove_base64=("Remove base64-encoded image data before storing example in the DB. (Caution: if enabled, make sure to keep original files!)",
    "flag", "R", bool))

def image2_manual(
    dataset: str,
    source: str,
    loader: str = "jsonl",
    label: Optional[List[str]] = None,
    exclude: Optional[List[str]] = None,
    darken: bool = False,
    no_fetch=("Don't fetch images as base64", "flag", "NF", bool),
    remove_base64: bool = False,
):
    # Load a stream of images from a directory and return a generator that
    # yields a dictionary for each example in the data. All images are
    # converted to base64-encoded data URIs.
    def get_stream():
        stream = Images(source)
        options = [
            {"id": 0, "text": "1"},
            {"id": 1, "text": "2"},
            {"id": 2, "text": "3"},
            {"id": 3, "text": "4"},
            {"id": 4, "text": "5"},
            {"id": 5, "text": "6"},
            {"id": 6, "text": "7"},
            {"id": 7, "text": "8"},
            {"id": 8, "text": "9"},
            {"id": 9, "text": "10+"},
        ]
        for eg in stream:
            eg["options"] = options
            yield eg
        
    blocks = [
        {"view_id": "image_manual"},
        {"view_id": "choice", "image": None, "text": None},
        ]
    
    """
    Manually annotate images by drawing rectangular bounding boxes or polygon
    shapes on the image.
    """

    # with open('./template.html', 'r') as f:
    #     html_template = f.read()
    
    def before_db(examples):
        # Remove all data URIs before storing example in the database
        for eg in examples:
            if eg["image"].startswith("data:"):
                eg["image"] = eg.get("path")
            return examples
    
    
    return {
        
        "view_id": "blocks",  # Annotation interface to use
        "dataset": dataset,  # Name of dataset to save annotations
        "stream": get_stream(),  # Incoming stream of examples
        "before_db": before_db if remove_base64 else None,
        "exclude": exclude,  # List of dataset names to exclude
        "config": {  # Additional config settings, mostly for app UI
            "blocks": blocks,
            "label": ", ".join(label) if label is not None else "all",
            "labels": label,  # Selectable label options,
            "darken_image": 0.2 if darken else 0,
            "show_bounding_box_center": True,
            "show_bounding_box_size": True,
            "choice_style": "single", 
            "show_stats": True
        },
    }
    


However, everything is simply just 50% wide but the choices are not in the second column next to the image.

hi @c00lcoder!

Thanks for your question!

Have you seen the captcha example in the Prodigy docs?

What's nice is that you only need to specify your images within the options like:

{
  "label": "CARS",
  "options": [
    {"id": 1, "image": "https://i.imgur.com/qVhETd4.jpg"},
    {"id": 2, "image": "https://i.imgur.com/HdrE9pv.jpg"},
    {"id": 3, "image": "https://i.imgur.com/KqxpIrc.jpg"}
  ]
}

So if you specify only two images, you'll get the two horizontal images.

Related, have you seen the extension of the captcha to a 3x3 grid by my colleague @koaning for the Bad Image Data tutorial video?

The code can be found in the accompanying project folder and you can learn more from the video itself:

Let me know if this does (or doesn't) solve your problem!

Hi Ryan, I'm not really sure if the caption version will work for my use case. I've been trying to think how I could apply it. For my first example, I'm simply making a selection of # of people within an image; ideally, I could simply make a selection using the 'choice' UI and go through the 1k images pretty quickly.

I can have quite a bit of options so being able to setup the choice selection next to the image would be perfect.

something like this is what I'm trying to achieve.

hi @c00lcoder!

Thanks for the context -- that makes sense.

Yes, for the layout you want, custom CSS would be the best route. I would need to take some time to think about it more in detail.

However, there may be an alternative layout too to accomplish the same task. Can you describe your choice columns a bit more--Would they be integers from 0 to 10 like your example? If so, why do you have two columns? Perhaps to avoid too long of a choice list for numbers > 10.

If this is the case, perhaps an alternative layout may be easier.

Option 1: Add a Slider

Option 2: Input Box

A Prodigy community member asked something similar. They needed four choice columns below an image.

I showed them how to convert it to multiple text_input (open-ended text boxes):

arm

Since they were selecting categories, the post includes how to use field suggestions which may not be relevant for you since you have a numerical (integer) input.

Let me know if this helps. If not, I can try to look at the CSS later on to debug for the original layout.

Thanks Ryan. For my initial use case of # of people, It will be 0-10+ ... for simplicity, I'll make use of the keyboard shortcuts for this use case (Initially I had to keep scrolling down). As I go through the dataset and add additional keyword annotations, that's when options can be anywhere from 15 to 25 or more options. Figuring out a faster UI would be very helpful. A CSS or HTML example would be useful; since, I haven't been able to figure it out, yet; based on what I was able to find.

Preference is still choice to limit any potential misspellings or errors. Thank you