Draw a shape and label a person and their behavior in a video frame


I have a video that I plan to separate into frames for the analysis. I need to do manual image labeling (such as drawing a box around a person on a video, i.e. person #1, person #2, etc). But I also need to label what the person is doing, i.e. person #1, standing, person #2 jumping, person #3 touching person #1, etc. Is it possible to do with Prodigy manual image labeling? In other words I will need to draw a box (or another shape) around the object in a frame and give it not one, but 2 labels (label #1=person #1; label #2=standing)

Hi! Are the labels in the second dimension (standing, touching etc.) pre-defined, or is this more of a free-form thing, depending on what's happening in the frame? If it's more like a free-form caption, you could use an interface with two blocks: image_manual to annotate the bounding boxes and text_input for the caption. See here for an example: https://prodi.gy/docs/custom-interfaces#blocks

If you have pre-defined labels for the second dimension, you could make two passes over the data: first, annotate the objects (person etc.), and review the annotations/data quality if needed. Next, stream in each bounding box and the second layer of labels, e.g. as multiple choice options (choice UI). This also makes it easier to validate and evaluate the two layers separately, and fix mistakes early (because you know that if the bounding box is wrong, any subsequent annotation will most likely be wrong, too).

Great, thank you so much! I have one more question, is there a way to point annotate? For example, I have an image of a person and I need point annotate with predefined labels, i.e. left eye x=185.25173502386923, y=610.3622760638572, nose x=208.25876751706778 y=638.481982444433 etc and save it in .csv file.

Can freehand shape just mark a single point in x, y format?

At the moment, the annotations are expected to be shapes, so they need at least two points – but you could use the freehand tool to draw a super tiny line and then just take the first point (which is where you first clicked).

1 Like

ok, great, thank you for your reply!