Pass additional metadata in DB commits

I’m using a custom recipe for image labeling and a custom loader to serve the images into the web application. They are sensitive images, so I have to encrypt them first. I would like to store the filename of the image the user has labelled in addition to the encrypted image byte string.

I have used the technique mentioned elsewhere to capture the image to adjust it before passing it into the DB

How can I pass the filename and possibly other metadata like IDs and Creation Dates for the image file into the prodigy DB for use later?

Below I have reposted a similar version of my recipe with the encryption steps turned into pseudocode

def custom_image(
Manually annotate images by drawing rectangular bounding boxes or polygon
shapes on the image.
log(“RECIPE: Starting recipe custom-image”, locals())
db = connect()
stream = get_stream(source, api=api, loader=loader, input_key=“image”)
stream = fetch_images(stream)

def encrypt(string):
    return string

def update(answers):
    for eg in answers:
        eg['image'] = encrypt(eg['image'])
    # add the answers to the dataset
    db.add_examples(answers, [dataset])

return {
    "view_id": "image_manual",
    "dataset": dataset,
    "stream": stream,
    "exclude": exclude,
    "config": {"labels": label, "darken_image": 0},
    "db": False,
    "update": update

For anyone who wanted to know, the answer is actually simple. The recipe doesn’t need anything special, it’s actually the loader.

add this to your custom loader and you can include anything you want, just make sure you’ve stringified it, byte formats like something you could generate from str(“a string”).encode(“UTF-8”) will cause an error.

yield {“image”: file_path, “meta”: {“file_name”: str(image_file), “id”: str(image_id)}}

1 Like

Thanks for updating with your solution – that’s exactly what I would have suggested :+1: The dict representing the example is saved in the database, so you can add any custom properties you like. Data added under the "meta" key will also be displayed in the UI.

1 Like