# Clarification on annotation capabilities

**URL:** <https://support.prodi.gy/t/clarification-on-annotation-capabilities/4721>\
**Category:** Uncategorized\
**Tags:** usage, image\
**Created:** [September 26, 2021, 2:19am UTC](https://support.prodi.gy/t/clarification-on-annotation-capabilities/4721 "2021-09-26T02:19:42Z")\
**Posts on this page:** 1\
**Showing post:** 8

<div class="post-metadata">

**Author:** ![ryanwesslen](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ryanwesslen/32/2969_2.png) [@ryanwesslen](https://support.prodi.gy/u/ryanwesslen)\
**Post date:** [January 4, 2023, 4:22pm UTC](https://support.prodi.gy/t/clarification-on-annotation-capabilities/4721/8 "2023-01-04T16:22:03Z")

</div>

hi @c00lcoder!

Thanks for the clarification.

> [@c00lcoder](#):
>
> So to clarify, if I have an image with two persons...person A and person B...I want the attributes for person A to be saved in the span with the person A bounding box and likewise for person B. If the "accept" answer is saved in the overall jsonl dictionary, it is associated with the image and not the actual person bounding box.

The [`before_db` callback](https://prodi.gy/docs/custom-recipes#before_db) could help with the 2nd step to modify the data.

Like described previously, it would need to be a two-step process. First you create your bounding boxes, then add the attributes to the spans by iterating over each span separately and adding the choices with the before\_db method to the spans.

It's important to know that `before_db` callback should be used sparingly and with caution. The [docs](https://prodi.gy/docs/custom-recipes#before_db) mention this:

> The `before_db` callback modifies the annotations and Prodigy will place whatever it returns in the database. You should therefore **use it cautiously** , since a small bug in your code could lead to data loss.

# Step 1: Get person bounding boxes

Let's say you start with a `.jsonl` file that has the path to your images:

```python
# image-sample.jsonl
{"image": "person-image.png"}

```

You first create the bounding boxes for the person by running:

```python
python -m prodigy image.manual image-sample image-sample.jsonl --loader jsonl --label PERSON

```

 ![localhost_8080_ (14)](https://us1.discourse-cdn.com/flex020/uploads/prodigy/original/2X/e/eb9c0b04253bd576594d01376aacc426bddb2ab2.jpeg)

Save your annotations so now your bounding box person annotations are in the Prodigy dataset `image-sample`.

# Step 2: Custom recipe for additional attributes

Here's a custom recipe to provide four choice attributes for each person (e.g., female/male, brown/black hair).

```python
# image-nesting.py
import prodigy
from prodigy.components.db import Database
from prodigy.components.db import connect
from prodigy import set_hashes

@prodigy.recipe(
    "image.nesting",
    dataset=("The dataset to use", "positional", None, str),
    origin_dataset=("The original dataset to get the images from", "positional", None, str),
)
def image_nesting(dataset: str, origin_dataset):    
    db = connect()      
    data = db.get_dataset(name=origin_dataset)
    
    def get_stream():
        for image in data:
            if "spans" in image:
                for span in image["spans"]:
                    image_copy = image.copy()
                    
                    image_copy["spans"] = [span]
                    image_copy["options"] = [
                        {"id": "female", "text": "female"},
                        {"id": "male", "text": "male"},
                        {"id": "brown", "text": "brown hair"},
                        {"id": "black", "text": "black hair"},
                        ]
                    image_copy["id"] = span["id"]
                    yield set_hashes(image_copy, task_keys=("spans", "image", "id"), input_keys=("spans", "image", "id"), overwrite=True)
    
    stream = get_stream()
    
    def before_db(examples):
        for eg in examples:
           if "spans" in eg and "accept" in eg:
               eg["spans"][0]["additional_info"] = eg["accept"]
        return examples
        
    return {
        "dataset": dataset,
        "stream": stream,
        "view_id": "choice",
        "config": {
            "choice_style": "multiple"
        },
        "before_db": before_db,
    }

```

You can then run this recipe as:

```python
python -m prodigy image.nesting image-nesting image-sample -F image-nesting.py

```

 ![localhost_8080_ (15)](https://us1.discourse-cdn.com/flex020/uploads/prodigy/original/2X/9/914fed63d3c7d343f8597ea8bc9dd5f29d545212.jpeg)

We can then look at your new annotation:

```python
python3 -m prodigy db-out image-nesting > image-nesting.jsonl

```

```python
#image-nesting.jsonl
{
  "image": "data:image/png;base64, ...", # removed actual base64 for example
  "_input_hash": 1875068863,
  "_task_hash": -354213639,
  "_is_binary": false,
  "path": "person-image.png",
  "_view_id": "choice",
  "width": 400,
  "height": 267,
  "spans": [
    {
      "id": "5b63d3f8-e4c8-4160-8a7f-d6579ce210ad",
      "label": "PERSON",
      "color": "yellow",
      "x": 56.3,
      "y": 9,
      "height": 247,
      "width": 178,
      "center": [
        145.3,
        132.5
      ],
      "type": "rect",
      "points": [
        [
          56.3,
          9
        ],
        [
          56.3,
          256
        ],
        [
          234.3,
          256
        ],
        [
          234.3,
          9
        ]
      ],
      "additional_info": [
        "female",
        "brown"
      ]
    }
  ],
  "answer": "accept",
  "_timestamp": 1672848904,
  "options": [
    {
      "id": "female",
      "text": "female"
    },
    {
      "id": "male",
      "text": "male"
    },
    {
      "id": "brown",
      "text": "brown hair"
    },
    {
      "id": "black",
      "text": "black hair"
    }
  ],
  "id": "5b63d3f8-e4c8-4160-8a7f-d6579ce210ad",
  "config": {
    "choice_style": "multiple"
  },
  "accept": [
    "female",
    "brown"
  ]
}

```

Does this solve your problem?

---

_[View the full topic](https://support.prodi.gy/t/clarification-on-annotation-capabilities/4721)._