Prepopulating the image.manual .jsonl data

[alphie]

I have pdfs many of which are identical and I am looking for a way to speed up annotations
I am interested in knowing more about the idea of “pre-populating them in the data”
As mentioned in

Prodigy - text extraction from forms

Hi! You can definitely label data for this type of task, but you'd have to decide which OCR solution or tool you want to use to go from image + bounding boxes to text. For example, you can use a workflow like image.manual and draw the boxes: https://prodi.gy/docs/recipes#image-manual If the boxes are mostly in the same place and have similar pixel coordinates, you could even pre-populate them in the data so you only have to adjust the box if the position changed. The data you export gives you t…

Lets say I have 50 pdfs with an identical layout.
Is there a workflow by which I could annotate one of my identical pdfs and save it as .jsonl.
and then put the same bounding boxes and labels on pdfs 2-50?

[magdaaniol]

Hi @alphie,

The easiest way to "prepopulate" your input jsonl with bounding boxes from the "reference" annotation would be via a python script outside Prodigy.
This script should read your "reference" example and copy the value of the spans key to all the other input examples.
spans is where the bounding boxes are stored. Please note that the bounding boxes defined as pixel offsets relative to the the image height and width so if the PDFs have different heights and widths the bounding boxes will not fall in exactly the same spots.
The script could look something like this:

from typing import Dict, List

import srsly
from prodigy.components.stream import get_stream
from prodigy.types import StreamType
from wasabi import msg


def add_spans(stream: StreamType, spans: List[Dict]) -> StreamType:
    for example in stream:
        if "spans" in example:
            msg.warn(
                f"Example with _task_hash {example.get('_task_hash')} already contains bounding boxes. Leaving as is."
            )
        else:
            example["spans"] = spans
        yield example


# Load the reference annotation
reference = next(srsly.read_jsonl("template_annotation.jsonl"), None)
if reference is None or "spans" not in reference:
    msg.error("The reference annotation does not contain `spans`", exits=1)

reference_spans = reference["spans"]

# Get the stream and apply spans
stream = get_stream("input_data.jsonl")
stream.apply(add_spans, stream=stream, spans=reference_spans)

# Write the annotated stream to a file
output_path = "./preannotated_data.jsonl"
srsly.write_jsonl(output_path, list(stream))
msg.info(f"Preannotated dataset saved at {output_path}")

The preannotated jsonl (preannotated_data.jsonl) should be ready to use with image.manual for curation.