Free text annotation with model suggestions

I have a project with free text annotations, however I want to use a generative model to suggest that free text for the text_input interface. Is there any way I can pre-populate that field from a model?

Hi Chris.

You can totally do this, as the docs of the text input interface explain here, you're free to add field_placeholder properties to the example stream. I made a quick custom recipe to demonstrate how this might work.

import prodigy
from prodigy.util import set_hashes

# I am pretending that this function is a generative model
# You'd have to replace this with your own.
def fake_model(text):
    return text + " ... but now I'm fake generated!"

@prodigy.recipe(
    "custom-text-recipe",
    dataset=("Dataset to save answers to", "positional", None, str),
)
def my_custom_recipe(dataset, view_id="text"):
    # You probably have a file on disk instead, but just as an example
    stream = [
        {"text": "this is an example"}, 
        {"text": "and this is another one"}
    ]

    # This is a function that "pretends" to apply a generative model
    def apply_model(examples):
        for ex in examples:
            yield {
                **set_hashes(ex), 
                "field_placeholder": fake_model(ex['text']),
                "field_label": f"Original text: {ex['text']}",
                "field_rows": 2
            }

    return {
        "dataset": dataset,
        "view_id": "text_input",
        "stream": apply_model(stream)
    }

When I run this, I can see "generated" text appear in the model card.

Let me know if this works for you :slightly_smiling_face:

By the way, if you're interested in evaluating generative models, you might also gain some inspiration from our A/B featured page.

2 Likes

Great! Thank you. It's almost there. The problem is the placeholder text can only be replaced, not edited. My generative model outputs a list and I'm hoping to just remove/add/alter that list.

Could you give a tanglible example of what you'd like? Maybe an example of what your generative model would produce together with the editing operation you're thinking of? There are other interfaces that might help, but an example would help me understand what might work best here.