Allowing for additional comments in the rel.manual recipe

1

Hi all,

I recently started playing around with Prodigy and I must say that I am really enjoying myself. The sheer volume of available recipes, online documentation and active support community made creating my own annotation pipeline really easy. However, I have reached the point where I'm stuck and I wondering if I'm missing something.

The problem is that I want the annotator to be able to select the span of an entity, add links between these spans, and also provide the option for additional comments by the annotator. So essentially, I would like a recipe like the rel.manual recipe, but with an additional comment box.

I have tried making a custom recipe using the newly added blocks feature. In my mind, using the following block structure should get me what I want:

blocks = [
        {"view_id": "relations", "span-label": ["SPAN1", "SPAN2"], "labels": ["REL1","REL2"] },
        {"view_id": "text_input", "field_rows": 3, "field_label": "Add description"}
    ]

However, the problem is that unlike the rel.manual recipe, the block with the relations view ID does not allow for creating new spans. It only allows for creating relations between existing tokens. I have tried looking at different view IDs but none seem to be more applicable than what I want to achieve (see attached screenshot for the list of permitted view IDs).

image
image1306×75 3.72 KB

So, my question is whether I am missing something. Is is possible to add a comment box to the rel.manual recipe using the blocks feature? Or has this not been implemented (yet)?

Also, if you have any suggestions for a different approach, I am happy to learn!

Thanks in advance!

Cheers,
Ritten

2

hi @Ritten!

Thanks for your thoughtful question and welcome to the Prodigy community :wave:

That's wonderful to hear. We really appreciate it :pray: We've got lots of new updates on the way :slight_smile:

So are you looking for this?

localhost_8080_ (31)
localhost_8080_ (31)1932×1708 205 KB

Here's how I created it.

First, you can find all of the built-in recipes locally. Run prodigy stats and then find the Location: path of where Prodigy is installed. Open that folder, and then look for the recipes/rel.py. This is the rel.manual recipe.

I'd recommend you copy that file (say rel_modified.py) and create a custom recipe off it, so place it somewhere convenient.

Next, open your copied recipe and in bottom of the recipe add your blocks:

# added these blocks
blocks = [
    {"view_id": "relations"},
    {"view_id": "text_input", "field_id": "comments", "field_rows": 3, "field_label": "Explain your decision"}
]

return {
    "view_id": "blocks",
    "dataset": dataset,
    "stream": stream,
    "exclude": exclude,
    "config": {
        "lang": nlp.lang,
        "labels": label,
        "blocks": blocks, # added this blocks
        "relations_span_labels": span_label,
        "exclude_by": "input",
        "wrap_relations": wrap,
        "custom_theme": {"cardMaxWidth": "90%"},
        "hide_relation_arrow": hide_arrow_heads,
        "auto_count_stream": True,
    },
}

Then at the top, modify your prodigy imports to:

# previously was ..components.stream import get_stream
from prodigy.components.stream import get_stream
from prodigy.core import recipe
from prodigy.models.matcher import (
    create_matcher,
    create_phrase_matcher,
    parse_pattern_name,
    parse_patterns,
)
from prodigy.types import RecipeSettingsType, StreamType
from prodigy.util import get_labels, load_model, log, msg, split_string

Then boom, you should be good to go:

python3 -m prodigy rel.manual rel-ex blank:en news_headlines.jsonl --label REL1,REL2 --span-label ENTY1,ENTY2 -F rel_modified.py

I didn't post the full recipe because I ran this on the new v1.12 alpha (which I highly recommend) and the stream in the recipes has changed a little and I don't want to confuse anyone with an older version.

Hope this helps!

3

@ryanwesslen Thank you! This is precisely what I was looking for. And after updating to version v1.12 (I was working with v1.11.4), it is working like a charm!

1 Like