Hello there!
I am writing summaries for some (long) text data in a text-input interface. I wonder if there is a way to freeze the text-input interface/box which I put above the text data, so that when I scroll through the text data, especially for long ones, I can still look at my current summary, in a way similar to the frozen blue banner of entity names in the NER interface?
Thank you!
Hi Solo,
I'm assuming you're using a custom recipe here? I think the easiest method to get what you want might be use a custom HTML template with some CSS rules that make the text to summarise scrollable, but not your text input.
Let's say, for demo purposes, that this is my data stored in an examples.jsonl file.
{"text": "this text is very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very long"}
{"text": "this text is short"}
The first example is meant to demonstrate that we're dealing with a long example that we'd like a scrollbar to appear for. This can be achieved by rendering it inside of an HTML element that uses a fixed height and a overflow: auto; css property.
Here's a custom recipe that uses a template for this.
import prodigy
from prodigy.components.loaders import JSONL
@prodigy.recipe(
"summary",
dataset=("Dataset to save answers to", "positional", None, str),
examples=("Examples to load from disk", "positional", None, str),
)
def summary(dataset, examples):
stream = JSONL(examples)
def add_html(examples):
for ex in examples:
ex["html"] = f"""
<b>Main Text</b>
<div style="overflow: auto; height: 200px;" >
<p>{ex['text']}</p>
</div>
<b>Summary</b>
"""
yield ex
return {
"dataset": dataset,
"stream": add_html(stream),
"view_id": "blocks",
"config": {
"blocks": [
{"view_id": "html"},
{"view_id": "text_input", "field_rows": 3},
]
}
}
There's a few things to note in this recipe.
add_html function I'm building some HTML to render using an f-string. The text is placed in an element that has overflow: auto; set and has a height predefined as well.view_id to blocks at the bottom of the recipe and in the config.blocks setting I'm showing a list of elements to render. The HTML will be rendered together with a text field that has 3 rows.I can now run this recipe like so;
python -m prodigy summary issue-5948 examples.jsonl -F recipe.py
This is what the interface now looks like:
When you save your annotation you can run db-out to see what's been stored on our behalf.
> python -m prodigy db-out issue-5948 | jq
This will yield:
{
"text": "this text is very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very long",
"html": "\n <b>Main Text</b>\n <div style=\"overflow: auto; height: 200px;\" >\n <p>this text is very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very very long</p>\n </div>\n <b>Summary</b>\n ",
"_input_hash": 1123455936,
"_task_hash": 2085830771,
"_view_id": "blocks",
"user_input": "yeh this is long",
"answer": "accept",
"_timestamp": 1663762192
}
You'll notice that we are storing both the text and the generated HTML. This can be a bit "heavy" since we're storing the same information twice. You could implement a before_db-callback if you want to filter out the HTML element after it's been annotated if you want to prevent this.
Hi Vincent,
Thank you so much for the detailed instructions (and the tips on filtering using before_db callback). This is most helpful!
I have a quick follow-up question: how can I also make the text in a spans_manual interface fixed height and scrollable? In the config of my custom recipe, I tried both
"global_css": ".prodigy-root[data-prodigy-view-id='spans_manual'] .prodigy-content { overflow: auto; height: 400px}"
and
"global_css": ".prodigy-spans { overflow: auto; height: 400px}"
, but neither of them worked.
This one
"global_css": ".prodigy-content { overflow: auto; height: 400px}"
does work (i.e. making the text in spans_manual scrollable), but it made every other interface the same height too (which is undesired). How can I make just the spans_manual scrollable?
FYI, the spans_manual interface is among many others in a block:
blocks = [
{"view_id": "text", "text": "some text here"},
{"view_id": "html"},
{"view_id": "text_input", "field_id": "text_input_1", "field_rows": 3, "field_label": "summary", "field_placeholder": "write a summary"},
{"view_id": "spans_manual"},
]
Thank you!
With regards to making the UI scrollable for NER, I think you may have indeed stumbled apon a bug, which we've now logged internally.
One comment though; having to scroll while annotating tends to be a terrible experience, so you might be interested in preprocessing your data upfront so that the texts are much smaller before they are annotated. Specifically, you might want to consider the split_sentences function.
To use the function you'll either need a custom recipe to apply this step on the fly on your stream, but you can also apply it to your data beforehand in a Python script to generate a file like split_examples.jsonl.
The split_sentence suggestions sounds great! But in my use case (customer calls), I would like to have the whole text data/call displayed at once so that I can refer to the context and go back and forth between different sentences. Anyway, for now I can work fine without the scrolling feature.
Thank you again for all the suggestions! I look forward to the next release.
Hello there. It seems that the spans_manual interface is still not scrollable in 1.11.1. Any chance that you plan to fix it in the future? Thanks!
We're working on a v1.12 release right now but we've decided that this issue will be addressed in Prodigy v2. That version will get a remake of the front-end which will make it much easier for us to fix issues like this.