Annotating dialogues with Prodigy

[ines (Ines Montani)]

@n.elvira

Although we have a script that makes these calculations, it would be interesting to simplify this part of the workflow, e.g. being able to have multiple messages within the interface without having to use multiple blocks or joining all messages in a single text.

Hi! If your messages are all separate and your goal is to display multiple of them together, I do think those are the best options – at least, I couldn't think of an alternative way to present the content with the existing or custom interfaces.

If you want to avoid recalculating the offsets, it might be simpler to use plain text / HTML blocks for the text shown before and after. You can format those however you like and you could use an html_template to reference specific fields in the data. So your input could be {"before": "...", "after": "...", "text": "..."}.

@n.elvira

Also, when we tried to add pre-annotated entities with our own NER model, it identifies entities correctly but the labels are misplaced when we open Prodigy and we suspect this might have something to do with the tokens that we have disabled (?).

The disabled tokens shouldn't affect the rendered entity spans. Maybe it's an off-by-one error? The token_end index of the span should be inclusive, and maybe also double-check that the spans are provided in the order they occur in the text.

@n.elvira

The entities + relations recipe could be useful for this task, but these dialogues are not very easy to work with in the current recipe. Newline breaks would help, but they do not seem to work in our texts when we wrap them (we're using version 1.10.5).

Prodigy v1.10.6 supports rendering newlines in the relations UI: newlines in relations annotation - #17 by ines

Also, on the topic of annotating and linking sentences: a similar use case came up in another thread a while ago and I suggested an experimental approach of using symbols/numbers to represent sentences or sentence fragments. So instead of connecting very long blocks with very little visual gain, you'd be looking at the full dialog and then annotating relationships in an abstract representation of it. Using relations interface for large texts - #4 by ines

@ines

I think if the goal is to connect these long text fragments, there's not really a benefit in placing arrows on top of the actual inline text. The visual gain from this is absolutely minimal (if not negative) and once you have more than one relation, it's pretty much always going to get messy.

One simple approach I just thought of that could work and that you could easily implement with the existing interfaces: split the content and alignment/relation annotation into two blocks so you can read the original text separately and then connect references to the sentences/paragraphs instead of the actual text.

It's possible that this is less relevant for your use case if your texts are fairly short. But it could still be worth experimenting with.