Label multiple text at the same time

Hi I'm working on a text classification problem where I take into account history of the data and label the status. I have 2 labels (Active/ Wait) and 7 sentences that belong to one instance.

In prodigy I want to look at all those 7 sentences at once and label each of those with either (Active/Wait). Is there a custom recipe that I can use to label it and further use it for a text classification problem?

hi @AakankshaP,

Thanks for your question and welcome to the Prodigy community :wave:

I'm not sure I understand your setup. I have several questions:

  • What do you mean by "instance"? I don't understand what it means for 7 sentences to belong to one instance. Did you mean label?
  • How is your input data currently structured? For example, are all of your input data the same, e.g., all on the sentence? Is this how your data comes raw or did you do any pre-processing steps for this?
  • Why 7 sentences? 7 seems like an arbitrary number.
  • Related, is there anything that links these 7 sentences together? For example, are all of the sentences from the same paragraph? Do you have metadata (e.g., a paragraph ID or page ID) that links the sentences?
  • What's the purpose of annotating the 7 sentences together? Are you trying to "save time" by having the labeler annotate multiple sentences at a time? Any context for why you want this setup would be extremely helpful.
  • Is it possible to do this step as a pre-processing step, i.e., you combine documents within your input (source) file? This would likely be the easiest way to implement.

Alternatively, if you could give hypothetical examples, that'll likely help us a lot.

It's important to mention that Prodigy was optimized for fast, simple decisions one at a time. This is especially true in binary text classification where the annotator can leverage the keyboard shortcuts to speed through examples. By adding multiple sentences, you may find it actually slows down annotators. For binary classification problems, you can even modify Prodigy to have swipes -- kinda like dating apps -- that can be helpful on mobile devices.

Hi thank you so much for your reply. Here are the answers for your questions.

When I say instance it means history of a particular event.
For eg:
Patient is under observation [I want to label this as wait]
Patient is being treated [I want to label this as active]

The data is updated in real-time and there are time stamp attached to it. Each patient has a unique id and the details are being updated.

The number 7 is variable, depending upon the number of entries

No sentences are not structured as paragraphs, rather they are just 1-2 line sentences

The purpose of annotating the text together is to keep into account of the history of events as some complicated text cannot be easily labeled without the context

The purpose of this activity is to understand wait/active time, combining the text will not help us identify the difference between wait/active because in meta we include the timestamp to label.

hi @AakankshaP!

Thanks for the background.

Ah gotcha. That makes sense.

This reminds me of this custom recipe I created. A user wanted to annotate sentences with a binary classification (like your wait/active) but wanted to provide the full context of the paragraph. In addition, their documents were sequential, so I created this custom recipe to start from the first paragraph, loop through each sentence with a highlight so the annotator knew which sentence was being annotated, then move to the next paragraph.


I know you mentioned your sentences are not structured as paragraphs -- but perhaps you could combine the sentences together you want to be shown together in the UI, but then use the highlighting to represent which sentence the user is actually annotating at a time.

The link includes a GitHub gist I created with the code and example.

Does this help?

Hi Thank you so much for your reply I came across this recipe, but we do not want the highlighted interface.

I was thinking some interface like this but without the title and each text has 2 options to select (active/wait) or maybe under each sentence we have the options available.

I came across this issue and picked up the recipe from here

1 Like

Hi is there a way to update the above custom recipe to have 2 options across each sentence? My task is not a binary classification that's why I need two labels as the third one is pre-defined