Prodigy for labeling similar sentences

1

Hi, I was wondering can we use the prodigy to label whether two sentences are semantically similar to each other or not. I have been searching through to find a similar problem but was not able to. Can anyone please guide me is there a way in prodigy for such a problem base.
For eg.
Sentence 1: Hi, How are you today?
Sentence 2: Hi, How are doing today?

So they should be labelled as similar.
Any help will be appreciated.

2

This sound for me like an A/B task. It is mentioned on the following site:

Maybe that is a starting point?

Furthermore, for me your usecase sounds a bit like machine translation annotation. Here and there one wants to know whether two texts match each other semantically. Maybe this is a helpful search term too.

1 Like
3

Yes, the A/B interface @MaxG87 mentioned should be useful for this task!

Alternatively, you could also build your own interface using the "html" view and a custom recipe. For example, let’s imagine your input looks like this:

{"sentence1": "Hi, How are you today?", "sentence2": "Hi, How are doing today?"}

You could then use a html_template that looks like this:

<strong>Sentence 1:</strong><br />
{{sentence1}}
<br /><br />
<strong>Sentence 2:</strong><br />
{{sentence2}}

The HTML above is obviously just dummy code to show the possibilities – you could also lay it out in a table or however else you want. The {{sentence1}} and {{sentence2}} variables will be automatically populated from your data, and the result you get back in your dataset will be the original input, plus an "answer" mapping to either "accept", "reject" or "ignore".

2 Likes