# Natural Language Inference (Sentence Pair Evaluation)

**URL:** <https://support.prodi.gy/t/natural-language-inference-sentence-pair-evaluation/2568>\
**Category:** Uncategorized\
**Tags:** usage, textcat, custom\
**Created:** [February 19, 2020, 10:12pm UTC](https://support.prodi.gy/t/natural-language-inference-sentence-pair-evaluation/2568 "2020-02-19T22:12:51Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![r-vi](https://avatars.discourse-cdn.com/v4/letter/r/bbe5ce/32.png) [@r-vi](https://support.prodi.gy/u/r-vi)\
**Post date:** [February 19, 2020, 10:12pm UTC](https://support.prodi.gy/t/natural-language-inference-sentence-pair-evaluation/2568/1 "2020-02-19T22:12:51Z")

</div>

Hello,

I am trying to use Prodigy to annotate a sentence-pair dataset for NLI (Entailment).  
The data is formatted as a jsonl file with two separate sentences per line.

```
{"sentence1":"This is the first sentence I want to compare","sentence2":"This is the second sentence I want to compare"}

```

I then want to label the relation between these two sentences with 1 of 3 labels: A, B, and C.

I am trying to use the choice recipe for this, but I am having trouble getting it set up.  
Does anyone have any tips/pointers for this? I am pretty unfamiliar with prodigy so anything will help. Thanks!

---

<div class="post-metadata">

**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\
**Post date:** [February 20, 2020, 11:20am UTC](https://support.prodi.gy/t/natural-language-inference-sentence-pair-evaluation/2568/2 "2020-02-20T11:20:40Z")

</div>

Hi! In case you haven't seen it yet, the custom recipes docs have some examples with multiple-choice options: [https://prodi.gy/docs/custom-recipes#example-choice](https://prodi.gy/docs/custom-recipes#example-choice)

You should be able to just adapt the sentiment annotation example and include your labels. An easy way to present your two sentences would be to add a `"html"` key to each task. For example, here you show them separated by line breaks:

```python
task["html"] = f"{task['sentence1']}<br /><br />{task['sentence2']}"

```

Of course, you could also add a divider line in between, or show the texts in two columns – this depends on what you prefer and how important nice formatting is to you 🙂