# Prodigy review recipe not entirely clear to me

**URL:** <https://support.prodi.gy/t/prodigy-review-recipe-not-entirely-clear-to-me/6044>\
**Category:** Uncategorized\
**Created:** [October 25, 2022, 8:28am UTC](https://support.prodi.gy/t/prodigy-review-recipe-not-entirely-clear-to-me/6044 "2022-10-25T08:28:55Z")\
**Posts on this page:** 1\
**Showing post:** 4

<div class="post-metadata">

**Author:** ![koaning](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/koaning/32/230_2.png) [@koaning](https://support.prodi.gy/u/koaning)\
**Post date:** [October 27, 2022, 1:51pm UTC](https://support.prodi.gy/t/prodigy-review-recipe-not-entirely-clear-to-me/6044/4 "2022-10-27T13:51:42Z")

</div>

So let's extend the example.

I'll be annotating this on-theme example:

```python
{"text": "a wood chuck could chuck a lot of wood if a wood chuck could chuck wood"} 

```

Again, I'll run:

```python
PRODIGY_CONFIG_OVERRIDES='{"feed_overlap": true}' python -m prodigy textcat.manual issue-6044 examples.jsonl --label truthy

```

 ![CleanShot 2022-10-27 at 15.54.03](https://us1.discourse-cdn.com/flex020/uploads/prodigy/original/2X/4/42f041c66a578489cc1526da7f81cf3de1caf1dc.png)

And now I'll annotate this with the `guybrush` user. This user did not appear before. And for good measure, I'll show the annotation from `db-out`:

```python
> python -m prodigy db-out issue-6044 | grep guybrush
{"text":"a wood chuck could chuck a lot of wood if a wood chuck could chuck wood","_input_hash":-1690856185,"_task_hash":1885086500,"label":"truthy","_view_id":"classification","answer":"accept","_timestamp":1666878830,"_annotator_id":"issue-6044-guybrush","_session_id":"issue-6044-guybrush"}

```

Let's now see what happens when we review this item.

## Without auto accept

```python
prodigy review issue-6044-reviewed issue-6044

```

 ![CleanShot 2022-10-27 at 16.01.21](https://us1.discourse-cdn.com/flex020/uploads/prodigy/original/2X/c/c89a719c362b82445c94bd5229e0dfdac8e9caf8.png)

I don't make an annotation, but the interface does just show a single annotator just fine. Note that `db-out`, as expected, doesn't have anything from Guybrush.

```python
python -m prodigy db-out issue-6044-reviewed | grep guybrush
# EMPTY! 

```

## With auto accept.

```python
prodigy review issue-6044-reviewed issue-6044 --auto-accept

```

It doens't show the annotation now!

 ![CleanShot 2022-10-27 at 16.03.32](https://us1.discourse-cdn.com/flex020/uploads/prodigy/original/2X/c/c0530512c33077561bbac805783bca2cf82b1957.png)

But! Does it appear in the reviewed dataset automatically, like before?

```python
python -m prodigy db-out issue-6044-reviewed | grep guybrush
# STILL EMPTY! 

```

The example with "wood chucks" doesn't appear in `db-out` because it's never been annotated by more than one person.

## Back to Your Issue

It could be that there are hard duplicates in your data because the data got merged in a wrong way earlier. If that's the case, you might be able to alleviate the pain if you try out the `--rehash` flag in the [db-merge](https://prodi.gy/docs/recipes#db-merge) recipe and re-run.

Another thing you can consider is to just do some analysis in a Jupyter notebook. If you're savvy with Pandas, you should be able to load in the jsonl file via;

```python
import pandas as pd 

pd.read_json("path.jsonl", lines=True)

```

Alternatively, you might enjoy my [clumper util library](https://koaning.github.io/clumper/api/clumper.html#clumper.clump.Clumper.read_jsonl). It's a lot slower than pandas, but it's typically more expressive for nested lists of dictionaries.

---

_[View the full topic](https://support.prodi.gy/t/prodigy-review-recipe-not-entirely-clear-to-me/6044)._