Annotated Dataset and NER task with Prodigy

[ryanwesslen]

hi @sudarshan85!

Sorry for the delay. We're trying to close out old tickets.

@sudarshan85

ner.correct seems to be doing sentence segmentation and showing only one sentence at a time. I would like it to show the whole document as it would in ner.manual. How do I achieve this?

By default, ner.correct does sentence segmentation (unlike ner.manual. You can turn it off by adding --unsegmented.

@sudarshan85

Despite adding --exclude option in the original command, I see samples from the original annotations which gives a feeling of "starting from scratch" and unfortunately SME time is valuable. Why is this happening?

That's tough to confirm. Let me go through a reproducible example of what should happen.

Start with this source file:
nyt_text_dedup.jsonl (18.5 KB)

Step 1: Label 10 records into dataset ner_correct1

python -m prodigy ner.correct ner_correct1 en_core_web_sm nyt_text_dedup.jsonl --label LOC

I then labeled the first 10 records. You can see them by running:

$ python -m prodigy print-dataset ner_correct1

Step 2: Rerun but use --exclude to exclude records in ner_correct1

python3 -m prodigy ner.correct ner_correct2 en_core_web_sm data/nyt_text_dedup.jsonl --exclude ner_correct1 --label LOC

Notice it starts on record 10 (see metadata in bottom right). Therefore, it skipped the first 10 records.

@sudarshan85

What is the sequence of documents that is displayed when running ner.manual vs ner.correct? Do they follow the same sequence as presented in the jsonl file. I know that ner.manual does, but not sure about ner.correct.

Yes. ner.manual and ner.correct will use based on order of documents. This is different than ner.teach, which uses active learning and will alter the order of the documents based on uncertainty scoring.

Let us know if you have any other questions!