KeyError: 'score' in ner.teach While trying to do this on pre-annotated data.

[kunalr (Kjr)]

Hi @ines

I am following this to load the pre-annotated data.
Does Prodigy load pre-annotated data? - Prodigy Support

the data format I have pushed in db is same as mentioned in the link.
"{
"text":"Pumped 18 M3 1,45 SG EZ spot pill, displaced same with 18 M3 1,4 SG mud. Spotted 10,3 M3 around BHA, left 7,7 M3 inside pipe.",
"spans":[
{"start":24,"end":33,"token_start":6,"token_end":7,"label":"EQUIPMENT"},
{"start":120,"end":124,"token_start":28,"token_end":28,"label":"DEVICE"}
]
}"

• Since ner.batch-train is depreciated I am using ner train.
• and ner.teach as:

Blockquote

prodigy ner.teach scrapinghub_golden trained_models/scrapinghub_model_v1/ scrapinghub_goldendata.jsonl --label CAST_CREW,PERSON,EXECUTIVE_NAME,CONTENTNAME,COMPANY,DATE,LOCATION,LICENSE_TERR,MONEY,COUNTRY

Blockquote
And getting KeyError: 'score' for this.

had a look on this, but eliminating spans from .jsonl will not bring my pre-annotations.

KeyError: 'score' in ner.teach

Hi: I'm trying to run ner.teach on a custom ner model with some custom tags. command line: prodigy ner.teach <dataset> <model_name> data/<annotation_file>.jsonl I've tried appending the explicit labels. I've also loaded the model in a terminal and validated that it was able to successfully run nlp.evaluate() on some gold docs. the ner.correct model runs successfully with the same model and same dataset. Irrespectively of anything I do, prodigy will ultimately throw the following stac…

2. I had also tried using:
   PRODIGY_PORT=5000 prodigy ner.manual scrapinghub_golden trained_models/scrapinghub_model_v1/ scrapinghub_goldendata.jsonl --label CAST_CREW,PERSON,EXECUTIVE_NAME,CONTENTNAME,CARDINAL,ORDINAL,COMPANY,DATE,LOCATION,DEAL_VAL,LICENSE_TERR,COUNTRY,MONEY,BOOK_NAME,WORKING_TITLE

this generates "No available task" page.

Could you please help?

BR,
K

[ines (Ines Montani)]

@kunalr

had a look on this, but eliminating spans from .jsonl will not bring my pre-annotations.

The problem here goes a bit deeper: the ner.teach recipe is designed to show you binary suggestions based on the model's predictions. So there's not really a point in feeding it pre-annotated data because it's all about showing you very specific selected predictions. If you want to annotate pre-labelled examples and correct them, you should use a recipe like ner.manual instead.

@kunalr

this generates "No available task" page.

This is shown if there are no examples left to annotate. This can happen if your dataset (e.g. scrapinghub_golden) already contains annotations for all of the examples you're loading in. In that case, Prodigy will skip the annotations that already exist. If you want to re-annotate your data, you should use a new and empty dataset.