Prodigy NER model evaluation and custom evaluation scripts

[ines (Ines Montani)]

This example of a custom recipe might also be relevant:

Evaluating Precision and Recall of NER

In case others like me come looking for a basic scoring recipe, here is what I cooked up. It doesn’t consider threshold, but it evaluates model accuracy without re-training and can output either PRF or the standard prodigy score scheme. import spacy import spacy.scorer from prodigy.components.db import connect from prodigy.core import recipe, recipe_args from prodigy.models.ner import EntityRecognizer, merge_spans from prodigy.util import log from prodigy.components.preprocess import split_sen…