Can we open the Spacy's NER model predictions in the annotation tool

sriiitr · August 19, 2019, 6:44am

We built an NER model using Prodigy's ner.batch-train recipe. I am just thinking if I can open the predictions made by the model on the annotation tool again so that I can fix the prediction and start using them for improving the models further.

Any thoughts?

ines · August 19, 2019, 9:09am

Yes, that's pretty much exactly the idea of the ner.make-gold workflow: you load a model, run it over the text, add the predicted entities as "spans" so you can view them all in the interface and edit them if necessary.

You can use the built-in recipe, or check out the implementation here to see how it works (and customise it if you want):

github.com

explosion/prodigy-recipes/blob/master/ner/ner_make_gold.py

# coding: utf8
from __future__ import unicode_literals

import prodigy
from prodigy.components.loaders import JSONL
from prodigy.components.preprocess import add_tokens
from prodigy.util import split_string, set_hashes
import spacy
import copy


def make_tasks(nlp, stream, labels):
    """Add a 'spans' key to each example, with predicted entities."""
    # Process the stream using spaCy's nlp.pipe, which yields doc objects.
    # If as_tuples=True is set, you can pass in (text, context) tuples.
    texts = ((eg['text'], eg) for eg in stream)
    for doc, eg in nlp.pipe(texts, as_tuples=True):
        task = copy.deepcopy(eg)
        spans = []
        for ent in doc.ents:

This file has been truncated. show original

Topic		Replies	Views
Detailed evaluation of NER model trained from Prodigy annotations usage , ner , training	6	700	December 14, 2021
uploading spacy model for ner.make-gold annotation usage , ner , spacy , solved	4	485	August 8, 2019
Formatting Prodigy annotations for evaluation of external NER models using spaCy usage , ner , spacy	4	584	April 13, 2022
How do I use prodigy as a purely annotation tool with no underlying SpaCy model? usage	1	1581	April 27, 2018
spacy model predictions as training input on prodigy usage , ner , spacy	2	550	October 2, 2020

Can we open the Spacy's NER model predictions in the annotation tool

Related Topics