Can we open the Spacy's NER model predictions in the annotation tool

sriiitr · August 19, 2019, 6:44am

We built an NER model using Prodigy's ner.batch-train recipe. I am just thinking if I can open the predictions made by the model on the annotation tool again so that I can fix the prediction and start using them for improving the models further.

Any thoughts?

ines · August 19, 2019, 9:09am

Yes, that's pretty much exactly the idea of the ner.make-gold workflow: you load a model, run it over the text, add the predicted entities as "spans" so you can view them all in the interface and edit them if necessary.

You can use the built-in recipe, or check out the implementation here to see how it works (and customise it if you want):

github.com

explosion/prodigy-recipes/blob/master/ner/ner_make_gold.py

# coding: utf8
from __future__ import unicode_literals

import prodigy
from prodigy.components.loaders import JSONL
from prodigy.components.preprocess import add_tokens
from prodigy.util import split_string, set_hashes
import spacy
import copy


def make_tasks(nlp, stream, labels):
    """Add a 'spans' key to each example, with predicted entities."""
    # Process the stream using spaCy's nlp.pipe, which yields doc objects.
    # If as_tuples=True is set, you can pass in (text, context) tuples.
    texts = ((eg['text'], eg) for eg in stream)
    for doc, eg in nlp.pipe(texts, as_tuples=True):
        task = copy.deepcopy(eg)
        spans = []
        for ent in doc.ents:

This file has been truncated. show original

Topic		Replies	Views
Detailed evaluation of NER model trained from Prodigy annotations usage , ner , training	6	708	December 14, 2021
How does model-in-the-loop in ner.correct work? usage , ner , solved	2	340	November 13, 2022
uploading spacy model for ner.make-gold annotation usage , ner , spacy , solved	4	487	August 8, 2019
Updating an NER model using the annotation tool ner , spacy	6	393	June 5, 2023
Prodigy with AllenNLP model usage , allennlp	3	917	March 4, 2021

Can we open the Spacy's NER model predictions in the annotation tool

Related Topics