Prodigy Output Visualization, Dependencies Structure Training Help

[ines (Ines Montani)]

@hmousa961

I would like about 2 things. First, I trained a model and would like to test it on plain text. How can I do that and is there a recipe to visualize the output text?

Hi! If you've trained a spaCy model, I think the easiest way would be to just load it with spaCy directly and look at the annotations you're interested in. spaCy also comes with built-in visualizers that you can run from your terminal or in a Jupyter notebook: Visualizers · spaCy Usage Documentation

import spacy
from spacy import displacy

nlp = spacy.load("/path/to/model/you/just/trained")
doc = nlp("Some plain text goes here")
# Print the output (choose the attributes you're interested in)
print([(token.text, token.pos_, token.dep_) for token in doc])
print([(ent.text, ent.label_) for ent in doc.ents])
# Visualize
displacy.serve(doc, style="dep")  # or displacy.render if you're in a notebook
displacy.serve(doc, style="ent")

If you want a more interactive demo you can play with or share with others, check out the spacy-streamlit package:

GitHub - explosion/spacy-streamlit: 👑 spaCy building blocks and visualizers...

@hmousa961

Secondly, I have a text file with dependencies for ontonotes dataset for arabic language. Is there a specific structure to follow in order to create the json file for dependencies to be imported to prodigy?

Yes, see my comment on this thread:

Ontonotes 5 Prodigy Training JSON Format

Hi! You can see an example of the JSON format that Prodigy uses here: https://prodi.gy/docs/api-interfaces#ner_manual It uses character offsets and lets you provide a list of "tokens" as well that the spans can reference. If you need to convert token-based tags to offsets, you could use the helper functions spaCy provides: https://prodi.gy/docs/named-entity-recognition#tip-biluo-offsets I'd say that importing your data into Prodigy really only makes sense if you're planning on annotating it, e…