(originally posted on https://stackoverflow.com/questions/65218606/is-it-possible-to-find-uncertainties-of-spacy-pos-tags)
I am looking for a way to figure out the probabilities of certain tags set by spacy. So far, with some help from StackOverflow, I got to the following piece of code:
import spacy
nlp = spacy.load('en_core_web_sm')
text = "This is an example sentence for the Spacy tagger."
doc = nlp(text)
docs = nlp(text, disable=['tagger'])
scores, tensors = nlp.tagger.predict([docs])
print(scores)
probs = tensors[0]
for p in probs:
print(p, max(p), p.tolist().index(max(p)))
Now this outputs a long list of numbers (96 of them, in this case) that look like probabilities, but I have no idea what these numbers mean exactly, and I have no idea how I can then turn this into an actual tag and the uncertainty attached to that tag. Is there any resource I can check to find what these numbers mean?