Added the following cat labels using your train_textcat.py example.
cats = [
{"Positive": 5,
"Somewhat Positive": 4,
"Neutral": 3,
"Somewhat Negative": 2,
"Negative": 1
} for y in labels]
For the evaluate code section, how should I be testing the scores?
for i, doc in enumerate(textcat.pipe(docs)):
gold = cats[i]
for label, score in doc.cats.items():
if label not in gold:
continue
if label == "NEGATIVE":
continue
if score >= 0.5 and gold[label] >= 0.5:
tp += 1.0
elif score >= 0.5 and gold[label] < 0.5:
fp += 1.0
elif score < 0.5 and gold[label] < 0.5:
tn += 1
elif score < 0.5 and gold[label] >= 0.5:
fn += 1