Fine-grained Sentiment Analysis

Added the following cat labels using your train_textcat.py example.
cats = [
{"Positive": 5,
"Somewhat Positive": 4,
"Neutral": 3,
"Somewhat Negative": 2,
"Negative": 1
} for y in labels]

For the evaluate code section, how should I be testing the scores?

for i, doc in enumerate(textcat.pipe(docs)):
    gold = cats[i]
    for label, score in doc.cats.items():
        if label not in gold:
            continue
        if label == "NEGATIVE":
            continue
        if score >= 0.5 and gold[label] >= 0.5:
            tp += 1.0
        elif score >= 0.5 and gold[label] < 0.5:
            fp += 1.0
        elif score < 0.5 and gold[label] < 0.5:
            tn += 1
        elif score < 0.5 and gold[label] >= 0.5:
            fn += 1

Hi,

In general we can't support spaCy usage questions on this forum, so in future you'll need to ask this type of question on a forum such as StackOverflow. I've answered it this time but I've also unlisted the thread, and in future we won't be able to help here.

The classifier assumes that the labels are independent categories, rather than an ordinal scale. There are different loss functions for things like ordinals, but in fact for a small number of categories I would usually stick to classification as it's simpler and usually performs better.