we have built a workflow where we use custom spacy model configurations to initialise models from the very start of an annotation project (for example to include custom tokenisation). We are initialising the models with blank textcat components if it is a textcat annotation, and we are running into a problem.
If you use
textcat.correct with 2 exclusive classes and a blank model in the loop, the model suggests both labels with 0.5 score. While the UI will only display one radio button selected, the
getChoices function in the React code will in fact return both labels, so if the user doesn't make any change and just clicks accept, the saved annotation will have both labels. This of course causes errors down the line when trying to train a model.
Currently we are using a workaround of setting the threshold to 0.51, but I wanted to report it anyway as it's clearly a bug, even if it's perhaps a relatively unlikely edge case for most users.