feature request: pre-trained model evaluation recipe

Can you add a pre-trained model evaluation recipe that prints the F1 score? I have several evaluation datasets and want to quickly get an F1 score for each after I train a new model. I realize the code is straightforward, but I bet I am not the only one who would appreciate a built-in recipe for this.

Yes, that's a good point!

For NER, here's a custom recipe written by a user – this is pretty much what you're looking for, right?

Yeah, pretty much. Essentially the output of the batch-train recipes minus the training bits.

1 Like