In this example the score is it the f1 score between the predicted label and the true label ?
And for the batch size, should I set it to 1?
Hi! The score is whatever your model predicts and what you want to use to score the examples and select/filter them based on the score, e.g. focusing on uncertain, high or low scores. Typically, this would be the F1 score (but in theory, you could user other computed scores here).
The batch size is the number of examples that will be processed and used to update the model at a time. You normally want to use a larger batch size than 1 here because otherwise, your model would only ever be updated with one single example at a time. For most model implementations, this isn't very efficient. 10 is a good default to start with. Depending on the model implementation, you might have to experiment a bit here to find the batch size that works best.
1 Like