Training recipe displays inconsistent epoch intervals between lines for evaluation metrics

,
1

I'm using the train recipe on prodigy to train an NER model. I tried using en_core_web_md as a baseline model, but I noticed that the epoch interval between lines of evaluation metrics during training is quite inconsistent.

Below are the metrics displayed when running python -m prodigy train ./ --ner master -m en_core_web_md -es 0.1 --training.max_epochs 40:

E    #       LOSS TOK2VEC  LOSS NER  ENTS_F  ENTS_P  ENTS_R  SCORE
---  ------  ------------  --------  ------  ------  ------  ------
  0       0          0.00     57.19    0.00    0.00    0.00    0.00
 12    1000          0.00  20532.62   71.58   72.63   70.55    0.72

Here's the metrics when running python -m prodigy train ./ --ner master -es 0.1 --training.max_epochs 40, without the base en_core_web_md model:

E    #       LOSS TOK2VEC  LOSS NER  ENTS_F  ENTS_P  ENTS_R  SCORE
---  ------  ------------  --------  ------  ------  ------  ------
  0       0          0.00     63.73   11.20    7.63   21.06    0.11
  1     200        434.25   8324.10   61.81   68.32   56.44    0.62
  3     400        729.20   4525.82   62.86   60.65   65.24    0.63
  6     600        803.97   2809.32   65.91   66.81   65.03    0.66
  8     800        759.58   1685.51   67.16   70.16   64.42    0.67
 12    1000        796.65   1334.16   64.39   63.19   65.64    0.64
 17    1200        765.09   1127.31   70.06   71.25   68.92    0.70
 22    1400        745.81    906.06   69.40   69.47   69.33    0.69
 29    1600        753.50    878.62   70.38   71.04   69.73    0.70
 37    1800        661.92    806.50   70.14   70.14   70.14    0.70

Why is there a difference in the epoch display interval? I don't see an option to adjust this interval in the teach recipe.

2

Hi! I just had a look and I think what's happening here is that when you use a base model, the training config settings are copied over from the base model. In the en_core_web_md pipeline, the eval_frequency in the [training] block is set to 1000. spaCy's default, which is used when you generate a new config from scratch, however is 200.

You can overwrite this setting on the CLI via --training.eval_frequency, so you can set it to be more or less detailed, depending on what you prefer.