Annotation score drops

21

I'm confused. You're using the same eval for both OS. I'm looking for files like:

  • train_data_vnc.jsonl
  • eval_data_vnc.jsonl
  • train_data_rdp.jsonl
  • eval_data_rdp.jsonl
22

eval_data_rdp.jsonl (488.4 KB)
eval_data_vnc.jsonl (701.2 KB)
train_data_rdp.jsonl (560.1 KB)
train_data_vnc.jsonl (517.9 KB)

python3 -m prodigy train --ner train_data_vnc,eval:eval_data_vnc

E # LOSS TOK2VEC LOSS NER ENTS_F ENTS_P ENTS_R SCORE

0 0 0.00 654.69 0.00 0.00 0.00 0.00
20 200 3864.20 7380.13 7.59 9.09 6.52 0.08
40 400 320.77 144.65 8.82 13.64 6.52 0.09
60 600 2.64 3.25 8.96 14.29 6.52 0.09
80 800 8.56 6.42 9.23 15.79 6.52 0.09
100 1000 0.00 0.00 12.31 21.05 8.70 0.12
120 1200 0.00 0.00 12.31 21.05 8.70 0.12
140 1400 0.08 0.07 12.50 22.22 8.70 0.12
160 1600 0.00 0.00 12.50 22.22 8.70 0.12
180 1800 7.31 3.99 14.93 23.81 10.87 0.15
200 2000 363.29 150.25 9.88 11.43 8.70 0.10
220 2200 70.98 26.37 2.99 4.76 2.17 0.03
240 2400 0.00 0.00 3.08 5.26 2.17 0.03
260 2600 0.00 0.00 3.08 5.26 2.17 0.03
280 2800 0.00 0.00 3.08 5.26 2.17 0.03
300 3000 0.00 0.00 3.08 5.26 2.17 0.03
320 3200 0.00 0.00 3.08 5.26 2.17 0.03
340 3400 56.03 9.65 9.38 16.67 6.52 0.09

python3 -m prodigy train --ner train_data_rdp,eval:eval_data_rdp

E # LOSS TOK2VEC LOSS NER ENTS_F ENTS_P ENTS_R SCORE

0 0 0.00 348.15 0.00 0.00 0.00 0.00
20 200 9447.36 9589.19 22.45 37.93 15.94 0.22
40 400 1.93 8.19 19.61 30.30 14.49 0.20
60 600 0.00 0.00 19.42 29.41 14.49 0.19
80 800 0.00 0.00 18.37 31.03 13.04 0.18
100 1000 0.00 0.00 20.00 32.26 14.49 0.20
120 1200 0.00 0.00 20.00 32.26 14.49 0.20
140 1400 0.00 0.00 18.35 25.00 14.49 0.18
160 1600 0.00 0.00 18.18 30.00 13.04 0.18
180 1800 0.00 0.00 18.18 30.00 13.04 0.18