ValueError: could not broadcast input array from shape (128) into shape (96)

, , ,
6

Hi @ines,

I was following along thener-food-ingredients tutorial. Installed prodigy and en_core_web_lg using the following commands in a virtual environment.

python -m pip install ./models/prodigy-1.9.9-cp36.cp37.cp38-cp36m.cp37m.cp38-macosx_10_13_x86_64.whl

python -m spacy download en_core_web_lg

Command

I came across the same issue when running the following command:

prodigy train ner food_data en_core_web_lg --init-tok2vec ./models/tok2vec_cd8_model289.bin --output ./models/tmp_model --eval-split 0.2

Output

Traceback (most recent call last):
  File "/Users/raqib/.pyenv/versions/3.7.0/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/Users/raqib/.pyenv/versions/3.7.0/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/raqib/nlp/ner-food-ingredients/.venv/lib/python3.7/site-packages/prodigy/__main__.py", line 60, in <module>
    controller = recipe(*args, use_plac=True)
  File "cython_src/prodigy/core.pyx", line 213, in prodigy.core.recipe.recipe_decorator.recipe_proxy
  File "/Users/raqib/nlp/ner-food-ingredients/.venv/lib/python3.7/site-packages/plac_core.py", line 367, in call
    cmd, result = parser.consume(arglist)
  File "/Users/raqib/nlp/ner-food-ingredients/.venv/lib/python3.7/site-packages/plac_core.py", line 232, in consume
    return cmd, self.func(*(args + varargs + extraopts), **kwargs)
  File "/Users/raqib/nlp/ner-food-ingredients/.venv/lib/python3.7/site-packages/prodigy/recipes/train.py", line 139, in train
    load_pretrained_tok2vec(pipe, init_tok2vec, require=True)
  File "cython_src/prodigy/util.pyx", line 520, in prodigy.util.load_pretrained_tok2vec
  File "/Users/raqib/nlp/ner-food-ingredients/.venv/lib/python3.7/site-packages/thinc/neural/_classes/model.py", line 396, in from_bytes
    copy_array(dest, param[b"value"])
  File "/Users/raqib/nlp/ner-food-ingredients/.venv/lib/python3.7/site-packages/thinc/neural/util.py", line 145, in copy_array
    dst[:] = src
ValueError: could not broadcast input array from shape (128) into shape (96)

Other details

pip show spacy
Name: spacy
Version: 2.2.4
Location: /Users/raqib/nlp/ner-food-ingredients/.venv/lib/python3.8/site-packages
Required-by: prodigy, en-core-web-lg

pip show en-core-web-lg
Name: en-core-web-lg
Version: 2.2.0
Location: /Users/raqib/nlp/ner-food-ingredients/.venv/lib/python3.8/site-packages
Requires: spacy

pip show prodigy
Name: prodigy
Version: 1.9.9
Location: /Users/raqib/nlp/ner-food-ingredients/.venv/lib/python3.8/site-packages

What have I tried thus far:

  1. Tried version 2.2.5 and 2.2.0 of en-core-web-lg
  2. Tried changing python version from 3.8 to 3.7.

Would appreciate any pointers.