I have a model directory that works wonderfully in the anaconda environment I built it in - if I try to load that model as either the raw directory (tarred and zipped) or as a package (following pkg build instructions) I get the error in the subject line - for python3.6 and python2.7 - looking for a clue here as I’m stumped.
File “…/site-packages/thinc/neural/util.py”, line 48, in copy_array
dst[:] = src
ValueError: could not broadcast input array from shape (64) into shape (64,1)
Hmm! Questions:
What model is it – the text classifier, NER, …?
Was it trained and loaded on CPU, or is a GPU involved somewhere?
Are any vectors loaded?
What version of Thinc? Are you sure it’s the same version in your environment and when you load back later?
Don’t know if it’s significant or not - but the base environment is Ubuntu 16, anaconda py3 - non-working environments - Ubuntu 16 anaconda 2.7, and Ubuntu 14 anaconda py2 3.6 virtual environment.
Seems to me it would be some change in numpy - but I’ve tried to ensure consistent versions (1.11.3) there as well.
Full trace on fresh 3.6 env -
import spacy
nlp = spacy.load("models/my-model")
Traceback (most recent call last):
File "", line 1, in
File "/opt/anaconda/envs/py36/lib/python3.6/site-packages/spacy/init.py", line 19, in load
return util.load_model(name, **overrides)
File "/opt/anaconda/envs/py36/lib/python3.6/site-packages/spacy/util.py", line 117, in load_model
return load_model_from_path(Path(name), **overrides)
File "/opt/anaconda/envs/py36/lib/python3.6/site-packages/spacy/util.py", line 159, in load_model_from_path
return nlp.from_disk(model_path)
File "/opt/anaconda/envs/py36/lib/python3.6/site-packages/spacy/language.py", line 638, in from_disk
util.from_disk(path, deserializers, exclude)
File "/opt/anaconda/envs/py36/lib/python3.6/site-packages/spacy/util.py", line 522, in from_disk
reader(path / key)
File "/opt/anaconda/envs/py36/lib/python3.6/site-packages/spacy/language.py", line 634, in
deserializers[name] = lambda p, proc=proc: proc.from_disk(p, vocab=False)
File "pipeline.pyx", line 212, in spacy.pipeline.Pipe.from_disk
File "/opt/anaconda/envs/py36/lib/python3.6/site-packages/spacy/util.py", line 522, in from_disk
reader(path / key)
File "pipeline.pyx", line 205, in spacy.pipeline.Pipe.from_disk.load_model
File "/opt/anaconda/envs/py36/lib/python3.6/site-packages/thinc/neural/_classes/model.py", line 352, in from_bytes
copy_array(dest, param[b'value'])
File "/opt/anaconda/envs/py36/lib/python3.6/site-packages/thinc/neural/util.py", line 48, in copy_array
dst[:] = src
ValueError: could not broadcast input array from shape (64) into shape (64,1)
Installed models (spaCy v2.0.9)
/opt/anaconda/envs/py36/lib/python3.6/site-packages/spacy
TYPE NAME MODEL VERSION
package en-vectors-web-lg en_vectors_web_lg 2.0.0 ✔
package en-core-web-sm en_core_web_sm 2.0.0 ✔
package en-core-web-lg en_core_web_lg 2.0.0 ✔
link en_core_web_sm en_core_web_sm 2.0.0 ✔
link en_core_web_lg en_core_web_lg 2.0.0 ✔
link en_vectors_web_lg en_vectors_web_lg 2.0.0 ✔
Of note - above trace is python 3.6.4 on ubuntu 14 - same model loads fine ubuntu 16 python 3.6.0
Python 3.6.0 |Anaconda custom (64-bit)| (default, Dec 23 2016, 12:22:00)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
import spacy
nlp = spacy.load("models/my-model")
Python 3.6.4 |Anaconda, Inc.| (default, Mar 13 2018, 01:15:57)
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
import spacy
nlp = spacy.load("models/my-model")
Traceback (most recent call last):....
Exported my Ubuntu 16 3.6.0 environment to yml file and was able to get it working on Ubuntu 14 - let me know if you need anything to narrow the issue down.
Hi Matthew
I tried to run spacy train to train NER model with en_vectors_web_lg as a base model and tok2vec_cd8_model289.bin tok2vec model as a pre-trained language model but I got the following error:
Please advise
Thanks 
