Seeing the same issue installing spacy 2.0.0a15 on a clean install of python 3.6, reverting to 2.0.0a13 also fixed the problem.
We just pushed a new spaCy update yesterday,
v2.0.0a16, which uses Thinc
v6.9.0 and should fix the compatibility issues. It requires the latest version of Prodigy that was released yesterday,
v0.3.0, so if you install that cleanly, it should use all the latest dependencies!
Hi @ines, thank for your answer.
Do I have to install prodigy to use spacy
pip install spacy-nightly
in a clean (conda) python 3.6 environment on MacOS install spacy
v2.0.0a16 with thinc
spacy download en
still fail as above.
End of log:
Successfully installed certifi-2017.7.27.1 chardet-3.0.4 cymem-1.31.2 cytoolz-0.8.2 dill-0.2.7.1 ftfy-4.4.3 html5lib-0.999999999 idna-2.6 msgpack-numpy-0.4.1 msgpack-python-0.4.8 murmurhash-0.28.0 numpy-1.13.3 pathlib-1.0.1 plac-0.9.6 preshed-1.0.0 regex-2017.4.5 requests-2.18.4 six-1.11.0 spacy-nightly-2.0.0a16 termcolor-1.1.0 thinc-6.9.0 toolz-0.8.2 tqdm-4.19.1.post1 ujson-1.35 urllib3-1.22 wcwidth-0.1.7 webencodings-0.5.1 wrapt-1.10.11 (analytics) MacBook-Pro-de-Thomas:hf_analytics thomaswolf$ spacy download en Traceback (most recent call last): File "/Users/thomaswolf/miniconda3/envs/analytics/lib/python3.6/runpy.py", line 183, in _run_module_as_main mod_name, mod_spec, code = _get_module_details(mod_name, _Error) File "/Users/thomaswolf/miniconda3/envs/analytics/lib/python3.6/runpy.py", line 142, in _get_module_details return _get_module_details(pkg_main_name, error) File "/Users/thomaswolf/miniconda3/envs/analytics/lib/python3.6/runpy.py", line 109, in _get_module_details __import__(pkg_name) File "/Users/thomaswolf/miniconda3/envs/analytics/lib/python3.6/site-packages/spacy/__init__.py", line 4, in <module> from .cli.info import info as cli_info File "/Users/thomaswolf/miniconda3/envs/analytics/lib/python3.6/site-packages/spacy/cli/__init__.py", line 6, in <module> from .train import train File "/Users/thomaswolf/miniconda3/envs/analytics/lib/python3.6/site-packages/spacy/cli/train.py", line 17, in <module> from ..tokens.doc import Doc File "/Users/thomaswolf/miniconda3/envs/analytics/lib/python3.6/site-packages/spacy/tokens/__init__.py", line 1, in <module> from .doc import Doc File "morphology.pxd", line 25, in init spacy.tokens.doc SystemError: initialization of morphology failed without raising an exception
Ah, no – but you need spaCy v2.0.0a16 to use the latest version of Prodigy.
And this is strange – just as a sanity check, do you get the same error when you run:
python -m spacy download en
Or compare the output of both:
spacy info python -m spacy info
If this does the trick, there might be an issue with the
spacy executable that should actually only alias
python -m spacy. For example, it could have ended up on the wrong Python path. (So maybe we should update the docs again to always recommend
python -m as we used to, since it still seems to be the safest option, especially when working with virtual environments etc.)
Hmm all these commands gives me the same error.
Actually opening python (3.6) and trying
import spacy (before the model is downloaded) also fails as
(analytics-test) MacBook-Pro-de-Thomas:hf_analytics thomaswolf$ python Python 3.6.2 |Continuum Analytics, Inc.| (default, Jul 20 2017, 13:14:59) [GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import spacy Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/Users/thomaswolf/miniconda3/envs/analytics-test/lib/python3.6/site-packages/spacy/__init__.py", line 4, in <module> from .cli.info import info as cli_info File "/Users/thomaswolf/miniconda3/envs/analytics-test/lib/python3.6/site-packages/spacy/cli/__init__.py", line 6, in <module> from .train import train File "/Users/thomaswolf/miniconda3/envs/analytics-test/lib/python3.6/site-packages/spacy/cli/train.py", line 17, in <module> from ..tokens.doc import Doc File "/Users/thomaswolf/miniconda3/envs/analytics-test/lib/python3.6/site-packages/spacy/tokens/__init__.py", line 1, in <module> from .doc import Doc File "morphology.pxd", line 25, in init spacy.tokens.doc SystemError: initialization of morphology failed without raising an exception
Tried in a clean conda environment again.
@ines, I tried in a clean env on ubuntu and it worked without any problem so I am suspecting it has something to do with MacOS (maybe in the line of this Cython issue: https://github.com/cython/cython/issues/1551 (I am a Cython newbie) ?
Do you have any recommendation in terms of MacOS setup? I use Miniconda env.
Hmm! So to summarize the chain here, we go:
.pyx source files -> Cython = .cpp files .cpp files -> GCC/clang = .so extensions .so extensions + python = import spacy
.cpp step happens on our server, and you download the generated
.cpp files in the spaCy package. When you do
pip install spacy-nightly, your compiler then turns them into then
.so extensions, so you can import the library.
Until recently, we’ve been using a fairly old version of Cython to generate the
.cpp files. I suspect that the newer version of Cython is generating code that your compiler is unhappy with. The suggestion in that issue looks like a likely solution — if we can change the compiler flag used when installing the module, that might work.
Thanks @honnibal for the Cython introduction!
So I tried to install spaCy from source by compiling the
develop branch of the repo.
I switched the
-O0 as advised in the above github issue and also tried to remove the open_mp flags.
But that didn’t change the error messages which is still
File "morphology.pxd", line 25, in init spacy.tokens.doc SystemError: initialization of morphology failed without raising an exception
Here is the part of the pip install log that relates to the compilation of morphology.pxd (I can post the full log if you want)
building 'spacy.morphology' extension gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/thomaswolf/miniconda3/envs/analytics-test/include -arch x86_64 -I/Users/thomaswolf/miniconda3/envs/analytics-test/include -arch x86_64 -I/Users/thomaswolf/miniconda3/envs/analytics-test/include/python3.6m -I/private/var/folders/yx/cw8n_njx3js5jksyw_qlp8p00000gn/T/pip-pp89oel9-build/include -I/Users/thomaswolf/miniconda3/envs/analytics-test/include/python3.6m -c spacy/morphology.cpp -o build/temp.macosx-10.7-x86_64-3.6/spacy/morphology.o -O0 -Wno-strict-prototypes -Wno-unused-function In file included from spacy/morphology.cpp:505: In file included from /private/var/folders/yx/cw8n_njx3js5jksyw_qlp8p00000gn/T/pip-pp89oel9-build/include/numpy/arrayobject.h:15: In file included from /private/var/folders/yx/cw8n_njx3js5jksyw_qlp8p00000gn/T/pip-pp89oel9-build/include/numpy/ndarrayobject.h:17: In file included from /private/var/folders/yx/cw8n_njx3js5jksyw_qlp8p00000gn/T/pip-pp89oel9-build/include/numpy/ndarraytypes.h:1728: /private/var/folders/yx/cw8n_njx3js5jksyw_qlp8p00000gn/T/pip-pp89oel9-build/include/numpy/npy_deprecated_api.h:11:2: warning: "Using deprecated NumPy API, disable it by #defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-W#warnings] #warning "Using deprecated NumPy API, disable it by #defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" ^ 1 warning generated. g++ -bundle -undefined dynamic_lookup -L/Users/thomaswolf/miniconda3/envs/analytics-test/lib -L/Users/thomaswolf/miniconda3/envs/analytics-test/lib -arch x86_64 build/temp.macosx-10.7-x86_64-3.6/spacy/morphology.o -L/Users/thomaswolf/miniconda3/envs/analytics-test/lib -o build/lib.macosx-10.7-x86_64-3.6/spacy/morphology.cpython-36m-darwin.so -Wl,-rpath,@loader_path/../spacy/platform/darwin/lib clang: warning: libstdc++ is deprecated; move to libc++ with a minimum deployment target of OS X 10.9
I am on MacOSX Sierra 10.12.4.
Shall I revert to your previsously used versino of cython ?
I am also having the issue with Spacy-Nightly V2.0.0A16 as reported above.
I recently attempted to upgrade to the newest version, which resulted in the errors described above. Now I can not reinstall spacy and prodigy without the following error thrown by prodigy:
/Users/crivera5/.virtual_envs/spacy_python3/bin/python: Error while finding spec for 'prodigy.__main__' (<class 'ValueError'>: thinc.extra.search.MaxViolation has the wrong size, try recompiling); 'prodigy' is a package and cannot be directly executed
I have the issue when installing prodigy 0.4.0 with spacy-nightly v.2.0.17
clean environment. python 3.6. macos 10.11.6
I get the
SystemError: initialization of morphology failed without raising an exception error when executing
prodigy or executing
spacy or importing
We’ve got this fixed in spaCy now, so the next update will take care of the problem.
Morphology file had a a long
enum, listing out the morphological features in the Universal Dependencies schema. Recent versions of Cython cause a lot of code to be generated from this construction, which causes the error on some platforms. I’ve found an easy workaround, that also reduces our compile times a bit.
How do we fix that problem? I am having it too. Just updated spacy to current and downloaded newest version of Prodigy. It downloaded nightly. When I run that it gives me the error.
@crivera The current version of Prodigy is not yet compatible with the latest spaCy – only with
spacy-nightly==v2.0.0a17. But we already have the new Prodigy wheels ready – just need to test them, then we’ll send out another update!
See this thread for more details: Installing problem with prodigy-0.4.0 on Ubuntu 16.0.4 LTS
I tried reinstalling with a fresh environment. I still get
mod_name, mod_spec, code = _get_module_details(mod_name) File "/usr/local/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/runpy.py", line 118, in _get_module_details return _get_module_details(pkg_main_name) File "/usr/local/Cellar/python3/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/runpy.py", line 104, in _get_module_details spec = importlib.util.find_spec(mod_name) File "/Users/crivera5/.virtual_envs/spacy_python3/lib/python3.5/importlib/util.py", line 88, in find_spec parent = __import__(parent_name, fromlist=['__path__']) File "/Users/crivera5/.virtual_envs/spacy_python3/lib/python3.5/site-packages/prodigy/__init__.py", line 4, in <module> from . import recipes, about # noqa File "/Users/crivera5/.virtual_envs/spacy_python3/lib/python3.5/site-packages/prodigy/recipes/__init__.py", line 4, in <module> from . import ner, textcat, compare, terms, generic # noqa File "/Users/crivera5/.virtual_envs/spacy_python3/lib/python3.5/site-packages/prodigy/recipes/ner.py", line 7, in <module> import spacy File "/Users/crivera5/.virtual_envs/spacy_python3/lib/python3.5/site-packages/spacy/__init__.py", line 4, in <module> from .cli.info import info as cli_info File "/Users/crivera5/.virtual_envs/spacy_python3/lib/python3.5/site-packages/spacy/cli/__init__.py", line 6, in <module> from .train import train File "/Users/crivera5/.virtual_envs/spacy_python3/lib/python3.5/site-packages/spacy/cli/train.py", line 17, in <module> from ..tokens.doc import Doc File "/Users/crivera5/.virtual_envs/spacy_python3/lib/python3.5/site-packages/spacy/tokens/__init__.py", line 1, in <module> from .doc import Doc File "morphology.pxd", line 25, in init spacy.tokens.doc```
@crivera This is a very strange error indeed We just sent out a new version that makes Prodigy compatible with spaCy v2.0. Could you download this one and try again?
Which did you update? Prodigy or Spacy? I will try tomorrow. Thanks!!!
@crivera Both – and Thinc But as I said, you don’t have to worry about any dependencies. Just download the new Prodigy wheel, install it in a fresh environment, and everything else will be installed automatically. This is also the best way to avoid installation issues and dependency mismatches.
Installation finally worked again with the newest version. Thanks!