Installing prodigy in conda on Windows 10?

Installing prodigy in conda on windows 10 works fine up until the mmh3 package which requires compilation.

Running

pip install prodigy-1.3.0-cp35.cp36-cp35m.cp36m-win_amd64.whl

After

conda install spacy

Leads to

Failed building wheel for mmh3

Is there any way around this (does the pymmh3 work as well for the dependency? does mmh3 have a conda recipe for amd64 somewhere?) or does windows 10 need to be setup to compile all of the packages?

Not that this is off any use but my conda list also you can see mumurhash is already installed

anaconda-client           1.6.3                    py36_0    conda-forge
Augmentor                 0.1.6                     <pip>
bkcharts                  0.2                      py36_0    conda-forge
bleach                    1.5.0                    py36_0    conda-forge
bokeh                     0.12.4                   py36_0
bolt-python               0.7.1                     <pip>
bottleneck                1.2.0               np112py36_0    conda-forge
ca-certificates           2017.4.17                     0    conda-forge
certifi                   2017.4.17                py36_0    conda-forge
cffi                      1.10.0                   py36_0    conda-forge
click                     6.7                      py36_0    conda-forge
cloudpickle               0.3.1                    py36_0    conda-forge
clyent                    1.2.1                    py36_0    conda-forge
colorama                  0.3.9                    py36_0    conda-forge
curl                      7.52.1                   vc14_0  [vc14]  conda-forge
cycler                    0.10.0                   py36_0    conda-forge
cymem                     1.31.2              py36_vc14_0  [vc14]  conda-forge
cytoolz                   0.8.2                    py36_0    conda-forge
dash                      0.17.5                    <pip>
dask                      0.15.0                   py36_0    conda-forge
datashader                0.4.0                    py36_0
datashape                 0.5.4                    py36_0
decorator                 4.1.2                     <pip>
decorator                 4.0.11                   py36_0    conda-forge
dill                      0.2.7.1                  py36_0    conda-forge
distributed               1.17.1                   py36_0    conda-forge
entrypoints               0.2.2                    py36_1    conda-forge
falcon                    1.4.1                     <pip>
Flask                     0.12.2                    <pip>
Flask-Compress            1.4.0                     <pip>
Flask-SeaSurf             0.2.2                     <pip>
freetype                  2.7                      vc14_1  [vc14]  conda-forge
ftfy                      4.4.2                    py36_0    conda-forge
future                    0.16.0                    <pip>
graphviz                  2.38.0                        4    conda-forge
graphviz                  0.8.1                     <pip>
h5netcdf                  0.3.1                    py36_0    conda-forge
h5py                      2.7.0               np112py36_1    conda-forge
hdf4                      4.2.12                   vc14_0  [vc14]  conda-forge
hdf5                      1.8.18                   vc14_0  [vc14]  conda-forge
heapdict                  1.0.0                    py36_0    conda-forge
holoviews                 1.6.2                    py36_0    ioam
html5lib                  0.999                    py36_0    conda-forge
hug                       2.4.0                     <pip>
hug-middleware-cors       1.0.0                     <pip>
icu                       58.1                     vc14_1  [vc14]  conda-forge
imageio                   2.2.0                    py36_0    conda-forge
ipykernel                 4.6.1                    py36_0    conda-forge
ipyrmd                    0.4.3                     <pip>
ipython                   6.1.0                    py36_0    conda-forge
ipython_genutils          0.2.0                    py36_0    conda-forge
ipywidgets                6.0.0                    py36_0    conda-forge
itsdangerous              0.24                      <pip>
jedi                      0.10.0                   py36_0    conda-forge
jinja2                    2.9.5                    py36_0    conda-forge
jpeg                      9b                       vc14_0  [vc14]  conda-forge
jsonschema                2.5.1                    py36_0    conda-forge
jupyter                   1.0.0                    py36_0    conda-forge
jupyter_client            5.0.1                    py36_0    conda-forge
jupyter_console           5.1.0                    py36_0    conda-forge
jupyter_core              4.3.0                    py36_0    conda-forge
Keras                     2.0.5                     <pip>
libgpuarray               0.6.5           np112py36_vc14_0  [vc14]  conda-forge
libnetcdf                 4.4.1.1                  vc14_5  [vc14]  conda-forge
libpng                    1.6.28                   vc14_0  [vc14]  conda-forge
libtiff                   4.0.6                    vc14_7  [vc14]  conda-forge
libwebp                   0.5.2                    vc14_7  [vc14]  conda-forge
llvmlite                  0.19.0                   py36_0
locket                    0.2.0                    py36_1    conda-forge
mako                      1.0.6                    py36_0    conda-forge
markupsafe                1.0                      py36_0    conda-forge
matplotlib                2.0.0               np112py36_2    conda-forge
mistune                   0.7.4                    py36_0    conda-forge
mkl                       2017.0.1                      0
msgpack-numpy             0.4.1                      py_0    conda-forge
msgpack-python            0.4.8                    py36_0    conda-forge
multipledispatch          0.4.9                    py36_0    conda-forge
murmurhash                0.28.0              py36_vc14_0  [vc14]  conda-forge
nbconvert                 5.2.1                    py36_1    conda-forge
nbformat                  4.3.0                    py36_0    conda-forge
netcdf4                   1.2.9               np112py36_0    conda-forge
networkx                  2.0                       <pip>
networkx                  1.11                     py36_0    conda-forge
nibabel                   2.1.0                     <pip>
notebook                  5.0.0                    py36_0    conda-forge
numba                     0.34.0              np112py36_5
numpy                     1.13.0                    <pip>
numpy                     1.12.1                   py36_0
observations              0.1.4                     <pip>
odo                       0.5.0                    py36_1
olefile                   0.44                     py36_0    conda-forge
opencv                    3.2.0             np112py36_203    conda-forge
openssl                   1.0.2h                   vc14_3  [vc14]  conda-forge
pandas                    0.20.2              np112py36_1    conda-forge
pandoc                    1.19.2                        0    conda-forge
pandocfilters             1.4.1                    py36_0    conda-forge
param                     1.5.1                    py36_0    ioam
partd                     0.3.8                    py36_0    conda-forge
peewee                    2.10.2                    <pip>
pickleshare               0.7.3                    py36_0    conda-forge
pillow                    4.1.1                    py36_0    conda-forge
pip                       9.0.1                    py36_0    conda-forge
plac                      0.9.6                     <pip>
plac                      0.9.6                    py36_0    conda-forge
plotly                    2.0.11                    <pip>
preshed                   1.0.0               py36_vc14_0  [vc14]  conda-forge
prompt_toolkit            1.0.14                   py36_0    conda-forge
psutil                    5.2.1                    py36_0    conda-forge
pycparser                 2.17                     py36_0    conda-forge
pygments                  2.2.0                    py36_0    conda-forge
pyparsing                 2.2.0                    py36_0    conda-forge
pyqt                      4.11.4                   py36_2    conda-forge
pyreadline                2.1                      py36_0
pyro-ppl                  0.1.1                     <pip>
python                    3.6.1                         3    conda-forge
python-dateutil           2.6.0                    py36_0    conda-forge
python-mimeparse          1.6.0                     <pip>
pytorch                   0.2.1           py36he6bf560_0.2.1cu80    peterjc123
pytz                      2017.2                   py36_0    conda-forge
pywavelets                0.5.2               np112py36_0    conda-forge
pyyaml                    3.12                     py36_1    conda-forge
pyzmq                     16.0.2                   py36_2    conda-forge
qt                        4.8.7                    vc14_7  [vc14]  conda-forge
qtconsole                 4.3.0                    py36_0    conda-forge
regex                     2017.11.09               py36_0    conda-forge
requests                  2.13.0                   py36_0    conda-forge
scikit-image              0.13.0              np112py36_0    conda-forge
scikit-learn              0.18.1              np112py36_1
scipy                     0.19.0              np112py36_0
seaborn                   0.7.1                     <pip>
setuptools                33.1.1                   py36_0    conda-forge
simplegeneric             0.8.1                    py36_0    conda-forge
SimpleITK                 1.0.1                     <pip>
sip                       4.18                     py36_1    conda-forge
six                       1.10.0                   py36_1    conda-forge
sortedcontainers          1.5.3                    py36_0    conda-forge
spacy                     2.0.7               py36_vc14_2  [vc14]  conda-forge
tblib                     1.3.2                    py36_0    conda-forge
termcolor                 1.1.0                    py36_1    conda-forge
testpath                  0.3                      py36_0    conda-forge
theano                    0.9.0                    py36_0    conda-forge
thinc                     6.10.2              py36_vc14_0  [vc14]  conda-forge
toolz                     0.8.2                    py36_0    conda-forge
torchbiomed               0.0.1                     <pip>
torchvision               0.1.9                     <pip>
tornado                   4.5.1                    py36_0    conda-forge
tqdm                      4.19.5                     py_0    conda-forge
tqdm                      4.14.0                    <pip>
traitlets                 4.3.2                    py36_0    conda-forge
ujson                     1.35                     py36_0    conda-forge
vc                        14                            0    conda-forge
vs2015_runtime            14.0.25420                    0    conda-forge
waitress                  1.1.0                     <pip>
wcwidth                   0.1.7                    py36_0    conda-forge
webencodings              0.5                      py36_0    conda-forge
Werkzeug                  0.12.2                    <pip>
wheel                     0.29.0                   py36_0    conda-forge
widgetsnbextension        2.0.0                    py36_0    conda-forge
wincertstore              0.2                      py36_0    conda-forge
wrapt                     1.10.11                  py36_0    conda-forge
wrapt                     1.10.11                   <pip>
xarray                    0.9.6                    py36_0    conda-forge
yaml                      0.1.6                    vc14_0  [vc14]  conda-forge
zict                      0.1.2                    py36_0    conda-forge
zlib                      1.2.11                   vc14_0  [vc14]  conda-forge

Thanks for reporting this. It’s been a long journey to make sure we keep spaCy wheels maintained on conda-forge, to help exactly your use-case: making sure Windows users don’t need to install a build environment. Having this little library missing definitely feels like snatching defeat from the jaws of victory!

As a workaround, you could create an mmh3.py file that exposes a function hash(string) — that should be all you need. Prodigy only imports mmh3 in a couple of places, and only calls that one function.

I think the following should work:

# Call this mmh3, and place it somewhere it can be imported, e.g. in your current working directory.
from spacy.strings import hash_string as hash

The only difference is that mmh3.hash returns 32 bit signed integers, while spacy.strings.hash_string() returns 64 bit unsigned integers. We used mmh3.hash because there were some problems serialising and deserialising the 64 bit values on some platforms. In particular, you should check that the hashes are coming back the same after the examples are passed to the update() callback.

If you have problems, you should be able to use any other library or method for hashing strings into 32 bit ints. The exact hashing isn’t referenced anywhere, it just needs to be internally consistent.