Error running on aws / beanstalk

Hi. I am troubleshooting an issue I’m running into in attempting to import prodigy from a django app view on an ec2 instance in an elastic beanstalk environment on aws.

prodigy==1.5.0

Installation of the wheel appears to succeed, but as soon as the line

import prodigy

is encountered, I get the following error:

ImportError “cannot import name ‘DependencyParser’”

from prodigy init.py -> recipes.py

And after moving some of the imports around inside the prodigy source it seems that none of the cython-compiled parts of the library are importing. I of course fresh-installed after this experimentation. SSHing into the instance does not trigger these errors, only when served from the webserver.

ImportError at /dataset/25/pgy_tasks/
cannot import name 'PRODIGY_HOME'
Request Method:	GET
  Django Version:	2.1.1
Exception Type:	ImportError
Exception Value:	
cannot import name 'PRODIGY_HOME'
Exception Location:	/opt/python/run/venv/local/lib64/python3.6/site-packages/prodigy/components/db.py in <module>, line 8
Python Executable:	/opt/python/run/venv/bin/python3
Python Version:	3.6.5

Any advice or speculation as to how to approach this would be would be highly appreciated.

Thanks for the report! First, could you try upgrading to the latest v1.5.1 and see if the erorr persists?

Your analysis makes sense and in general, that type of error indicates that Python can’t find the compiled source. This can happen if the platform / version identifiers of the files don’t match – e.g. if you were running the whole thing with Python 3.7 (while the file names only specify 3.5 and 3.6). But that doesn’t seem to be the case here :thinking:

Yes, upgraded to 1.5.1 and am seeing the same error.

Another issue I came across was that the 3rd party package pathlib, required by spacy, ends up throwing an exception from within prodigy when prodigy calls pathlib.Path.home().

prodigy is assuming the stdlib api of pathlib- pathlib is in the python stdlib in 3.5+, but the external packaged version does not implement the home classmethod.

I have not encountered this problem locally, only on ec2/amazon linux.

I really think you’re using a different version of Python when you execute pip, from when you’re actually running the command. I’m not sure quite how it’s happening, but that’s definitely what’s going on.