Does prodigy work in a completely airtight environment (i.e. AWS instance w/o internet access)?

, ,
1

We have a task which requires us to work on text in a super-secure environment, without any internet access whatsoever.

The question is, after all prerequisites are installed, will Prodigy be able to work without outside internet connection?

Thanks!

2

Yes, once it’s installed, Prodigy won’t make any outgoing requests and will work completely offline in an airtight environment :slightly_smiling_face:

3

Hi @ines,

Is it possible to install Prodigy without internet connection at all? I’m working on a secure server without access to internet (apart from white listed channels like conda and pip). While I can start chasing our IT admins to install Prodigy for me, it normally takes ages.

Is there a simple way to install Prodigy myself without internet connection? I can upload to the secure server any packages I want and then install them locally. Thanks!

4

@Andrey Yes, that should be no problem – you’d just have to install all of Prodigy’s dependencies separately first, before you install the Prodigy wheel with --no-deps. If you’re using the latest version of Prodigy, that would be the following packages:

# Our dependencies
spacy>=2.1.0,<2.2.0
wasabi>=0.1.3,<1.1.0
srsly>=0.0.6,<1.1.0
# Other dependencies
toolz>=0.8.2,<=1.0.0
plac>=0.9.6,<1.0.0
hug>=2.4.1,<2.4.9
waitress>=1.0.0,<1.3.0
requests>=2.0.0,<3.0.0
peewee>=2.10.1,<3.0.0
jsonschema>=2.6.0,<3.0.0
cachetools>=2.1.0
pyjwt>=1.6.1,<2.0.0

However, what can this pretty tedious is that many of those packages also have their own dependents, and so on. For an accurate picture of all dependencies, you can install Prodigy in a clean virtualenv and run pip list.

To make this easier, you might want to check out .pex files. We use them a lot internally and they essentially package a whole Python environment into a single file. So you can install Prodigy and whatever else you need locally, create a .pex, copy that single file (!) over to your airtight machine and then execute it instead of the Python interpreter. (The only thing your machine needs is a system Python installation). For example:

your_environment.pex -m prodigy stats
1 Like
5

Hi @ines, fantastic and many thanks for detailed answer! I will try to do it myself and let you know if I run into problems. Being dependent on IT admins it’s a nightmare.