I'm used to deploying my microservices behind AWS Lambda because of the scalability and cost-control it gives. I haven't been able to run Prodigy behind Lambda, and I'm going to explain why.
Why this matters
If you can't deploy Prodigy behind a FaaS, it greatly limits its scaling potential at least for the web/inference parts of the app.
The training part could (arguably ?) be better-suited behind some sort of CRON-based PaaS (e.g. AWS Batch), although with the new 10GB RAM / 6 vCPU Lambdas most "small" datasets should fit easily in Lambda as well.
There are "half-baked" solutions (such as AWS Fargate or equivalent PaaS solutions), but it seems the only trivial way to deploy Prodigy right now is in a serverful fashion (e.g. on AWS EC2 or ECS), in which case it begs the question : what server size should I choose ? how will that scale with my usage and data growth ?
The better solution would be to be able to deploy Prodigy in a serverless architecture.
What I tried
From what I understand, Prodigy pretty much has to run via its CLI :
I dug a bit into the
app.py file inside the wheel and I see that on top of the FastAPI
App (which could easily be served behind Lambda), you've defined some mandatory
That means that catching the app (e.g.
from prodigy.app import app) and somehow running it behind something other than
uvicorn will fail because the DB won't get initialized properly... shame, that seemed like a good try
Is there a proven way to run the whole Prodigy
app (or even just the REST API, which is what I mostly care about) behind a different Python webserver than uvicorn ?