I am using Prodigy to annotate datasets for a sentiment classification task. I am getting the following issue:
I run the prodigy command to start a server for annotating responses from a file called 'sentiment.csv'
The prodigy server shown running status.
I try to access the server from the {server-ip}:5100 endpoint. The interface works when loaded the first time.
If I open the same endpoint next time, it shows "Oops, something went wrong" and then the command that was running the server is killed. It shows no logs whatsoever.
I have already tried changing the prodigy port, enable verbose logging and even running it from a docker container, but the same issue persists. Also tried it on different python versions, virtualenv/conda. Currently, port is set to 5100 from the prodigy.json config file.
Hi! That's definitely strange. And this only happens when you deploy it on Azure, not locally? If you set PRODIGY_LOGGING=basic, is there anything in the log that looks suspicious before it dies? And I guess you're not loading a recipe with a large model that could cause the machine to run out of memory, right?
Yeah, it only happens on Azure server. The same command and flow works fine when used locally.
No, I'm using the builtin "prodigy textcat.manual ..." recipe for annotation only. The spacy command to auto-create models based on this are not used. It's a simple sentiment classification task.
There is a file called sentiment.csv with a column "text" and sentences in that column. A dataset called "sentiment" is already present with old annotations.
Thanks for checking. I don't think it has anything to do with the recipes – it's possibly related to the server Prodigy starts, or the async, or maybe something Azure-specific. We definitely have users running Prodigy successfully on Azure, so it's a bit puzzling.
So what does it say in the terminal before it dies? Nothing? If you enable logging, it should at least output something when it starts up the server and when you open the web app.
Okay, but you are seeing logs from before the server dies? If the server starts and you're able to access the web app, you should see all the logs for that. If you don't see anything printed to the terminal, maybe it's directing the logging output somewhere else (to a file etc.)?
Any updates on this @amitness? I am having the same issue with my setup. I think it is related to how Azure Web Apps handles ports & how it scales availability of the app
It sounds like this is an issue that's specific to Azure, and I don't think it relates to something we're doing wrong with Prodigy. You'll need to seek support from Azure about how to find out what's happening.
Old reply, but I did eventually figure this out and wrote up a quick guide on how to do it: Deploying Prodigy to Azure using Docker (hevia.github.io) involves changing some settings on the Azure Web App. Just in case anyone stumbles onto this in the future.
@Hevia Awesome, thanks so much for sharing We definitely want to add some docs to our install page with instructions for different cloud providers, so we might take some inspiration from your post.
We definitely want to add some docs to our install page with instructions for different cloud providers, so we might take some inspiration from your post.