Prodigy server automatically killed when run from Azure server


I am using Prodigy to annotate datasets for a sentiment classification task. I am getting the following issue:

  1. I run the prodigy command to start a server for annotating responses from a file called 'sentiment.csv'
  2. The prodigy server shown running status.
  3. I try to access the server from the {server-ip}:5100 endpoint. The interface works when loaded the first time.
  4. If I open the same endpoint next time, it shows "Oops, something went wrong" and then the command that was running the server is killed. It shows no logs whatsoever.

I have already tried changing the prodigy port, enable verbose logging and even running it from a docker container, but the same issue persists. Also tried it on different python versions, virtualenv/conda. Currently, port is set to 5100 from the prodigy.json config file.

Prodigy version: prodigy-1.9.1
Python environment: 3.7

Hi! That's definitely strange. And this only happens when you deploy it on Azure, not locally? If you set PRODIGY_LOGGING=basic, is there anything in the log that looks suspicious before it dies? And I guess you're not loading a recipe with a large model that could cause the machine to run out of memory, right?

Yeah, it only happens on Azure server. The same command and flow works fine when used locally.

No, I'm using the builtin "prodigy textcat.manual ..." recipe for annotation only. The spacy command to auto-create models based on this are not used. It's a simple sentiment classification task.

There are no logs with PRODIGY_LOGGING=basic as well. Getting the same error.

This is the command I'm using

prodigy textcat.manual sentiment ./sentiment.csv --loader csv --label Positive,Negative,Neutral -E -e sentiment

There is a file called sentiment.csv with a column "text" and sentences in that column. A dataset called "sentiment" is already present with old annotations.

Thanks for checking. I don't think it has anything to do with the recipes – it's possibly related to the server Prodigy starts, or the async, or maybe something Azure-specific. We definitely have users running Prodigy successfully on Azure, so it's a bit puzzling.

So what does it say in the terminal before it dies? Nothing? If you enable logging, it should at least output something when it starts up the server and when you open the web app.

There are no logs printed whatsoever. The process is killed by itself without any messages. I've verified that the server has enough memory.

Okay, but you are seeing logs from before the server dies? If the server starts and you're able to access the web app, you should see all the logs for that. If you don't see anything printed to the terminal, maybe it's directing the logging output somewhere else (to a file etc.)?