when i start the teach process, i have the following messages:
Task queue depth is 1
Task queue depth is 1
Task queue depth is 2
Task queue depth is 2
Task queue depth is 2
Task queue depth is 3
Task queue depth is 4
Task queue depth is 4
Task queue depth is 4
Task queue depth is 2
what do they mean?
I am refreshing my browser from time to time.
is this related in any way?
No, this is logging output from one of our dependencies (I think waitress) and not related to Prodigy. We’ve been trying to suppress it, but it keeps coming back. This should be gone once we switch our REST API dependencies over to FastAPI.
These messages are coming up because the server is synchronous, with a single process and a single thread. Usually this doesn’t matter because the server is fairly fast and there are few users, but sometimes it does. We’ve been working on this, first by switching to a different API library (FastAPI) which will allow us to make the server asynchronous, and second by allowing Prodigy to use multiple worker threads or processes while still updating the same model state.
The message means that requests are queueing on the server, but the server is blocked doing work processing a previous request. This basically means you’re clicking faster than the model can keep up.