Setting the source argument to - will read from standard input – so you can pipe the output of one process forward to Prodigy. See here for examples: Loaders and Input Data · Prodigy · An annotation tool for AI, Machine Learning & NLP
The streams are processed as generators and if it's exhausted, you'd typically have to restart the server. However, I think you could work around that by sending a stream of None while nothing new is available, e.g. something like:
while not new_tasks:
yield None
yield from new_tasks
That said, we typically comment starting and stopping instances as you need them instead of having one long-running session like this.