After annotating and saving on an input file. Can I able to open a new terminal and run an annotation task with the new input file, could I able to continue in the prodigy session. by just refreshing? I tried and it throws me an bind error. Is there any other way to solve this issue?
Thanks in advance.
Hi! In general, this should be no problem – but you do need to quit the other server, or choose a different port (e.g. using the PRODIGY_PORT environment variable). Otherwise, your system will complain, because you're already running an instance of Prodigy on that port.
But you can just exit the server, start it again on the same port with a new input file and refresh the app to start the new annotation session.
Thank you INES for you swift reply. I was actually looking for updating the input stream. Can I update the stream input using custom recipes such that whenever I refresh the prodigy session, it will add new data to the stream based on the updated input file.
Yes, that should be no problem, either – streams are generators and Prodigy will only ask for one batch at a time. Just make sure you're streaming data in a format that can be read line by line (otherwise, the whole file has to be parsed and read into memory) and that you don't confuse Python if it has unclosed files etc.
Dependin on how your data is stored, you could also write a custom loader: https://prodi.gy/docs/api-loaders#loaders-custom
And if you're not doing any active learning etc., you can also set "force_stream_order": true in your recipe "config" to make sure examples are always sent out in order and Prodigy keeps sending out unanswered batches until they're answered and doesn't wait for them to come back, even if you refresh the app a lot.