EC2 instance was recently moved from public to private subnet. Since then, the prodigy service is broken.
Login mechanism - login to ec2, port forwarded to port 8080, host the job using the prodigy command, hosting is successful.
When i open the browser and send a request to localhost:8080, it opens the prodigy UI but the content isn't loaded and a message appears - "ERROR:Can't fetch project. Make sure the server is running correctly." Upon checking the browser's developer console - I saw a 403 response from localhost:8080/project.
I am attaching the relevant screenshots. Can someone help here and share any possible fixes or resolutions?
Thanks for your question and welcome to the Prodigy community
Could you check your Network tab in the browser? There might be more info on why the request failed, rather than just the console line.
Typically, a 403 should only happen if there's a JWT error. Have you modified any JWT settings?
If not, one possibility is that JWT cookie can't be reached due to something CORS related, then the token isn't set on the request AUTHORIZATION header so the 403 appears.
But if it were a CORS problem, we'd expect to see a CORS warning and then the failed request.
To test if this is the problem, can you try setting "cors": false in prodigy.json. It's not really recommended to turn that off but they could determine if it is a CORS issue through that test. If this does seem to be the issue, you can set the PRODIGY_CORS_ORIGIN env var with the proper URLS for CORS.
One other possibility is to try to modify your "host" in prodigy.json from "localhost" to either "127.0.0.1" or "0.0.0.0". However, this is usually a fix for a different, but somewhat related error: OSError: [Errno 99] Cannot assign requested address:
Last, another user had a somewhat similar 403 issue with the same Prodigy error message ERROR:Can't fetch project .... Seemed like they forgot to pass a session_id which would be required if you set PRODIGY_ALLOWED_SESSIONS in your prodigy.json. Any chance this could be the problem?