# Prodigy crashing

**URL:** <https://support.prodi.gy/t/prodigy-crashing/6141>\
**Category:** Uncategorized\
**Tags:** usage, textcat\
**Created:** [November 23, 2022, 6:20pm UTC](https://support.prodi.gy/t/prodigy-crashing/6141 "2022-11-23T18:20:11Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![ryanwesslen](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ryanwesslen/32/2969_2.png) [@ryanwesslen](https://support.prodi.gy/u/ryanwesslen)\
**Post date:** [November 23, 2022, 7:19pm UTC](https://support.prodi.gy/t/prodigy-crashing/6141/2 "2022-11-23T19:19:05Z")

</div>

hi @bev.manz!

Thanks for your question and welcome to the Prodigy community 👋

> [@bev.manz](#):
>
> My team never had this issue historically but it seems that now whenever we launch a teach (prodigy textcat.teach) labelling session.

You mentioned this problem wasn't an issue in the past. What changed?

Did you have this same architecture on local machines and only now moved to AWS (since you mentioned EC2)?

> [@bev.manz](#):
>
> we launch a teach (prodigy textcat.teach) labelling session

Or alternatively, is it `textcat.teach`? Did this same problem happen with non-model-in-the-loop recipes like `textcat.manual` or `ner.manual`? Just as a test, get some sample data and run through a few examples for manual recipes, trying to hit multiple sessions at the same time. If that works, then this would suggest it's something wrong with running the model.

Are your documents large or very long?

> [@Prodigy crashes on large documents](https://support.prodi.gy/t/prodigy-crashes-on-large-documents/220/2):
>
> I think the problem isn’t that all documents are processed at once – internally, Prodigy works with generators to process the stream, so examples are processed as they come in. It seems like the difficulty here is that ner.teach uses beam search, so it’s trying to find the best parses for each 20k token document – instead of only the one best parse like ner.pipe. At each step, it has a beam of states, and has to create a new state on the next token. That state representation involves a copy tha…

Granted you're using `textcat.teach` which may not be as intensive as `ner.teach` due to beam search.

Any [possible issues with memory](https://phoenixnap.com/kb/linux-commands-check-memory-usage) too?

> [@Prodigy server crashes on some ports during the annotation process](https://support.prodi.gy/t/prodigy-server-crashes-on-some-ports-during-the-annotation-process/3847/2):
>
> Hi! Do you have some more details on what exactly happens? How does it "go down", is there an error message? Does the machine run out of memory, and is there anything else running on it? Do you see anything in the logs that could be relevant? Under the hood, Prodigy is a fairly straightforward Python service with a REST API powered by FastAPI. There are many different reasons why a service like this could die – memory constraints, an actual error in the Python process, something else interferin…

> [@bev.manz](#):
>
> port forward

Are you modifying the port or using the default `8080` port? Any chance there's anything else that runs on that port on your machine?

Have you also started using [Prodigy logging](https://prodi.gy/docs/install#debugging-logging)?

You can add `PRODIGY_LOGGING=basic` or `PRODIGY_LOGGING=verbose` to your Prodigy command to add more detailed logs. I typically recommend always running this in development environments.

Related, you can also add `/docs` to view the [interactive API](https://prodi.gy/docs/api-web-app#rest-api) so you can try to figure out if one endpoint is crashing.

Also, it may not be relevant, could there be any issues with security since you're running on `http://`, not `https://`?

> [@Prodigy annotation - Starting the web server at http://localhost:8080](https://support.prodi.gy/t/prodigy-annotation-starting-the-web-server-at-http-localhost-8080/3690/2):
>
> Hi! See here for the recommended approach: If you're using Prodigy in Jupyterlab, see this thread:

> [@bev.manz](#):
>
> We use this format to start sessions:  
> [http://MYIP](http://MYIP):MYPORT/?session=username
> 
> but lately I've had to advise to just ignore the crashed session and use a number with your name like so:  
> [http://MYIP](http://MYIP):MYPORT/?session=username1  
> [http://MYIP](http://MYIP):MYPORT/?session=username2  
> [http://MYIP](http://MYIP):MYPORT/?session=username3  
> [http://MYIP](http://MYIP):MYPORT/?session=username4

As a best practice, you may want to avoid having users try different session names. I know you're doing this for now b/c of the crashing as a short term solution.

But I mention this in the future if you need to analyze annotator quality (e.g., track inter-annotator agreement) you'll appreciate having correct session names. In fact, a good practice is setting `PRODIGY_ALLOWED_SESSIONS` so that you have a finite choices of what sessions are allowed. For instance, `PRODIGY_ALLOWED_SESSIONS=alex,jo` would only allow `?session=alex` and `?session=jo` , and other parameters would raise an error. This prevents fat finger errors with invalid session names.

This is secondary to your key problem so you can ignore for now. Hopefully if some of the ideas above can help identify the crashing problem, this may be something good to build into your process afterwards.

Hope this provides some avenues to explore. Let us know if this works (or doesn't). Also, feel free to provide logs or other details.

---

_[View the full topic](https://support.prodi.gy/t/prodigy-crashing/6141)._
