Delete all session information but keep annotations

[kmahelona (Keoni Mahelona)]

Aloha,

Is it possible to remove all session information from datasets or the database but keep the annotations/examples each session did? I'm wanting to do this because of a memory error I've encountered. I figured there may be others who would want to do something like this so making a separate topic for it made sense.

Mahalo,
Keoni.

[koaning (Vincent D Warmerdam)]

For other folks looking for an answer, this question seems answered in this thread:

Database performance/connection issues with a remote postgresql DB

Thanks for all the extra info Keoni! The 24million sessions thing is definitely a red flag and beyond the scope of what I was testing. That certainly makes some more sense as to why you have so much RAM usage. I'll try working this amount of sessions into my testing and yes, we'll definitely have an option in v2 for returning a Python generator when querying examples from a dataset. We're updating our schema for Prodigy v2 to fix this but have also made a bunch of Database performance fixes in …