We are using prodigy as a a rest-api to render the annotation in an iframe
We have a button to start the prodigy annotation process in the web app. This sends a request to the prodigy vm/rest api.
It takes takes too long to load the en_core_web_md model each time you hit start annotate. We need a way to speed this up… pre-load the model or something.
What do you mean by “too long”? How long does it take? The md and lg models do take longer to load, because they’re larger and have word vectors. And when you start up a Python process to annotate, you do need to load it once, there’s not really a way around that. However, depending on hwat you need an what you’re trying to do, there might be ways to make the model smaller by disabling components you don’t need, pruning the vectors etc.
Hi Ines,
I am also working together with Austin (original poster). Basically, when we start prodigy from the command line, it takes about 30 seconds for the server to start up (when we use md and lg models). We have a button called “start annotate” on our app, and we have to wait 30 seconds between hitting “start annotate” and having prodigy show up. If we just use the sm model, the server seems to start up instantly and we can use prodigy almost instantly.
Instead of having en_core_web_md(lg) as a command line argument, would it be possible to have it pre-loaded as part of prodigy.json or something? We just dont want to have the delay when starting the server, so any shortcut would be very helpful. Thanks!
Hi! Yes, 30 seconds makes sense for a larger model with word vectors. The sm model doesn't have vectors, which is why it's so much faster.
There are several reasons why this isn't as easy as it sounds: When you execute Prodigy from the command line, it starts a new Python process – that's where all the loading happens, and that's also where all the preloading would happen. So preloading wouldn't really make a difference here.
Another thing to consider is that if you're using any recipe that updates a model in the loop, it's very important that the recipe controls the model. If you use a pre-loaded, shared instance here, you can very easily and up in a bad state where the active learning wouldn't be effective anymore.
That said, one thing you could do is wrap the whole thing in Python and call prodigy.serve with the recipe and recipe arguments, instead of calling it from the command line. You could also write your own custom recipe function that takes a loaded model as an argument, and then execute that with prodigy.serve when the user hits the button. This way, you'd have a Python process running that already has the models pre-loaded, and when the user hits "start annotate", you're only starting up the Prodigy server and passing in pre-loaded objects.