These recipes are successors to our original
*openai* recipes and our recommended way to leverage LLMs for annotation going forward. To find out more about the advantages of
spacy-llm recipes please check our docs.
terms.llm.fetch recipe offers a similar functionality to
terms.openai.fetch but it allows for plugging in an LLM backend of choice (including models running locally) as long as it can be configured via
You can use this recipe generate phrases and terms and convert them into Prodigy patterns for bootstrapping NER and spancat annotations. If you're interested in this workflow you can check an example here.
ab.llm.tournament recipe offers two improvements over its predecessor
ab.openai.tournament: since it's powered by
spacy-llm you can use a custom LLM backend, and you can include the LLM backend in the tournament as well!
As you annotate you can see the scores in the terminal so that you can take an informed decision about which combination of a prompt/LLM works best!
If you're interested in the algorithm behind the tournament recipe and happen to be at
PyData Amsterdam do check out our colleague's Andy's talk on the topic. You can learn all about "Promptly Evaluating Prompts with Bayesian Tournaments"!
Talk link: https://amsterdam2023.pydata.org/cfp/talk/HZB8JU/.
In this release we have also added an
llm_io annotation interface to facilite the development of custom LLM recipes.
We also included a fix to task router issue related to server restarts.
Finally, we have treat for those who would you like to explore these new features but have an old version of Prodigy.
You can get 10% off a new personal license, but also off any upgrade. Make sure you use PRODIGY10 as a discount code at checkout. The code is valid all through September!
Looking forward to hearing what you think