Tournament Recipe Examples?

1

Question: Where can I find an example tournament recipe? The recipe repo doesn’t have any.

Use Case: I have pre-generated documents that I want to evaluate with Prodigy. Tournaments seem like a great way to do this, but the included recipes assume I’m generating documents on the fly. I already have 3 variations of each document, I want to chug through a bunch of pairwise comparisons to measure which variation is the best.

2

I’d hoped to start with a simple example and build up, but I’ll try starting with site-packages\prodigy\recipes\openai\ab.py and slimming it down.