I have a medical text files where I want to perform question answering. There are several questions for each text file and list of answers for each question. I am currently looking into purchasing a tool for labelling this data for question answering. I am wondering if I can customise Prodigy application to label the medical text for several questions from one UI.
I will greatly appreciate if there is any documentation of some sort.
Thanks for your post and I apologize for the delay on getting back to you.
While we don't have detailed documentation on a case study (it's something we've been thinking about), there are several links that list out ideas on how to approach this problem.
First, we have a custom prodigy-recipe for question-and-answering. Assuming you have input candidate q-a, it allows users to "accept", "reject", or "ignore" each question-answer pair.
It assumes the source data (qa pairs) are in this jsonl format:
{"question": "What color is the sky?", "question_answer": "blue"}
Also the compare recipe could work if you wanted to frame the task as an A/B evaluation (e.g., provide the question, provide the QA model's answer and a dummy answer and allow the user to choose the "best answer").
If you're interested in something where the user has to provide a free form response (e.g., answer a question), text_input recipe is one option.
):