Customise Prodigy interface for NLP Q&A Task with Multiple Questions

Hi All,

I have a medical text files where I want to perform question answering. There are several questions for each text file and list of answers for each question. I am currently looking into purchasing a tool for labelling this data for question answering. I am wondering if I can customise Prodigy application to label the medical text for several questions from one UI.

I will greatly appreciate if there is any documentation of some sort.

Looking forward to hearing from you.

Many thanks and

Kind Regards,

hi @nlp-guy!

Thanks for your post and I apologize for the delay on getting back to you.

While we don't have detailed documentation on a case study (it's something we've been thinking about), there are several links that list out ideas on how to approach this problem.

First, we have a custom prodigy-recipe for question-and-answering. Assuming you have input candidate q-a, it allows users to "accept", "reject", or "ignore" each question-answer pair.

It assumes the source data (qa pairs) are in this jsonl format:

{"question": "What color is the sky?", "question_answer": "blue"}

Also the compare recipe could work if you wanted to frame the task as an A/B evaluation (e.g., provide the question, provide the QA model's answer and a dummy answer and allow the user to choose the "best answer").

If you're interested in something where the user has to provide a free form response (e.g., answer a question), text_input recipe is one option.

If the prodigy-recipe for question-and-answering doesn't meet your need, here's a post that has an alternative.

It shows how to use blocks and JavaScript for a custom extractive QA recipe (details in the :thread:):


Let us know if you have any further questions!