Downstream Task QnA

Hey,

in V2.1 of Spacy you added BERT-style language model pretraining. I wanted to ask if you planning to add the feature for creating a question & answering down steam task. Furthermore, would these kind of models be support labeling Q&A pairs within Prodigy.

Cheers,

You can already do pairwirse prediction (e.g. for question answering) using our spacy-pytorch-transformers library. See the GLUE tasks for details. For the spaCy core library, we are also thinking about user-facing APIs to support this type of task – this is just a little trickier, because you want to make predictions in batches, so an API like spaCy's current .similarity methods wouldn't work.

If you have a model for question answering, you can always collect annotations for it in Prodigy (independently of spaCy). For instance, render a question and an text and annotate whether the text answers the question. Or select one or more sentences that are answers to a given question (e.g. using the choice interface). You can then download the data and train your model with it.

1 Like