Integration with Prolific (URL query parameters and redirect url)

, ,
1

Hi,

we want to use Prodigy with Prolific (https://www.prolific.co/) to source annotators. Prolific is similar to Amazon MT, but the actual annotation logic has to be implement in an external web app (typically Qualtrics, Google Forms etc). To integrate the two apps, one needs to do two things:

  1. Hand over URL query parameters from Prolific to the annotation tool to identify the user, and optionally a study and session (e.g. http://www.myurl.com/?PROLIFIC_PID={{%PROLIFIC_PID%}}&STUDY_ID={{%STUDY_ID%}}&SESSION_ID={{%SESSION_ID%}}).
  2. After finishing the annotation, either redirect them to a specific Prolific URL with a code in a parameter (e.g., Prolific) or simply show the annotators the code (here: 3943E9FB) after finishing the annotation. The code is static for a given study (i.e., not user-dependent).

Do you think this integration is possible with Prodigy? What would be the easiest/best way to implement 1 and 2?

Best,
Oliver

2

Hi Oliver,

I don't think this integration would be possible, no. Prodigy isn't designed around this sort of use-case, and depending on how you did it, it might be against the license terms as well.

You should use Prodigy with annotators you engage with directly. This is by far the best way to get machine learning training and evaluation data, because you'll be able to talk to the annotators about the annotation decisions, and you'll be able to ensure that the annotations are more consistent and higher quality. You'll also be able to refine your annotation scheme over time, to make sure that edge-cases have a consistent solution.