How to customize error message for validate_answer function?

1

We are using validate_answer function in our project which is very helpful!

However, when validation fails, the error message always includes the line 'To skip this example, you can click the ignore button'. This could confuse our annotators as 'ignore button' is not enabled in our project.

Is there any way we can remove or customize this message?

Thanks!

2

hi @jinshuai!

Thanks for your question and welcome to the Prodigy community :wave:

Unfortunately, the code with the warning message for this is embedded in our compiled react app. Therefore, it's not possible for you to modify that part of the code since it is compiled. I have made a note on this and we could see about providing an easier way to modify in a future version.

I'm curious - can you explain why you removed the "ignore" button? We've seen some users do this and we're interested in the use case where you may not want users to skip. Thanks for the insight!

3

Hi @ryanwesslen,

Thanks for your response!

As for removing the "ignore" button:

We are doing manual annotation for text classification and implemented a feature called 'Attention checks' to reduce the number of invalid labels. There will be a certain amount of questions (around 10% of total) being "attention check" questions which already have correct answers, but the ui looks exactly the same as normal questions. So that we don't want annotators to ignore/skip questions and if they select wrong answer, we know that they might lose attention or might not fully understand the coding instructions. Instead of the "ignore" button, we use the "flag" button to let annotators mark uncertain questions.