Annotation UI supporting typed text input with ner.make-gold or other recipes

,
1

Hi there,

We are working on an application where we would like to support the following scenario.

  1. The user types a query (say in a text box).
  2. The UI uses a spacy (eg. ner) model that is running in the background to display the model’s predictions.
  3. If needed, the user corrects the model’s predictions and submits.

Is there a way that this can be done in the current version?

Cheers and thanks for the awesome product.

2

While you could add user input fields via the (currently still experimental) JavaScript support, I’m not really sure this makes sense here. The problem is that a workflow like this would require a completely separate endpoint for processing the text the user types in and you also wouldn’t be able to use any batch processing. (Normally, Prodigy will process batches and then send them out for annotation.)

I’m not sure what your end goal is and there are certainly use cases where you want free-form input and don’t care so much about the predictability of the data, annotations and experiments. But by design, Prodigy tries to limit the modifications made by the annotator to a few selected properties (answer, spans, label). There should always be an input (the data to be annotated) and an output (the input plus annotations). In the scenario you describe, this is different: you’d have nothing coming in, and arbitrary unpredictable data coming out.

This makes it super difficult to run predictable and structured experiments, so we’d usually advise against workflows like this. Instead, it’s often more efficient to move the unpredictable, free-form data collection out into a separate step: for example, start by collecting the data from the user and then annotate it with Prodigy.