Can Prodigy give users dynamic annotation options based on custom function?

Prodigy looks very promising! I have two questions:

  1. Can Prodigy support a workflow that includes
    A: first pass annotations of raw text based on SPACY based processing.
    B. Support dynamic annotation options for the user, based on a custom python function? See pic for example:

  2. Can Prodigy support Named Entities (manual) + Text Classification (multiple) in 1 page?

Hi! Detailed answers below – but the short summary is, yes, both of these things should work :slightly_smiling_face:

While you typically define the full label set once via the "config" returned by the recipe, you can also override them on a per-task basis. The same question actually came up the other day and I shared an example here:

So depending on the example, you'd just add "config": {"labels": [...]} to the outgoing task dictionary. If no overrides are provided, the default labels are used as a fallback.

Yes, you can use the blocks UI to combine multiple interfaces. Here's an example of a custom interface consisting of ner_manual and choice (multiple choice, e.g. for text classification): Custom Interfaces · Prodigy · An annotation tool for AI, Machine Learning & NLP

In the dictionary you send out, you would then include the "text", "tokens" and "spans", as well as the "options" that can be rendered by the choice block.

Thank you for your quick response! It definitely answered my inquiry, thanks.
@ines I have a follow-up question:

I read somewhere prodigy can apply annotation on to the raw text before it is shown to the users. Is this true? What I'm trying to do is to use Spacy to find some annotations, then have a human reader annotate what the system missed, and feed it back into the NER model again to improve it over time - a feedback system.

Yes, definitely – in fact, this is a very common use case :slightly_smiling_face: The easiest solution is to use the built in recipe ner.correct that takes a spaCy model and pre-highlights the predicted entities for you: Built-in Recipes · Prodigy · An annotation tool for AI, Machine Learning & NLP

Prodigy's input and output formats are identical, so you can always load in data in the same format with pre-defined "spans", or even plug in a custom model. See here for an example: Named Entity Recognition · Prodigy · An annotation tool for AI, Machine Learning & NLP

1 Like