Prodigy with AllenNLP model

[ines (Ines Montani)]

Thanks, that's nice to hear!

Here are some examples of custom Prodigy recipes with custom models in the loop: Named Entity Recognition · Prodigy · An annotation tool for AI, Machine Learning & NLP There are essentially two parts your recipe needs to return: a stream of examples to annotate (a generator, so the suggestions can change as the model is updated) and an update callback that receives answers and updates the model. The specifics here depend on your model implementation: you want to choose a model that is sensitive enough to small updates in small batches, but also not too sensitive so a single decision won't throw it off. This may take some experimentation.

Also, here are some related recent threads on similar topics:

active learning and update function

Hi! The update callback is called by Prodigy whenever a batch of examples comes back from the web app. It receives a list of annotated examples in Prodigy's JSON format – so basically whatever was sent out via the stream, with the added annotations (e.g. manually added spans) and the "answer". See here for the API docs: https://prodi.gy/docs/custom-recipes#update Prodigy's binary annotation recipes use a more complex annotation model (e.g. the EntityRecognizer class implemented by Prodigy) to…

Custom recipe to teach a DistilBERT model with custom labels

Hi! We don't currently have an NER implementation that uses transformer weights in spaCy v2.x, so your approach wouldn't work – but once spaCy v3 is out, we'll have an updated version of Prodigy that will let you use transformer-based pipelines, pipelines with custom models in PyTorch/TF and pretty much everything else that spaCy v3 offers. (The error you came across here btw looks like a different problem: internally, the NER annotation model deepcopies/pickles the nlp object and it looks like…