Using Stanford CoreNLP during active learning

[jloverich (john)]

Is it possible to use an arbitrary machine learning tool instead of SpaCy during active learning. For example, if we are ultimately wanting to create a Stanford CoreNLP model can we use a python wrapper and then use CoreNLP during active learning phase?

[ines (Ines Montani)]

Yes, you should be able to implement this with a similar approach to the one I outline here:

Custom model Requirements

To make any model integrate with Prodigy’s active learning workflow, you mainly need to expose two functions: a predict function that takes an iterable stream of examples in Prodigy’s JSON format, scores them and yields (score, example) tuples an update callback that takes a list of annotated examples and updates the model accordingly Here’s a pseudocode example of how this could look in a custom text classification recipe. How you implement the individual components of course depends on the…

One thing that's important to keep in mind for active learning-powered NER is that your model needs to handle updates with sparse labels. For example, if you receive an accepted entity annotation back, your model shouldn't accidentally assume that all other tags are O (outside an entity) and instead treat them as missing values.