Adding patterns to entity ruler in the loop

[ines (Ines Montani)]

Hi! You can definitely update your matcher/entity ruler within the recipe – you could do this in a custom recipe with an update callback that adds patterns for newly annotated terms so they're automatically highlighted in future batches. I actually outlined a very similar approach in this thread a while ago:

Prodigy Custom Model; Model in the Loop (matcher)

Hi! This is definitely a cool idea And yes, for that to work you should implement your own function that adds annotated spans to the matcher as patterns. The PatternMatcher currently only sets the pattern-based annotations and is updated with the accept/reject information to update the scores assigned to the patterns (so you can use it in an active learning context where you filter based on certain/uncertain predictions). It doesn't add annotated spans to the patterns, since this isn't alw…

@1danjordan

Also, is this the best way to do this? Should I be doing a session with ner.manual and then another session with ner.correct or something like that?

Depending on the data, you could also do this in two steps: annotate a (maybe random) selection of examples manually and then use a simple script that loads the annotations and creates match patterns from them.