I'm building a custom recipe following
What is the relation between PatternMatcher and EntityRecognizer? (the one from prodigy.models.ner) EntityRecognizer isn't documented anywhere
If i add the same patterns to both by doing something like
and then combine the models
predict, update = combine_models(model, matcher)
what is happening in combine_models?
Is defining patterns redundant for EntityRecognizer? Should i just use EntityRecognizer and not a PatternMatcher? OR just a PatternMatcher and not an EntityRecognizer. (I want the resulting model to update as i'm annotating like in ner.teach)
Also I'm using a custom tokenizer on my spacy model (nlp.tokenizer = nlp.tokenizer.tokens_from_list) and passing a list of tokens to ["text"] instead of string. I have managed to get everything else working but model
update(answers) -> where answers["text"] is the list of tokens is giving me this error:
cython_src/prodigy/models/ner.pyx in prodigy.models.ner.EntityRecognizer.update( cython_src/prodigy/models/ner.pyx in prodigy.models.ner.merge_spans() TypeError: unhashable type: 'list'
pattern_matcher.update(answers) is working though i'm not sure what it is that it is updating since it is just a pattern matcher?