I have managed to fix the model.update error by changing my tokenizer logic (made a custom tokenizer class) but now I have another problem/question
If i start with no patterns (nothing even in the nlp model) can PatternMather add patterns through the update logic, annotating for multiple labels at a time with combine_matches=True.
PatternMatcher(
vanilla_nlp.nlp, filter_labels=vanilla_nlp.labels, task_hash_keys=("text"), all_examples=True, combine_matches=True
)
The documentation says the spans must have a pattern key for it to be updatable. but the answer spans I see returned don't set that automatically.
(Edit: the "pattern" fields disappear on the answers' spans! opened another issue for it-> Pattern Ids disappear from spans upon modification)
Expects the examples to have an
"answer"
key ("accept"
,"reject"
or"ignore"
) and will use all"spans"
that have a"pattern"
key, which is the ID of the pattern assigned byPatterMatcher.__call__
Would adding a pattern to the PatternMatcher manually before the update then setting the pattern id for the span work?
The ner.teach example recipe teaches one label at a time so I'm not sure if it is possible for multiple labels?
So my main question shortly is: can PatternMatcher.update add patterns or only modify the existing patterns' scores?