a question about custom recipe

[ines (Ines Montani)]

@gwd

If I have raw data and patterns, how to automatically create a dataset like one that is saved by the annotation process?

Check out these threads for details:

How to perform automatically NER annotation based on patterns?

Hi! In that case, you can just go directly via spaCy, for example, using the entity ruler: https://spacy.io/usage/rule-based-matching#entityruler It lets you add your patterns and will add all matches to the doc.ents, just like an entity recognizer. You can then use that nlp object to process your texts and extract the pattern-based NER annotations. In theory, you don't even have to go through Prodigy at all and you could just export the data and train with spaCy directly. But if you want to mi…

Store the annotation obtained by ner.manual and --patterns at once

@ines Thanks a lot ! it works now Here I share my experience: My source data is in jsonl format and look like: {"text":"abcd","meta":{"source":"doc1"}} . . I wrote a code (compatible with SpaCy 2.5) based on your explanation to read a set of documents and annotate them based on patterns file: # path of jsonl file contains the performed annotation to be loaded in the db db_jsonl_path='db_jsonl.jsonl' nlp = English() ruler = EntityRuler(nlp) # the patterns file ruler.from_disk('patterns.…