Correct way to "accept" examples in custom recipe?

1

I'm using recipe from Dependencies and Relations · Prodigy · An annotation tool for AI, Machine Learning & NLP like this

prodigy rel.manual ner_rels blank:en ./tmp.jsonl --label EQUALS --span-label QTY,AMOUNT

which works great!

But I'm only interested in reviewing and fixing relations manually for sentences which have some labels (eg PERCENT)

What would be right way to pre-accept all other sentences from the source document to the database without writing my own custom recipe?

I mean I can preprocess my files and already save ("accept") sentences without labels as positive examples which have 0 relations and only send rest of the sentences to the prodigy

2

or should i just yield such examples with these attributes from my custom loader:

ex = {"text": text, 
        "tokens": tokens,

        # set this to "accept" ? 
        "spans": [],
        "relations": [],
        "answer": "accept",
        }
yield ex
3

Yeah this does feel like the kind of thing where I might recommend writing a custom script to handle the logic, mainly because the logic might also change over time.

With that in mind, it might also be good to perhaps consider multiple datasets. You might want to have a dataset with reviewed items and another with automatically accepted items just to ensure that they are separate.

1 Like