ner.silver-to-gold is a nice one, do we have textcat.silver-to-gold recipe ? or similar function for textcat?
shooting from the hip here, but isn't review able to achieve this?
shooting from the hip here, but isn't review able to achieve this?
That was my initial thinking too. @linb is there a reason why the review recipe wouldn't work for you?
review recipe always creates a new dataset, I want to merge a silver dataset into an existing gold dataset.
I'm not aware of a recipe that directly does that. The best way forward seems to be a custom Python script/recipe. The benefit of this approach is that you can manually control the data such that it becomes exactly what you're interested in.
You can fetch the data as a list of dictionaries in a Python script via:
from prodigy.components.db import connect
db = connect()
dataset = db.get_dataset("test_dataset")
After processing the data, you could store it locally as a processed.jsonl file which can then be loaded into Prodigy as a dataset or as input for another labelling task.
More information on helper functions to fetch data out of Prodigy can be found on our documentation here.
1 Like