Prodigy input stream as MySQL

,
1

Hi,

I have data collector jobs creating (to be) annotation batches in MySQL, is there an example or reference to look up for the customs recipes, where I can load data directly into annotation tool, instead of JSONP.

Any guidance would be great.

2

Prodigy streams are Python generators that yield examples – so if you can write a function that loads data from your MySQL database and yields out a dictionary for each example, you can plug it into Prodigy. You can do this via a custom recipe or with a custom loader script whose output you pipe forward.

Here’s a pseudocode example:

def custom_stream():
    texts = load_data_from_your_mysql_database()
    for text in texts:
        yield {'text': text}
3

Thanks, really appreciate.

1 Like