Is there a way to combine Prodigy with Google Cloud Vision?
I would just love to use Prodigy as frontend - export the data in the right format for Google Cloud vision - train a model - download it and insert it into Tensorflow.js done.
Is that possible?
Hi! So I haven't really used Google Cloud Vision myself but in theory, it should be possible and pretty straightforward – how you set it up just depends on the types of annotations you're working with and the data format you need.
If you just want to annotate manually and without any automation or model in the loop, you can just use Prodigy's image.manual (or any other workflow you need), and then convert the annotations as a post-processing step. For example, for object detection, Prodigy gives you the x/y/width/height and label of all bounding boxes, which is typically all you need to train a model. You can find an example of the format here: https://prodi.gy/docs/computer-vision#box-data-format
If you want to annotate with a cloud API in the loop and source the suggestions from the API (so you only have to correct mistakes), you can integrate this via a custom recipe and just make API calls from Python. Here's an example of a custom stream processing images: https://prodi.gy/docs/computer-vision#custom-model But instead of loading a model and running it over the examples, you'd send the examples to the API, and add the returned annotations to the outgoing examples in Prodigy's JSON format.
Thank you for your quick reply. Hmm I think it might not work -> the data needs to be exported like this:
[set,]image_path[,label,x1,y1,,,x2,y2,,]
TRAIN,gs://My_Bucket/sample1.jpg,cat,0.125,0.25,,,0.375,0.5,,
VALIDATE,gs://My_Bucket/sample1.jpg,cat,0.4,0.3,,,0.55,0.55,,
TEST,gs://My_Bucket/sample1.jpg,dog,0.5,0.675,,,0.75,0.875,,
[set,]image_path[,label,x1,y1,x2,y1,x2,y2,x1,y2]
TRAIN,gs://My_Bucket/sample2.jpg,dog,0.56,0.25,0.97,0.25,0.97,0.50,0.56,0.50
Is there any way to make an export like this? I would theoretically need to upload the images that I annotated with prodigy and then export it in this way.
That format looks very similar, though? It's just the concatenated bounding box information, so for each box you've annotated in "spans", you can create one line with the file name, label name and the coordinates of the bounding box, which corresponds to the "points" exported by Prodigy.
You could pretty much just dump this as a string line by line in Python. Or put it in a dataframe and export it as a CSV with pandas or something, whatever you're most familiar with. You can also connect to the database programmatically in Python (see here) so you can do the export and conversion all in one step.