How to access and remodel a dataset that has already been annotated with prodigy for images ?

[ryanwesslen]

hi @Mat!

Thanks for your question and welcome to the Prodigy community

@Mat

How to access and remodel a dataset that has already been annotated with prodigy for images

Have you seen this post:

prodigy mark: images from JSONL not showing

Hi! That's expected, due to how modern browsers handle images from local paths. You can read more about this here for instance: If you don't want to store the image data with the tasks, the easiest way to load images is by providing URLs. Modern browsers typically block images from local paths for security reasons. So you can either host the images with a local web server (e.g. Prodigy's image-server loader or in an S3 bucket (or similar). So instead of the local paths, use URLs or use --loa…

As it mentions, you may want to use load images by URL's using image-server loader.

Also related, are you aware of base 64 encodings?

Using base64-encoded data URIs and storing the image data with the annotation task is the safest way to ensure that you never lose the reference to the original data. If you only store URLs or file names and the original files are ever renamed or get lost, your annotations will be useless. However, keep in mind that all task data will be also be stored in the database – including the base64-encoded images. In some cases, this can lead to unexpected results and database bloat .

Therefore, you may want to turn off base 64 encodings when annotating. If you do that, then your annotations will only look like this (that is, "image" is the URL, not base64 encoding):

{
  "image": "https://images.unsplash.com/photo-1554415707-6e8cfc93fe23?w=400",
  "spans": [{"points": [[155, 15], [305, 15], [305, 160], [155, 160]], "label": "LAPTOP"}]
}

Here's a related post that includes a snippet you can use in a custom recipe to prevent the base 64 encodings in order to use the mark recipe:

Labelling a set of images (classification)

Hi! It sounds like you're definitely on the right track Instead of using the mark recipe, which really just streams in what you give it, you might actually find it easier to just implement a custom recipe for this, since it'll make it more obvious what's going on and lets you add your own custom logic (e.g. for shuffling, removing base64 and maybe other stuff). This example recipe actually goes in a very similar directon: https://prodi.gy/docs/computer-vision#classifica…

Hopefully these two posts can help you out. Let me know if you have any further questions!