So we have used the prodigy to annotate page images with the layout labels Text-Area, Margin, Header, Footer, Table, hole, Illustration, Caption, other. we used that annotations of the images to train an image model that identify the layout annotations of an page image. now we want to inject that model in the workflow loop so that model will do a rough annotations on the images that are stream to the prodigy, so that the annotators can just review that annotations and correct if need be. I have looked at the documentations and found that right now it seems it only works with the language models or spacy models not all models, how can I inject my .h5 model in the loop of the workflow ? do I need to convert my .h5 model to spacy model format ? do I have to use ner.correct recipe or I can use a custom recipe ?
This is the format that the image interface requires to render the card. Once you have a .jsonl file in this format you should be able to pass it to image.manual and see the annotations.
Custom Recipe Route
The other path you could take is to write a custom recipe that re-uses your model, like explained on the docs here. It would involve defining a recipe which will have code similar to this:
from prodigy.components.loaders import Images
from prodigy.util import b64_uri_to_bytes
def get_stream(source_dir):
stream = Images(source_dir)
custom_model = load_your_custom_model()
for eg in stream:
image_bytes = b64_uri_to_bytes(eg["image"])
# Pass the image (bytes) to your model and get its predictions
predictions = your_model(image_bytes)
# 🚨 Add the predictions to the task, e.g. as "label" or "spans"
# ...
yield eg
In this case the predictions will be applied one batch at a time, just before the example needs to be annotated.