Text spans and Image spans simultaneously

I'm trying to build a tool that will allow me to draw boxes in an image (using "image_manual") and highlight corresponding portions of a text input (would be similar to NER). However, when I try to use these two views together, I run into an issue because the "spans" field is used by both views. Any suggested workarounds or fixes?

Hi! This is definitely unideal and I regret not naming the key "image_spans" :weary: I'm not sure there's an easy workaround at the moment but I'll keep thinking... For the next version, I'll see if we can implement a setting that lets you customise the name of the key to save the image spans to. Then you could set it to "image_spans" (or whatever) in your config to prevent the clash.

Thanks for your response! It seems like as a general rule, it would be best for each view to have a unique name for its output field? Also, roughly when do you think the next version will be released?

Yeah, it definitely makes sense if you're mixing interfaces – but that feature is pretty new, and when the data formats were initially designed, we made them as similar as possible for consistency. And formats are always difficult to change without breaking backwards compatibility. So I think the setting is a good solution here.

I don't want to promise an ETA for the next release that includes the pending enhancements we want to implement. I'd love to have another release later this month, but it depends on how things go :slightly_smiling_face:

1 Like

Since this is not possible at present, is there a way to have a 2-step workflow where the annotator (1) draws N bounding boxes (where N is not pre-determined) for an image, (2) clicks a "submit" or "next" button, and (3) N text fields appear, one for each box, and the annotator enters text in each? It would be nice if all the text boxes could appear on the same page with the image and overlayed boxes.

You could implement that with some custom JavaScript in a HTML block: when the user clicks the button, you can access window.prodigy.content.spans to get the annotated bounding boxes and append a textarea or input for each box to the .prodigy-content element. And then you could have another button that the user clicks when they're done, and that reads the content of each textarea or input, and calls window.prodigy.update to add the text content to the current JSON task under a new key. See here for details and examples: https://prodi.gy/docs/custom-interfaces#js-css

1 Like

Update: Just released Prodigy v1.10.4, which introduces a new "image_manual_spans_key" setting (defaults to "spans") so you can change the key used by the interface to store the annotated spans.

Hi! Jumping in on this since I'm using a similar interface.

Is it possible to "link" the labels that are used for the "image_manual" and "spans_manual" blocks?
That is, I have the same labels for both blocks (eg: "Cat", "Dog") and when I click on "Cat" in the image_manual box, I would like that "Cat" in the spans_manual gets also selected.

It would be even better to just have one bar for labels (eg at the top), that are then applied to both blocks. Is this possible?

Thank you!

I don't think so because both components are independant of eachother in the JS codebase.

Could you explain your use-case in more detail though? I may be able to think along if I understand your use-case better.

Hi!
Jumping in on this as well. I would like to do the same thing as in the comment above - either have just one labels bar for both blocks (my blocks are audio_manual and spans_manual), or, if that's not possible, link the label bars, so that when selecting a label in one, the same label would automatically be selected in the other (It could also be fine for me if just selecting a label in the audio labels bar would guide the selection in for the text annotation, and not the other way around).

Thank you,
Gal

Hi @Gal_R,

I'm afraid it's still the case that labels corresponding to different blocks or pages are separate data structures. In most of the cases this is actually what you'd expect so we're dealing with a bit of an edge case here.
We are discussing internally what the most flexible solution could be. We're leaning toward exposing a new js event listener that would be triggered on the label change. This could then be mapped to custom js behavior that should affect other labels in other blocks or pages.
We'll definitely keep you posted!

1 Like