Custom view templates with scripts

[ines (Ines Montani)]

@SofieVL Yay, that's nice to hear! And yes, the html_template is currently limited to the html – although I think you can kinda trick the app into using it to render the content in interfaces like choice or classification by passing it an empty "html" property in the task.

@SofieVL

Like is there an obvious way to combine this with presets like the choice interface or NER annotations in the text?

Custom CSS and JavaScript are now supported for all interfaces, so you can always do something like this and append elements to the annotation card:

const content = document.querySelector('.prodigy-content')
const button = document.createElement('button')
button.textContent = 'Click me!'
// add event listener / onclick handler etc.
contend.appendChild(button)

[SofieVL (Sofie)]

@ines: you're right, it does kind of work when specifying the html tags in the choice interface properly:

However it looks like you can't actually type in the box. Because as soon as your mouse goes there and selects the box, the radio button is selected instead, and focus moves aways from the text box.

[ines (Ines Montani)]

Ah yeah, the choice options weren't really designed to have interactive content in them. So you probbly want to just add the field at the bottom below the options (instead of actually making it an option itself).

[Din0_21 (michael)]

Hi @ines,
my window.prodigy.content is pointing to the wrong content when using view_id blocks.
My html_template consist of buttons and the js_templates holds the onClick functions. The OnClick are supposed to highlight some words. I planned on doing this by appending Spans to window.prodigy.content.spans and then updating it. However, it is not showing on the text visualization of the ner_manual block. May I know where I was wrong?
Thanks!

recipe.py
    "config": {
      "block": [
       { "view_id":"ner_manual", "labels": ["PERSON","ORG"] },
       { "view_id": "html", "html_template": get_html_template()}
      ],
      "javascript": get_js_template()
      }

[ines (Ines Montani)]

@Din0_21

I planned on doing this by appending Spans to window.prodigy.content.spans and then updating it.

Hi! The ner_manual interface currently keeps its own copy of the spans – calling update does update the task content, but it doesn't necessarily trigger a re-render of a built-in component. I do want to change this, though, since use cases like the one you describe are pretty cool. It's just not implemented yet.

You can read more about it here:

JavaScript: prodigy.update(...) doesn't trigger a re-render

I'm currently playing around with Prodigy's JavaScript API and ran into a problem when using the update(...) function. The documentation says that it will trigger a shallow merge of the current task object. Since the frontend is based on React/Redux and I get back the corresponding UPDATE_TASK action object, I was expecting the UI to re-render based on the state update I provided, but this was not the case. I'm running the latest Prodigy version and am using the ner_manual interface, my goal wa…

[Din0_21 (michael)]

Hi @ines,
Thanks! I understand!
As a temporary workaround, may I ask if there is any way for me to trigger React to re-render? Perhaps using a pseudo state change?

[ines (Ines Montani)]

@Din0_21

As a temporary workaround, may I ask if there is any way for me to trigger React to re-render? Perhaps using a pseudo state change?

Ah, good idea! It's hacky, but you could make it think that it's receiving a new task by updating the _task_hash of the current task. Just tested it and the following works for me:

const taskHash = window.prodigy.content._task_hash
window.prodigy.update({ _task_hash: 0 })
window.prodigy.update({ _task_hash: taskHash })

Just make sure to reset it to its original value again – otherwise, you'll end up with bad task hashes in your dataset (which impacts what Prodigy considers duplicates etc.). Maybe also try it with a test dataset first to check that there are no unintended side-effects on the client.

[lucinvitae (Luc Cary)]

Hello @ines, do you know how I can update the selected label using custom view templates?

Here's what I have so far:

Custom dropdown HTML:

<div class="custom-dropdown" onchange="updateLabel()">
  <select id="labels">
    {label_options_str}
  </select>
</div>

Custom JS:

function updateLabel() {
    const selectedLabel = document.querySelector('select').value;
    console.log(selectedLabel)
    window.prodigy.update({ label: selectedLabel });
}

I am able to confirm that the custom javascript is correctly logging the selected label, but I'm not sure how to actually set the selected label in prodigy, since I don't see any examples of updating the selected label in the window.prodigy interface.

The .update() call in my custom JS was a wild guess, and sure enough, it didn't work (it's just using the first item in the dropdown, which is NO_LABEL in the below screenshot):

So, in summary, my question is: Is it possible to update the label from a custom HTML interface like this?

Thanks in advance!

-Luc

P.S. The reason I'm using a custom dropdown is that the default dropdown for a list of labels provided by the --dropdown prodigy option is not flexible enough for our image_manual use case, which requires using several <optgroup>'s in the select bar.

[ines (Ines Montani)]

Glad you got it to work so far

@lucinvitae

The .update() call in my custom JS was a wild guess, and sure enough, it didn't work (it's just using the first item in the dropdown, which is NO_LABEL in the below screenshot):

The window.prodigy.update function will update the current task dictionary with whatever data you pass in. So in your code, you're adding the top-level property "label" to the dictionary – but I think that's not what you want, right? You'd want the label to be added to the entry in the "spans" that was just added.

[lucinvitae (Luc Cary)]

Hi @ines, thank you for the quick reply.

You'd want the label to be added to the entry in the "spans" that was just added.

The order of operations is reversed for us; during image.manual tasks, we need to select the label via the dropdown and then create a bounding box (or multiple boxes) for that label.

In other words, we are selecting the label first (e.g. INSURANCE_CARD), and then using it to highlight which parts of the image contain the data that need to be labeled (e.g. the actual sections of the image that contain the cards).

For some of our image.manual tasks, there is also the possibility that there are spans in the image for different types of labels (e.g. INSURANCE_CARD_FRONT and INSURANCE_CARD_BACK within the same image). This is one reason why we can't automatically apply a selected label to all spans when the user selects an item in the labels dropdown.

Is there any Javascript interface for selecting the label that will be used when annotating the following/next span(s) in Prodigy?

---

P.S. I inspected the minified JS source code and noticed that there is a function named handleLabelSelect() which appears to call something like setState({label}) in what I'm guessing is the frontend component that allows the user to select a label. Since this appears to determine which selected label is used when labeling spans in the image, I believe this is the interface I'm trying to hook into from our custom JS code. I understand if/why the actual setState call would not be exposed in the JS API (for security and state encapsulation reasons) but I'm wondering if there's a way to modify the selected label via the window.prodigy API documented in this support thread and in the PRODIGY_README.html.

[ines (Ines Montani)]

@lucinvitae

but I'm wondering if there's a way to modify the selected label via the window.prodigy

At the moment, the call is not exposed and the prodigyspanselected event is currently only used in the manual NER interface, so you couldn't use that one. If you're not using the built-in logic to select labels, this is probably how I would go about it, though: when the task is updated and a span is added, you can update it with the currently selected label.

One workaround could be to listen on prodigyupdate and keep track of the "spans". It's not as clean, but you know a span was added if the task hash is the same (e.g. the current task hasn't changed) and there are more spans than on the last update

@lucinvitae

in the PRODIGY_README.html.

Btw, as of v1.9, the PRODIGY_README.html has been replaced by the docs live on the website: Prodigy 101 – everything you need to know · Prodigy · An annotation tool for AI, Machine Learning & NLP. So this will always be the most up-to-date reference.

[lucinvitae (Luc Cary)]

@ines

One workaround could be to listen on prodigyupdate and keep track of the "spans". It's not as clean, but you know a span was added if the task hash is the same (e.g. the current task hasn't changed) and there are more spans than on the last update

Thanks for the tip, I'll try that!

@ines

Btw, as of v1.9, the PRODIGY_README.html has been replaced by the docs live on the website: Prodigy 101 – everything you need to know · Prodigy · An annotation tool for AI, Machine Learning & NLP. So this will always be the most up-to-date reference.

Ah, good to know. I wasn't sure which should be the source of truth.

At the moment, the call is not exposed and the prodigyspanselected event is currently only used in the manual NER interface, so you couldn't use that one.

That makes sense that not all of this is exposed in the current version of the app... although some of our company's image labeling projects have some very specific labeling UI/UX requests. Right now we have access to the minified UI code in our browser, but if we wanted to extend the frontend to support other types of events, it could be very useful to have the non-minified source code. That way we could potentially fork the frontend, add features, build the app with our extensions internally, and coordinate with your team regarding whether or not some features make it upstream. Could that (sending us a copy of the unminified code) be an option for our team? Or does the Prodigy team deny such requests?

[ines (Ines Montani)]

Yeah, it should be pretty easy to fire prodigyspanselected for image spans as well, so I'll put this on my enhancement list for the next version

@lucinvitae

Could that (sending us a copy of the unminified code) be an option for our team? Or does the Prodigy team deny such requests?

We want everyone who purchases a Prodigy license to get the same thing, so this wouldn't really be an option for us.

[dijana-sagit (Dijana)]

Hi @ines,
I came across this thread as it was referenced in the documentation. I think my problem is somewhat similar to the previous comments, but I wanted to check if there is a workaround. Fig. 1 roughly shows what I want to achieve. In the custom html area I would like to listen to the selected span and print some of the metadata (text and label in my case). Also, as I change the span, I would like to trigger the event and "recalculate" the text in that span. I don't think I can use prodigyupdate as this event doesn't trigger on span selection.

Figure 1. Visual example of the intended functionality

I would appreciate your input.

Thanks,
Dijana

[ines (Ines Montani)]

@dijana-sagit

Fig. 1 roughly shows what I want to achieve. In the custom html area I would like to listen to the selected span and print some of the metadata (text and label in my case).

Ohh, this is super cool

I just tested it and for me, the prodigyupdate event fires every time an update is made to the UI, including new boxes that are added or boxes that are edited. The most recent update (v1.10.5) also adds the prodigyspanselected event to the image_manual UI that should fire every time a new span is added.

The only information that's currently not easily exposed is the span that's currently selected in the UI (as in, editable)

[dijana-sagit (Dijana)]

Thank you, I appreciate it!

[dijana-sagit (Dijana)]

@ines

I just tested it and for me, the prodigyupdate event fires every time an update is made to the UI, including new boxes that are added or boxes that are edited. The most recent update (v1.10.5) also adds the prodigyspanselected event to the image_manual UI that should fire every time a new span is added.

Regarding the prodigyspanselected event, it doesn't seem that if fires for me for image_manual view (I am using Windows wheel v1.10.5), while prodigyypdate works.
Another issue that I am facing (maybe I should write about it in a different thread) is the Mustache rendering.
For example, with the following html template:

{{#spans}}
<strong>{{text}}</strong>
<br>
{{/spans}}

I get:
{{spans}}{{text}}
{{/spans}}

So, the html gets interpreted, but the variables from the task not. I feel like I am doing some rookie mistake.

[ines (Ines Montani)]

@dijana-sagit

Another issue that I am facing (maybe I should write about it in a different thread) is the Mustache rendering.

That's definitely strange, yes! If you have time, maybe you could open a separate thread and include some more details on how you're integrating the HTML template etc.

[dijana-sagit (Dijana)]

Thanks @ines, I figured out the problem. I was using the template config in a wrong place!