Custom view templates with scripts

@SofieVL Yay, that's nice to hear! And yes, the html_template is currently limited to the html – although I think you can kinda trick the app into using it to render the content in interfaces like choice or classification by passing it an empty "html" property in the task.

Custom CSS and JavaScript are now supported for all interfaces, so you can always do something like this and append elements to the annotation card:

const content = document.querySelector('.prodigy-content')
const button = document.createElement('button')
button.textContent = 'Click me!'
// add event listener / onclick handler etc.
contend.appendChild(button)
1 Like

@ines: you're right, it does kind of work when specifying the html tags in the choice interface properly:

However it looks like you can't actually type in the box. Because as soon as your mouse goes there and selects the box, the radio button is selected instead, and focus moves aways from the text box.

Ah yeah, the choice options weren't really designed to have interactive content in them. So you probbly want to just add the field at the bottom below the options (instead of actually making it an option itself).

Hi @ines,
my window.prodigy.content is pointing to the wrong content when using view_id blocks.
My html_template consist of buttons and the js_templates holds the onClick functions. The OnClick are supposed to highlight some words. I planned on doing this by appending Spans to window.prodigy.content.spans and then updating it. However, it is not showing on the text visualization of the ner_manual block. May I know where I was wrong?
Thanks!

recipe.py
    "config": {
      "block": [
       { "view_id":"ner_manual", "labels": ["PERSON","ORG"] },
       { "view_id": "html", "html_template": get_html_template()}
      ],
      "javascript": get_js_template()
      }

Hi! The ner_manual interface currently keeps its own copy of the spans – calling update does update the task content, but it doesn't necessarily trigger a re-render of a built-in component. I do want to change this, though, since use cases like the one you describe are pretty cool. It's just not implemented yet.

You can read more about it here:

Hi @ines,
Thanks! I understand!
As a temporary workaround, may I ask if there is any way for me to trigger React to re-render? Perhaps using a pseudo state change?

Ah, good idea! It's hacky, but you could make it think that it's receiving a new task by updating the _task_hash of the current task. Just tested it and the following works for me:

const taskHash = window.prodigy.content._task_hash
window.prodigy.update({ _task_hash: 0 })
window.prodigy.update({ _task_hash: taskHash })

Just make sure to reset it to its original value again – otherwise, you'll end up with bad task hashes in your dataset (which impacts what Prodigy considers duplicates etc.). Maybe also try it with a test dataset first to check that there are no unintended side-effects on the client.

Hello @ines, do you know how I can update the selected label using custom view templates?

Here's what I have so far:

Custom dropdown HTML:

<div class="custom-dropdown" onchange="updateLabel()">
  <select id="labels">
    {label_options_str}
  </select>
</div>

Custom JS:

function updateLabel() {
    const selectedLabel = document.querySelector('select').value;
    console.log(selectedLabel)
    window.prodigy.update({ label: selectedLabel });
}

I am able to confirm that the custom javascript is correctly logging the selected label, but I'm not sure how to actually set the selected label in prodigy, since I don't see any examples of updating the selected label in the window.prodigy interface.

The .update() call in my custom JS was a wild guess, and sure enough, it didn't work (it's just using the first item in the dropdown, which is NO_LABEL in the below screenshot):

So, in summary, my question is: Is it possible to update the label from a custom HTML interface like this?

Thanks in advance!

-Luc

P.S. The reason I'm using a custom dropdown is that the default dropdown for a list of labels provided by the --dropdown prodigy option is not flexible enough for our image_manual use case, which requires using several <optgroup>'s in the select bar.

1 Like

Glad you got it to work so far :smiley:

The window.prodigy.update function will update the current task dictionary with whatever data you pass in. So in your code, you're adding the top-level property "label" to the dictionary – but I think that's not what you want, right? You'd want the label to be added to the entry in the "spans" that was just added.

Hi @ines, thank you for the quick reply.

You'd want the label to be added to the entry in the "spans" that was just added.

The order of operations is reversed for us; during image.manual tasks, we need to select the label via the dropdown and then create a bounding box (or multiple boxes) for that label.

In other words, we are selecting the label first (e.g. INSURANCE_CARD), and then using it to highlight which parts of the image contain the data that need to be labeled (e.g. the actual sections of the image that contain the cards).

For some of our image.manual tasks, there is also the possibility that there are spans in the image for different types of labels (e.g. INSURANCE_CARD_FRONT and INSURANCE_CARD_BACK within the same image). This is one reason why we can't automatically apply a selected label to all spans when the user selects an item in the labels dropdown.

Is there any Javascript interface for selecting the label that will be used when annotating the following/next span(s) in Prodigy?


P.S. I inspected the minified JS source code and noticed that there is a function named handleLabelSelect() which appears to call something like setState({label}) in what I'm guessing is the frontend component that allows the user to select a label. Since this appears to determine which selected label is used when labeling spans in the image, I believe this is the interface I'm trying to hook into from our custom JS code. I understand if/why the actual setState call would not be exposed in the JS API (for security and state encapsulation reasons) but I'm wondering if there's a way to modify the selected label via the window.prodigy API documented in this support thread and in the PRODIGY_README.html.

At the moment, the call is not exposed and the prodigyspanselected event is currently only used in the manual NER interface, so you couldn't use that one. If you're not using the built-in logic to select labels, this is probably how I would go about it, though: when the task is updated and a span is added, you can update it with the currently selected label.

One workaround could be to listen on prodigyupdate and keep track of the "spans". It's not as clean, but you know a span was added if the task hash is the same (e.g. the current task hasn't changed) and there are more spans than on the last update :sweat_smile:

Btw, as of v1.9, the PRODIGY_README.html has been replaced by the docs live on the website: Prodigy 101 – everything you need to know · Prodigy · An annotation tool for AI, Machine Learning & NLP. So this will always be the most up-to-date reference.

Thanks for the tip, I'll try that!

Ah, good to know. I wasn't sure which should be the source of truth.

At the moment, the call is not exposed and the prodigyspanselected event is currently only used in the manual NER interface, so you couldn't use that one.

That makes sense that not all of this is exposed in the current version of the app... although some of our company's image labeling projects have some very specific labeling UI/UX requests. Right now we have access to the minified UI code in our browser, but if we wanted to extend the frontend to support other types of events, it could be very useful to have the non-minified source code. That way we could potentially fork the frontend, add features, build the app with our extensions internally, and coordinate with your team regarding whether or not some features make it upstream. Could that (sending us a copy of the unminified code) be an option for our team? Or does the Prodigy team deny such requests?

Yeah, it should be pretty easy to fire prodigyspanselected for image spans as well, so I'll put this on my enhancement list for the next version :+1:

We want everyone who purchases a Prodigy license to get the same thing, so this wouldn't really be an option for us.

1 Like

Hi @ines,
I came across this thread as it was referenced in the documentation. I think my problem is somewhat similar to the previous comments, but I wanted to check if there is a workaround. Fig. 1 roughly shows what I want to achieve. In the custom html area I would like to listen to the selected span and print some of the metadata (text and label in my case). Also, as I change the span, I would like to trigger the event and "recalculate" the text in that span. I don't think I can use prodigyupdate as this event doesn't trigger on span selection.


Figure 1. Visual example of the intended functionality

I would appreciate your input.

Thanks,
Dijana

Ohh, this is super cool :star_struck:

I just tested it and for me, the prodigyupdate event fires every time an update is made to the UI, including new boxes that are added or boxes that are edited. The most recent update (v1.10.5) also adds the prodigyspanselected event to the image_manual UI that should fire every time a new span is added.

The only information that's currently not easily exposed is the span that's currently selected in the UI (as in, editable) :thinking:

Thank you, I appreciate it!

Regarding the prodigyspanselected event, it doesn't seem that if fires for me for image_manual view (I am using Windows wheel v1.10.5), while prodigyypdate works.
Another issue that I am facing (maybe I should write about it in a different thread) is the Mustache rendering.
For example, with the following html template:

{{#spans}}
<strong>{{text}}</strong>
<br>
{{/spans}}

I get:
{{spans}}{{text}}
{{/spans}}

So, the html gets interpreted, but the variables from the task not. I feel like I am doing some rookie mistake.

That's definitely strange, yes! If you have time, maybe you could open a separate thread and include some more details on how you're integrating the HTML template etc.

1 Like

Thanks @ines, I figured out the problem. I was using the template config in a wrong place!

1 Like