Thanks – this is actually something I’ve been thinking about a lot, and I still don’t have a perfect solution. The main problem is that in order to interact with the current annotation task that’s displayed, user scripts would also need to be able to call the pre-compiled app’s internals (more specifically, dispatch an action). This is pretty difficult.
Because Prodigy is a developer tool and only intended to be hosted internally, preventing cross-site scripting isn’t such a high priority here. So executing random user code wouldn’t be a problem either – but it also wouldn’t be very useful.
I did experiment with a few possible solutions, including some crazy stuff like evaling raw JSX with Babel. To my surprise, it worked – but it still feels very wrong.
In theory, we could also allow users to add custom hooks to their markup to tell Prodigy how to pass values forward (like <input data-prodigy-field="user_text">). But markup added via dangerouslySetInnerHTML is essentially a black box, so it would always require the front-end to actually parse the HTML and convert it to React elements. This seems doable – I’m just not sure if it’s a reasonable approach. However, It still feels more natural to me than reinventing the wheel and coming up with our own templating API – like, an arbitrary JSON format or something like that. After all, this API already exists: it’s HTML and JavaScript.
Another question is how opinionated we want the Prodigy app to be, and to what extent the front-end should be allowed to modify the incoming data. Allowing it to freely edit annotation tasks is potentially problematic – a small bug or inconsistency can easily ruin an entire dataset without the user even noticing. So initially, the front-end’s editing privileges were limited to the "accept" property only. (With the choice and manual NER interface, we ended up adding "accept" and "spans" as well.)
Anyway, I’m curious to hear your thoughts on this! (Here’s a related discussion on user input fields in Prodigy btw.)