Questions on difficulty of implementing the following features in a custom recipe

[ines (Ines Montani)]

@tmpr

Could you, given that you have a good overview of the tool, give us some insight as to how difficult the following would be to code up?

Hi! Most of what you describe definitely sounds possible so it's a question of deciding how things should work and how comfortable you are implementing a bit of JavaScript.

@tmpr

Relation- and entity-attributes: we would like to, for example, add a ’negated’ attribute to entities:

By attribute, do you mean an additional label? Whether the entity was negated? Depending on what you want to do here and what the end goal is, you could probably model this as a single flat label scheme.

@tmpr

Lookups: Suppose a text contains an abbreviation like “HTN”: Can one add a button, which automatically, after highlighting the respective abbreviation, googles “medical abbreviation ”?

This is something you could definitely do with a bit of custom JavaScript: Custom Interfaces · Prodigy · An annotation tool for AI, Machine Learning & NLP Assuming you're using an interface like ner_manual or spans_manual, You can listen to the prodigyspanselected (fired when span is selected) or update event (fired when current annotation changes), look at the window.prodigy.content.spans for a list of all spans and insert a button or link that points to the Google search results for the span.text.

@tmpr

Fragmented entities: Suppose there is the text “abdominal- and chest-pain”. Is there a way to mark two entities, “abdominal-pain” and “chest-pain” here? If not, how difficult would it be to implement this behavior?

Non-contgiuous entities are a tricky challenge in general and solving this probably goes beyond just customising an interface and comes down to the annotation scheme for it as well. I've posted som thoughts on this in the following thread:

Annotation of non-contiguous entities

Hi! Non-contiguous and potentially overlapping spans can definitely be a little tricky, especially if you want the annotation process to be clear and efficient. One option could be to just model this as a relation annotation task instead of span annotation and only focus on connecting tokens with the given entity label. For example, in your case, you would annotate "Type" + "1" → "diabetes" and "Type" + "2" → "diabetes". This way, a token or entity fragment can be part of multiple spans of diff…

@tmpr

Knowledge base integration: How difficult is it to save URI to an entity: Say we find out that “headache” has a certain URI “C1”, I would like to

You should be able to do this via a custom recipe, a Python script that lets you control how data is streamed in, what to pre-annotate (e.g. using a model) and how to present the examples for annotation. I think it'll mostly come down to deciding how you want to present the information, e.g. if you want to use a general-purpose html interface, or a combination of different interfaces using blocks: Custom Interfaces · Prodigy · An annotation tool for AI, Machine Learning & NLP

If you're working with entity linking, you might also find the following example project and tutorial useful: https://github.com/explosion/projects/tree/v3/tutorials/nel_emerson It also includes a Prodigy recipe for annotation entity links with a multiple-choice UI that shows what's possible:

explosion/projects/blob/v3/tutorials/nel_emerson/scripts/el_recipe.py

@tmpr

Javascript embedding: Can one generate outputs from dataset which can then be embedded in other applications? Say Jupyter, or something we write ourselves

Prodigy comes with a JupyterLab extension for annotation: GitHub - explosion/jupyterlab-prodigy: 🧬 A JupyterLab extension for annotating data with Prodigy In general, you can also embed the Prodigy web app via an iframe if needed, but it's not necessarily that useful: Prodigy is really designed for annotation and annotation sessions can also be stateful – so you typicall want to spin up the annotation server, create a small dataset, stop the server and run training experiments.