javascript in views other then html - add position information to span elements

[pat]

@ines

Just to make sure I understand what you’re trying to do, could you share an example of the data you want to feed in?

Sure, I’m happy to. I have attached a random invoice example from google picture search (it doesn’t reflect or domain, but its good enough for illustration purpose).

Sample input: could be a picture or real pdf

prodigy: some output text (only about 1/4 of the whole) how its rendered in via ner.manual in prodigy
As you can imagine someone who has to annotate this will hate me

some poc how I would like it to render for the people who have to annotate.
the tokens does currently not exactly match those tokens generated from prodigy but that’s ok so far as well does it not show newline chars.
some tokens are already selected with selection handler and that stuff (marked in yellow)
Its by far not perfect as you can clearly see in the first line but someone who has to annotate should be way faster then with a wall of text.

What i want to archive: picture -> ocr -> annotate or pdf -> convert -> annotate. it nearly doesn’t matter if its image or pdf, the intermediate after ocr/converting is nearly in the same for both.

@ines

And is there any way you can do more preprocessing on the server side to extract only the text you’re looking for, annotate that in smaller chunks in the manual interface and then put it back together based on the character positions and meta? This means you won’t have to rely on both the front-end and back-end to do data transformation, which introduces a much higher error potential and risks if they’re out-of-sync.

Thought about that as well. It would be cool to analyse the content of the document and better detect tables, letterheads ... for example I have read this post [tables and text(Tables and Text) in the forum where honnibal said it could be possible to use the image annotation tools which prodigy provides.

I think this is a really cool idea but 1.) my lack of knowledge is an issue 2.) it currently works well with the wall of text. So i think its better to focus now to ship the product asap with maybe a not perfect, but reasonable extraction accuracy and refine it later witch is definitely on the to-do list

@ines

Do you mean pre-defined spans in the input data? If so, it’s possible that the manual interface currently doesn’t retain custom metadata on spans, as it assumes they’ll be added/removed/edited anyways. But I’ll need to double-check this!

Ah no sorry, i meant the html span tags in the annotation interface. I have attached 2 images, one with the mentioned ids and one without them after the tokens are selected.

Spans with id:

Spans without id:

@ines

This is a good point – I definitely want to make all interfaces expose the task data and callbacks if custom JavaScript is enabled, and also allow executing custom scripts across interfaces.

That would be great! Maybe it would be even possible to create on annotation interfaces like diff/html and so on for specific tasks, e.g. annotate PDFs and expose even more internal functions or allow some sort of plugin system? (I recently stumbled over gatsy.js and short experience with it the plugin system is great - https://www.gatsbyjs.org/plugins/ maybe something like this would be also possible? - in the far future of course )