Documents annotations (from .pdf,.doc,.docx resumes)

[ines (Ines Montani)]

@KMJ

Due to those problems, I have some low scores on entity recognition, and I will need A LOONG LONG time to be able to get a usable model.

So, my idea is to transform doc to HTML to preserve the structure of my documents, and then annotate them in Prodigy. But, I don't know if it's the best solution, does someone already had those problems ?

I think the problem here goes a bit deeper: ultimately, what you'll be updating your model with is raw text. No matter how you present or annotate your data, at the end of it, you need to feed the model raw text and something you want it to predict (labels, character offsets into the text etc.).

If your data is HTML and you're rendering that, there's no clear answer for how to resolve annotations you create back to the original text, or how to deal with more complex markup. And at the end of it, you still have the same raw fragmented text, with or without added markup. I've explained some of the considerations and reasoning behind this in the following threads:

Using ner.manual on HTML Input

Hi! I think the problem here is that "html" input only works with the "html" annotation interface. If you’re annotating data with ner.manual, you’ll be selecting and labelling tokens – and that really only works on raw text. That’s why the recipe will only use the "text" key that’s present in your data. If you pass in "<strong>hello</strong>", there’s no clear solution for how this should be handled. How should it be tokenized, and what are you really labelling here? The underlying markup or ju…

Does Prodigy support HTML annotation for NER

Annotating rendered HTML might sound appealing at first, but there’s actually not really an easy answer for how the annotations should be resolved back to the underlying raw text and how to ensure that annotations are consistent. After all, what your model will get to see is the raw text. I discuss some of these considerations in more detail on this thread: One common solution is to write a function that takes raw HTML, strips out the markup, tokenizes the text and stores each token’s charact…

If you're working with tabular data with very little natural language texts, it's possible that approaching this as a basic sequence tagging / NER problem just isn't a good fit. NER works well for tasks where you need to predict exact boundaries based on the surrounding tokens – like mentions of names and concepts in text. But if you have no context and no real text, it's not surprising that you're seeing poor results.

For the specific use case here, I think some more preprocessing and extraction rules can make a big difference. You don't need deep learning to figure out what time periods like "2010 - 2020" are. And if you know that you have a table, you can use a PDF extraction tool that extracts the tables as a CSV (or similar), so you know what text belongs together and is part of the same column and work from there. [job title] at [company] is probably a super common construction, so you can easily cover those and focus on the more difficult cases, and maybe that's where you actually want to start predicting custom things.

An alternative approach that I've been seeing more often is framing the whole problem differently and as a computer vision task. This seems to be especially effective if the visual strucutre of the documents holds a lot of important clues, like in an invoice. So the model would then predict where the recipient or total amount is, and in the next step, you'd use OCR to convert the contents of the bounding box to text. This approach is more involved, though, and potentially overkill for this specificl use case.