Adding a helper image

[jordandavis]

I'm annotating a textcat model for text originating from PDFs. The visual format of each document is very different which makes it much easier to annotate based on the image rather than the plain text. Therefore, I'd like to be able to see the page as an image but have the text of the page saved in the db.

I'm a little stuck on how to implement this, please let me know if anyone has any suggestions.

[ryanwesslen]

hi @jordandavis!

Have you seen my colleague @ljvmiranda921's recent blog where he created a PDF processing workflow with Prodigy?

A framework for designing document processing solutions

There is also an accompanying GitHub repo with a spaCy project and custom Prodigy recipes.

I have only dabbled with the project so I don't know all of the details, but it has been very popular with many Prodigy users. What's cool about the project is that it also uses HuggingFace's LayoutLMv3 that combines both text and image masking and fine tunes the model. The project uses the FUNDU dataset so likely to adapt this you'll need to learn how that dataset is structured and mimic it for your own data.

While this may not be a perfect solution, hopefully it provides a concrete idea of an approach. As you've probably seen, we typically recommend (see below) for pdfs either to OCR text and use that text in Prodigy or treat them as images.

Taking a Computer Vision Approach (leveraging image.manual) to build a custom NER model on PDFs

Hi Jetson, here are the answers to your questions: Yes, correct. Prodigy doesn't natively support PDFs. However, you can choose to write your own custom recipe that is able to use them. That way, you could consider using a Python package that can parse .pdf files. You might be able to consider this if your pdfs follow a very strict structure, but the image OCR path seems like a more common approach. An annotated image will have bounding boxes with the data format described here. You may apprec…

Using prodigy with PDF documents

Thanks for your question! I actually wrote a simple PDF loader a while ago using PyPDF2. It’s probably not the most elegant solution, and also just extracts the verbatim text of the page: from pathlib import Path import PyPDF2 def get_pdf_stream(pdf_dir): pdf_dir = Path(pdf_dir) for pdf_file in pdf_dir.iterdir(): file_obj = pdf_file.open('rb') reader = PyPDF2.PdfFileReader(file_obj) page_count = reader.numPages text = '' for page in range(page_co…

Prodigy UI Customization

Hi! If you're working with PDFs and are looking to do train text-based models, the first step would be to extract the raw text from the PDF files. There are different libraries and solutions for this, and here's an example using PyPDF to extract a JSON file in Prodigy's format: Using prodigy with PDF documents - #2 by ines If you want a more UI-based workflow, you could put a mini service in front of Prodigy that includes an upload field and then starts Prodigy in a subprocess on the server usi…

Hope this helps and definitely keep us informed on whatever direction you go!

[Khadke_C (Chetan)]

Currently we are working on an invoice processing problem where we need pdf annotation tools. We plan to use LayoutLM post annotation.

1. Can we directly import the pdf and pdf's ocr file directly into Prodigy?
2. Do you provide an export option which is compatible with LayoutLM?
3. Do you provide active learning while doing the annotation?
4. Do you support the training option for LayoutLM directly?

[ryanwesslen]

hi @Khadke_C,

I moved your post about @ljvmiranda921's post/project because I think my answer yesterday answers a lot of your questions.

Let us know if you have questions!

[jordandavis]

Thanks Ryan,

The blog post is super cool. I've been interested in getting into the Layout models but am trying to get a better understanding of Prodigy's workflow before I delve into more complicated projects. I managed to get the images to appear by adding the fetch_images wrapper on my stream.