So, after a bit of experimentations, we settled (for now) on a sliding window of around 10 lines of text, to annotate our documents. Line breaks in our documents are significant so we cannot just throw them away, and there is an issue on relations. Some relations are between entities that are really far apart (sometimes up to 5 lines) which means we cannot annotate all relations in one sliding window as some might be outside the example while others are present and must be annotated...
I don't know whether to mark examples where we only have some values and relations but not all of them as rejected examples, or if we should annotate them anyway, even if they are missing relations.
I foresee some issues trying to train a relation model with all these separate annotation of 10 lines windows, given that on some windows we will have relations with out of the window entities... I am not sure about how to make this workflow better. Do you have any idea ? I'm thinking about maybe trying to add a pseudo-sentencizer that separates blocks of significant value (where the relations are sure to be) but it's a lot of extra work...