Best way to format text for annotation?

[ines (Ines Montani)]

@Sarah Hi! There's not really an easy solution, because it's more of a conceptual problem and there's no easy answer for how to treat HTML markup. I've also posted about this in more detail on this thread, and why it's difficult to render HTML if you're highlighting manually and planning on training a statistical model using the resulting data:

Using ner.manual on HTML Input

Hi! I think the problem here is that "html" input only works with the "html" annotation interface. If you’re annotating data with ner.manual, you’ll be selecting and labelling tokens – and that really only works on raw text. That’s why the recipe will only use the "text" key that’s present in your data. If you pass in "<strong>hello</strong>", there’s no clear solution for how this should be handled. How should it be tokenized, and what are you really labelling here? The underlying markup or ju…