Segmentation and newlines in ner.manual

Quick update: I tested the "ā†µ plus line break" solution and it's been working well ā€“ so we will be able to ship this update with the next release :tada:

I've also been experimenting with solutions for use cases like this one and how to allow adding more visual clues to the manual interface:

In the upcoming version, you'll be able to mark individual "tokens" in the input data as "disabled": true. This will render them in grey and will prevent the user from selecting those tokens (or any text spanning across them). The disabled tokens can be used for whitespace characters, list bullets and other tokens purely intended for formatting, and they can also help the annotator identify what's important quicker. You can also use them to prevent highlighting mistakes (e.g. by disabling all newline tokens to not allow entities spanning over two paragraphs). The "disabled" property can also make it easier later on to separate annotator-only markup from the annotated text.

1 Like