Anotation task format for ner_manual interface

Hi! What happens if you just load in your data the way it is? If no "tokens" are present, Prodigy will use the model’s tokenizer to generate them automatically for you :slightly_smiling_face:

The only thing that’s important is that your entity spans align with the token boundaries – for instance, “Apple” is fine, because that’ll be one token. But the span “Apple up” wouldn’t because “up” isn’t a standalone token. If the pre-annotated spans don’t align, Prodigy will raise an error and show you the mismatch, so you can fix it. It usually shouldn’t happen, though – unless your data is very noisy.