# revising annotation by prodigy--here only one label (DATE)

**URL:** <https://support.prodi.gy/t/revising-annotation-by-prodigy-here-only-one-label-date/1518>\
**Category:** Uncategorized\
**Tags:** usage, ner, solved\
**Created:** [May 14, 2019, 12:26pm UTC](https://support.prodi.gy/t/revising-annotation-by-prodigy-here-only-one-label-date/1518 "2019-05-14T12:26:55Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\
**Post date:** [May 14, 2019, 1:33pm UTC](https://support.prodi.gy/t/revising-annotation-by-prodigy-here-only-one-label-date/1518/2 "2019-05-14T13:33:28Z")

</div>

The "mismatched tokenization" error means that one or more `"spans"` you've added to your data do not map to the tokens assigned by the model's tokenizer. For example, let's say you have a sentence like `"Next Monday~"` and you've labelled the span for `Monday` – but the tokenizer only splits the text into two tokens: `["Next", "Monday~"]`. Then your annotations will never be "true", because there will never be a token `"Monday"` in that example. If you're training a model with those examples, that would be pretty bad, because it wouldn't learn anything.

**TL;DR:** All `"spans"` should map to token boundaries. If you know what the tokenization should be (e.g. you already have the pre-tokenized text), you can also provide a `"tokens"` key in the data that tells Proidgy how the text should be split.

You can also use the method described here to find the examples with mismatched tokenization – maybe it turns out it's only one or two texts with very specific punctuation or extra whitespace.

> [@Anotation task format for ner\_manual interface](https://support.prodi.gy/t/anotation-task-format-for-ner-manual-interface/1501/2):
>
> Hi! What happens if you just load in your data the way it is? If no "tokens" are present, Prodigy will use the model’s tokenizer to generate them automatically for you slightly_smiling_face The only thing that’s important is that your entity spans align with the token boundaries – for instance, “Apple” is fine, because that’ll be one token. But the span “Apple up” wouldn’t because “up” isn’t a standalone token. If the pre-annotated spans don’t align, Prodigy will raise an error and show you t…

> [@myeghaneh](#):
>
> python -m prodigy ner.manual ner\_01 en\_core\_web\_sm NER\_01.jsonl
> 
> then I had all labels like ORG. and which is not my case…could you let me know what is wrong

That's expected. If you do not specify a `--label` on the command line, Prodigy will use the labels that are present in the model. If you're running `ner.manual`, you usually always want to specify one or more label options via the `--label` argument. For example, you could have your examples pre-labelled with `DATE`, and then manually add another label `PERSON`. In that case, you could write `--label DATE,PERSON`, and you'd be able to select those labels.

---

_[View the full topic](https://support.prodi.gy/t/revising-annotation-by-prodigy-here-only-one-label-date/1518)._