# Question about example data during ner.batch-train **URL:** \ **Category:** Uncategorized\ **Tags:** ner, spacy\ **Created:** [July 26, 2019, 3:32pm UTC](https://support.prodi.gy/t/question-about-example-data-during-ner-batch-train/1833 "2019-07-26T15:32:09Z")\ **Posts on this page:** 1\ **Showing post:** 2
**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\ **Post date:** [July 28, 2019, 4:09pm UTC](https://support.prodi.gy/t/question-about-example-data-during-ner-batch-train/1833/2 "2019-07-28T16:09:42Z")
There are several steps here that change the total number of individual records: 1. Merging entity spans of annotations on the same data. So if you've accepted/rejected several entities on the same text, those will be combined into one example. 2. Splitting sentences. If you don't set `--unsegmented`, long texts will be split into sentences. 3. Filtering out ignored examples with `"answer": "ignore"`. 4. Filtering out duplicates or otherwise invalid annotations. > [@kbarresi](#): > > Likewise, what does “Skipping mismatched tokens” mean? This means that the tokenization of the model doesn't match the entity offsets annotated in the data. For example, if your entity is `"hello"` in `"hello-world"`, but the tokenizer doesn't produce a separate token for `"hello"`, the entity doesn't map to valid tokens and the model can't be updated in a meaningful way. Did you import annotations created with a different process / model / tokenization? If your data was created with different tokenization, you can always provide your own `"tokens"` object on the data (see the README for format details). If you set `PRODIGY_LOGGING=verbose`, Prodigy will also show you the spans that it's skipping. In your case, it seems to be only 2, so it doesn't seem very significant given your dataset size. --- _[View the full topic](https://support.prodi.gy/t/question-about-example-data-during-ner-batch-train/1833)._