# Lengthy newlines etc rendering oddly in UI

**URL:** <https://support.prodi.gy/t/lengthy-newlines-etc-rendering-oddly-in-ui/557>\
**Category:** Uncategorized\
**Tags:** done, front-end\
**Created:** [May 22, 2018, 6:36pm UTC](https://support.prodi.gy/t/lengthy-newlines-etc-rendering-oddly-in-ui/557 "2018-05-22T18:36:51Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![hannahlindsley](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/hannahlindsley/32/285_2.png) [@hannahlindsley](https://support.prodi.gy/u/hannahlindsley)\
**Post date:** [May 22, 2018, 6:36pm UTC](https://support.prodi.gy/t/lengthy-newlines-etc-rendering-oddly-in-ui/557/1 "2018-05-22T18:36:51Z")

</div>

Not sure if it’s just long tokens or only these particular character repetitions. When we highlight them, they line-wrap as expected. Prior to that, however, they look like this:

 ![10 PM](https://us1.discourse-cdn.com/flex020/uploads/prodigy/original/1X/e1f193edcd0551b6b0a3539ecb4aba1274b9e657.png) ![28 AM](https://us1.discourse-cdn.com/flex020/uploads/prodigy/original/1X/80a17cee39801b136d6d1bbbe68ef509b3c8cf87.png)

---

<div class="post-metadata">

**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\
**Post date:** [May 22, 2018, 6:50pm UTC](https://support.prodi.gy/t/lengthy-newlines-etc-rendering-oddly-in-ui/557/2 "2018-05-22T18:50:13Z")

</div>

Thanks for the report, this should definitely render better! Are those long sequences one token or multiple tokens in a row?

Btw [this thread](https://support.prodi.gy/t/segmentation-and-newlines-in-ner-manual/494/14?u=ines) has some more background on newlines in general, why the manual mode replaces them with symbols and the changes and new options that are coming in the next version.

---

<div class="post-metadata">

**Author:** ![hannahlindsley](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/hannahlindsley/32/285_2.png) [@hannahlindsley](https://support.prodi.gy/u/hannahlindsley)\
**Post date:** [May 22, 2018, 7:17pm UTC](https://support.prodi.gy/t/lengthy-newlines-etc-rendering-oddly-in-ui/557/3 "2018-05-22T19:17:07Z")

</div>

Thanks!  
Our custom tokenizer treats them as a single token; we don’t end up with the same problem if we split them as multiples, fwiw.

---

<div class="post-metadata">

**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\
**Post date:** [May 22, 2018, 7:25pm UTC](https://support.prodi.gy/t/lengthy-newlines-etc-rendering-oddly-in-ui/557/4 "2018-05-22T19:25:13Z")

</div>

Ah, okay! Maybe Prodigy should just force line breaks for _very_ long tokens. Alternatively, I’ll also experiment with a solution that adds a space behind the replacement symbols.

For your use case, you’d probably also want an option to turn off added “real” line breaks when we ship the feature [I describe here](https://support.prodi.gy/t/segmentation-and-newlines-in-ner-manual/494/14?u=ines), so you don’t end up with 3828342 newlines, right? I might add an option for this, just in case.

---

<div class="post-metadata">

**Author:** ![hannahlindsley](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/hannahlindsley/32/285_2.png) [@hannahlindsley](https://support.prodi.gy/u/hannahlindsley)\
**Post date:** [May 22, 2018, 7:26pm UTC](https://support.prodi.gy/t/lengthy-newlines-etc-rendering-oddly-in-ui/557/5 "2018-05-22T19:26:10Z")

</div>

Yep, that sounds great.

---

<div class="post-metadata">

**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\
**Post date:** [June 7, 2018, 5:56pm UTC](https://support.prodi.gy/t/lengthy-newlines-etc-rendering-oddly-in-ui/557/6 "2018-06-07T17:56:49Z")

</div>

Just released `v1.5.0`, which includes the following fixes:

- newline characters within a token should now be separated by spaces, which means that long lines should wrap
- by default, newline tokens will now come with added line breaks – but you can turn this behaviour off by setting `"hide_true_newline_tokens": true`
- plus everything [in this post](https://support.prodi.gy/t/segmentation-and-newlines-in-ner-manual/494/14)
