# HTML Source Sentence Boundary Detection Prodigy

**URL:** <https://support.prodi.gy/t/html-source-sentence-boundary-detection-prodigy/1451>\
**Category:** Uncategorized\
**Tags:** usage, spacy\
**Created:** [April 26, 2019, 4:56pm UTC](https://support.prodi.gy/t/html-source-sentence-boundary-detection-prodigy/1451 "2019-04-26T16:56:47Z")\
**Posts on this page:** 1\
**Showing post:** 3

<div class="post-metadata">

**Author:** ![nix411](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/nix411/32/2982_2.png) [@nix411](https://support.prodi.gy/u/nix411)\
**Post date:** [November 28, 2019, 11:45am UTC](https://support.prodi.gy/t/html-source-sentence-boundary-detection-prodigy/1451/3 "2019-11-28T11:45:28Z")

</div>

> [@SmoothRecursion](#):
>
> Does anyone have any good strategies on preprocessing html to get it to a usable form in Prodigy?

I've made a parser that segments html into a list of paragraphs (if you see a new line on the HTML page then you get a new paragraph). Is that something you could use? However note what Matthew Honnibal said

> [@honnibal](#):
>
> Mark-up languages like HTML often provide important non-text information in the formatting. If you extract to plain text, you might lose information about what’s a list, what’s a section, etc. Obviously spaCy expects to work on raw text — but that doesn’t mean the non-text aspects aren’t going to be important in other ways.

I have this exact issue. I actually want some of the non-text aspects as features but I don't know how to achieve that in a spaCy framework. I've laid out my challenge in another [thread](https://support.prodi.gy/t/teaching-a-spacy-model-to-attend-to-the-right-n-gram/2265).

---

_[View the full topic](https://support.prodi.gy/t/html-source-sentence-boundary-detection-prodigy/1451)._
