# SpanCat Training Error on Custom Preprocessed Dataset **URL:** \ **Category:** Uncategorized\ **Tags:** usage, training, spancat\ **Created:** [March 3, 2023, 10:25am UTC](https://support.prodi.gy/t/spancat-training-error-on-custom-preprocessed-dataset/6405 "2023-03-03T10:25:46Z")\ **Posts on this page:** 1\ **Showing post:** 2
**Author:** ![ryanwesslen](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ryanwesslen/32/2969_2.png) [@ryanwesslen](https://support.prodi.gy/u/ryanwesslen)\ **Post date:** [March 3, 2023, 12:56pm UTC](https://support.prodi.gy/t/spancat-training-error-on-custom-preprocessed-dataset/6405/2 "2023-03-03T12:56:07Z")
hi @daffahilmyf! Thanks for your question and welcome to the Prodigy community 👋 Thanks as well for providing your examples. This helped a lot! I noticed that in your updated file it looks like you stemmed and removed stop words. Is this correct? - Original (worked): "text": "In order to fulfil the requirements of some railways, it should be possible to provide an alternative means of link assurance indication." - New (didn't work): "text": "in order fulfil requir some railway possibl provid altern mean link assur indic" Can you explain more on why the pre-processing is needed? There's no need to do pre-processing like this and we generally recommend against doing it. For example, this is a good post on the background on stop words: > [@Do stop words affect text classification in Prodigy?](https://support.prodi.gy/t/do-stop-words-affect-text-classification-in-prodigy/3645/2): > > Hey, In general I would advise against stop-word removal, and I'd definitely advise against storing data that's been transformed (especially a lossy transformation like stop-word removal). The history of stop-word removal goes like this. Early work on text classification was heavily influenced by information retrieval (search engines, basically). In early search engines, you started out by making a term-document matrix, a table where you could look up a term and get all the documents mentionin… --- _[View the full topic](https://support.prodi.gy/t/spancat-training-error-on-custom-preprocessed-dataset/6405)._