Can i train my models with smaller sentences?

usage
(Damiano) #1

Hello,
i am getting memory error when try to train my model with big documents. So the question is simple, can i create small chucks of my documents…lets say: 10 words on the left 10 words on the right
If yes, then…what happen when i use the original document to get the .ents?

Thanks

0 Likes

(Matthew Honnibal) #2

Creating arbitrary slices is probably unideal. Can’t you use some natural unit of the document, like paragraphs?

0 Likes