Add vectors to nlp model using terms.train-vectors

[beckerfuffle (Michael Becker)]

Not sure if this will help you but I think I found a bug in the recipe:

terms.train-vectors reads in entire dataset into memory

I was reviewing the terms.train-vectors recipe and I noticed that it reads in the entire dataset before calling word2vec: for doc in nlp.pipe((eg['text'] for eg in stream)): for sent in doc.sents: sentences.append([w.text for w in sent]) print("Extracted {} sentences".format(len(sentences))) w2v = Word2Vec(sentences, size=size, window=window, min_count=min_count, sample=1e-5, iter=n_iter, workers=n_workers, negative=negative…