📺 Video: NER with Prodigy & Transfer Learning

I recorded a new video :tada: In this video, I'm training a named entity recogntion model from scratch, using semi-automatic annotation with sense2vec vectors and improving a model in the loop, plus some cool transfer learning stuff! The goal is to analyze 2m+ comments posted to Reddit's r/Cooking subreddit to find out how mentions of ingredients change over time, and to create a cool bar chart race animation.

Annotation took about 2.5 hours and the results are pretty interesting. I've open-sourced all code, raw data, annotations and results so you can check it out and play with it. I'm sure there's a lot more to explore in the data.

5 Likes

https://github.com/explosion/projects/ner-food-ingredients is invalid.
Can you post it again? Thank you!

Ah, sorry, I mistyped. It's the ner-food-ingredients directory in that repo, so: https://github.com/explosion/projects/tree/master/ner-food-ingredients Also updated the link above.

1 Like

Thank you, this video is very useful for me, for it is an example of NER from scratch.

1 Like

Great video, thanks for sharing! I was hoping to hear a bit more about the use case for --init-tok2vec and the process you took to create that component-- is that something you might add to the video or add a bit more color to here?

Thanks again!

The pretraining process itself is not that exciting, to be honest :sweat_smile: It's really just running spacy pretrain on lots of raw text from Reddit for a while (~8 hours on GPU). The tok2vec_cd8_model289.bin artifact was trained with a depth of 8 and for 289 iterations. You can read more about spacy pretrain here: https://spacy.io/api/cli#pretrain

Thanks! I’ll read the docs to better understand the benefits offered by pretraining.

I know this might be too much of a basic question, but I still can't figure out how to pretrain using the GPU. There doesn't appear to be any way to do this via CLI. Please let me know what I'm missing, @ines! Hopefully I am not the only one confused on this.

Ah, so what's the problem, does it not detect the GPU? If you have a GPU available, spacy pretrain should "just work", detect the GPU and train on GPU.