# ๐Ÿ“บ Video: NER with Prodigy & Transfer Learning **URL:** \ **Category:** Uncategorized\ **Tags:** ner, project, best-practices\ **Created:** [March 16, 2020, 12:05pm UTC](https://support.prodi.gy/t/video-ner-with-prodigy-transfer-learning/2662 "2020-03-16T12:05:21Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\ **Post date:** [March 16, 2020, 12:05pm UTC](https://support.prodi.gy/t/video-ner-with-prodigy-transfer-learning/2662/1 "2020-03-16T12:05:21Z")
I recorded a new video ๐ŸŽ‰ In this video, I'm training a named entity recogntion model from scratch, using semi-automatic annotation with [sense2vec](https://github.com/explosion/sense2vec) vectors and improving a model in the loop, plus some cool transfer learning stuff! The goal is to analyze 2m+ comments posted to Reddit's r/Cooking subreddit to find out how mentions of ingredients change over time, and to create a cool bar chart race animation. [![](https://img.youtube.com/vi/59BKHO_xBPA/hqdefault.jpg "Training a NAMED ENTITY RECOGNITION MODEL with Prodigy and Transfer Learning") ](https://www.youtube.com/watch?v=59BKHO_xBPA) Annotation took about 2.5 hours and the results are pretty interesting. I've open-sourced all code, raw data, annotations and results so you can check it out and play with it. I'm sure there's a lot more to explore in the data. > **[explosion/projects](https://github.com/explosion/projects/tree/master/ner-food-ingredients)** > > ๐Ÿ’ Example projects for various NLP tasks with datasets, scripts and results - explosion/projects ---
**Author:** ![frenet](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/frenet/32/1268_2.png) [@frenet](https://support.prodi.gy/u/frenet)\ **Post date:** [March 16, 2020, 7:45pm UTC](https://support.prodi.gy/t/video-ner-with-prodigy-transfer-learning/2662/2 "2020-03-16T19:45:04Z")
[https://github.com/explosion/projects/ner-food-ingredients](https://github.com/explosion/projects/ner-food-ingredients) is invalid. Can you post it again? Thank you! ---
**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\ **Post date:** [March 16, 2020, 7:54pm UTC](https://support.prodi.gy/t/video-ner-with-prodigy-transfer-learning/2662/3 "2020-03-16T19:54:48Z")
Ah, sorry, I mistyped. It's the `ner-food-ingredients` directory in that repo, so: [https://github.com/explosion/projects/tree/master/ner-food-ingredients](https://github.com/explosion/projects/tree/master/ner-food-ingredients) Also updated the link above. ---
**Author:** ![frenet](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/frenet/32/1268_2.png) [@frenet](https://support.prodi.gy/u/frenet)\ **Post date:** [March 16, 2020, 8:01pm UTC](https://support.prodi.gy/t/video-ner-with-prodigy-transfer-learning/2662/4 "2020-03-16T20:01:45Z")
Thank you, this video is very useful for me, for it is an example of NER from scratch. ---
**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\ **Post date:** [March 18, 2020, 9:55am UTC](https://support.prodi.gy/t/video-ner-with-prodigy-transfer-learning/2662/5 "2020-03-18T09:55:08Z")
---
**Author:** ![adamkgoldfarb](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/adamkgoldfarb/32/741_2.png) [@adamkgoldfarb](https://support.prodi.gy/u/adamkgoldfarb)\ **Post date:** [March 28, 2020, 12:15am UTC](https://support.prodi.gy/t/video-ner-with-prodigy-transfer-learning/2662/6 "2020-03-28T00:15:23Z")
Great video, thanks for sharing! I was hoping to hear a bit more about the use case for --init-tok2vec and the process you took to create that component-- is that something you might add to the video or add a bit more color to here? Thanks again! ---
**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\ **Post date:** [March 29, 2020, 9:43am UTC](https://support.prodi.gy/t/video-ner-with-prodigy-transfer-learning/2662/7 "2020-03-29T09:43:50Z")
> [@adamkgoldfarb](#): > > I was hoping to hear a bit more about the use case for --init-tok2vec and the process you took to create that component-- is that something you might add to the video or add a bit more color to here? The pretraining process itself is not that exciting, to be honest ๐Ÿ˜… It's really just running `spacy pretrain` on lots of raw text from Reddit for a while (~8 hours on GPU). The [`tok2vec_cd8_model289.bin`](https://github.com/explosion/projects/releases/tag/tok2vec) artifact was trained with a depth of `8` and for `289` iterations. You can read more about `spacy pretrain` here: [Command Line Interface ยท spaCy API Documentation](https://spacy.io/api/cli#pretrain) ---
**Author:** ![adamkgoldfarb](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/adamkgoldfarb/32/741_2.png) [@adamkgoldfarb](https://support.prodi.gy/u/adamkgoldfarb)\ **Post date:** [March 30, 2020, 2:13am UTC](https://support.prodi.gy/t/video-ner-with-prodigy-transfer-learning/2662/8 "2020-03-30T02:13:18Z")
Thanks! Iโ€™ll read the docs to better understand the benefits offered by pretraining. ---
**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\ **Post date:** [April 8, 2020, 9:46am UTC](https://support.prodi.gy/t/video-ner-with-prodigy-transfer-learning/2662/11 "2020-04-08T09:46:58Z")
---
**Author:** ![baxtersapp](https://avatars.discourse-cdn.com/v4/letter/b/f05b48/32.png) [@baxtersapp](https://support.prodi.gy/u/baxtersapp)\ **Post date:** [June 9, 2020, 3:07pm UTC](https://support.prodi.gy/t/video-ner-with-prodigy-transfer-learning/2662/12 "2020-06-09T15:07:19Z")
I know this might be too much of a basic question, but I still can't figure out how to pretrain using the GPU. There doesn't appear to be any way to do this via CLI. Please let me know what I'm missing, @ines! Hopefully I am not the only one confused on this. ---
**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\ **Post date:** [June 10, 2020, 8:11am UTC](https://support.prodi.gy/t/video-ner-with-prodigy-transfer-learning/2662/13 "2020-06-10T08:11:47Z")
> [@baxtersapp](#): > > I know this might be too much of a basic question, but I still can't figure out how to pretrain using the GPU. There doesn't appear to be any way to do this via CLI. Ah, so what's the problem, does it not detect the GPU? If you have a GPU available, `spacy pretrain` should "just work", detect the GPU and train on GPU. ---
**Author:** ![ysz](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ysz/32/3411_2.png) [@ysz](https://support.prodi.gy/u/ysz)\ **Post date:** [January 7, 2023, 3:23pm UTC](https://support.prodi.gy/t/video-ner-with-prodigy-transfer-learning/2662/14 "2023-01-07T15:23:25Z")
@ines thank you for sharing this, quick one, you use `prodigy sense2vec.teach` to then `prodigy terms.to-patterns` to compile a list of patterns which is in fact just a plain text jsonl Does downstream NER model which you then train the video make any use of that sense2vec model trained under the hood of the NER maybe? or its just the word verbatim patterns which are actually used? ---
**Author:** ![koaning](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/koaning/32/230_2.png) [@koaning](https://support.prodi.gy/u/koaning)\ **Post date:** [January 9, 2023, 9:46am UTC](https://support.prodi.gy/t/video-ner-with-prodigy-transfer-learning/2662/15 "2023-01-09T09:46:26Z")
The video is a bit dated and the commands are slightly different now, but terms are just terms used at verbatim. Their main use-case is to help pre-fill annotations which makes annotating much easier, which in turns makes it much easier to get your first model ready via `prodigy train`. If you're interested in a more recent tutorial with sense2vec, you might enjoy this video where I use it to detect video games. [![](https://img.youtube.com/vi/EoYHbUHr0fM/maxresdefault.jpg "Finding Video Games with Sense2Vec") ](https://www.youtube.com/watch?v=EoYHbUHr0fM)