I recorded a new video In this video, I'm training a named entity recogntion model from scratch, using semi-automatic annotation with sense2vec vectors and improving a model in the loop, plus some cool transfer learning stuff! The goal is to analyze 2m+ comments posted to Reddit's r/Cooking subreddit to find out how mentions of ingredients change over time, and to create a cool bar chart race animation.
Annotation took about 2.5 hours and the results are pretty interesting. I've open-sourced all code, raw data, annotations and results so you can check it out and play with it. I'm sure there's a lot more to explore in the data.
Great video, thanks for sharing! I was hoping to hear a bit more about the use case for --init-tok2vec and the process you took to create that component-- is that something you might add to the video or add a bit more color to here?
The pretraining process itself is not that exciting, to be honest It's really just running spacy pretrain on lots of raw text from Reddit for a while (~8 hours on GPU). The tok2vec_cd8_model289.bin artifact was trained with a depth of 8 and for 289 iterations. You can read more about spacy pretrain here: Command Line Interface · spaCy API Documentation
I know this might be too much of a basic question, but I still can't figure out how to pretrain using the GPU. There doesn't appear to be any way to do this via CLI. Please let me know what I'm missing, @ines! Hopefully I am not the only one confused on this.
Ah, so what's the problem, does it not detect the GPU? If you have a GPU available, spacy pretrain should "just work", detect the GPU and train on GPU.
@ines thank you for sharing this, quick one, you use prodigy sense2vec.teach to then prodigy terms.to-patterns to compile a list of patterns which is in fact just a plain text jsonl
Does downstream NER model which you then train the video make any use of that sense2vec model trained under the hood of the NER maybe? or its just the word verbatim patterns which are actually used?
The video is a bit dated and the commands are slightly different now, but terms are just terms used at verbatim. Their main use-case is to help pre-fill annotations which makes annotating much easier, which in turns makes it much easier to get your first model ready via prodigy train. If you're interested in a more recent tutorial with sense2vec, you might enjoy this video where I use it to detect video games.
In this video, I'm training a named entity recogntion model from scratch, using semi-automatic annotation with sense2vec vectors and improving a model in the loop, plus some cool transfer learning stuff! The goal is to analyze 2m+ comments posted to Reddit's r/Cooking subreddit to find out how mentions of ingredients change over time, and to create a cool bar chart race animation.
It's really just running 