Using Fastext vector model in Prodigy?

ines · November 16, 2017, 3:59pm

Yes! All you have to do is add the vectors to spaCy, save out the model and load it with Prodigy. We actually have an end-to-end example script for adding fastText vectors to spaCy: see this page for the code and more details. Here’s the script:

github.com

explosion/spacy/blob/master/examples/vectors_fast_text.py

#!/usr/bin/env python
# coding: utf8
"""Load vectors for a language trained using fastText
https://github.com/facebookresearch/fastText/blob/master/pretrained-vectors.md
Compatible with: spaCy v2.0.0+
"""
from __future__ import unicode_literals
import plac
import numpy

import spacy
from spacy.language import Language


@plac.annotations(
    vectors_loc=("Path to .vec file", "positional", None, str),
    lang=("Optional language ID. If not set, blank Language() will be used.",
          "positional", None, str))
def main(vectors_loc, lang=None):
    if lang is None:

This file has been truncated. show original

After adding the vectors, you can simply call nlp.to_disk('/path/to/model'), which saves the full model including the vectors to a directory. You can then load it into Prodigy using that path – for example:

prodigy terms.teach fruits_dataset /path/to/model --seeds apple,pear,banana

Topic		Replies	Views
Loading fasttext vectors to spacy/prodigy ner , spacy , solved	9	1399	February 13, 2022
biomedical nlp models in spacy usage , spacy , solved , gensim	4	2328	February 28, 2018
Do the outputted models using textcat.batch-train make use of word vectors? usage , textcat , spacy	2	560	March 28, 2019
German sense2vec model usage , spacy , sense2vec	4	608	September 23, 2021
Convert Gensim FastText to spaCy-readable Word2Vec format for terms.teach recipe spacy , terms , solved , gensim	4	1369	September 11, 2020

Using Fastext vector model in Prodigy?

Related Topics