Training a grammar tool

,
22

One more thing, when i try to use a model trained with the custom classifier. i get this error:

>>> import spacy
>>> nlp = spacy.load("./data/models/grammar7")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ohenrik/.pyenv/versions/upfeed_analyser/lib/python3.6/site-packages/spacy/__init__.py", line 19, in load
    return util.load_model(name, **overrides)
  File "/home/ohenrik/.pyenv/versions/upfeed_analyser/lib/python3.6/site-packages/spacy/util.py", line 116, in load_model
    return load_model_from_path(Path(name), **overrides)
  File "/home/ohenrik/.pyenv/versions/upfeed_analyser/lib/python3.6/site-packages/spacy/util.py", line 158, in load_model_from_path
    return nlp.from_disk(model_path)
  File "/home/ohenrik/.pyenv/versions/upfeed_analyser/lib/python3.6/site-packages/spacy/language.py", line 638, in from_disk
    util.from_disk(path, deserializers, exclude)
  File "/home/ohenrik/.pyenv/versions/upfeed_analyser/lib/python3.6/site-packages/spacy/util.py", line 521, in from_disk
    reader(path / key)
  File "/home/ohenrik/.pyenv/versions/upfeed_analyser/lib/python3.6/site-packages/spacy/language.py", line 634, in <lambda>
    deserializers[name] = lambda p, proc=proc: proc.from_disk(p, vocab=False)
  File "pipeline.pyx", line 211, in spacy.pipeline.Pipe.from_disk
  File "/home/ohenrik/.pyenv/versions/upfeed_analyser/lib/python3.6/site-packages/spacy/util.py", line 521, in from_disk
    reader(path / key)
  File "pipeline.pyx", line 204, in spacy.pipeline.Pipe.from_disk.load_model
  File "/home/ohenrik/.pyenv/versions/upfeed_analyser/lib/python3.6/site-packages/thinc/neural/_classes/model.py", line 351, in from_bytes
    dest = getattr(layer, name)
AttributeError: 'FeedForward' object has no attribute 'Q'
23

So based on this answer:

I managed to load the model by adding this first:

class GrammarTextClassifier(spacy.pipeline.TextCategorizer):
    @classmethod
    def Model(nr_class=1, width=64, **cfg):
        print("Building custom textcat model")
        return build_text_classifier(nr_class=nr_class, width=width, **cfg)

spacy.load("./data/models/small7/model7")
spacy.lang.nb.Language.factories["textcat"] = lambda nlp, **cfg: GrammarTextClassifier(nlp.vocab, **cfg)

However it still does not seem to train correctly.

24

Do you have a pre-trained parser and tagger in your model, and pre-trained vectors?

If so, then another possibility is simply that the neural network is having a tough time getting off the ground, given there’s not much data. The default architecture’s linear model does help with that. It’s also possible that fiddling with the hyper-parameters could help – it’s tough to say.

25

I’m using the Norwegian model trained to create tags, pos and dep. It is what i have used to create the matcher logic and find potential word split errors. Is it important to add a vector model to this? (I’m assuming you mean Word2Vec or glove).

Maybe i just need to add more data

26

You don’t necessarily need pre-trained vectors, but if you don’t have them loaded, remove the static_vectors part from the model.

The network is wired up so that before the CNN, we get the word embeddings two ways: we’re training word vectors, now with dependencies and POS tags, and we also concatenate in the static vectors. If you don’t have any vectors set, that might be a problem – I think it might be taking random values? Not sure.