Error using GPU with an existing model

,
1

Hello @ines and @honnibal,
I tried to train the model using GPU but I couldn't able to run properly with an existing model while there is no problem running it with an empty spaCy model.

Also, when I run locally my train_new_entity_type.py for 3 labels, there isn't any problem but when I tried it with GPU I got this following error:

Loaded model 'en_core_web_lg'
Traceback (most recent call last):
  File "train_new_entity_type.py", line 4603, in <module>
    plac.call(main)
  File "/root/miniconda3/envs/ner/lib/python3.7/site-packages/plac_core.py", line 328, in call
    cmd, result = parser.consume(arglist)
  File "/root/miniconda3/envs/ner/lib/python3.7/site-packages/plac_core.py", line 207, in consume
    return cmd, self.func(*(args + varargs + extraopts), **kwargs)
  File "train_new_entity_type.py", line 4573, in main
    nlp.update(texts, annotations, sgd=optimizer, drop=0.20, losses=losses)
  File "/root/miniconda3/envs/ner/lib/python3.7/site-packages/spacy/language.py", line 449, in update
    proc.update(docs, golds, drop=drop, sgd=get_grads, losses=losses)
  File "nn_parser.pyx", line 423, in spacy.syntax.nn_parser.Parser.update
  File "_parser_model.pyx", line 199, in spacy.syntax._parser_model.ParserModel.begin_update.finish_parser_update
  File "_parser_model.pyx", line 323, in spacy.syntax._parser_model.ParserStepModel.make_updates
  File "/root/miniconda3/envs/ner/lib/python3.7/site-packages/thinc/neural/_classes/feed_forward.py", line 53, in continue_update
    gradient = callback(gradient, sgd)
  File "/root/miniconda3/envs/ner/lib/python3.7/site-packages/thinc/api.py", line 300, in finish_update
    d_X = bp_layer(layer.ops.flatten(d_seqs_out, pad=pad), sgd=sgd)
  File "/root/miniconda3/envs/ner/lib/python3.7/site-packages/thinc/neural/_classes/feed_forward.py", line 53, in continue_update
    gradient = callback(gradient, sgd)
  File "/root/miniconda3/envs/ner/lib/python3.7/site-packages/thinc/neural/_classes/resnet.py", line 37, in residual_bwd
    dX = bp_y(d_output, sgd)
  File "/root/miniconda3/envs/ner/lib/python3.7/site-packages/thinc/neural/_classes/feed_forward.py", line 53, in continue_update
    gradient = callback(gradient, sgd)
  File "ops.pyx", line 101, in thinc.neural.ops.Ops.dropout.wrap_backprop.finish_update
  File "cupy/core/core.pyx", line 940, in cupy.core.core.ndarray.__mul__
  File "cupy/core/_kernel.pyx", line 811, in cupy.core._kernel.ufunc.__call__
  File "cupy/core/_kernel.pyx", line 89, in cupy.core._kernel._preprocess_args
TypeError: Unsupported type <class 'numpy.ndarray'>

Could you please help me to understand the error and using GPU successfully cause I don't understand why I received type error while I didn't get any error of running it locally.
thanks

2

I think the most likely cause here is that you'll need to run the spacy.prefer_gpu() function before you call spacy.load(). I suspect you've already loaded your model on CPU before activating the GPU.