deserializers[name] = lambda p, proc=proc: proc.from_disk(p, vocab=False)
File "pipeline.pyx", line 643, in spacy.pipeline.Tagger.from_disk
File "/Users/Abhishek/Projects/Python-Projects/Python-VEs/NM-NLP/lib/python3.6/site-packages/spacy/util.py", line 511, in from_disk
reader(path / key)
File "pipeline.pyx", line 625, in spacy.pipeline.Tagger.from_disk.load_model
File "pipeline.pyx", line 534, in spacy.pipeline.Tagger.Model
ValueError: [T008] Bad configuration of Tagger. This is probably a bug within spaCy. We changed the name of an internal attribute for loading pre-trained vectors, and the class has been passed the old name (pretrained_dims) but not the new name (pretrained_vectors).
```
It would be extremely helpful if you could point me to any place I can write a custom tokenizer with multiple pattern matching capabilities, so that I can avoid this problem entirely.
---
### Author:  [@honnibal](https://support.prodi.gy/u/honnibal)
#### Post date: [January 17, 2019, 4:37pm UTC](https://support.prodi.gy/t/basic-question-about-prodigy-annotations-and-model-training/1127/7 "2019-01-17T16:37:40Z")
Ah, understood. I’m pretty sure I’ll be able to help you get that working without running a fork. Let’s focus on the textcat problem first though.
Could you paste the contents of the `meta.json` in the output model? Also, can you run `python -m spacy validate` and paste the output? I just want to check that the right version of the model is there.
---
### Author:  [@adwaraki](https://support.prodi.gy/u/adwaraki)
#### Post date: [January 17, 2019, 5:10pm UTC](https://support.prodi.gy/t/basic-question-about-prodigy-annotations-and-model-training/1127/9 "2019-01-17T17:10:37Z")
Spacy Validate:
```python
[Abhishek:~/Projects/Git-Repositories/spaCy] [NM-NLP] master(+25/-6) 4s ± python -m spacy download en
Requirement already satisfied: en_core_web_sm==2.0.0 from https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.0.0/en_core_web_sm-2.0.0.tar.gz#egg=en_core_web_sm==2.0.0 in /Users/Abhishek/Projects/Python-Projects/Python-VEs/NM-NLP/lib/python3.6/site-packages (2.0.0)
Linking successful
/Users/Abhishek/Projects/Python-Projects/Python-VEs/NM-NLP/lib/python3.6/site-packages/en_core_web_sm
--> /Users/Abhishek/Projects/Git-Repositories/spaCy/spacy/data/en
You can now load the model via spacy.load('en')
[Abhishek:~/Projects/Git-Repositories/spaCy] [NM-NLP] master(+25/-6) 4s ± python -m spacy validate
Installed models (spaCy v2.0.18)
/Users/Abhishek/Projects/Git-Repositories/spaCy/spacy
TYPE NAME MODEL VERSION
package en-core-web-sm en_core_web_sm 2.0.0 ✔
package en-core-web-lg en_core_web_lg 2.0.0 ✔
link en_core_web_lg en_core_web_lg 2.0.0 ✔
link en_core_web_sm en_core_web_sm 2.0.0 ✔
link en en_core_web_sm 2.0.0 ✔
```
meta.json contents:
```python
{
"lang":"en",
"name":"model",
"version":"0.0.0",
"spacy_version":">=2.0.18",
"description":"",
"author":"",
"email":"",
"url":"",
"license":"",
"vectors":{
"width":0,
"vectors":0,
"keys":0,
"name":null
},
"pipeline":[
"sbd",
"textcat"
]
}
```
I added in “tagger”, “parser” and “ner” as @ines directed me to, but everytime I train, it is rewritten to the above. I have not tried it with `ner.batch-train`, but this is with `textcat.batch-train`.
---
### Author:  [@honnibal](https://support.prodi.gy/u/honnibal)
#### Post date: [January 17, 2019, 5:28pm UTC](https://support.prodi.gy/t/basic-question-about-prodigy-annotations-and-model-training/1127/10 "2019-01-17T17:28:33Z")
Thanks! I really feel like there must be something I’m missing. Could you give the command you’re running to `textcat.batch-train`?
---
### Author:  [@adwaraki](https://support.prodi.gy/u/adwaraki)
#### Post date: [January 17, 2019, 5:53pm UTC](https://support.prodi.gy/t/basic-question-about-prodigy-annotations-and-model-training/1127/11 "2019-01-17T17:53:41Z")
```python
prodigy textcat.batch-train event_labels --output-model /tmp/model --eval-split 0.8
```
I am actually following most of the commands from the documentation. 🙂
And what I am using to annotate a new label.
```python
prodigy textcat.teach event_labels en_core_web_lg /Users/Abhishek/Downloads/training_dataset.txt --label LOGIN_RELATED_FAIL
```
---
### Author:  [@adwaraki](https://support.prodi.gy/u/adwaraki)
#### Post date: [January 18, 2019, 7:19am UTC](https://support.prodi.gy/t/basic-question-about-prodigy-annotations-and-model-training/1127/12 "2019-01-18T07:19:40Z")
Also, do you want me to create a new thread for the tokenization problem?
---
### Author:  [@honnibal](https://support.prodi.gy/u/honnibal)
#### Post date: [January 18, 2019, 12:45pm UTC](https://support.prodi.gy/t/basic-question-about-prodigy-annotations-and-model-training/1127/13 "2019-01-18T12:45:37Z")
Aha! Okay you’re missing the second positional argument, which specifies the input model. Try:
```python
prodigy textcat.batch-train event_labels en_core_web_sm --output-model /tmp/model --eval-split 0.8
```
Without the `en_core_web_md` argument, the model was defaulting to a blank model, which didn’t have the POS tagger loaded.
Yes, I think a new thread for the tokenization problem will be best. I’ll give you some code for it, so it’s best if it’s easily searchable.
---
### Author:  [@adwaraki](https://support.prodi.gy/u/adwaraki)
#### Post date: [January 18, 2019, 2:06pm UTC](https://support.prodi.gy/t/basic-question-about-prodigy-annotations-and-model-training/1127/14 "2019-01-18T14:06:37Z")
That was such a stupid mistake. I cannot believe I missed something as elementary as that. My apologies. Thank you so much and I will create a new thread for the tokenization question. You folks are amazing!