# textcat affects NER

**URL:** <https://support.prodi.gy/t/textcat-affects-ner/257>\
**Category:** Uncategorized\
**Tags:** textcat, done\
**Created:** [January 28, 2018, 6:30pm UTC](https://support.prodi.gy/t/textcat-affects-ner/257 "2018-01-28T18:30:22Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![WeinstockShahar](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/weinstockshahar/32/115_2.png) [@WeinstockShahar](https://support.prodi.gy/u/WeinstockShahar)\
**Post date:** [January 28, 2018, 6:30pm UTC](https://support.prodi.gy/t/textcat-affects-ner/257/1 "2018-01-28T18:30:22Z")

</div>

I used `textcat` to train a new model based on `en_core_web_lg`.  
Apparently the new model does nice text classification but really bad NER.

Is it possible that the NER model is affected by `TextCategorizer`?

---

<div class="post-metadata">

**Author:** ![honnibal](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/honnibal/32/35_2.png) [@honnibal](https://support.prodi.gy/u/honnibal)\
**Post date:** [January 28, 2018, 11:33pm UTC](https://support.prodi.gy/t/textcat-affects-ner/257/2 "2018-01-28T23:33:52Z")

</div>

Hmm, it shouldn’t!

The NER and textcat models shouldn’t share any weights except the pre-trained vectors, but the pre-trained vectors should be static. So updates to the model shouldn’t be changing the NER. My guess is that the NER model is being updated during the textcat updates, even though that shouldn’t happen.

For sanity, could you try copying the textcat model directory within the original directory? Assuming this works, this also gives you a workaround until we figure out what’s wrong.

---

<div class="post-metadata">

**Author:** ![WeinstockShahar](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/weinstockshahar/32/115_2.png) [@WeinstockShahar](https://support.prodi.gy/u/WeinstockShahar)\
**Post date:** [January 29, 2018, 6:54am UTC](https://support.prodi.gy/t/textcat-affects-ner/257/3 "2018-01-29T06:54:47Z")

</div>

I just noticed that the `ner` directory is actually missing in my new model. 🤦🏼‍♂️  
Sorry for the bad problem explanation...

> [@honnibal](#):
>
> try copying the textcat model directory within the original directory

Did you mean copying the `textcat` from the created model into the original `en_core_web_lg`?  
I tried it but the model don't recognize it.

---

<div class="post-metadata">

**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\
**Post date:** [January 29, 2018, 3:13pm UTC](https://support.prodi.gy/t/textcat-affects-ner/257/4 "2018-01-29T15:13:20Z")

</div>

> [@WeinstockShahar](#):
>
> I just noticed that the ner directory is actually missing in my new model. 🤦🏼‍♂️

Ah, this explains a lot – so it looks like the `textcat` training process disabled the `ner` pipeline component. So when you save out the model, it saves without NER weights. Will have a look at this issue!

**Edit:** Just had a look and the most likely explanation is the following line in the `textcat.teach` recipe:

```python
if input_model is not None:
    nlp = spacy.load(input_model, disable=['ner'])

```

I think we originally added this for efficiency, since the recipe always keeps a serialized copy of the best model, which can easily take long for large models. But maybe this is a bad default, since it's pretty unintuitive. You should be able to change this behaviour by removing `disable=['ner']`. (If you do so, keep us updated on the speed and performance!)

> [@WeinstockShahar](#):
>
> Did you mean copying the textcat from the created model into the original `en_core_web_lg`? I tried it but the model don’t recognize it.

Yes – but since the original model didn't have a text classifier, you'll also have to add `"textcat"` to the pipeline in the model's `meta.json`. For example:

```json
{
    "pipeline": ["parser", "tagger", "ner", "textcat"]
}

```

---

<div class="post-metadata">

**Author:** ![WeinstockShahar](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/weinstockshahar/32/115_2.png) [@WeinstockShahar](https://support.prodi.gy/u/WeinstockShahar)\
**Post date:** [January 29, 2018, 7:50pm UTC](https://support.prodi.gy/t/textcat-affects-ner/257/5 "2018-01-29T19:50:01Z")

</div>

Perfect, Thanks a lot.

> [@ines](#):
>
> If you do so, keep us updated on the speed and performance

Will do!