# Should I be using --base-model when training my model?

**URL:** <https://support.prodi.gy/t/should-i-be-using-base-model-when-training-my-model/5626>\
**Category:** Uncategorized\
**Tags:** ner, training\
**Created:** [May 10, 2022, 7:35pm UTC](https://support.prodi.gy/t/should-i-be-using-base-model-when-training-my-model/5626 "2022-05-10T19:35:38Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![tomw](https://avatars.discourse-cdn.com/v4/letter/t/b5ac83/32.png) [@tomw](https://support.prodi.gy/u/tomw)\
**Post date:** [May 10, 2022, 7:35pm UTC](https://support.prodi.gy/t/should-i-be-using-base-model-when-training-my-model/5626/1 "2022-05-10T19:35:38Z")

</div>

Using patterns and then a new model I've been training (without `--base-model` specified), I've developed some gold label data for 3,000 documents and a single NER category.

The end goal is to use this model on discussions indexed from the web, first with English documents.

Should I be training this model using my gold label data and specifying one of spacy's en\_core\_web models as a `--base-model`?

I've tried it and I _think_ the model is improving, but I experience something strange, so wondering if maybe this is the wrong approach. The strangeness is that when I run ner.correct using the newly base-model-trained pipeline on my remaining dataset to label, it ends up warning:

> The model you're using isn't setting sentence boundaries (e.g. via the parser or sentencizer). This means that incoming examples won't be split into sentences.

And when I look at the data in prodigy, sure enough it has started over at the beginning of my dataset, and everything is just the first sentence of each document.

I have to specify `--unsegmented` if I want to see the full paragraphs again.

Conversely, when I run `ner.correct` on a model I trained _without_ any --base-model set, the documents are rendered completely and I don't get the warning.

And, if I should be using a --base-model, how does one decide which one to use? I see that en\_core\_web\_trf is more accurate than en\_core\_web\_sm, but what does that typically mean in practice? Is it just training the model that's slower and memory intensive, or will using the trained model also be slower and require more resources?

---

<div class="post-metadata">

**Author:** ![koaning](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/koaning/32/230_2.png) [@koaning](https://support.prodi.gy/u/koaning)\
**Post date:** [May 11, 2022, 6:39am UTC](https://support.prodi.gy/t/should-i-be-using-base-model-when-training-my-model/5626/2 "2022-05-11T06:39:08Z")

</div>

Could you share the exact commands that you tried running? That might help me reproduce the error that you're seeing. Also, just to confirm, are you running recent versions of Prodigy/spaCy/Python?

In general, you can switch the base model, which is the starting point for the training procedure, and this could result in better/worse-performing models. In particular, the `en_core_web_trf` typically yields good accuracy, but it does come at the cost of speed. This pipeline contains a transformer model which is typically slower to train and _much_ slower to run in production, typically on the order of 10x. You can read more about the speed details [here](https://spacy.io/usage/facts-figures#benchmarks-speed).

---

<div class="post-metadata">

**Author:** ![tomw](https://avatars.discourse-cdn.com/v4/letter/t/b5ac83/32.png) [@tomw](https://support.prodi.gy/u/tomw)\
**Post date:** [May 11, 2022, 4:30pm UTC](https://support.prodi.gy/t/should-i-be-using-base-model-when-training-my-model/5626/3 "2022-05-11T16:30:23Z")

</div>

@koaning Sure, first I started with a source containing a bunch of blobs of text in jsonl format (paragraphs of text, but just spaces for separators, no line breaks or anything), and patterns also in a jsonl format:

```python
ner.manual mymodel en_core_web_sm ./source.jsonl --label MYCATEGORY --patterns patterns.jsonl

```

I then labeled probably 500 source documents this way before training `mymodel` like this:

```python
prodigy train ./modeltrain1 --ner mymodel

```

It would then ultimately result in a statement like:

```python
✔ Saved pipeline to output directory
modeltrain1/model-last

```

Then I would further label using something like:

```python
prodigy ner.correct mymodel modeltrain1/model-last ./source.jsonl --label MYCATEGORY

```

And would just keep bouncing back and forth between train and ner.correct, incrementing the training directory each time, e.g. modeltrain2, modeltrain3, etc, in case I wanted to return to a previously trained model (I never did, ultimately).

As I got more and more labels, I experimented with using --base-model:

```python
prodigy train ./modeltrain9 --ner mymodel --base-model en_core_web_sm

```

It seemed to have higher scoring, so I decided to continue forward and use it to label some more, so I tried my usual follow-up `ner.correct`:

```python
prodigy ner.correct mymodel modeltrain9/model-last ./source.jsonl --label MYCATEGORY

```

And that's when I would get the warning:

> ⚠ The model you're using isn't setting sentence boundaries (e.g. via the parser or sentencizer). This means that incoming examples won't be split into sentences.

When I'd load [http://localhost:8080](http://localhost:8080) to label, I noticed it started over on my source documents, back to the first one, and only the first sentence of that document was shown. It was like it saw the data only as the first sentence of each document now.

I was then able to work around this by adding the `--unsegmented` param:

```python
prodigy ner.correct mymodel modeltrain9/model-last ./source.jsonl --label MYCATEGORY --unsegmented

```

However, I began wondering if I was using --base-model when I shouldn't be, and I don't really understand what's happening under the covers when training with or without the base model.

---

<div class="post-metadata">

**Author:** ![ryanwesslen](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ryanwesslen/32/2969_2.png) [@ryanwesslen](https://support.prodi.gy/u/ryanwesslen)\
**Post date:** [May 16, 2022, 3:57pm UTC](https://support.prodi.gy/t/should-i-be-using-base-model-when-training-my-model/5626/4 "2022-05-16T15:57:06Z")

</div>

Hi @tomw!

I think the confusion is because each spaCy model can have different components for each model pipeline.

Your first model, `modeltrain1`, didn't use `en_core_web_sm` , so by default the base model was a blank model. The output model had only two components: `tok2vec` and a new `ner`, which used only the `MYCATEGORY` category. Notice it doesn't have a `parser` or `senter` (sentence model).

However, you did use the `en_core_web_sm` for `modeltrain9` which kept the original `ner` model in `en_core_web_sm` and added a new ner category (`MYCATEGORY`) to the existing ner levels. I suspect the model accuracy could have been masked by the pre-existing labels (e.g., `ORG`) in the `ner` model in `en_core_web_sm`.

The `en_core_web_sm` also had a `parser` component, what is causing the difference in sentence boundaries warning as this does have a `parser` component in its pipeline.

I would suggest running `print(nlp.pipe_names)` for each of your models so you can see all of the components of your pipeline. You can [disable components you may not want manually](https://spacy.io/models#design-modify).

Here is an excellent recent spaCy short on disabling spaCy components by @koaning:

[![](https://img.youtube.com/vi/nnBUjDC7psU/maxresdefault.jpg "How to Disable Unused Pipeline Components - spaCy shorts") ](https://www.youtube.com/watch?v=nnBUjDC7psU)

Let me know if this makes sense and helps!

---

<div class="post-metadata">

**Author:** ![tomw](https://avatars.discourse-cdn.com/v4/letter/t/b5ac83/32.png) [@tomw](https://support.prodi.gy/u/tomw)\
**Post date:** [May 24, 2022, 5:15pm UTC](https://support.prodi.gy/t/should-i-be-using-base-model-when-training-my-model/5626/5 "2022-05-24T17:15:13Z")

</div>

@ryanwesslen Thank you, this does clear up a lot of confusion.

However, I'm still wondering what is better when training an NER model ... should I be starting from scratch and just training on my gold annotations, or should I use a base model like en\_core\_web\_sm?

Also, I see how to disable pipeline components in my python scripts when using the nlp, but I'm not seeing how (or whether I even _should_) disable components when running `train` or `ner.correct`?

---

<div class="post-metadata">

**Author:** ![ryanwesslen](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ryanwesslen/32/2969_2.png) [@ryanwesslen](https://support.prodi.gy/u/ryanwesslen)\
**Post date:** [May 24, 2022, 8:14pm UTC](https://support.prodi.gy/t/should-i-be-using-base-model-when-training-my-model/5626/6 "2022-05-24T20:14:25Z")

</div>

Hi @tomw!

> [@tomw](#):
>
> I'm still wondering what is better when training an NER model ... should I be starting from scratch and just training on my gold annotations, or should I use a base model like en\_core\_web\_sm?

Likely a base model will help by providing word vectors but you'll want to turning off the `ner` component as your `ner` model will be trained from scratch. If you don't turn off the `ner` then you will be [adding your new entity to the existing `ner` entities](https://support.prodi.gy/t/work-flow-for-extending-an-ner-model-with-new-entity-types/1603/2).

This post shows how to do it (fyi some syntax has changed but general idea is the same):

> [@NER and blank models](https://support.prodi.gy/t/ner-and-blank-models/1795/4):
>
> Ah, yes – that looks like what I was referring to. The error message here isn't very nice, but I'm pretty sure it fails because the blank NER component isn't initialized. No, I just meant in the code you use to create blank\_ner\_en\_core\_web\_lg slightly_smiling_face Instead of calling create\_pipe and replace\_pipe, you just call remove\_pipe and get rid of the old entity recognizer: nlp = spacy.load("en\_core\_web\_lg") nlp.remove\_pipe("ner") nlp.to\_disk("en\_core\_web\_lg\_without\_ner") Prodigy shou…

```python
nlp = spacy.load("en_core_web_sm")
nlp.remove_pipe("ner") # can remove other not used components too
nlp.to_disk("en_core_web_sm_without_ner")

```

This also solves your second question because by using `to_disk()`, you can now call this model like any other model for `train` or `ner.correct`:

```bash
prodigy ner.correct gold_ner en_core_web_sm_without_ner ./news_headlines.jsonl --label PERSON,ORG

```

As @koaning mentioned, you do have a choice between which base models (word vectors) where speed vs. accuracy may come into play.

If you use [`en_core_web_sm`](https://spacy.io/models/en#en_core_web_sm), this will be the fastest/most compact, but have lower performance. Alternatively [`en_core_web_trf`](https://spacy.io/models/en#en_core_web_trf) will give you the greatest accuracy, but you should be cautious as putting transformers into production can be challenging (e.g., need GPU).

One compromise could be to use [`en_core_web_lg`](https://spacy.io/models/en#en_core_web_lg) model like in the post above. Like the small model, it is very fast but has better performance than the small model due to a larger set of word vectors. Given the larger set of word vectors, the large model is larger in size (382 MB). Ideally, you can see [our experiments with different NER models](https://github.com/explosion/projects/tree/v3/benchmarks/speed) and you could mimic do your own experiments to determine which performs best on your model.

Let me know if this helps or if you have any other questions!

---

<div class="post-metadata">

**Author:** ![tomw](https://avatars.discourse-cdn.com/v4/letter/t/b5ac83/32.png) [@tomw](https://support.prodi.gy/u/tomw)\
**Post date:** [May 26, 2022, 4:10pm UTC](https://support.prodi.gy/t/should-i-be-using-base-model-when-training-my-model/5626/7 "2022-05-26T16:10:18Z")

</div>

> [@ryanwesslen](#):
>
> This also solves your second question because by using `to_disk()`, you can now call this model like any other model for `train` or `ner.correct`:

Thank you for your help @ryanwesslen! I tried this approach, removing the `ner` from both the \_sm and \_lg models, and then writing to disk and using that new model as my base-model when training. However, when I do this I get this error:

> ✘ Config validation error  
> Bad value substitution: option 'width' in section 'components.ner.model.tok2vec' contains an interpolation key 'components.tok2vec.model.encode.width' which is not a valid option name. Raw value: '${components.tok2vec.model.encode.width}'

> [@ryanwesslen](#):
>
> If you use [`en_core_web_sm`](https://spacy.io/models/en#en_core_web_sm), this will be the fastest/most compact, but have lower performance. Alternatively [`en_core_web_trf`](https://spacy.io/models/en#en_core_web_trf) will give you the greatest accuracy, but you should be cautious as putting transformers into production can be challenging (e.g., need GPU).

I've been getting ready to move off training on my personal computer and onto an AWS VM of some sort, so I wanted to just experiment with training \_trf locally before I started the process. (My understanding is that \_trf will work without GPU, it's just much slower?)

However, when I use that `en_core_web_trf` as my base model for `train`, I get this error:

> Token indices sequence length is longer than the specified maximum sequence length for this model (670 \> 512). Running this sequence through the model will result in indexing errors

I don't experience this error when using \_sm or \_lg. Do I just need to run this on a machine with a GPU, or is something else going on here?

Thanks!

---

<div class="post-metadata">

**Author:** ![ryanwesslen](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ryanwesslen/32/2969_2.png) [@ryanwesslen](https://support.prodi.gy/u/ryanwesslen)\
**Post date:** [May 26, 2022, 5:19pm UTC](https://support.prodi.gy/t/should-i-be-using-base-model-when-training-my-model/5626/8 "2022-05-26T17:19:26Z")

</div>

hi @tomw!

We're making progress but still some work left 🙂 .

Let me think about your first question a little more. I just tried to remove `ner` and I had no issues. Can you run these commands to double check your spacy/prodigy versions? Also make sure what's in your pipeline. Maybe worst case try to remove everything except your `tok2vec` component and try again.

```bash
>>> import spacy
>>> spacy. __version__'3.2.4'
>>> import prodigy
>>> prodigy. __version__'1.11.7'
>>> nlp = spacy.load("en_core_web_sm_without_ner")
>>> print(nlp.pipe_names)
['tok2vec', 'tagger', 'parser', 'attribute_ruler', 'lemmatizer']

```

There was a [similar issue](https://github.com/explosion/spaCy/issues/9581) with `textcat` but for that the recommendation was removing `tok2vec`, which likely won't help you. If you remove `tok2vec` with the other components you get right back to the `blank:en` model. It's the `tok2vec` is what we want to keep.

For your 2nd question, there was a similar [GitHub issue on it](https://github.com/explosion/spaCy/issues/6939) and related [GitHub discussion](https://github.com/explosion/spaCy/discussions/9280). This is a warning, not an error, as what it is truncating any of your documents that are over 512 tokens long. Looks like you had record that was 670 tokens. This is a [known limitation for most transformers](https://github.com/huggingface/transformers/issues/1791) and isn't an issue with running on a CPU.

You are correct that you don't absolutely need a GPU (see [this spaCy issue discussion](https://github.com/explosion/spaCy/discussions/10543)) but likely you'll have issues if you try to put into production. So if you're looking for your model as a test case, then you can try a CPU for now. Perhaps run a few experiments on varying compute time vs. number of records. I would be curious where the bottleneck would occur!

---

<div class="post-metadata">

**Author:** ![tomw](https://avatars.discourse-cdn.com/v4/letter/t/b5ac83/32.png) [@tomw](https://support.prodi.gy/u/tomw)\
**Post date:** [May 27, 2022, 12:33am UTC](https://support.prodi.gy/t/should-i-be-using-base-model-when-training-my-model/5626/9 "2022-05-27T00:33:28Z")

</div>

@ryanwesslen Thanks again! Output I'm getting from those commands:

I took your advice and removed everything except tok2vec:

```python
>>> import spacy
>>> nlp = spacy.load("en_core_web_lg")
>>> nlp.remove_pipe("ner")
>>> nlp.remove_pipe("parser")
>>> nlp.remove_pipe("tagger")
>>> nlp.remove_pipe("lemmatizer")
>>> nlp.remove_pipe("attribute_ruler")
>>> nlp.to_disk("en_core_web_lg_light")

```

Yet when I run the `train` again, it still generates that warning ("Bad value substitution: option 'width' in section...").

Here's the output from the questions you asked related to the other warning:

```python
>>> import spacy
>>> spacy. __version__'3.3.0'
>>> import prodigy
>>> prodigy. __version__'1.11.7'
>>> nlp = spacy.load("en_core_web_lg_light")
>>> print(nlp.pipe_names)
['tok2vec']

```

In terms of the other warning ("Token indices sequence length is longer than the specified maximum sequence length for this model (670 \> 512)"), does this mean when the model is trained, I won't be able to run NER on blocks of text that are over 512 characters in length?

Or is it just choking on "word" sequences that are over 512 characters, such as a really long URL or something?

Thanks!