# combine\_models - the effect of batch\_size?

**URL:** <https://support.prodi.gy/t/combine-models-the-effect-of-batch-size/2205>\
**Category:** Uncategorized\
**Tags:** usage, solved\
**Created:** [November 10, 2019, 1:34pm UTC](https://support.prodi.gy/t/combine-models-the-effect-of-batch-size/2205 "2019-11-10T13:34:44Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![JanP](https://avatars.discourse-cdn.com/v4/letter/j/a587f6/32.png) [@JanP](https://support.prodi.gy/u/JanP)\
**Post date:** [November 10, 2019, 1:34pm UTC](https://support.prodi.gy/t/combine-models-the-effect-of-batch-size/2205/1 "2019-11-10T13:34:44Z")

</div>

Hello team!

I have been trying to get the `prefer_high_scores` sorter to look over the larger window of examples to present the bootstrapped (although relatively rare) texts first. I have came across this thread where  
the function `combine_models` is discussed.

> [@Best way to annotate rare labels for classification](https://support.prodi.gy/t/best-way-to-annotate-rare-labels-for-classification/1113/5):
>
> Ah, sorry about that. Here’s the source for `combine_models` , it’s pretty straightforward:
> 
> ```python
> from toolz.itertoolz import interleave, partition_all
> 
> def combine_models(one, two, batch_size=32):
> """Combine two models and return a predict and update function. Predictions
> of both models are combined using the toolz.interleave function. Mostly
> used to combine an EntityRecognizer with a PatternMatcher.
> one (callable): First model. Requires a ` __call__ ` and `update` method.
> two (callable): Second model. Requires a ` __call__ ` and `update` method.
> batch_size (int): The batch size to use for predicting the stream.
> RETURNS (tuple): A `(predict, update)` tuple of the respective functions.
> """
> 
> def predict(stream):
> for batch in partition_all(batch_size, stream):
> batch = list(batch)
> stream1 = one(batch)
> stream2 = two(batch)
> yield from interleave((stream1, stream2))
> 
> def update(examples):
> loss = one.update(examples) + two.update(examples)
> return loss
> 
> return predict, update
> 
> ```

Would setting a higher batch\_size to combine model achieve just that? I have found nothing in the docs so maybe it was already deprecated. The goal is simply to get the examples containing the patterns to come up first (now I have to run the rule-based matcher in a separate script to pre-assign the scores, which is not very convenient).

Thank you in advance!  
Best wishes,  
Jan

---

<div class="post-metadata">

**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\
**Post date:** [November 11, 2019, 12:23pm UTC](https://support.prodi.gy/t/combine-models-the-effect-of-batch-size/2205/2 "2019-11-11T12:23:02Z")

</div>

Hi! There are two batch sizes here: first, the batch size used to partition the two generators and interleave them, and second, the batch size that Prodigy uses to divide the final stream into batches.

The `batch_size` on `combine_models` is less relevant here and mostly used for efficiency. In the end, the `predict` function still just yields `(score, example)` tuples. The `batch_size` setting in Prodigy is what decides about how many examples are fetched from the stream at once, how many are sent to the web app and how many are sent back to the server.

See my reply here for more ideas on how you could solve your problem:

> [@prefer\_high\_scores(stream) not prioritising the bootstrapped texts during cold start](https://support.prodi.gy/t/prefer-high-scores-stream-not-prioritising-the-bootstrapped-texts-during-cold-start/2177/4):
>
> Yeah, I think what you're observing here is that there are no pattern matches in the stream, only some initial suggestions from the model. Because the sorters operate over a (potentially infinite) stream and only get to see batches, they use an exponential moving average to decide what to send out. The main objectives here are: make sure there's always something to annotate and make sure the stream never gets stuck because the scores are too high/low. One option would be to use a larger batch\_…

---

<div class="post-metadata">

**Author:** ![JanP](https://avatars.discourse-cdn.com/v4/letter/j/a587f6/32.png) [@JanP](https://support.prodi.gy/u/JanP)\
**Post date:** [November 11, 2019, 3:39pm UTC](https://support.prodi.gy/t/combine-models-the-effect-of-batch-size/2205/3 "2019-11-11T15:39:54Z")

</div>

Oh, I see!

Currently, I have compared bootstrapped textcat.teach with textcat.manual stream and since the patterns were not so common, the stream were almost identical. I will try using higher `batch-size` setting.

Thank you so much!