When is the model called and the scores updated in the textcat teach method

[AnshulP10 (Anshul Padhi)]

Hi,
I am trying to build a custom recipe for binary classification. I used the custom-recipe code as a template. On checking the logs I saw that the model call function was only being called once while the update function was being called on receiving every 10 answers. It wasn't being called on the updated model to get new scores. I was wondering how I can check the overall flow of the code-base to understand the process better. This was also the case when I directly used the custom-recipe code prodigy's repo

[ryanwesslen]

hi @AnshulP10!

Thanks for your question and welcome to the Prodigy community

The textcat.teach recipe will only suggest you examples to annotate. This is different than the textcat.correct which if you add in --update flag, it will update the model.

textcat.teach vs correct wh

Both recipes are indeed similar. The difference are explained in more detail in the textcat.correct docs and the textcat.teach docs but the main difference is that in the base behaviour the teach recipe has a model in the loop that updates as you give it labels while the correct recipe does not. That said, you can configure textcat.correct to also allow the model to update and you can configure a --threshold to tune how examples are pre-selected for annotation. The textcat.teach recipe, on the …

There are other posts that have mention ways to use the update callback to update the textcat.teach model:

Textcat.teach with Multiple Choice & Update Model

Hi! To update the annotation model, you can call model.update with a batch of examples in Prodigy's format. Since your recipe is creating predict/update functions depending on whether you're using patterns or not, you probably want to be using that update function. For example: if patterns is None: # etc. predict = model update = model.update else: # etc. predict, update = combine_models(model, matcher) def review_answers(answers): update(answers) Alternatively, you co…

@AnshulP10

I was wondering how I can check the overall flow of the code-base to understand the process better.

The best way to understand is by logging, which it sounds like you were already doing.

However, if you want to see/modify those recipes, you can look in the folder of where you've installed Prodigy. You can find the location of your Prodigy installation by running python -m prodigy stats and find the Location folder printed out (for you'd look for the recipes/textcat.py)

Hope this helps!