# Evaluation data for ner model **URL:** \ **Category:** Uncategorized\ **Tags:** ner\ **Created:** [October 10, 2023, 3:09pm UTC](https://support.prodi.gy/t/evaluation-data-for-ner-model/6839 "2023-10-10T15:09:33Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![kaiser](https://avatars.discourse-cdn.com/v4/letter/k/dfb087/32.png) [@kaiser](https://support.prodi.gy/u/kaiser)\ **Post date:** [October 10, 2023, 3:09pm UTC](https://support.prodi.gy/t/evaluation-data-for-ner-model/6839/1 "2023-10-10T15:09:33Z")
Heyy, I have a question regarding the train command. I´ve successfully trained a ner model and received afterwards a valid prec acc and f1 score. Now I´m struggling to understand whether these scores are on the trainingdata or on the evaluation data since with the train command the dataset was split. If the score is on the trainingdata, how can i evaluate the model? Is it possible with Prodigy? Thank you! Best regards Jonas ---
**Author:** ![ryanwesslen](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ryanwesslen/32/2969_2.png) [@ryanwesslen](https://support.prodi.gy/u/ryanwesslen)\ **Post date:** [October 10, 2023, 7:31pm UTC](https://support.prodi.gy/t/evaluation-data-for-ner-model/6839/2 "2023-10-10T19:31:40Z")
hi @kaiser! Thanks for your question. The Precision/Accuracy/F1 are on the evaluation data. Here's a good post that describes more: > [@precision, recall and F-Score](https://support.prodi.gy/t/precision-recall-and-f-score/5884/2): > > hi @arashmlk! Thanks for your question and welcome to the Prodigy community wave The scoring is on the evaluation. The default scoring all happens within [spaCy](https://spacy.io/api/scorer/#_title) and doesn't depend on Prodigy. You can view the code (for spaCy 3.0) here: We also have several posts that explain the scoring (e.g., what does it mean) more in detail like: That includes this post on understanding the output: > [@understanding the different terminology in the command line output of a training pipeline](https://support.prodi.gy/t/understanding-the-different-terminology-in-the-command-line-output-of-a-training-pipeline/5719/2): > > Hi @nanyasrivastav! Thank you for your questions! These concepts are very important. I'm glad you've asked because it's hard to effectively use Prodigy without understanding them slight_smile. Initial learn rate: The initial learning rate used. The learning rate is a [tuning parameter](https://en.wikipedia.org/wiki/Hyperparameter_(machine_learning)) in an [optimization algorithm](https://en.wikipedia.org/wiki/Mathematical_optimization) that determines the step size at each iteration while moving toward a minimum of a [loss function](https://en.wikipedia.org/wiki/Loss_function). This is a default value but can be optimized for training. Optimizing the learning … It's important to mention that `prodigy train` is just a wrapper for `spacy train`. In fact, it's just a quick-and-dirty way to train using spaCy with smart defaults; however, it obscures a lot of important concepts like created spaCy binary files (i.e., dedicated training / evaluation dataset) along with using spaCy config files. This post describes a bit more (and why you get `best-model` and `last-model` folders): > [@Explanation Custom NER model folders](https://support.prodi.gy/t/explanation-custom-ner-model-folders/5854/4): > > Hi @stefan.bartell, By default with only a ner component, then yes. But this can be configured and may require a little explanation. When your run prodigy train, you're really running spacy train. To run spacy train, you need a [spaCy config file](https://spacy.io/api/data-formats#config). To simplify this, prodigy train will use spaCy's default config, very similar to running: python -m spacy init config config.cfg --lang en --pipeline ner This then produces this config file: [paths] train = null dev = null vectors = null init\_tok… In general, I recommend moving towards using [`data-to-spacy`](https://prodi.gy/docs/recipes#data-to-spacy) then `spacy train` rather than `prodigy train` once you start developing a more serious pipeline. This post shows either results in the same model if setup correctly: > [@prodigy train result is different with the spacy train result, why?](https://support.prodi.gy/t/prodigy-train-result-is-different-with-the-spacy-train-result-why/5325/8): > > hi @sigitpurnomo! Sorry for the delayed response. We're trying to close out old issues. Regarding comparing prodigy train and spacy train, I recommend anyone interested to check out the Prodigy sample project: If you clone this repo, you can run two examples to compare spacy train and prodigy train. Using sample fashion data, you can run spacy train by running python -m spacy project run all This will load the data (db-in), export the data and config file (data-to-spacy), and spacy trai… You can create dedicated test/eval datasets using `prodigy train` but your eval will need to be its own dataset and you'll need to call `prodigy train` like this: ```python prodigy train --ner train_data,eval:eval_data ... ``` See this post for more context: > [@questions on Multi NERs Annotation & Training at Once in a Sentence](https://support.prodi.gy/t/questions-on-multi-ners-annotation-training-at-once-in-a-sentence/5960/2): > > hi @ruiye! Thanks for your message! This is excellent. We're happy to help coach you along the way. I'll provide a lot of detailed links that can point you in the right direction. Some you may already know, but hopefully it can help other community members for context. Also, keep searching in the wealth of information in our [spaCy documentation](https://spacy.io/usage/spacy-101), [Prodigy documentation](https://prodi.gy/docs), and in this community and [spaCy GitHub community](https://github.com/explosion/spaCy/discussions). Can I rephrase this to align with Prodigy's terminology? I would suggest … Hope this at least gets you started! Sorry if I included things you knew or you didn't directly ask for, but trying my best to give you a few extra resources that'll answer future questions before you ask 🙂 ---
**Author:** ![kaiser](https://avatars.discourse-cdn.com/v4/letter/k/dfb087/32.png) [@kaiser](https://support.prodi.gy/u/kaiser)\ **Post date:** [October 11, 2023, 9:33am UTC](https://support.prodi.gy/t/evaluation-data-for-ner-model/6839/3 "2023-10-11T09:33:35Z")
This is a great response, it helps a lot. i´ve searched for similar posts but didnt found anything.. well i obv could have done better Thank you!