# NER document Labeling

**URL:** <https://support.prodi.gy/t/ner-document-labeling/1741>\
**Category:** Uncategorized\
**Tags:** ner, solved\
**Created:** [July 4, 2019, 10:42pm UTC](https://support.prodi.gy/t/ner-document-labeling/1741 "2019-07-04T22:42:04Z")\
**Posts on this page:** 6\
**Page:** 2

<div class="post-metadata">

**Author:** ![mystuff](https://avatars.discourse-cdn.com/v4/letter/m/a88e4f/32.png) [@mystuff](https://support.prodi.gy/u/mystuff)\
**Post date:** [July 18, 2019, 11:29pm UTC](https://support.prodi.gy/t/ner-document-labeling/1741/22 "2019-07-18T23:29:31Z")

</div>

i finally fixed my issue by adding all white space chars like \r\t. not just " ". when i ran ner.batch-train below is the output. I have used default batch size. Also there is no duplicates in data.

Correct 420  
Incorrect 419  
Baseline 0.000  
Accuracy 0.501

How do i improve accuracy? is it by adding more data(currently it has 150.)?

---

<div class="post-metadata">

**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\
**Post date:** [July 19, 2019, 8:39am UTC](https://support.prodi.gy/t/ner-document-labeling/1741/23 "2019-07-19T08:39:19Z")

</div>

Yes, adding data should definitely be the first step. 150 examples is very low, so you won’t be seeing very reliable results.

---

<div class="post-metadata">

**Author:** ![mystuff](https://avatars.discourse-cdn.com/v4/letter/m/a88e4f/32.png) [@mystuff](https://support.prodi.gy/u/mystuff)\
**Post date:** [July 19, 2019, 11:00am UTC](https://support.prodi.gy/t/ner-document-labeling/1741/24 "2019-07-19T11:00:34Z")

</div>

Thought so. but just want to confirm. Thanks for the reply. Your reply means a lot, you gave me confidence that i am on right direction.

---

<div class="post-metadata">

**Author:** ![mystuff](https://avatars.discourse-cdn.com/v4/letter/m/a88e4f/32.png) [@mystuff](https://support.prodi.gy/u/mystuff)\
**Post date:** [July 31, 2019, 9:03am UTC](https://support.prodi.gy/t/ner-document-labeling/1741/25 "2019-07-31T09:03:51Z")

</div>

Hello @ines , I have increased the dataset from 150 to 300 using ner.manual.  
Annotated new 150 and merged those with previous 150.  
python -m prodigy ner.batch-train dataset\_300 en\_core\_web\_sm --output model\_300 --label ........  
The accuracy only increased 0.8%. May i know where i am doing wrong?. Is there a way to debug the accuracy?

**dataset\_150:**  
Correct 420  
Incorrect 419  
Baseline 0.000  
Accuracy 0.501

**dataset\_300:**  
Correct 831  
Incorrect 597  
Baseline 0.000  
Accuracy 0.582

---

<div class="post-metadata">

**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\
**Post date:** [August 1, 2019, 8:55am UTC](https://support.prodi.gy/t/ner-document-labeling/1741/26 "2019-08-01T08:55:33Z")

</div>

300 examples is still a very low number of examples. To really be able to trust your results, you typically want a lot more - maybe like 1000 or 2000.

If you haven't seen it yet, check our my NER flowchart for some more tips:

> [@Annotation Flowchart: Named Entity Recognition](https://support.prodi.gy/t/annotation-flowchart-named-entity-recognition/1556):
>
> Just shared this on Twitter – if you find flowcharts like this useful, I'm happy to make more smiley Updated on October 18, 2022: New versions of the NER flowcharts! See our [Twitter thread](https://twitter.com/explosion_ai/status/1582389799646023680?s=20&t=JoiOj44gnIExW9DffomOig) for more details.

---

<div class="post-metadata">

**Author:** ![mystuff](https://avatars.discourse-cdn.com/v4/letter/m/a88e4f/32.png) [@mystuff](https://support.prodi.gy/u/mystuff)\
**Post date:** [August 1, 2019, 10:21am UTC](https://support.prodi.gy/t/ner-document-labeling/1741/27 "2019-08-01T10:21:52Z")

</div>

Thanks for the detailed Flowchart. In that flow chart, it says 1000 sentences not 1000 documents. am i right?. I have more than 4000 sentences in my dataset.

[Previous page](https://support.prodi.gy/t/ner-document-labeling/1741.md?page=1)
