How to use output model?

, ,
1

image

Please give me a sample code snippet to use my NER model for text:

"Ngày 20/3/2019, Công ty nhập kho hàng hóa trị giá 100.000.000 đồng và thanh toán bằng tiền mặt, đã bao gồm thuế GTGT".

Processing: output_dir.zip...

Thank you

2

Hi @donhuvy,

You can use the output model just like any other spaCy model. For example, if you want to load the best model and use its functions, you can do:

import spacy
nlp = spacy.load("path/to/model-best/")

text = "Hi, welcome to Prodigy!"
doc = nlp(text)

Prodigy's training system is the same as spaCy v3, making it much more convenient to work between these two tools :slight_smile: This means you can use commands like spacy evaluate , etc. etc. and more

1 Like
3

Thank you very much. I run it, no syntax error, but no result.

import spacy
nlp = spacy.load("D:\\github\\thesis\\temp_prodigy\\output_dir\\model-best")

# text = "30.000.000 đồng thanh toán tiền hàng GTGT"
text = "5.000.000.000 đồng"
doc = nlp(text)
for entity in doc.ents:
    print(entity.text, entity.label_)

image
image1513×485 29.2 KB

Seemly, in my traning dataset, my sentences is too long (it is exactly paragraph in about 30 lines). I will revise it.

4

Hi @donhuvy,

To fully evaluate the performance of your model (and what it can and cannot do), it is recommended to look at multiple examples. You can probably run everything first in a decently-sized test set and make use of spacy evaluate to get the full extent of your model's performance.

Afterwards, there are many ways you can proceed to debug/improve your training:

  • Check the training data. A model's performance is highly-dependent on the dataset it was trained on. You can always check for imbalanced labels, etc.
  • Evaluation results. You can estimate if your model will work well based on its performance in the evaluation/dev dataset.