Why there is no token_start and token_end in the mark file?How to solve it.

1

I use the following command to mark a file and name it ner-d3

python -m prodigy ner.manual ner-d3 ./ner-d3.txt --label MD,GD,DD,RD,PHON

Then I trained the dataset and generated model-best

Then I marked a new file with the following command and named it the dataset ner-d1

python -m prodigy ner.correct ner-d1 ./ner-d2/model-best/ ./ner-d1.txt

After labeling, I downloaded jsonl .

python -m prodigy db-out ner-d1 > ner-d1.jsonl

but I found that there are no token-start and token-end attributes in the span of ner-d1.jsonl. I need the two attributes. How can I solve this problem? Thank you.

image
image970×65 4.86 KB

2

Hi there!

When I look at your first command:

python -m prodigy ner.manual ner-d3 ./ner-d3.txt --label MD,GD,DD,RD,PHON

Then I can see that you've added a --label to the command. However, this seems missing from your second command.

python -m prodigy ner.correct ner-d1 ./ner-d2/model-best/ ./ner-d1.txt

I wonder if this is causing the issue, could you confirm?