NER document Labeling

[mystuff]

I have annotated 150 htmls, the raw text is separated by new lines. Now i need to run "prodigy ner.batch-train ". am i right?.

python -m prodigy ner.batch-train company_details_dataset en_core_web_sm --output company_model --label COMPANY_TYPE,COMPANY_INFORMATION, COMPANY_NAME,COMPANY_DEPARTMENT,COMPANY_ADDRESS,COMPANY_COUNTRY_USA,EMAIL,NAME

when i run above train command that i am getting below error:
File "transition_system.pyx", line 148, in spacy.syntax.transition_system.TransitionSystem.set_costs
ValueError: [E024] Could not find an optimal move to supervise the parser. Usually, this means the GoldParse was not correct. For example, are all labels added to the model?

It seems that there are some white spaces issue so i followed below post to fix it.

ner.batch-train after ner.maual results error (Value error : [E024])

You can export your dataset by running the db-out command and then check the JSONL file: prodigy db-out resume_ner > resume_ner.jsonl After you’ve removed the problematic spans or have corrected them, you can then reimport the data to a new dataset: prodigy db-in resume_ner_fixed resume_ner.jsonl You can probably also write a script to find the problematic entities automatically and then exclude them, and add the result to a new dataset. I haven’t tested this yet, but something like this sh…

Still getting same error. I can see many of these in the dataset {"text":"\n","start":3359,"end":3360,"id":594},. do you think its a tokenization issue?. If so, how do i pass new line tokenizer while running "prodigy ner.batch-train "