SpanCat Training Error on Custom Preprocessed Dataset

[ryanwesslen]

Thanks for your response.

@daffahilmyf

After downgrading the version, it started working. However, I am now facing a new error like this

Were your annotations altered with any pre- or post-processing? You should avoid any modifications to your annotations if you want to use prodigy train.

That message is a bit too vague for me to diagnose without more details. Was this all of the error message? If not, can you provide the full stack error message?

The closest I found was relating to tokenization:

Error with pos.batch-train

Hi @ines/@honnibal, I trained a PROPN tag for my dataset because I would like the model to recognize certain network entities as proper nouns. Since I had to use my tokenization scheme, I copied the pos.teach recipe from Prodigy over to my recipe and just edited it to use my custom tokenizer. When I use pos.batch-train to train the model, it is failing with an error in the first iteration of the run. I looked on the forums, but the only other issue with a similar error was some issue that was …

I'm wondering if this is a tokenization problem because of some pre- or post-processing you may have done.

Alternatively if not, can you provide a small sample of your data like you did previously?

Also, moving forward, please avoid screen shots of code - you can instead copy/paste it directly. This enables it be searchable for the next user (e.g., now others could search by the same error message and find this post)