NER tag capitalization question

[ryanwesslen]

hi @BenHolmes,

Thanks for your question and welcome to the Prodigy community

By "tags", I assume you mean label names.

@BenHolmes

When I investigated the annotations with db-out, I saw that some of the tags had been saved as (lowercase) biomarker_name and biomarker_result.

Do you have the exact commands you ran during annotation?

For future purposes, please be sure to provide full Prodigy commands (and ideally Prodigy version too) as this can help debug.

@BenHolmes

I never used the lowercase label when doing any training. Any ideas what's going on?

Yes, but it sounds like you did use lowercase when running annotation as your labels are lower case in your data (i.e., ran prodigy ner.manual ... you used lowercase. How did you do training? Can you provide the full command (e.g., prodigy train vs spacy train)?

@BenHolmes

When I did something like ner.correct asking for the BIOMARKER_NAME or BIOMARKER_RESULT tags, none of the lowercased ones were being displayed.

I'm suspecting you're running into this issue:

Fine-tuned ner transformer model not labelling in Prodigy ner.correct

Hi @ryanwesslen - thank you for the helpful feedback! Wehn I checked the labeles in the model it appears they were all lowercase, when I switched the labels to lowercase in the command line the labels appeared! Just out of curiosity, is it a best practice to keep labels in all caps for spaCy and Prodigy? Thanks so much for your help!!

Probably your best bet now is to output your data, write a Python script to change the labels from lower case to upper case in your .jsonl (e.g., try ChatGPT, it's really great if you give it an example input and an example output for generic Python scripting), then reload your data with db-in. Sorry for the hassle, but as mentioned in that previous post, it's best to always annotation with capital letters for --label.

Hope this helps!