How can I compare my ner model with a huggingface transformer model?

Hi @giovaninb , welcome to Prodigy! :slight_smile:

How can I compare using seqeval (GitHub - chakki-works/seqeval: A Python framework for sequence labeling evaluation(named-entity recognition, pos tagging, etc...))?

How about running spacy evaluate on BioBERT's test set, obtain the Precision/Recall/F-score, and comparing it with the reported results in the paper? It might also be done the other way around: run the trained model to a test set, get the predictions, and use those predictions with the seqeval repo. :thinking:

How can I import my annotated databases to huggingface data?

This thread should be useful: Exporting NER annotations for HF datasets - #4 by ChrisM
It talks about a user project that uses Prodigy annotations in the datasets library and training a transformer with transformers.

1 Like