# \#transformers **URL:** https://support.prodi.gy/tag/transformers/50.md [Latest](https://support.prodi.gy/latest.md) · [Categories](https://support.prodi.gy/categories.md) · [Tags](https://support.prodi.gy/tags.md) --- ## [Prodigy vs DistillBERT model](https://support.prodi.gy/t/prodigy-vs-distillbert-model/4293)
**Author:** [@ElisonSherton](https://support.prodi.gy/u/ElisonSherton)\ **Replies:** 2\ **Last updated:** [February 9, 2025, 6:01pm UTC](https://support.prodi.gy/t/prodigy-vs-distillbert-model/4293 "2025-02-09T18:01:08Z")
Hello everyone! I have observed that prodigy is a really good tool to start with building awesome baselines for text classification/ner tasks. However, when it comes to production level specific tasks where accuracy and… --- ## [\--label-stats for spaCy train](https://support.prodi.gy/t/label-stats-for-spacy-train/7295)
**Author:** [@fsa](https://support.prodi.gy/u/fsa)\ **Replies:** 2\ **Last updated:** [July 7, 2024, 2:03pm UTC](https://support.prodi.gy/t/label-stats-for-spacy-train/7295 "2024-07-07T14:03:20Z")
following the steps described in training with transformer model, I wanted to train my custom ner model in spaCy V3 (involving transformer models) All steps from data conversion, training work fine: python -m prodigy d… --- ## [How to do relation annotation after using bert.mer.manual](https://support.prodi.gy/t/how-to-do-relation-annotation-after-using-bert-mer-manual/6612)
**Author:** [@Jiachen](https://support.prodi.gy/u/Jiachen)\ **Replies:** 2\ **Last updated:** [December 12, 2023, 10:47pm UTC](https://support.prodi.gy/t/how-to-do-relation-annotation-after-using-bert-mer-manual/6612 "2023-12-12T22:47:59Z")
Hi, I'm using Prodigy to do annotation for fine-tuning BERTs, which aims to do the NER and RE tasks. I've annotated the NER with the 'bert.ner.manual' recipe. I wondered if there is a recipe for relation annotation using… --- ## [Training after annotating with custom tokenizer](https://support.prodi.gy/t/training-after-annotating-with-custom-tokenizer/6846)
**Author:** [@AakankshaP](https://support.prodi.gy/u/AakankshaP)\ **Replies:** 3\ **Last updated:** [November 8, 2023, 9:48am UTC](https://support.prodi.gy/t/training-after-annotating-with-custom-tokenizer/6846 "2023-11-08T09:48:18Z")
Hi, I used a custom\_tokenizer to annotate the data, I have saved the model using nlp.to\_disk. def custom\_tokenizer(nlp): infix\_re = compile\_infix\_regex(nlp.Defaults.infixes + \[r'(?\<=\[0-9\])(?=\[a-zA-Z\])', r'\\.'\]) … --- ## [Can prodigy be used to automatically train/predict on any BERT model in huggingface's cloud?](https://support.prodi.gy/t/can-prodigy-be-used-to-automatically-train-predict-on-any-bert-model-in-huggingfaces-cloud/3872)
**Author:** [@rubmz](https://support.prodi.gy/u/rubmz)\ **Replies:** 4\ **Last updated:** [October 25, 2023, 8:49am UTC](https://support.prodi.gy/t/can-prodigy-be-used-to-automatically-train-predict-on-any-bert-model-in-huggingfaces-cloud/3872 "2023-10-25T08:49:54Z")
Hi :slightly\_smiling\_face: I am considering to use prodigy for my next token classification task. I wish to do some fine tuning on a BERT model off from huggingface repository. It's a Hebrew model (avichr/heBERT · Huggi… --- ## [Integrating SpanCat with HuggingFace, specifically AutoTrain](https://support.prodi.gy/t/integrating-spancat-with-huggingface-specifically-autotrain/6821)
**Author:** [@gmp](https://support.prodi.gy/u/gmp)\ **Replies:** 2\ **Last updated:** [October 25, 2023, 8:48am UTC](https://support.prodi.gy/t/integrating-spancat-with-huggingface-specifically-autotrain/6821 "2023-10-25T08:48:38Z")
I am new to the SpaCy/ Prodigy ecosystem so this is more of general question: can the annotated files created while using Spancat, specifically overlapping spans, be used to fine tune specialized models (like BiomedBERT)… --- ## [Transform annotations to match tokenization required for SpanBERT/BERT](https://support.prodi.gy/t/transform-annotations-to-match-tokenization-required-for-spanbert-bert/5923)
**Author:** [@Yan](https://support.prodi.gy/u/Yan)\ **Replies:** 19\ **Last updated:** [July 30, 2023, 12:21pm UTC](https://support.prodi.gy/t/transform-annotations-to-match-tokenization-required-for-spanbert-bert/5923 "2023-07-30T12:21:05Z")
Hi team, We are using prodigy a lot in my place of work and we are trying to evaluate how much better transformer models can be compared to the ones we already have (built using the default prodigy native ones). We alre… --- ## [Multi-user sessions using a custom transformer-based span-cat pipeline](https://support.prodi.gy/t/multi-user-sessions-using-a-custom-transformer-based-span-cat-pipeline/6602)
**Author:** [@coltonflowers1](https://support.prodi.gy/u/coltonflowers1)\ **Replies:** 1\ **Last updated:** [June 14, 2023, 3:57pm UTC](https://support.prodi.gy/t/multi-user-sessions-using-a-custom-transformer-based-span-cat-pipeline/6602 "2023-06-14T15:57:05Z")
My team is trying to use the span.correct recipe from a transformer-based pipeline using multi-user sessions with feed\_overlap set to true. However, after starting the server using the following recipe: prodigy spans.co… --- ## [Span vs NER, compatibility with transformers models](https://support.prodi.gy/t/span-vs-ner-compatibility-with-transformers-models/6543)
**Author:** [@Alvaro8gb](https://support.prodi.gy/u/Alvaro8gb)\ **Replies:** 2\ **Last updated:** [May 17, 2023, 10:44am UTC](https://support.prodi.gy/t/span-vs-ner-compatibility-with-transformers-models/6543 "2023-05-17T10:44:36Z")
Hey what a good community, we are trying to strcutured information in medical domain and he have a good cuestions that maybe more people will benefict. Our aims is from this clinical note : "Multifocal unspecified and … --- ## [mismatched structure when using tranformers model to train textcat (en\_core\_web\_trf)](https://support.prodi.gy/t/mismatched-structure-when-using-tranformers-model-to-train-textcat-en-core-web-trf/5015)
**Author:** [@mwade625](https://support.prodi.gy/u/mwade625)\ **Replies:** 16\ **Last updated:** [March 29, 2023, 10:08pm UTC](https://support.prodi.gy/t/mismatched-structure-when-using-tranformers-model-to-train-textcat-en-core-web-trf/5015 "2023-03-29T22:08:42Z")
I am using Ubuntu Linux version 20.14, Python 3.9, Prodigy 1.11.6, Spacy 3.2.0. I also downloaded the latest version of en\_core\_web\_trf. I created my model using prodigy the following command: prodigy train ./output … --- ## [NER training produces invalid config.cfg?](https://support.prodi.gy/t/ner-training-produces-invalid-config-cfg/4642)
**Author:** [@bob\_ln](https://support.prodi.gy/u/bob_ln)\ **Replies:** 13\ **Last updated:** [March 21, 2023, 9:02am UTC](https://support.prodi.gy/t/ner-training-produces-invalid-config-cfg/4642 "2023-03-21T09:02:49Z")
Hi, not sure if this question goes here or spaCy support, so here goes. Prodigy version: 1.11.2 Training command: prodigy train ner\_20210830 --ner \ --base-model en\_core\_web\_trf --label-stats The training we… --- ## [Train Curve not using GPU even tho -g is set.](https://support.prodi.gy/t/train-curve-not-using-gpu-even-tho-g-is-set/6068)
**Author:** [@turulix](https://support.prodi.gy/u/turulix)\ **Replies:** 3\ **Last updated:** [March 14, 2023, 1:44pm UTC](https://support.prodi.gy/t/train-curve-not-using-gpu-even-tho-g-is-set/6068 "2023-03-14T13:44:23Z")
Hey! I'm not sure if i'm just simply missing something or if this is an actual bug. When I run prodigy train-curve -g 0 --spancat Dataset -c .\\config.cfg it will still train on the CPU. prodigy train -g 0 --spancat Da… --- ## [config.cfg for bert.ner.manual](https://support.prodi.gy/t/config-cfg-for-bert-ner-manual/5970)
**Author:** [@ruiyeNLP](https://support.prodi.gy/u/ruiyeNLP)\ **Replies:** 5\ **Last updated:** [September 30, 2022, 8:33am UTC](https://support.prodi.gy/t/config-cfg-for-bert-ner-manual/5970 "2022-09-30T08:33:26Z")
Hi, I annotated my file with the command prodigy bert.ner.manual data\_5\_trf ./input/data\_5\_ground\_truth\_1.0.jsonl --label RIGHTV,RIGHTN,ACCESSV,ACCESSN --tokenizer-vocab ./bert-base-uncased-vocab.txt --lowercase --hide-… --- ## [Training BERT on prodigy](https://support.prodi.gy/t/training-bert-on-prodigy/6339)
**Author:** [@Mohammad](https://support.prodi.gy/u/Mohammad)\ **Replies:** 3\ **Last updated:** [February 2, 2023, 3:09pm UTC](https://support.prodi.gy/t/training-bert-on-prodigy/6339 "2023-02-02T15:09:24Z")
After I did the annotation on prodigy how I can run training on prodigy for BERT python3 -m prodigy bert.ner.manual correctd\_drsv1 ./model\_drsv1 ./CV\_json\_English\_Format.jsonl --label Skill,MaritalStatus,University,De… --- ## [Can I do annotation for BERT on prodigy](https://support.prodi.gy/t/can-i-do-annotation-for-bert-on-prodigy/6300)
**Author:** [@Mohammad](https://support.prodi.gy/u/Mohammad)\ **Replies:** 8\ **Last updated:** [January 31, 2023, 7:47am UTC](https://support.prodi.gy/t/can-i-do-annotation-for-bert-on-prodigy/6300 "2023-01-31T07:47:20Z")
Can I do annotation and convert to BERT on prodigy --- ## [Can one leverage zero-shot classifiers for textcat tasks?](https://support.prodi.gy/t/can-one-leverage-zero-shot-classifiers-for-textcat-tasks/4885)
**Author:** [@davidefiocco](https://support.prodi.gy/u/davidefiocco)\ **Replies:** 3\ **Last updated:** [January 15, 2023, 8:42pm UTC](https://support.prodi.gy/t/can-one-leverage-zero-shot-classifiers-for-textcat-tasks/4885 "2023-01-15T20:42:18Z")
Hi! I understand that textcat.teach can use pattern-matching to bootstrap the labeling of (rare) classes in text classification tasks, but I would like to know your thoughts about using zero-shot classifiers (e.g. Huggin… --- ## [Will using transformers improve textcat\_multilabel score/accuracy?](https://support.prodi.gy/t/will-using-transformers-improve-textcat-multilabel-score-accuracy/6199)
**Author:** [@joebuckle](https://support.prodi.gy/u/joebuckle)\ **Replies:** 2\ **Last updated:** [January 3, 2023, 1:43am UTC](https://support.prodi.gy/t/will-using-transformers-improve-textcat-multilabel-score-accuracy/6199 "2023-01-03T01:43:32Z")
We are currently training on a CPU and we are getting 70% on the textcat score. Is there a guarantee that using transformers will improve our results a lot? We don't want to spend a lot on a GPU if the improvement is not… --- ## [TypeError: 'FullTransformerBatch' object is not iterable](https://support.prodi.gy/t/typeerror-fulltransformerbatch-object-is-not-iterable/4592)
**Author:** [@atesas](https://support.prodi.gy/u/atesas)\ **Replies:** 11\ **Last updated:** [November 22, 2022, 11:30am UTC](https://support.prodi.gy/t/typeerror-fulltransformerbatch-object-is-not-iterable/4592 "2022-11-22T11:30:17Z")
Hi! Firstly, thanks very much for this great product and support which is very useful and efficient for my research! I am trying to use en\_core\_web\_trf base model, but I keep receiving the following error (There is no … --- ## [data-to-spacy for transformers](https://support.prodi.gy/t/data-to-spacy-for-transformers/6019)
**Author:** [@jordandavis](https://support.prodi.gy/u/jordandavis)\ **Replies:** 6\ **Last updated:** [October 12, 2022, 9:18pm UTC](https://support.prodi.gy/t/data-to-spacy-for-transformers/6019 "2022-10-12T21:18:23Z")
I have an annotated dataset for ner and I've used trained a model successfully using the prodigy train command. I wanted to train a new model using transformers and used the starter config downloaded from spacy and fill… --- ## [mismatched structure when loading ner tranformers model (en\_core\_web\_trf)](https://support.prodi.gy/t/mismatched-structure-when-loading-ner-tranformers-model-en-core-web-trf/6013)
**Author:** [@ruiyeNLP](https://support.prodi.gy/u/ruiyeNLP)\ **Replies:** 1\ **Last updated:** [October 7, 2022, 4:16pm UTC](https://support.prodi.gy/t/mismatched-structure-when-loading-ner-tranformers-model-en-core-web-trf/6013 "2022-10-07T16:16:25Z")
I am using Python 3.8, Prodigy 1.11.8, Spacy 3.4.0. I also downloaded the latest version of en\_core\_web\_trf. I created my model 'en\_tagger\_parser\_trf' using prodigy the following code: The model was trained with the… --- ## [Can't find recipe or command 'bert.ner.manual'](https://support.prodi.gy/t/cant-find-recipe-or-command-bert-ner-manual/5965)
**Author:** [@ruiyeNLP](https://support.prodi.gy/u/ruiyeNLP)\ **Replies:** 4\ **Last updated:** [September 23, 2022, 5:07pm UTC](https://support.prodi.gy/t/cant-find-recipe-or-command-bert-ner-manual/5965 "2022-09-23T17:07:39Z")
Hey, I got the error when using prodigy bert.ner.manual My spacy environment is as follows and prodigy version is 1.11.8. Do you know how i can use the bert.ner.manual recipe? --- ## [ner.teach starts going wacky about 30 examples in](https://support.prodi.gy/t/ner-teach-starts-going-wacky-about-30-examples-in/5141)
**Author:** [@tdauria](https://support.prodi.gy/u/tdauria)\ **Replies:** 10\ **Last updated:** [January 14, 2022, 3:31pm UTC](https://support.prodi.gy/t/ner-teach-starts-going-wacky-about-30-examples-in/5141 "2022-01-14T15:31:30Z")
I'm using ner.teach to fine-tune a model to detect ORG entities. It starts off great and presents reasonable examples that are fairly accurate. About 20 examples in, it (correctly) shows a sequence of examples with no … --- ## [Convert spancat annotations for use with transformer model](https://support.prodi.gy/t/convert-spancat-annotations-for-use-with-transformer-model/5822)
**Author:** [@DGMS90](https://support.prodi.gy/u/DGMS90)\ **Replies:** 4\ **Last updated:** [August 11, 2022, 9:42am UTC](https://support.prodi.gy/t/convert-spancat-annotations-for-use-with-transformer-model/5822 "2022-08-11T09:42:14Z")
Hey, I would like to be able to use e.g. bert-large-cased or spanBERT as the embedding layer for a span categorisation task. Importantly, these spans will often be overlapping. However, I can't seem to find examples of … --- ## [Cannot train tagger on trf models](https://support.prodi.gy/t/cannot-train-tagger-on-trf-models/5734)
**Author:** [@joe](https://support.prodi.gy/u/joe)\ **Replies:** 3\ **Last updated:** [June 24, 2022, 7:30am UTC](https://support.prodi.gy/t/cannot-train-tagger-on-trf-models/5734 "2022-06-24T07:30:56Z")
Hi, it seems that for the Spanish trf pretrained model, there is no 'tagger' element, so it cannot be trained. How would one, then, improve the POS tagging from that model? Additionally, does the NER component rely on P… --- ## [pretrain weights for transfer learning](https://support.prodi.gy/t/pretrain-weights-for-transfer-learning/3334)
**Author:** [@raylite](https://support.prodi.gy/u/raylite)\ **Replies:** 7\ **Last updated:** [June 1, 2022, 2:06pm UTC](https://support.prodi.gy/t/pretrain-weights-for-transfer-learning/3334 "2022-06-01T14:06:53Z")
I am new to prodigy. I have an NER task of which I have used ner.manual to provide some initial labels. In order to test the prospect I want to use ner.train with pretrained weights. I tried using Scibert provided weight… --- ## [We are not able to get more-or-less similar F-score when upgrading prodigy and spacy](https://support.prodi.gy/t/we-are-not-able-to-get-more-or-less-similar-f-score-when-upgrading-prodigy-and-spacy/5490)
**Author:** [@echarles](https://support.prodi.gy/u/echarles)\ **Replies:** 0\ **Last updated:** [March 28, 2022, 9:15pm UTC](https://support.prodi.gy/t/we-are-not-able-to-get-more-or-less-similar-f-score-when-upgrading-prodigy-and-spacy/5490 "2022-03-28T21:15:56Z")
Upgrade of spacy from version 2.3.7 to 3.2.3 Upgrade of prodigy from version 1.10.8 to 1.11.7 Hi, We have been using prodigy V1.10.8 (spacy v2) to annotate and train NER model. Our entities (33) are "new" so we have … --- ## [How to keep newline when use bert.ner.manual?](https://support.prodi.gy/t/how-to-keep-newline-when-use-bert-ner-manual/5474)
**Author:** [@linb](https://support.prodi.gy/u/linb)\ **Replies:** 1\ **Last updated:** [March 28, 2022, 2:22am UTC](https://support.prodi.gy/t/how-to-keep-newline-when-use-bert-ner-manual/5474 "2022-03-28T02:22:58Z")
After I used bert.ner.manual, prodigy bert.ner.manual ner\_reddit ./small.jsonl --label PLATFORM-AWS,PLATFORM-AZURE --tokenizer-vocab ./bert-base-uncased-vocab.txt --lowercase --hide-wp-prefix --hide-special -F transform… --- ## [spacy.gold missing](https://support.prodi.gy/t/spacy-gold-missing/5379)
**Author:** [@aph61](https://support.prodi.gy/u/aph61)\ **Replies:** 2\ **Last updated:** [March 1, 2022, 10:16am UTC](https://support.prodi.gy/t/spacy-gold-missing/5379 "2022-03-01T10:16:30Z")
Installation of spacy==3.2.2 appears to be good (as in, I never had problems before) but now I continue to have the issue that spacy-pytorch-transformers requires spacy.gold PS: I re-installed everything in a clean, new… --- ## [Token indices sequence length is too long](https://support.prodi.gy/t/token-indices-sequence-length-is-too-long/5268)
**Author:** [@EM22](https://support.prodi.gy/u/EM22)\ **Replies:** 2\ **Last updated:** [February 3, 2022, 11:10am UTC](https://support.prodi.gy/t/token-indices-sequence-length-is-too-long/5268 "2022-02-03T11:10:25Z")
Hi, I'm trying to update an existing model with a new entity, but apparently the new dataset of annotated data I am training on includes token sequences that are exceeding the model's limit. When training, I get the fol… --- ## [train --spancat questions](https://support.prodi.gy/t/train-spancat-questions/5224)
**Author:** [@mhlucero](https://support.prodi.gy/u/mhlucero)\ **Replies:** 2\ **Last updated:** [January 26, 2022, 11:05am UTC](https://support.prodi.gy/t/train-spancat-questions/5224 "2022-01-26T11:05:02Z")
Hello, This is my first post :slight\_smile: in the forum. Sorry if I make mistakes, so here goes. I'm using prodigy 1.11.6, spacy 3.1.4, spacy-transformers 1.0.6 and python 3.7.3 for this project. I begin with some que… [Next page](https://support.prodi.gy/tag/transformers/50.md?match_all_tags=true&page=1&tags%5B%5D=transformers)