Can't find recipe or command 'bert.ner.manual'

ruiyeNLP · September 23, 2022, 9:48am

Hey, I got the error when using prodigy bert.ner.manual

My spacy environment is as follows and prodigy version is 1.11.8.

Do you know how i can use the bert.ner.manual recipe?

ryanwesslen · September 23, 2022, 9:57am

hi @ruiyeNLP!

Thanks for your question!

That recipe is posted on our GitHub repo for prodigy-recipes:

github.com

explosion/prodigy-recipes/blob/master/other/transformers_tokenizers.py

"""This recipe requires Prodigy v1.10+."""
from typing import List, Optional, Union, Iterable, Dict, Any
from tokenizers import BertWordPieceTokenizer
from prodigy.components.loaders import get_stream
from prodigy.util import get_labels
import prodigy


@prodigy.recipe(
    "bert.ner.manual",
    # fmt: off
    dataset=("Dataset to save annotations to", "positional", None, str),
    source=("Data to annotate (file path or '-' to read from standard input)", "positional", None, str),
    loader=("Loader (guessed from file extension if not set)", "option", "lo", str),
    label=("Comma-separated label(s) to annotate or text file with one label per line", "option", "l", get_labels),
    tokenizer_vocab=("Tokenizer vocab file", "option", "tv", str),
    lowercase=("Set lowercase=True for tokenizer", "flag", "LC", bool),
    hide_special=("Hide SEP and CLS tokens visually", "flag", "HS", bool),
    hide_wp_prefix=("Hide wordpieces prefix like ##", "flag", "HW", bool)
    # fmt: on

This file has been truncated. show original

The documentation has a button with the link but I can understand as it's easy to overlook. With this file, you can then run the recipe by referencing the name of the file with -F like:

prodigy bert.ner.manual ... -F transformers_tokenizers.py

Hope this helps!

ruiyeNLP · September 23, 2022, 10:50am

True, my command was as below. But the error occurred then.

prodigy bert.ner.manual data_5_trf ./input/data_5_ground_truth_1.0.jsonl --label RIGHTV,RIGHTN,ACCESSV,ACCESSN --tokenizer-vocab ./bert-base-uncased-vocab.txt --lowercase --hide-wp-prefix -F transformers_tokenizers.py

ryanwesslen · September 23, 2022, 12:28pm

hi @ruiyeNLP,

To be clear, in order to run that command did you take the python script I provided, renamed it transformers_tokenizers.py, and it is located in the current directory where you're running the command? If you have a script recipe, using the -F command will use that recipe.

ruiyeNLP · September 23, 2022, 5:07pm

Thanks. It worked now. The issue was that transformers_tokenizers.py was not in the current directory.

Topic		Replies	Views
bert.ner.manual with pattern matcher ner	1	263	February 8, 2023
config.cfg for bert.ner.manual usage , ner , transformers	5	813	September 30, 2022
transformers model for NER ner , spacy	6	394	October 31, 2023
Can I do annotation for BERT on prodigy transformers	8	443	January 31, 2023
Using transformers_tokenizers.py usage , solved	2	328	June 30, 2021

Can't find recipe or command 'bert.ner.manual'

Related topics