prodigy-hf throws error

1

Hello hello,

I am trying to use prodigy to annotate some texts for NER and then use prodigy-hf to fine-tune a model off huggingface for NER. Installing prodigy-hf as a plugin in the conda environment where I already have prodigy seems to break prodigy, however. After installation, if i issue any prodigy command, I get

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/azureuser/miniconda3/envs/pyspark/lib/python3.11/site-packages/prodigy/__main__.py", line 53, in <module>
    registry.recipes.get_entry_points()
  File "/home/azureuser/miniconda3/envs/pyspark/lib/python3.11/site-packages/catalogue/__init__.py", line 125, in get_entry_points
    result[entry_point.name] = entry_point.load()
                               ^^^^^^^^^^^^^^^^^^
  File "/home/azureuser/miniconda3/envs/pyspark/lib/python3.11/importlib/metadata/__init__.py", line 198, in load
    module = import_module(match.group('module'))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/azureuser/miniconda3/envs/pyspark/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1206, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1149, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/azureuser/miniconda3/envs/pyspark/lib/python3.11/site-packages/prodigy_hf/__init__.py", line 1, in <module>
    from .ner import hf_train_ner, hf_ner_correct
  File "/home/azureuser/miniconda3/envs/pyspark/lib/python3.11/site-packages/prodigy_hf/ner.py", line 16, in <module>
    from prodigy.core import Arg, recipe
ImportError: cannot import name 'Arg' from 'prodigy.core' (/home/azureuser/miniconda3/envs/pyspark/lib/python3.11/site-packages/prodigy/core.cpython-311-x86_64-linux-gnu.so)

Maybe someone knows what's happening? Perhaps it's because i'm on an old (v1.12.7) prodigy?
Thank you for prodigy!
Riccardo.

2

hi @riccardopinosio,

Thanks for your question.

I'm fairly certain yes, it's because you're using an older version of Prodigy (v1.12.7). The problem is in v1.14.0 we switched our CLI to radicli from plac and the Arg it's looking for is from radicli. You could try to install radicli since it's open source (pip install radicli).
But this still wouldn't read from prodigy.core -- you'd need to refactor this to import directly from radicli. I would also expect since we've made a ton of new updates too, you may hit other issues like this so it would require more refactoring.

Unfortunately, we decided with the plugins that we can only maintain with the most recent version of Prodigy. So at least for this, your best option is to install the latest version of Prodigy. Thanks for understanding!

3

Hello,

Ok that's clear, we'll just move to the most recent prodigy! Thank you!