Installing `transformers` without installing `torch` crashes prodigy

1

Hello! Installing transformers without installing torch completely breaks Prodigy. Running prodigy command fails with

None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/usr/local/lib/python3.11/site-packages/prodigy/__main__.py", line 50, in <module>
    main()
  File "/usr/local/lib/python3.11/site-packages/prodigy/__main__.py", line 44, in main
    controller = run_recipe(run_args)
                 ^^^^^^^^^^^^^^^^^^^^
  File "cython_src/prodigy/cli.pyx", line 111, in prodigy.cli.run_recipe
  File "cython_src/prodigy/cli.pyx", line 36, in prodigy.cli.get_cli
  File "/usr/local/lib/python3.11/site-packages/prodigy/recipes/__init__.py", line 1, in <module>
    from . import (
  File "/usr/local/lib/python3.11/site-packages/prodigy/recipes/audio.py", line 3, in <module>
    from ..components.preprocess import fetch_media as fetch_media_preprocessor
  File "cython_src/prodigy/components/preprocess.pyx", line 9, in init prodigy.components.preprocess
  File "/usr/local/lib/python3.11/site-packages/spacy_llm/__init__.py", line 1, in <module>
    from . import cache  # noqa: F401
    ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/spacy_llm/cache.py", line 11, in <module>
    from .ty import PromptTemplateProvider, ShardingLLMTask
  File "/usr/local/lib/python3.11/site-packages/spacy_llm/ty.py", line 14, in <module>
    from .models import langchain
  File "/usr/local/lib/python3.11/site-packages/spacy_llm/models/__init__.py", line 1, in <module>
    from .hf import dolly_hf, openllama_hf, stablelm_hf
  File "/usr/local/lib/python3.11/site-packages/spacy_llm/models/hf/__init__.py", line 7, in <module>
    from .stablelm import stablelm_hf
  File "/usr/local/lib/python3.11/site-packages/spacy_llm/models/hf/stablelm.py", line 11, in <module>
    class _StopOnTokens(transformers.StoppingCriteria):
  File "/usr/local/lib/python3.11/site-packages/spacy_llm/models/hf/stablelm.py", line 13, in _StopOnTokens
    self, input_ids: torch.LongTensor, scores: torch.FloatTensor, **kwargs
                     ^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'LongTensor'

I don't think that installing any kinds of requirements should impact the way the tool works. Installing transformers should not out of nowhere require installing torch.

Thanks!

1 Like
2

Hi @egor ,

This is actually spacy-llm requiring torch there. It's a bit tricky to catch as it is only at the import time that you know its required.
spaCy docs mention the fact that HF models via transformers require torch installed: Large Language Models · spaCy Usage Documentation. Admittedly, we should probably add a warning to our docs as well.

3

Hi, thanks for the reply! The issue is that we don't use spacy-llm. We just installed transformers for our own purposes and it automatically breaks anything with prodigy command unless we install torch.

I assume there is something like

if transformers_installed():
    import <spacy llm>

But the fact that transformers are installed does not guarantee that torch is installed.

So the check probably should be

if transformers_installed() and torch_installed():
    import <spacy llm>

What do you think?

1 Like
4

Hi @egor ,

Ah yeah, I can see now it's a big issue as we're importing spacy-llm in parts of the library that do not necessarily require them. Thanks again for reporting!
The conditional import is a bit tricky because you don't really need either to still use spacy-llm e.g. with models other than HF models.
We'll definitely try to mitigate the problem asap!

1 Like
5

Thank you!

6

Hi @egor, thanks for the report!

This was a bug in spacy-llm, which has now been fixed and released as part of spacy-llm 0.7.2. In your environment, if you update spacy-llm to this version, you shouldn't get this crash anymore. In a new environment, Prodigy will automatically pull in the updated version of spacy-llm and it hopefully shouldn't bother you anymore.

One more note. Your traceback actually contains two issues, the crash that's fixed, and above that a warning:

None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.

This warning is thrown by HuggingFace's transformers library if you install it without installing Torch, Tensorflow or Flax. Some related discussion is here: None of PyTorch, TensorFlow >= 2.0, or Flax have been found · Issue #27214 · huggingface/transformers · GitHub. This is an upstream issue that we can't do much about, but the warning shouldn't result in a crash either, so hopefully you'll be able to just continue working as is.

Let us know if you do run into any further issues!

1 Like