error : Unknown command: huggingface-hub while uploading spacy model on huggingface

, , , ,
1

Hi,
I am trying to upload my named entity recognition custom model on huggingface repository, prepared with prodi.gy spacy on my scientific text.
As prescribed on spacy (spaCy), I ran pip install spacy-huggingface-hub in my ubuntu (22.04) terminal.
I notice the module in the list of packages.
However when I run 'python -m spacy huggingface-hub --help' to check, the terminal gives me error 'Unknown command: huggingface-hub'.

Any suggestion?

2

hi @rahul1!

This is a forum for Prodigy-related issues. Unfortunately I haven't used that package. It sounds like that's just some installation issue. I just installed and it worked fine.

But a few questions:

  • Did you receive any warning/error messages when you installed spacy-huggingface-hub?
  • Did you install into a new venv?
  • Can you run python -m spacy info? If so, what version?
  • Per the README, can you try to run it within Python like:
from spacy_huggingface_hub import push

result = push("./en_ner_fashion-0.0.0-py3-none-any.whl")
print(result["url"])
  • Have you tried on another OS (e.g., Ubuntu 20.04)?

The issue may be with spaCy, not necessarily huggingface-hub. Alternatively, I wonder if Ubuntu 22.04 may have some issues too. I suspect the package wasn't designed for 22.04 since it's fairly new.

If none of this helps, your best bet would be to log an issue on the GitHub repo for the package. If you do this, please make sure to follow general rules for spaCy FAQ.

3

Thank you very much for your suggestion Ryan.
I figured out that I created my custom model with Prodi.gy with spacy V2. However, the commands on spacy (spaCy) are with spacy V3. I tried downgrading spacy to V2 and proceed with the commands, but that gives error as "command --build wheel is not found".
The other option is to find way to upgrade my custom model with Spacy v3. I will try that.
regards
Rahul

4

@ryanwesslen : Just let you know I am busy with a different approach, as described here under.

Header: How to reuse the prodigy.db to retrain the older (spacy v2) ner custom model

Hi,
Last year in June 2021 I created a ner custom model with Prodi.gy (and spacy 2.x.x.) on my windows laptop:
python -m prodigy train ner dataset,dataset_correct,dataset_correct1,dataset_correct3 en_vectors_web_lg — output C:\Users\myname\Documents\tmp_model — eval-split 0.2 — n-iter 40

I tried to upload this model to huggingface.co, but I couldnot, due to incompatibility between spacy2 used in the model vs spacy3 of the spacy-huggingface-hub. Therefore I have decided to install prodi.gy on my ubuntu laptop 22.04 to retrain the old model or rebuilt it, depending on the possibility.

I still have my .prodigy folder from my windows laptop from last year. It contains two files: prodigy.db (168 MB, 9 datasets) and prodigy.json (6B). I want to reuse this prodigy.db database to retrain or rebuilt the old model.

Can you please give suggestions on how to do it, with links to right code?

gr.
Rahul

5

Hi @rahul1 , we answered this question in your separate post:

1 Like