Multiple, separate text classifications in a single model

edward · December 11, 2019, 3:35pm

Thanks, that looks like what I want, but I've been having no fun trying to implement it.

I managed to save my "aggregated" model using the code you gave. But when I tried to import it I got:

KeyError: "[E002] Can't find factory for 'textcat2'. This usually happens when spaCy calls `nlp.create_pipe` with a component name that's not built in - for example, when constructing the pipeline from a model's meta.json. If you're using a custom component, you can write to `Language.factories['textcat2']` or remove it from the model meta and add it via `nlp.add_pipe` instead."

I Googled that and found this thread saying I needed to package with spacy package for it to work: adding custom attribute to doc, having NER use attribute

I tried that, but packaging didn't work:

shutil.Error: [('am-test/textcat', 'am-test-packaged/en_vectors_web_lg-2.1.0/en_vectors_web_lg/en_vectors_web_lg-2.1.0/textcat', "[Errno 13] Permission denied: 'am-test-packaged/en_vectors_web_lg-2.1.0/en_vectors_web_lg/en_vectors_web_lg-2.1.0/textcat'"), ('am-test/textcat3', 'am-test-packaged/en_vectors_web_lg-2.1.0/en_vectors_web_lg/en_vectors_web_lg-2.1.0/textcat3', "[Errno 13] Permission denied: 'am-test-packaged/en_vectors_web_lg-2.1.0/en_vectors_web_lg/en_vectors_web_lg-2.1.0/textcat3'"), ('am-test/vocab', 'am-test-packaged/en_vectors_web_lg-2.1.0/en_vectors_web_lg/en_vectors_web_lg-2.1.0/vocab', "[Errno 13] Permission denied: 'am-test-packaged/en_vectors_web_lg-2.1.0/en_vectors_web_lg/en_vectors_web_lg-2.1.0/vocab'"), ('am-test', 'am-test-packaged/en_vectors_web_lg-2.1.0/en_vectors_web_lg/en_vectors_web_lg-2.1.0', "[Errno 13] Permission denied: 'am-test-packaged/en_vectors_web_lg-2.1.0/en_vectors_web_lg/en_vectors_web_lg-2.1.0'")]

github.com

explosion/spaCy/blob/38e1bc19f4a1c28e27938f46bb3805df4919fedc/spacy/cli/package.py#L71


            shutil.rmtree(path2str(package_path))
        else:
            msg.fail(
                "Package directory already exists",
                "Please delete the directory and try again, or use the "
                "`--force` flag to overwrite existing "
                "directories.".format(path=path2str(package_path)),
                exits=1,
            )
    Path.mkdir(package_path, parents=True)
    shutil.copytree(path2str(input_path), path2str(package_path / model_name_v))
    create_file(main_path / "meta.json", srsly.json_dumps(meta, indent=2))
    create_file(main_path / "setup.py", TEMPLATE_SETUP)
    create_file(main_path / "MANIFEST.in", TEMPLATE_MANIFEST)
    create_file(package_path / "__init__.py", TEMPLATE_INIT)
    msg.good("Successfully created package '{}'".format(model_name_v), main_path)
    msg.text("To build the package, run `python setup.py sdist` in this directory.")




def create_file(file_path, contents):
    file_path.touch()

If I breakpoint at that line, do the copy manually, and then run the rest of the function, it seems to build - but then the resulting .tar.gz is only 4kb, and besides a mention in the meta.json, doesn't include my model.

Topic		Replies	Views
Use textcat and textcat_multilabel in the same model textcat , spacy	1	347	May 19, 2022
A model with multiple labels or multiple models with a single label? usage , textcat , solved	2	453	February 10, 2020
Heirarchical text classification usage , textcat , spacy	1	597	March 25, 2021
Textcat model with multiple classes usage , textcat	5	1535	November 1, 2019
Training multiple Spacy models in Prodigy usage , spacy	7	882	January 11, 2022

Multiple, separate text classifications in a single model

Related topics