# training data format for multiclass textcat **URL:** \ **Category:** Getting Started\ **Tags:** usage, textcat\ **Created:** [August 25, 2022, 10:45pm UTC](https://support.prodi.gy/t/training-data-format-for-multiclass-textcat/5882 "2022-08-25T22:45:06Z")\ **Posts on this page:** 1\ **Showing post:** 2
**Author:** ![ryanwesslen](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ryanwesslen/32/2969_2.png) [@ryanwesslen](https://support.prodi.gy/u/ryanwesslen)\ **Post date:** [August 26, 2022, 1:50pm UTC](https://support.prodi.gy/t/training-data-format-for-multiclass-textcat/5882/2 "2022-08-26T13:50:57Z")
Hi @n8te! Thanks for your question and welcome to the Prodigy community 👋 > [@n8te](#): > > I am very new to spacy Are you interested in only training in spaCy or Prodigy? It seems you're asking for both so I'll try to provide both answers. First for spaCy: If you're only interested in spaCy, here's an example of a standard format (see [spaCy tests](https://github.com/explosion/spaCy/blob/master/spacy/tests/pipeline/test_textcat.py#L34)): ```python TRAIN_DATA_MULTI_LABEL = [ ("I'm angry and confused", {"cats": {"ANGRY": 1.0, "CONFUSED": 1.0, "HAPPY": 0.0}}), ("I'm confused but happy", {"cats": {"ANGRY": 0.0, "CONFUSED": 1.0, "HAPPY": 1.0}}), ] ``` Here are more details from spaCy on the [details for setting up training](https://spacy.io/api/data-formats#binary-training). The rest of my response will assume the question is for Prodigy as this forum is for Prodigy. If you're interested in format to train in Prodigy, we have several examples in Prodigy Support that can help: > [@Multilabel JSONL format & active learning](https://support.prodi.gy/t/multilabel-jsonl-format-active-learning/5843/4): > > hi @suryaiitkgp ! My colleague @Jette16 reminded me there's an alternative data format for multilabel. This includes each tag as the "options" and then whichever are the accepted (selected) labels are in the "accept" {"options": [{"id": "OTHER"}, {"id": "baking"}, {"id":"bread"}, {"id":"chicken"}, {"id":"eggs"}, {"id":"equipment"}, {"id": "food-safety"},{"id":"meat"}, {"id":"sauce"}, {"id":"storage-method"}, {"id":"substitutions"}], "accept":["OTHER","baking"], "text": "How can I get chewy cho… And if you're doing binary classification, sometimes it can be confusing because some examples show `textcat_multilabel`. Here's a post where we try to convert binary data so that you can use `textcat` training instead: > [@textcat vs textcat\_multilabel](https://support.prodi.gy/t/textcat-vs-textcat-multilabel/4729/10): > > Thanks for the background! I assume that you can run --textcat-multilabel, right? python -m prodigy train ebf/ --textcat-multilabel ebf\_0,ebf\_1,ebf\_2 --eval-split 0.2 --base-model en\_core\_web\_lg I definitely understand the point that this is a bit confusing (why would binary classification use a textcat\_multilabel?) so maybe such a helper function could do the trick. As the thread above explains, some of this is due to how spaCy handles this and try to match up to it in Prodigy. A bit of a h… > [@n8te](#): > > I think it needs to be in json format that is then coverted to the spacy .doc format, but i can;t see any simple examples or tutorials showing how this is done. You can do this but it's optional if you're training in Prodigy. An alternative route is to get the data into a `.jsonl` format, then load it as a Prodigy dataset using the [`db-in` command](https://prodi.gy/docs/recipes#db-in). Then you can use [`prodigy train` by pointing to the dataset](https://prodi.gy/docs/recipes#train). > [@n8te](#): > > multiclass categoriser One key point to be careful. spaCy / Prodigy use slightly different terminology for text classification (below from [spaCy textcat documentation](https://spacy.io/api/textcategorizer/#_title)): > The text categorizer predicts **categories over a whole document**. and comes in two flavors: `textcat` and `textcat_multilabel` . When you need to predict exactly one true label per document, use the `textcat` which has mutually exclusive labels. If you want to perform multi-label classification and predict zero, one or more true labels per document, use the `textcat_multilabel` component instead. For a binary classification task, you can use `textcat` with **two** labels or `textcat_multilabel` with **one** label. Notice that there's not the term "multiclass". The key difference is whether you want your labels to be mutually exclusive (which you'd use `textcat`) or non-multually exclusive (use `textcat_multilabel`). This will be important as even after you format and load your data, you will need to select the appropriate type of model that you're training as an argument to your `prodigy train` command. Last, I **highly recommend** looking at some of the spaCy project templates. There are several for `textcat` like: > > > //github.com/explosion/projects/tree/v3/tutorials/textcat\_docs\_issues > > > //github.com/explosion/projects/tree/v3/tutorials/textcat\_goemotions > > > //github.com/explosion/projects/tree/v3/benchmarks/textcat\_architectures FYI these typically cover more of spaCy than Prodigy -- however, a few do include the process of loading `.jsonl` into Prodigy. Although it's for `ner`, there's also a helpful template on Prodigy-spaCy project integration: > > > //github.com/explosion/projects/tree/v3/integrations/prodigy Thanks again for your question! I can understand it's sometimes tough to navigate through all of the resources so I wouldn't be surprised if others have the same question. Let me know if you have any follow up questions! --- _[View the full topic](https://support.prodi.gy/t/training-data-format-for-multiclass-textcat/5882)._