# How to use loader to load a csv with text and label?

**URL:** <https://support.prodi.gy/t/how-to-use-loader-to-load-a-csv-with-text-and-label/2972>\
**Category:** Uncategorized\
**Tags:** usage, textcat\
**Created:** [June 2, 2020, 9:45am UTC](https://support.prodi.gy/t/how-to-use-loader-to-load-a-csv-with-text-and-label/2972 "2020-06-02T09:45:43Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![PlataleaMinor](https://avatars.discourse-cdn.com/v4/letter/p/5fc32e/32.png) [@PlataleaMinor](https://support.prodi.gy/u/PlataleaMinor)\
**Post date:** [June 2, 2020, 9:45am UTC](https://support.prodi.gy/t/how-to-use-loader-to-load-a-csv-with-text-and-label/2972/1 "2020-06-02T09:45:43Z")

</div>

Hi, I have a csv file labeled with person or company. The sample file is like below

text,label  
ABC Company, company  
BCD Company, company  
Peter Smith, Person  
Mary Smith, Person

I would like to load the csv into prodigy for text classification. Yet, I could not find much information on the implementation of loader. Thank you for your asssistance.

---

<div class="post-metadata">

**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\
**Post date:** [June 2, 2020, 2:00pm UTC](https://support.prodi.gy/t/how-to-use-loader-to-load-a-csv-with-text-and-label/2972/2 "2020-06-02T14:00:12Z")

</div>

Hi! Just to make sure I understand your question correctly: you want to use your examples in the CSV file to pre-select examples containing those company and person names? Because the example you posted doesn't look like the data you're annotating, right?

In that case, you can convert them to a [`patterns.jsonl` file](https://prodi.gy/docs/api-loaders#input-patterns) and then use the file as the `--patterns` argument. Here's an example:

```json
{"label": "company", "pattern": "ABC Company"}
{"label": "person", "pattern": "Peter Smith"}

```

If you just want to load a CSV file for annotation, the default [CSV loader](https://prodi.gy/docs/api-loaders#loaders-file) should also work fine if your examples have a column `text`. You can also find the docs on [custom loaders here](https://prodi.gy/docs/api-loaders#loaders-custom). If you're using a custom recipe, all you need is a generator that yields dictionaries, so you don't have to follow any specific implementation. How you set that up is up to you 🙂

---

<div class="post-metadata">

**Author:** ![PlataleaMinor](https://avatars.discourse-cdn.com/v4/letter/p/5fc32e/32.png) [@PlataleaMinor](https://support.prodi.gy/u/PlataleaMinor)\
**Post date:** [June 3, 2020, 7:58am UTC](https://support.prodi.gy/t/how-to-use-loader-to-load-a-csv-with-text-and-label/2972/3 "2020-06-03T07:58:02Z")

</div>

Thanks Ines. I have created a jsonl which has the exact format you mentioned. Just want to make sure I followed it correctly, the code should be

> prodigy textcat.manual name\_classification file.jsonl --label name,company --pattern file.jsonl

Is this correct? Thanks again

---

<div class="post-metadata">

**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\
**Post date:** [June 3, 2020, 11:21am UTC](https://support.prodi.gy/t/how-to-use-loader-to-load-a-csv-with-text-and-label/2972/4 "2020-06-03T11:21:37Z")

</div>

No, that wouldn't work – the [`textcat.manual` recipe](https://prodi.gy/docs/recipes#textcat-manual) doesn't take an argument for pattern files. It will just go through all examples in your data and will ask you to label every example. You can also see some examples of text classification workflows [in the docs](https://prodi.gy/docs/text-classification)

What are you trying to achieve with the patterns / the examples in your CSV file? What do they mean, and what do you want to use them for? If you want to use them to select examples to annotate, maybe a workflow like [`match`](https://prodi.gy/docs/recipes#match) with `--combine-matches` and `--label-taks` would be a better fit?

---

<div class="post-metadata">

**Author:** ![PlataleaMinor](https://avatars.discourse-cdn.com/v4/letter/p/5fc32e/32.png) [@PlataleaMinor](https://support.prodi.gy/u/PlataleaMinor)\
**Post date:** [June 9, 2020, 8:45am UTC](https://support.prodi.gy/t/how-to-use-loader-to-load-a-csv-with-text-and-label/2972/5 "2020-06-09T08:45:38Z")

</div>

Hi ines, I am thinking considering to transform the csv to jsonl format and use db-in to resolve the issue. I have db-out my existing db and see the format. Yet, I would like to know do I need to \_input\_hash, \_task\_hash,\_session\_id,\_view\_id and answer in my jsonl

sample jsonl  
{"text":"ABC Limited","label":"company"}  
{"text":"John Doe","label":"person"}

db out jsonl  
{"text":"ABC Limited","\_input\_hash":-6232565122,"\_task\_hash":-520270855,"label":"company","\_session\_id":null,"\_view\_id":"classification","answer":"accept"}  
{"text":"John Doe","\_input\_hash":-6232554122,"\_task\_hash":-510170855,"label":"person","\_session\_id":null,"\_view\_id":"classification","answer":"accept"}

Thanks again

---

<div class="post-metadata">

**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\
**Post date:** [June 10, 2020, 7:51am UTC](https://support.prodi.gy/t/how-to-use-loader-to-load-a-csv-with-text-and-label/2972/6 "2020-06-10T07:51:03Z")

</div>

> [@PlataleaMinor](#):
>
> Yet, I would like to know do I need to \_input\_hash, \_task\_hash,\_session\_id,\_view\_id and answer in my jsonl

`answer` is required, yes, because it indicates whether the example is a positive or negative example of that label, or whether it was skipped. The hashes are required, but they're generated automatically based on the data, so you don't have to pre-set them. `_session_id`, `_view_id` are also generated automatically when you annotate.

Btw, are you sure you want to use those single phrases to train your text classifier? Text classification typically works best if you have _actual_ text, like sentences or paragraphs, and you're typically predicting categories over the whole text. In your example, you seem to just have single phrases, more like named entities?

---

<div class="post-metadata">

**Author:** ![PlataleaMinor](https://avatars.discourse-cdn.com/v4/letter/p/5fc32e/32.png) [@PlataleaMinor](https://support.prodi.gy/u/PlataleaMinor)\
**Post date:** [June 17, 2020, 2:22am UTC](https://support.prodi.gy/t/how-to-use-loader-to-load-a-csv-with-text-and-label/2972/7 "2020-06-17T02:22:58Z")

</div>

For my use case, I would like to classify a single phrase eg. John Smith, Tesco Limited into either person or company. Are you suggesting I should use ner instead of textcat? Thank you so much

---

<div class="post-metadata">

**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\
**Post date:** [June 17, 2020, 9:37am UTC](https://support.prodi.gy/t/how-to-use-loader-to-load-a-csv-with-text-and-label/2972/8 "2020-06-17T09:37:26Z")

</div>

NER is the task of detecting names etc. _in context_, so that's an approach you would use if your input consists of whole texts with those phrases _in context_. But that doesn't seem to be the case here because there's no context, so there's also nothing for the model to learn.

If your input data only consists of single phrases, text classification might not work that well either because... there's not really any text. Text classification algorithms are typically designed to predict labels over text and use the text for clues.

At least for parts of what you're trying to do here, framing it as a prediction task doesn't seem that useful? For instance, do you really need to predict "Tesco Limited"? I doubt you'd get better results using a model than you would with a database lookup and some fuzzy matching. So you probably want to focus on a rule-based system first, also so you have a good baseline and know what you need to beat.
