# How to incorporate document metadata?

**URL:** <https://support.prodi.gy/t/how-to-incorporate-document-metadata/296>\
**Category:** Uncategorized\
**Tags:** usage, spacy\
**Created:** [February 8, 2018, 1:53pm UTC](https://support.prodi.gy/t/how-to-incorporate-document-metadata/296 "2018-02-08T13:53:08Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\
**Post date:** [February 8, 2018, 2:26pm UTC](https://support.prodi.gy/t/how-to-incorporate-document-metadata/296/2 "2018-02-08T14:26:40Z")

</div>

> [@hugoduncan](#):
>
> We then have to separately call `Language.make_doc` to create a `Doc` instance, add the required data, and then thread the document through a loop over the pipeline components (is there a better way of handling this in Spacy?).

You could use [`Language.pipe`](https://spacy.io/api/language#pipe) and set `as_tuples=True`, to process the documents as `(text, context)` tuples. For example:

```python
messages = [('This is a message', {'user_id': 123})]
for doc, meta in nlp.pipe(messages, as_tuples=True):
    # do something with the docs and meta, e.g. add it to a custom attribute
    doc._.meta_data = meta

```

> [@hugoduncan](#):
>
> I’m wondering how we can use Prodigy with this pipeline, as I see no way to pass in the information required.

Annotation tasks are simple dictionaries that can be structured however you like. Depending on the recipe and annotation interface you want to use, Prodigy expects some keys to be present (for example, `"text"`, `"spans"` or `"label"`) – but aside from that, you can freely add your own fields and data that should be stored with the task. For example:

```json
{"text": "This is a message", "user_id": 123}

```

When you annotate the task, the annotations are added and its saved to the database, together with your custom fields.

> [@hugoduncan](#):
>
> I see the Prodigy meta field, but don’t think that does what we need.

The `"meta"` field is mostly intended for metadata that should be displayed to the annotator (in the bottom right corner of the annotation card). You could consider adding some of your metadata here as well, especially in the early development / exploration phase. If you come across strange or interesting examples, it's often good to know their context.

---

_[View the full topic](https://support.prodi.gy/t/how-to-incorporate-document-metadata/296)._