# convert prodigy annotation file to iob format

**URL:** <https://support.prodi.gy/t/convert-prodigy-annotation-file-to-iob-format/2801>\
**Category:** Uncategorized\
**Tags:** usage, ner, solved, transformers\
**Created:** [April 16, 2020, 10:08am UTC](https://support.prodi.gy/t/convert-prodigy-annotation-file-to-iob-format/2801 "2020-04-16T10:08:58Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![codingnoobneedshelp](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/codingnoobneedshelp/32/1740_2.png) [@codingnoobneedshelp](https://support.prodi.gy/u/codingnoobneedshelp)\
**Post date:** [April 16, 2020, 10:08am UTC](https://support.prodi.gy/t/convert-prodigy-annotation-file-to-iob-format/2801/1 "2020-04-16T10:08:58Z")

</div>

Hello,

i want to run BERT-NER with PyTorch, which requires IOB tagging instead. Is there any conversion code from prodigy annotation data format to IOB?

---

<div class="post-metadata">

**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\
**Post date:** [April 16, 2020, 12:40pm UTC](https://support.prodi.gy/t/convert-prodigy-annotation-file-to-iob-format/2801/2 "2020-04-16T12:40:44Z")

</div>

Yes, check out the section here for examples of how to create IOB and BILUO tags from character offsets: [https://prodi.gy/docs/named-entity-recognition#tip-offsets-biluo](https://prodi.gy/docs/named-entity-recognition#tip-offsets-biluo)

The easiest way is to use spaCy, which lets you add entities as character offsets and then gives you easy access to each token's `ent_iob_` and `ent_type_` attributes. Just make sure that the tokenization matches the one you used during annotation and use `nlp.pipe` for faster conversion. So the whole end-to-end script that reads your Prodigy dataset and outputs IOB could look like this:

```python
from prodigy.components.db import connect
import spacy

db = connect()
prodigy_annotations = db.get_dataset("your_ner_dataset")
examples = ((eg["text"], eg) for eg in prodigy_annotations)
nlp = spacy.blank("en")
for doc, eg in nlp.pipe(examples, as_tuples=True):
    doc.ents = [doc.char_span(s["start"], s["end"], s["label"]) for s in eg["spans"]]
    iob_tags = [f"{t.ent_iob_}-{t.ent_type_}" if t.ent_iob_ else "O" for t in doc]
    print(doc.text, iob_tags) # do something here...

```

---

<div class="post-metadata">

**Author:** ![codingnoobneedshelp](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/codingnoobneedshelp/32/1740_2.png) [@codingnoobneedshelp](https://support.prodi.gy/u/codingnoobneedshelp)\
**Post date:** [April 16, 2020, 1:23pm UTC](https://support.prodi.gy/t/convert-prodigy-annotation-file-to-iob-format/2801/3 "2020-04-16T13:23:05Z")

</div>

Thanks a lot.