# How to convert IOB/BILOU format into jsonl **URL:** \ **Category:** Uncategorized\ **Tags:** ner, spacy\ **Created:** [May 28, 2021, 3:31pm UTC](https://support.prodi.gy/t/how-to-convert-iob-bilou-format-into-jsonl/4276 "2021-05-28T15:31:16Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![sh123](https://avatars.discourse-cdn.com/v4/letter/s/a9a28c/32.png) [@sh123](https://support.prodi.gy/u/sh123)\ **Post date:** [May 28, 2021, 3:31pm UTC](https://support.prodi.gy/t/how-to-convert-iob-bilou-format-into-jsonl/4276/1 "2021-05-28T15:31:16Z")
I have annotated dataset(for NER) in IOB format. Is there straighforward method to convert it to JSONL format ??? IOB file ```python Alex B-PER is O going O with O Marty B-PER Rick I-PER to O Los B-LOC Angeles I-LOC . O Alex B-PER birthday O is O April B-DATE 1996 I-DATE . O ``` Required output file ```python {"text":"Alex is goin with Marty Rick to Los Angeles.","_input_hash":-154867549,"_task_hash":-848148549,"tokens":[{"text":"Alex","start":0,"end":4,"id":0,"ws":true},{"text":"is","start":5,"end":7,"id":1,"ws":true},{"text":"goin","start":8,"end":12,"id":2,"ws":true},{"text":"with","start":13,"end":17,"id":3,"ws":true},{"text":"Marty","start":18,"end":23,"id":4,"ws":true},{"text":"Rick","start":24,"end":28,"id":5,"ws":true},{"text":"to","start":29,"end":31,"id":6,"ws":true},{"text":"Los","start":32,"end":35,"id":7,"ws":true},{"text":"Angeles","start":36,"end":43,"id":8,"ws":false},{"text":".","start":43,"end":44,"id":9,"ws":false}],"_session_id":null,"_view_id":"ner_manual","spans":[{"start":0,"end":4,"token_start":0,"token_end":0,"label":"PER"},{"start":18,"end":28,"token_start":4,"token_end":5,"label":"PER"},{"start":32,"end":43,"token_start":7,"token_end":8,"label":"LOC"}],"answer":"accept"} {"text":"Alex birthday is April 1996.","_input_hash":-1936698937,"_task_hash":-1676226294,"tokens":[{"text":"Alex","start":0,"end":4,"id":0,"ws":true},{"text":"birthday","start":5,"end":13,"id":1,"ws":true},{"text":"is","start":14,"end":16,"id":2,"ws":true},{"text":"April","start":17,"end":22,"id":3,"ws":true},{"text":"1996","start":23,"end":27,"id":4,"ws":false},{"text":".","start":27,"end":28,"id":5,"ws":false}],"_session_id":null,"_view_id":"ner_manual","spans":[{"start":0,"end":4,"token_start":0,"token_end":0,"label":"PER"},{"start":17,"end":27,"token_start":3,"token_end":4,"label":"DATE"}],"answer":"accept"} ``` ---
**Author:** ![ines](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/ines/32/3_2.png) [@ines](https://support.prodi.gy/u/ines)\ **Post date:** [June 2, 2021, 1:23am UTC](https://support.prodi.gy/t/how-to-convert-iob-bilou-format-into-jsonl/4276/2 "2021-06-02T01:23:59Z")
Hi! Prodigy's format uses simple character offsets into the text. If you still have the original text or tokenization anymore and only the IOB or BILUO tags, you could use spaCy's `offsets_from_biluo_tags` helper function to convert the token-based tags to offsets. See here for an example: [https://prodi.gy/docs/named-entity-recognition#tip-biluo-offsets](https://prodi.gy/docs/named-entity-recognition#tip-biluo-offsets) ---
**Author:** ![elazzouzi1080](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/elazzouzi1080/32/2523_2.png) [@elazzouzi1080](https://support.prodi.gy/u/elazzouzi1080)\ **Post date:** [November 3, 2021, 2:32pm UTC](https://support.prodi.gy/t/how-to-convert-iob-bilou-format-into-jsonl/4276/4 "2021-11-03T14:32:40Z")
biluo\_tags\_to\_offsets works only with BILUO tags and not with IOB tags ![image](https://us1.discourse-cdn.com/flex020/uploads/prodigy/original/2X/e/e2810b2a748e91d59b604cfab81d9eef59daa782.png) ![image](https://us1.discourse-cdn.com/flex020/uploads/prodigy/original/2X/b/b6085e03b743f4a1d8f18e9a4a84ad650deee13d.png) My ner data have this format in frensh BIO TAG words: ['Le', '11', 'octobre', '2018', ',', 'Monsieur', 'B\*\*\*\*', 'se', 'presente', 'a', 'la', 'consultation', 'avec', 'un', 'tableau', 'de', 'luxation', 'posterieure', 'de', 'genou', 'non', 'deficitaire', '.'] tags: ['O', 'B-DATE', 'I-DATE', 'I-DATE', 'O', 'O', 'B-PATIENT', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O'] and i need to let my model highlight suggestions for me ---
**Author:** ![adriane](https://avatars.discourse-cdn.com/v4/letter/a/46a35a/32.png) [@adriane](https://support.prodi.gy/u/adriane)\ **Post date:** [November 3, 2021, 2:45pm UTC](https://support.prodi.gy/t/how-to-convert-iob-bilou-format-into-jsonl/4276/5 "2021-11-03T14:45:18Z")
Hi, if you use `nlp()` to tokenize, the tokenization might not be aligned with your original annotation. This should work to create a doc from `words` + IOB tags: ```python doc = Doc(nlp.vocab, words=words, ents=tags) ``` And then you can access the entities with token and character offsets through `doc.ents`. ---
**Author:** ![elazzouzi1080](https://sea2.discourse-cdn.com/flex020/user_avatar/support.prodi.gy/elazzouzi1080/32/2523_2.png) [@elazzouzi1080](https://support.prodi.gy/u/elazzouzi1080)\ **Post date:** [November 3, 2021, 4:58pm UTC](https://support.prodi.gy/t/how-to-convert-iob-bilou-format-into-jsonl/4276/6 "2021-11-03T16:58:40Z")
Hi ADRIANE thank you for your feedback How can I get this expected format using my model ![image](https://us1.discourse-cdn.com/flex020/uploads/prodigy/original/2X/6/6cca7937ea7107e0c38b35e92b738b7955d070d3.png) my model prediction results have this format for exemple ['O', 'O', 'B-PATIENT', 'I-PATIENT', 'O', 'O', 'B-DATE'] so I'm trying Converting IOB tags to character offsets but in your documentation i find only the conversion BILUO ![image](https://us1.discourse-cdn.com/flex020/uploads/prodigy/original/2X/2/2c97a7b9bce415463d5c537dd9a4b226619ee00c.png) ---
**Author:** ![adriane](https://avatars.discourse-cdn.com/v4/letter/a/46a35a/32.png) [@adriane](https://support.prodi.gy/u/adriane)\ **Post date:** [November 4, 2021, 9:03am UTC](https://support.prodi.gy/t/how-to-convert-iob-bilou-format-into-jsonl/4276/7 "2021-11-04T09:03:03Z")
Ah, it looks like that example didn't get updated for spacy v3. We'll get the docs updated soon, but here's the updated example: ```python from spacy.tokens import Doc from spacy.vocab import Vocab from spacy.training import biluo_tags_to_offsets doc = Doc(Vocab(), words=["I", "like", "New", "York"]) tags = ["O", "O", "B-LOC", "L-LOC"] offsets = biluo_tags_to_offsets(doc, tags) # [(7, 15, 'LOC')] ``` If you have IOB tags instead of BILUO tags, you need to convert them first with: ```python from spacy.training import iob_to_biluo tags = iob_to_biluo(tags) ```