JSONL format to CONLL

[jonnyfoka (Mark)]

Hello,

I have used prodigy to annotate several news articles for a relation classification task. So my JSONL file contains meta information along with the spans and relation information, Now, I would like to convert this JSONL file into a CONLL format. Is there any way how to do it?

Thanks in advance.

[ryanwesslen]

hi @jonnyfoka!

Thanks for your question and welcome to the Prodigy community

@jonnyfoka

Is there any way how to do it?

We don't have an off-the-shelf converter but there are several posts that may help:

How to convert JSONL annotation file to CONLL BIO tags?

Hi! The JSON data specifies the character offsets and tokens so it's mostly just a question of converting character offsets to token-based tags. spaCy has some handy utilities for this that can do it automatically for you. See here for a code example: https://prodi.gy/docs/named-entity-recognition#tip-offsets-biluo

Prodigy JSONL (or spaCY Doc) to CoNLL 2003

Hello everyone, I want to convert from either a Prodigy JSONL sample (or a better yet, a spaCy Doc), to a CoNLL 2003 sample. In CoNLL 2003 format documentation, I see there are 4 columns or items: "The first item on each line is a word, the second a part-of-speech (POS) tag, the third a syntactic chunk tag and the fourth the named entity tag". If my understanding is correct, I could obtain these items as follows: 1st item: The original word. 2nd item: Could be obtained from "tag_" Token attri…

Also this one is a little older but goes step-by-step:

Ner format to CONLL

I’ve just acquired prodigy to work on a manual ner tagging task. After manually tagging a document, the tool exports the results to a json format, the stantard spacy format. We would like to have this data tagged in the CoNLL format, the column format like so: John PERSON works O for O Microsoft ORGANIZATION Is there an option to do this or should we opt to post-process the json file in order to do so? Thanks in advance

Then this one covers relations info:

CoNLL-U Format for Syntactic Annotation

Hi! We don't have an existing conveter script – although it'd be cool to have, if you end up writing one and want to share Assuming you've used the (semi-)manual relations UI to annotate dependencies, here's the format that Prodigy will produce: https://prodi.gy/docs/api-interfaces#relations It should give you all the information you need, so it should mostly come down formatting it as columns, depending on the CoNLL-U specification you're using. The easiest is probably …

I haven't used CONLL but I've been told there are different formats, or at least different things people call CONLL. So if those posts don't work out, if you can provide any more details we can try to help.

[jonnyfoka (Mark)]

Hi @ryanwesslen,

thank you very much for the quick and helpful answer. After spending some time in reading related questions, I found out that CONLL-U apparently is not suitable at all for my use case, as it does not support relation annotations.

However, I will try to convert the spacy doc into a more suitable format. In doing so, I noticed that my spacy Doc object does not contain the relations I annotated but only the NER labels.

Can you tell me if I need to add an additional argument to the data-to-spacy command?

Kind regards

[ryanwesslen]

@jonnyfoka

Can you tell me if I need to add an additional argument to the data-to-spacy command?

Unfortunately this isn't available:

prodigy data-to-spacy for relation extraction

At the time of writing, spaCy doesn't natively support relation extraction models. The example that we list on our docs here is meant to be a tutorial on how to set up a custom component, not a guide on a feature in spaCy. The crux of the issue is that the Doc object in spaCy currently has no support for relationships. That is also why, in turn, the .spacy object does not support them. The config file that you see can be changed via the --config flag (docs). If this flag is not set, which is y…