data-to-spacy losing annotations

7

Thank you! I had to modify merge function to deal with the fact that not every example has a span for a label:

def merge(examples):
    key_values = {}
    for ex in examples:
        key = ex['_input_hash']
        if key in key_values:
            if 'spans' in ex:
                if 'spans' in key_values[key]:
                    key_values[key]['spans'].extend(ex['spans'])
                else:
                    key_values[key]['spans'] = ex['spans']
        else:
            key_values[key] = ex
    return examples

I am now unblocked :slight_smile:

2 Likes