Incorporate Document metadata in built-in recipes

[vsocrates (Vimig Socrates)]

Is there a way to include document metadata directly from the textcat.manual and ner.manual built-in recipes? I'd like to keep some identifiers that would be helpful in downstream processing, stored in the sqlite annotation db. Or do these need to be incorporated using a custom recipe as below?:

How to incorporate document metadata?

You could use Language.pipe and set as_tuples=True, to process the documents as (text, context) tuples. For example: messages = [('This is a message', {'user_id': 123})] for doc, meta in nlp.pipe(messages, as_tuples=True): # do something with the docs and meta, e.g. add it to a custom attribute doc._.meta_data = meta Annotation tasks are simple dictionaries that can be structured however you like. Depending on the recipe and annotation interface you want to use, Prodigy expects some …

Thanks!

[ryanwesslen]

hi @vsocrates!

@vsocrates

Is there a way to include document metadata directly from the textcat.manual and ner.manual built-in recipes?

What do you mean by "include"? Do you mean to show that metadata In Prodigy? You likely know (as it's mentioned in Ines' post) that Prodigy will display any "meta" information by default in the bottom right corner of the UI. So I don't think it's that.

@vsocrates

I'd like to keep some identifiers that would be helpful in downstream processing, stored in the sqlite annotation db.

Can you give an example of what your use case, like what would be such identifiers?