Custom storage & custom text visualisation

1

Hi,

Still me :slight_smile: I opened a new thread as it’s for more specific questions than general feedback.

I tried to make a custom store function in order to save annotations results in mongodb. But seems not to be working for now… Here is my store function:

def store_mongo(dataset, session_id, answers, mongo_uri):
    """ """
    db = MongoClient(mongo_uri).news_feed
    #
    writes = []
    for answer in answers:
        if answer["answer"] == "accept" and "accept" in answer:
            writes.append(UpdateOne(
                {"_id": answer["_id"]},
                {"$set": {"category_ids": answer["accept"]}}))
    db.news.bulk_write(writes)

Which I then pass in the recipe components:

def my_recipe(...):
    [...]
    mstore = partial(store_mongo, mongo_uri=mongo_uri)
    return {
        'dataset': dataset,
        'view_id': 'choice',
        'stream': stream,
        'store': mstore,
        'config': {"choice_style": "multiple"}
    }

Actually nothing happens it sill saves to sqlite. can you point me to the right direction :slight_smile:

Secondly I’m wondering if it is possible to control how the text in rendered in the card. By default it seems that the text is centered and that every newline "\n" are removed. I’ve seen that the html mode allows to use a template but then I can’t use the choice mode anymore.

Thanks,

Thomas

2

Thanks for pointing this out – I'll look at this again in more detail, but I think the problem might be that the controller currently performs too aggressive checks and expects the store to follow Prodigy's database API with the same methods (see the docs on the db component). Sorry about the conflicting docs here – the original idea was definitely to allow simply replacing the store with a function, but the recent changes to the database made this a little more complicated.

If you mostly care about saving the annotations, the quickest workaround for now would be to implement the storage in the update function instead. The update function is called every time the REST API receives a batch of annotations, and takes a list of examples as its argument.

Currently, the choice interface (and all other interfaces that show text) will render the text as the text content of a React element – i.e. plain text, like the textContent property of a DOM element in JS.

However, choice should also support a html key instead of text. This should render the content via dangerouslySetInnerHTML. So doing something like this should work:

{
    "options": [
        {"id": 1, "html": "First line<br />Second line"},
        {"id": 2, "html": "<strong>bold text</strong>"},
        {"id": 3, "html": "<a href='https://google.com'>Google</a>"}
    ]
}

I wouldn't recommend going too overboard with this, though, as it can easily lead to unexpected visual results depending on the markup. But if you're mostly planning on using it for line breaks and simple formatting, this should be no problem.

(Some background on the text vs. html: The original idea behind this distinction here was to make the user explicitly decide to use "HTML mode". It also prevents leftover HTML markup from being rendered by accident. For example, if you're working with raw, uncleaned data, you usually want to see the stray markup, as it affects spaCy's predictions and lead to worse results when training the model.)

3

Concerning annotation storage, I confirm the doc is a bit confusing between the db and store components. Given the example I guessed the usage was to use store but I remember being confused when reading the doc… Anyway using the update does the trick thanks ! :slight_smile:

As for the text rendering the html key does also the work ! It’s good enough for the main text, to render title and break line by replacing the “\n” with <br/> :). However just to let you know, using html key in choice options results in “big button”…

I think your last paragraph about text vs. html could have its place in the documentation by the way :slight_smile:

Thanks a lot for your help !