Issue with prodigyanswer event

,
1

Hi, I think I've come across a bug with the javascript functionality (or I really don't understand how it's supposed to work). It seems like the prodigyanswer event which fires after completing a task is passing the info of the upcoming task, not the one that was just completed.

I made a POC recipe to show what I mean:

import prodigy

@prodigy.recipe('Barebones JS Test',
                dataset = ('Dataset ID', 'positional', None, str),
                )
def barebones_js_test(
        dataset
):
    stream = [
        {
            'text': 'text1',
            'label': 'label1'
        },
        {
            'text': 'text2',
            'label': 'label2'
        }, {
            'text': 'text3',
            'label': 'label3'
        }
    ]

    # language=JavaScript
    javascript = """
    document.addEventListener('prodigyanswer', testAnswer);

    function testAnswer(details) {
        console.log(details.detail.task)
    }
    """

    return {
        "view_id": "classification",
        "dataset": dataset,
        "stream": stream,
        "config": {
            'javascript': javascript,
        }
    }

Upon load, nothing special:

00
00.png1194×827 62.2 KB

After accepting text1/label1, I catch the prodigyanswer event and log the passed argument, but it's text2/label2, the upcoming task:

01
01.png1194×827 74 KB

Same on ignore or reject:

02
02.png1194×827 86.8 KB

03
03.png1194×827 91.4 KB

Am I misinterpreting the purpose of the event or is an index shifted somewhere?

Austin

1 Like
2

Thanks for the super detailed report :+1:

I think you’re right and there might be a race condition here: by the time Prodigy fires that event, the reference to the task object has already changed and is now the next task :woman_facepalming: My tests did not catch this, because I was only testing for the detail.answer, which is of course correct. (Haha, classic.) I need to test this in detail, but should be pretty easy to fix. Sorry about that!

Edit: Okay, can confirm, this was indeed the problem. Already fixed it and it will be included in the next release. If you’re currently blocking on this, feel free to email us and I’ll send you an updated bundle.js you can drop into your Prodigy installation.

3

You know, I’ve worked with some nice teams but I don’t think I’ve ever had a 20 minute fix turnaround. That’s awesome. An updated bundle.js would be great! I’m setting up a view with an autocompleting text box and updating elasticsearch in real time if the labeler inputs a new value.

4

Haha, thanks! To be fair, you also provided a great report and test case, which made it a lot easier. And once I know what’s wrong, I might as well fix it. (I just don’t want the 20 minute turnaround time to be the expectation going forward :wink:)

Feel free to send us an email to contact@explosion.ai and I’ll send over the bundle!

1 Like