Understanding the Update mechanism

#1

Hello,

I have a question regarding the update mechanism from the recipes.
From what I Understood, update function is called everytime prodigy received a batch of annotation (defined by the batch number option).

So I tried to play around that feature for a while, but I have an error, and I can’t really figure what is going wrongly. My original function being a bit complex, so i tried, So I tried to do the simplest update function, mainly print an ‘ok’ when it’s called.
This is what is given me :

Open the app in your browser and start annotating!

ok
Exception when serving /give_answers
Traceback (most recent call last):
File “/home/robin/anaconda3/lib/python3.6/site-packages/waitress/channel.py”, line 338, in service
task.service()
File “/home/robin/anaconda3/lib/python3.6/site-packages/waitress/task.py”, line 169, in service
self.execute()
File “/home/robin/anaconda3/lib/python3.6/site-packages/waitress/task.py”, line 399, in execute
app_iter = self.channel.server.application(env, start_response)
File “/home/robin/anaconda3/lib/python3.6/site-packages/hug/api.py”, line 421, in api_auto_instantiate
return module.hug_wsgi(*args, **kwargs)
File “/home/robin/anaconda3/lib/python3.6/site-packages/falcon/api.py”, line 242, in call
responder(req, resp, **params)
File “/home/robin/anaconda3/lib/python3.6/site-packages/hug/interface.py”, line 692, in call
self.render_content(self.call_function(input_parameters), request, response, **kwargs)
File “/home/robin/anaconda3/lib/python3.6/site-packages/hug/interface.py”, line 633, in call_function
return self.interface(**parameters)
File “/home/robin/anaconda3/lib/python3.6/site-packages/hug/interface.py”, line 99, in call
return __hug_internal_self._function(*args, **kwargs)
File “/home/robin/anaconda3/lib/python3.6/site-packages/prodigy/app.py”, line 66, in give_answers
controller.receive_answers(answers)
File “cython_src/prodigy/core.pyx”, line 72, in prodigy.core.Controller.receive_answers
TypeError: ‘NoneType’ object is not callable

If instead of ‘ok’ I decide to print(self) on the update function, I see the annotation correctly.

I encounter this behavior on different recipes, if the update parameter is to None on the recipe, then all is good.

So I’m not sure if the error I encounter is the expected behavior and therefore I missed something, or if it’s a bug?

Robin

#2

Thanks for the report and sharing your progress!

I think I know what’s going on here: If you provide a custom update() function, Prodigy currently also expects there to be a progress() function that receives an argument loss and returns a progress value. (This is not intentional btw – it’s just something we didn’t seem to have covered by our test cases, because always used both in combination. Sorry about that!)

As a quick workaround, try adding a simple function as the 'progress' key to the dictionary returned by your recipe, e.g.:

'progress': lambda loss: 0  # or actually compute some progress here, however you like