Opaque error from malformed prodigy.json

,
1

I commented out a line in my prodigy.json file with a #, and Prodigy really didn’t like that. I got the following error and it took me a while to figure out what the problem was. Is there a way of making problems with the prodigy.json more attributable? If not, hopefully this helps someone else.

Traceback (most recent call last):
  File "/home/andy/anaconda3/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/andy/anaconda3/lib/python3.6/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "multiuser_db_quiz.py", line 147, in serve
    port=port)  # port
  File "/home/andy/anaconda3/lib/python3.6/site-packages/prodigy/__init__.py", line 15, in serve
    controller = loaded_recipe(*args)
  File "cython_src/prodigy/core.pyx", line 161, in prodigy.core.recipe.recipe_decorator.recipe_proxy
  File "cython_src/prodigy/util.pyx", line 52, in prodigy.util.get_config
  File "cython_src/prodigy/util.pyx", line 375, in prodigy.util.read_json
  File "cython_src/prodigy/util.pyx", line 376, in prodigy.util.read_json
ValueError: Expected object or value
2

Yeah, json is a bit bad for config files in that respect. It would be nice to use something that supported comments, but we’re not really happy with any of the alternatives. YAML is popular, but we find the syntax too ambiguous. The PyYAML library has also made a number of not-great security decisions. We considered using json5, but it would require another library dependency.

We can wrap the read_json function in a try/except block, so that we can raise a better error here. That’s probably a good idea. Ines has also been working on json schema validation for the prodigy.json. While this wouldn’t catch a syntax error, it can help with a number of other cases.

1 Like
3

Sounds good! I assumed at first that the JSON error was related to the format of incoming tasks, not the prodigy.json. I think an exception would be helpful there. And I know your thoughts on YAML…