It would be great if we could specify one or more paths to Python files containing custom recipes in the prodigy.json config file. That would allow us to conveniently omit the -F in the CLI.
Yes, that's a nice idea. I think we didn't initially do this because we wanted to keep it explicit where a recipe comes from. But at this point, I do think it makes sense to introduce a "recipe_paths" config setting. Or maybe a PRODIGY_PATH env variable – not sure if this is great or terrible 
One way to solve the loading internally would be to allow a list of absolute (or relative) paths to Python files that will be imported as modules on startup. The decorator will take care of registering the commands, so all Prodigy needs to do is import the file. One concern is that adding lots of files can potentially make startup slower (since the files will be imported every time when you start up Prodigy) – but I think the import itself should be fast, and it'd be up to the user to make sure that the files don't do anything that slows down the import.
1 Like