db-in command can't handle blank lines in jsonl

If there are blank lines in the jsonl file then the db-in command throws a ValueError.

Hi! This is kind of expected, because strictly speaking, blank lines aren't valid JSON. The loader currently raises an error for everything that can't be loaded to point out potential problems in the data. I guess there could be an option to skip only blank lines if the data is provided as JSONL – but in the meantime, it should hopefully be quick to just edit the input file by hand and remove the newlines.

Ok, I have been deleting them by hand so far, but it would definitely be nice to have the option of skipping the blank lines like when using srsly directly. Especially because srsly writes a new line when appending to a file.

But will continue deleting the lines manually for now.

Good point and I agree, I do think this is probably something we should handle more generally on the srsly level. (If you have time to open a quick enhancement issue in the repo, that'd be helpful, just so we don't forget!)

Have done so and made a PR: https://github.com/explosion/srsly/pull/14

1 Like