Progress bar for audio folder data loader

I currently run Prodigy as follows:

prodigy audio.manual dataset_name dataset_folder --label LABEL1,LABEL2

Unfortunately, the UI doesn't display a total count on the progress. Looks like internally, the stream is an iterator and not a list of the filenames. Would it be possible to change that? I'd love to see the total progress in the sidebar.

Best,
Thomas

This is a good point and something I've wanted to add for a while: the file loaders should at least expose a __len__, since they'll know how many files they'll load from, and just enumerating the file names upfront isn't expensive.

In the meantime, you can work around this by just calling list around the stream in the recipe.

1 Like

Awesome, thanks so much!