variable audio_rate for audio annotation

I have an audio annotation task that is focused on applying multiple (occasionally overlapping) classes to speech regions. It would be extremely helpful to be able to accelerate or decelerate the playback rate to make it easier for annotators to carefully specify onset/offset boundaries, especially in areas of overlapping speakers.

Is this possible, whether out of the box or through some workaround? I've only been able to find this mention of an audio_rate configuration parameter, but no discussion of whether there's a way to update that on the fly as opposed to setting it once at the outset of the session and then just living with it.

Hi! At the moment, the audio_rate setting can only be defined once when you start up the server, and it's then configured on the interactive audio player when it's created.

I'll put this on my list of enhancement issues and see if we can at least make it adjust the rate if the current annotation task is updated. Then you'd be able to call something like window.prodigy.update from JavaScript and even implement your own custom controls using a custom HTML block with a button, slider, dropdown or whatever else you need.

1 Like

Update: Just released Prodigy v1.10.4, which exposes the underlying WaveSurfer instance via window.wavesurfer, so you can access it and implement custom controls (like audio rate, but also various other settings). wavesurfer.js is the library Prodigy uses under the hood, and it exposes different methods on its player.

For example, you could use a custom interface with two blocks: audio_manual for the audio UI and html for some custom controls. The "html_template" could look like this:

<button onclick="window.wavesurfer.setPlaybackRate(2)">2x speed</button>
<button onclick="window.wavesurfer.setPlaybackRate(1)">1x speed</button>
1 Like

This is fantastic news, @ines! Thank you for adding this, and doing so so quickly!

@ines - thank you for the new feature, and the guidance! I've been able to get something running, which is incredibly exciting.

It occurs to me, now that I've seen it in action: wavesurfer.js doesn't have a time-stretching feature, does it? It'll be pretty difficult to accomplish the annotation I want to do when the pitch is correspondingly transformed by playing the samples with an audio_rate setting of 0.5 or 0.1, especially since part of the tagging we need to do is dependent upon the pitch of the audio being listened to.

I am fairly certain that swapping out wavesurfer.js for some other library would be do-able, given how modularly you all built this thing...and it might even be worth my attempting to figure out (although I'd be painfully slow!). I just don't know the JS library ecosystem well enough to know what's out there that would already accomplish what you all have called upon wavesurfer to do AND ALSO then do this thing for me...

Looks like it's do-able in-ish WaveSurfer! (Paired with the soundtouch.js plugin, I guess?)

Here's their demo: http://wavesurfer-js.org/example/stretcher/

And the PR that added the feature, which helpfully links back to various associated issues and discussions: https://github.com/katspaugh/wavesurfer.js/pull/1214

So I guess the question is how to access whatever wavesurfer.js is doing within Prodi.gy to enable the appropriate soundtouch.js filter.

Cool to see that it's possible with Wavesurfer, that definitely makes things easier! (While it's theoretically possible to integrate another player as well, it would have been a pretty involved task because there are many moving parts to consider to really make the interactive audio annotation work.)

I just had a quick look at the integration of soundtouch.js here: http://wavesurfer-js.org/example/stretcher/app.js

It does require a bit of code to be added in different places, but afaik, Wavesurfer lets you add multiple handlers (e.g. wavesurfer.on('play', () => {})), so you could try just adding the time stretcher code to the existing window.wavesurfer object exposed by Prodigy via custom JavaScript and see what happens. You also need to make sure soundtouch is available, but if you're lazy and just want to test things, you could literally just copy-paste the code into your custom JavaScript string.