Adaptive Speed Audio

I like to listen to audiobooks and podcasts at several times speed. I find I am able to absorb the information well at a faster speed and I am able to enjoy more audiobooks and podcasts than I would otherwise be able to.

I use a different speed for different podcasts and audiobooks depending on how much thought is required to comprehend the material as well as how comprehensible the speaker is. However this is not optimized enough for me specifically, in podcasts different people speak and some are more comprehensible than others.

So how can this actually be implemented? Well automatic speech recognition (ASR) models exist and something being understandable to them should have at least something to do with things being understandable by a person so we can see how an ASR does when sped up to various speeds. Then we can formulate an objective based off of the ASR performance impact for different sections of audio and a penalty for how much the speed for different sections varies. Then we just use this data to stitch together some audio from the different sections at different speeds. We don't want it to be too abrupt so we'll make sure that switching just happens between words.

And indeed this is exactly what I did you can check it out here it seems to be working solidly for my purposes, though the main thing it is doing seems to be removing silence let me know what you all think.

This doesn't really address the other problem of sections requiring more thought to semantically comprehend. There is also unfortunately the matter that sometimes when people talk slow they are thinking about what they are going to say which is in turn indicative that those words may require additional thought process so it goes in exactly the opposite direction of what we want. That being said the thought required for composition is generally greater than for comprehension, and the difference in the differences is hard to say by intuition alone. Addressing this will have to wait until a later version.

It isn't the original audio you are losing information about the timing etc.. Yes but I am also not getting the original timing by listening to it faster in the first place. It is a tradeoff you do loose some information but you are able to listen to more overall, for some things say dramatic performances it is important, I think generally for audiobooks and podcasts it is less so.

Generally this also assumes that ASR models have similar difficulties in recognizing fast spoken words as humans do. Or more precisely that the differences in recognition at different speeds are reasonably comparable to our differences in recognition across different speeds. I think it is reasonably correlated but directly modeling it could almost assuredly do better.

Other further improvement that could be made would be sub-word adjustments, deferentially speeding up different parts of the word or enhancing the comprehensibility to allow that audio to be played and comprehended better at higher speeds. We can push to higher and higher speeds as we relax the constraint of how close we hue to the original audio.