Adjusting buffer size of WavStreamer

[meganetaaan]

The current WavStreamer buffers up to one second of audio, but there are situations where one may want to adjust the buffer size. When I run multiple types of processing with Stack-chan simultaneously, a memory full exception occurred during streaming, so the buffer size was halved (hard-coded).

I'm thinking about the ideal interface, whether it should indicate the number of seconds of audio to be held or the byte size of the buffer directly for use cases that require explicit buffer size. However, since some assertion is being made about the buffer size, there may not be complete freedom in choosing the buffer size?

[phoddie]

I agree that it would be useful to be able to adjust the amount of buffering. I have also adjusted it manually for projects -- though I typically increase the size, to provide more robustness against network delays. Of course, there are valid reasons to make it smaller too.

I generally prefer to use time for media streaming buffering rather than bytes. Time is what defines how resilient the connection is to network delays. It also is consistent regardless of the data rate of the stream (changing from a 44.1 kHz stereo 16-bit stream to an 8-bit mono stream doesn't require changing the time buffering value). I acknowledge there's no single correct answer, so we have to decide.

We could have a options.bufferDuration, for example. It would default to 1 second for compatibility. That's not difficult to implement. What do you think?

.... for use cases that require explicit buffer size...

Maybe you could elaborate on that case?

FWIW – the assertion is just to ensure that the buffers can hold an integer number of samples (e.g. a 5 byte buffer is for 2.5 16-bit samples , which won't work for playback). Instead of that assertion, it should probably just round the buffer size to a safe value.

[meganetaaan]

I agree with having a bufferDuration option.

Maybe you could elaborate on that case?

Stack-chan wants to allocate the maximum buffer for each function based on the rough memory amount planned, without hindering the execution of other functions. However, upon further consideration, specifying the duration in seconds seems more mainstream, and if the sample rate is known, converting between bufferSize and bufferDuration is a minor issue.
Rather than having both methods, I prefer a simpler implementation by using the time-based option. It is better to keep the class implementation simple than to make it more complex by accommodating both methods.

[phoddie]

Thanks for the additional background. I'm glad we don't need to do both.

I wrote the code today, including replacing the assert with some rounding. I also added it to SBCStreamer to be consistent. I want to test a little more before committing though.

[phoddie]

A follow-up note regarding WavStreamer & SBCStreamer buffering. I was recently doing some testing with WavStreamer and noticed that the memory use wasn't as stable as I expected. In theory, all the buffers should be allocated before audio playback begins. Those buffers should be recycled -- as one buffer finishes playing, it is then used to buffer the next audio data received. Because of timing inconsistencies -- in the delivery of audio from the network and invoking of audio buffer done callbacks -- sometimes an additional buffer was allocated during playback. That seems OK, but it led to breaking the recycling of memory blocks. Because more blocks are being allocated and freed, that eventually caused the system heap to fragment, which eventually caused allocation failures.

I solved that problem by updating the buffer recycling logic. Now there are no system allocations during playback. That's what was originally intended. The result is a bit lower CPU load and improved long-term stability when streaming.

The changes were applied to both WavStreamer and SBCStreamer. Because SBCStreamer uses much less memory, the issue is much was significant there.