GregSantulli GregSantulli - 1 month ago 26
Javascript Question

Web Audio API - Live Stream 'clicks' between chunks.

I am trying to stream audio through a websocket on a node.js (express) server to a web browser. The audio is coming from an iOS device as 16-bit, mono wav files sampled at 4k (4000 samples per second).

Here's my code:

Server Code:

webSocketServer.on('connection', function connection(client) {
client.on('message', function(message) {
webSocketServer.clients.forEach(function each(connection) {
connection.send(message, { binary: true }
);
});
});


Client Code:

webSocket = new WebSocket('ws://' + window.location.hostname + ':8080/');
webSocket.binaryType = 'arraybuffer'
webSocket.onmessage = function(message) {
var arrayBuffer = message.data // wav from server, as arraybuffer
var source = audioContext.createBufferSource();
audioContext.decodeAudioData(arrayBuffer, function(buffer){
source.buffer = buffer
source.connect(audioContext.destination)
source.start(time);
time += source.buffer.duration
}, function(){
console.log('error')
})
};


decodeAudioData()
appears to be working, however the audio buffer it returns is half the length I was expecting. (eg 4000 samples will only give me 0.5 seconds of audio. I originally thought this was because the wav is 16 bit and not 32, but switching to 32 caused
decodeAudioData()
to trigger it's error callback.

I figured this workaround could be added to the success callback:

source.playbackRate.value = 0.5 // play at half speed
time += source.buffer.duration * 2 // double duration


This gets the timing to work perfectly, but I am left with one problem: There is an audible 'click' or 'pop' between audio chunks. After spacing out the chunks by one second (
time += (source.buffer.duration * 2) + 1
), I was able to find that the click happens at the very beginning of each chunk.

So my main two head-scratchers are:

1) Why is the decoded audio playing at twice the speed I am expecting? Is my sampling rate of 4k too low for the Web Audio API? Why can't I decode 32-bit wav's?

2) I have some experience with digital audio workstations (ableton, logic) and I know that clicking sounds can arise if a wave 'jumps' from a sample back down to zero or vice versa (ie: starting/ending a sine wave in the midst of a phase). Is that what's going on here? Is there a way to get around this? Crossfading each individual sample seems silly. Why doesn't each chunk pickup where the last one left off?

Answer

1) The audio I was receiving was actually at 2k by mistake, but the wav header still said 4k, thus the double speed error.

2) See the last paragraph of Chris Wilsons answer here:

Finally - this is not going to work well if the sound stream does not match the default audio device's sample rate; there are always going to be clicks, because decodeAudioData will resample to the device rate, which will not have a perfect duration. It will work, but there will likely be artifacts like clicks at the boundaries of chunks. You need a feature that's not yet spec'ed or implemented - selectable AudioContext sample rates - in order to fix this.

Brion Vibbers AudioFeeder.js works great without any clicks but requires raw 32bit pcm data. Also be wary of upsampling artifacts!