Web Audio API

这篇翻译不完整。请帮忙从英语翻译这篇文章

Web Audio API 提供了在Web上控制音频的一个非常有效通用的系统 , 允许开发者来自选音频源,对音频添加作用,创建可视化音频,应用空间效果 (如平移),等等。

Web audio 概念与使用

Web Audio API 涉及音频环境内的音频操作, 并且已经设计为允许模块化路由。基础的音频操作在音频节点上进行, 它们连接在一起构成音频路由图。多个源 — 不同类型的通道布局 — 支持在单一的环境里。这种模块化设计提供了灵活地创建具有动态效果的复合音频的方法。

音频节点通过它们的输入输出相连,形成从一个或多个源开始的链,经过一个或多个节点,直到一个目地(虽然你可能并不需要一个目地,比如说,只是想可视化音频数据)一个简单、典型的 web audio 工作流是这样的:

  1. 创建音频环境
  2. 在音频环境里创建源 — 例如 , 振荡器, 流
  3. 创建效果节点,例如混响、双二阶滤波器、平移、压缩
  4. 为音频选择一个目地,例如你的系统扬声器
  5. 连接源到效果器,以及效果器和目地

一个简单的框图,一个外框标记为音频环境,以及三个内框分别标记为源、效果、输出。三个内框之间有从左向右的箭头,表示音频信息流。

时间可以被高精度低延迟地控制,允许开发者写代码来精确响应事件,并且能处理特定样本,即使是高采样率。所以像豉或者音序器之类的应用可以很容易地实现。

Web Audio API 也使我们能够控制如何音频空间化。使用基于源 - 侦听器模型的系统中,它允许控制平移模型和处理距离引起的衰减或移动源(移动侦听)引起的多普勒频移

注意: 你可以阅读我们关于 Web Audio API 的文章来了解更多细节 Web Audio API背后的基本概念

Web Audio API 接口

Web Audio API共有28个接口和相关的事件,我们已经分成了九类功能。

通用音频图定义

Web Audio API 中与生成音频图相关的定义与通用容器。

AudioContext
AudioContext接口代表由音频模块构成的音频处理图。音频环境控制其所包含节点的创建和音频处理、解码的执行。进行任何处理前你必需创建一个音频环境,一切操作都在这个环境里进行。
AudioNode
音频节点 接口是一个音频处理模块,包括音频源(例如:一个<audio> 或者 <video> 元素)、音频输出、中间处理模块(例如:滤波器 BiquadFilterNode 或者音量控制器 GainNode)。
AudioParam
AudioParam 接口代表音频相关的参数,比如一个 AudioNode。它可以设置为特定值或值的变化,并且可以在指定的时间之后以指定模式变更。
ended (event)
当媒体播放到结尾停止时,会触发ended事件。

音频源定义

Web Audio API 使用的音频源接口。

OscillatorNode
OscillatorNode接口表示一个正弦波。它是一个AudioNode音频处理模块,使用指定频率生成正弦波。
AudioBuffer
AudioBuffer接口表示在内存中驻留的短音频,使用AudioContext.decodeAudioData()方法从音频文件创建,或者使用AudioContext.createBuffer()从原始数据创建。解码完成后可以被放入一个AudioBufferSourceNode
AudioBufferSourceNode
AudioBufferSourceNode接口表示内存音频数据的音频源,存储在AudioBuffer。这是一个作为音频源的AudioNode
MediaElementAudioSourceNode
MediaElementAudioSourceNode接口表示一个存在于HTML5 <audio><video>元素的音频源。这是一个作为音频源的AudioNode
MediaStreamAudioSourceNode
MediaStreamAudioSourceNode接口表示一个存在于 WebRTC MediaStream(如网络摄像头或麦克风)的音频源。这是一个作为音频源的AudioNode

Defining audio effects filters

应用到音频源上的效果接口。

BiquadFilterNode
BiquadFilterNode 接口表示一个简单的低阶滤波器。它是一个AudioNode,可以表示不同种类的滤波器、调音器或图形均衡器。BiquadFilterNode 总是只有一个输入和一个输出。
ConvolverNode
ConvolverNode 接口是一个AudioNode对给定的 AudioBuffer 执行线性卷积,通常用于实现混响效果
DelayNode
DelayNode 接口表示延迟线;一个AudioNode 音频处理模块,使输入的数据延时输出。
DynamicsCompressorNode
DynamicsCompressorNode 提供了一个压缩效果,当多个音频在同时播放并且混合的时候,它可以通过降低音量最大的部分的音量来帮助避免发生削波和失真。
GainNode
GainNode 接口表示音量变更。它是一个 AudioNode 音频处理模块,在输出前使用给定 增益 应用到输入。
StereoPannerNode
StereoPannerNode 接口表示一个简单立体声控制节点,用来左右移动音频流。
WaveShaperNode
WaveShaperNode接口表示一个非线性的扭曲。AudioNode 利用曲线来应用waveshaping扭曲。在扭曲效果里,常被用来添加温暖的感觉。
PeriodicWave
用来定义周期性的waveform,可被用来重塑 OscillatorNode的输出.

Defining audio destinations

在你处理完音频之后,这些接口定义了输出到哪里。

AudioDestinationNode
The AudioDestinationNode interface represents the end destination of an audio source in a given context - usually the speakers of your device.
MediaStreamAudioDestinationNode
The MediaStreamAudioDestinationNode interface represents an audio destination consisting of a WebRTC MediaStream with a single AudioMediaStreamTrack, which can be used in a similar way to a MediaStream obtained from Navigator.getUserMedia. It is an AudioNode that acts as an audio destination.

Data analysis and visualisation

如果你想从音频里提取时间、频率或者其它数据,你需要 AnalyserNode。

AnalyserNode
The AnalyserNode interface represents a node able to provide real-time frequency and time-domain analysis information, for the purposes of data analysis and visualization.

Splitting and merging audio channels

你将使用这些接口来拆分、合并声道。

ChannelSplitterNode
The ChannelSplitterNode interface separates the different channels of an audio source out into a set of mono outputs.
ChannelMergerNode
The ChannelMergerNode interface reunites different mono inputs into a single outputs. Each input will be used to fill a channel of the output.

Audio spatialization

这些接口用来添加空间平移效果到音频源。

AudioListener
The AudioListener interface represents the position and orientation of the unique person listening to the audio scene used in audio spatialization.
PannerNode
The PannerNode interface represents the behavior of a signal in space. It is an AudioNode audio-processing module describing its position with right-hand Cartesian coordinates, its movement using a velocity vector and its directionality using a directionality cone.

使用 JavaScript 处理音频

如果你想使用一个外部脚本来处理音频源,下面的节点和事件可以实现。

Note: As of the August 29 2014 Web Audio API spec publication, these features have been marked as deprecated, and are soon to be replaced by Audio_Workers.

ScriptProcessorNode
The ScriptProcessorNode interface allows the generation, processing, or analyzing of audio using JavaScript. It is an AudioNode audio-processing module that is linked to two buffers, one containing the current input, one containing the output. An event, implementing the AudioProcessingEvent interface, is sent to the object each time the input buffer contains new data, and the event handler terminates when it has filled the output buffer with data.
audioprocess (event)
The audioprocess event is fired when an input buffer of a Web Audio API ScriptProcessorNode is ready to be processed.
AudioProcessingEvent
The Web Audio API AudioProcessingEvent represents events that occur when a ScriptProcessorNode input buffer is ready to be processed.

Offline/background audio processing

It is possible to process/render an audio graph very quickly in the background — rendering it to an AudioBuffer rather than to the device's speakers — with the following.

OfflineAudioContext
The OfflineAudioContext interface is an AudioContext interface representing an audio-processing graph built from linked together AudioNodes. In contrast with a standard AudioContext, an OfflineAudioContext doesn't really render the audio but rather generates it, as fast as it can, in a buffer.
complete (event)
The complete event is fired when the rendering of an OfflineAudioContext is terminated.
OfflineAudioCompletionEvent
The OfflineAudioCompletionEvent represents events that occur when the processing of an OfflineAudioContext is terminated. The complete event implements this interface.

Audio Workers

Audio workers provide the ability for direct scripted audio processing to be done inside a web worker context, and are defined by a couple of interfaces (new as of 29th August 2014.) These are not implemented in any browsers yet. When implemented, they will replace ScriptProcessorNode, and the other features discussed in the Audio processing via JavaScript section above.

AudioWorkerNode
The AudioWorkerNode interface represents an AudioNode that interacts with a worker thread to generate, process, or analyse audio directly.
AudioWorkerGlobalScope
The AudioWorkerGlobalScope interface is a DedicatedWorkerGlobalScope-derived object representing a worker context in which an audio processing script is run; it is designed to enable the generation, processing, and analysis of audio data directly using JavaScript in a worker thread.
AudioProcessEvent
This is an Event object that is dispatched to AudioWorkerGlobalScope objects to perform processing.

过时的接口

The following interfaces were defined in old versions of the Web Audio API spec, but are now obsolete and have been replaced by other interfaces.

JavaScriptNode
Used for direct audio processing via JavaScript. This interface is obsolete, and has been replaced by ScriptProcessorNode.
WaveTableNode
Used to define a periodic waveform. This interface is obsolete, and has been replaced by PeriodicWave.

示例

This example shows a wide variety of Web Audio API functions being used. You can see this code in action on the Voice-change-o-matic demo (also check out the full source code at Github) - this is an experimental voice changer toy demo; keep your speakers turned down low when you use it, at least to start!

The Web Audio API lines are highlighted; if you want to find more out about what the different methods, etc. do, have a search around the reference pages.

var audioCtx = new (window.AudioContext || window.webkitAudioContext)(); // define audio context
// Webkit/blink browsers need prefix, Safari won't work without window.

var voiceSelect = document.getElementById("voice"); // select box for selecting voice effect options
var visualSelect = document.getElementById("visual"); // select box for selecting audio visualization options
var mute = document.querySelector('.mute'); // mute button
var drawVisual; // requestAnimationFrame

var analyser = audioCtx.createAnalyser();
var distortion = audioCtx.createWaveShaper();
var gainNode = audioCtx.createGain();
var biquadFilter = audioCtx.createBiquadFilter();

function makeDistortionCurve(amount) { // function to make curve shape for distortion/wave shaper node to use
  var k = typeof amount === 'number' ? amount : 50,
    n_samples = 44100,
    curve = new Float32Array(n_samples),
    deg = Math.PI / 180,
    i = 0,
    x;
  for ( ; i < n_samples; ++i ) {
    x = i * 2 / n_samples - 1;
    curve[i] = ( 3 + k ) * x * 20 * deg / ( Math.PI + k * Math.abs(x) );
  }
  return curve;
};

navigator.getUserMedia (
  // constraints - only audio needed for this app
  {
    audio: true
  },

  // Success callback
  function(stream) {
    source = audioCtx.createMediaStreamSource(stream);
    source.connect(analyser);
    analyser.connect(distortion);
    distortion.connect(biquadFilter);
    biquadFilter.connect(gainNode);
    gainNode.connect(audioCtx.destination); // connecting the different audio graph nodes together

    visualize(stream);
    voiceChange();

  },

  // Error callback
  function(err) {
    console.log('The following gUM error occured: ' + err);
  }
);

function visualize(stream) {
  WIDTH = canvas.width;
  HEIGHT = canvas.height;

  var visualSetting = visualSelect.value;
  console.log(visualSetting);

  if(visualSetting == "sinewave") {
    analyser.fftSize = 2048;
    var bufferLength = analyser.frequencyBinCount; // half the FFT value
    var dataArray = new Uint8Array(bufferLength); // create an array to store the data

    canvasCtx.clearRect(0, 0, WIDTH, HEIGHT);

    function draw() {

      drawVisual = requestAnimationFrame(draw);

      analyser.getByteTimeDomainData(dataArray); // get waveform data and put it into the array created above

      canvasCtx.fillStyle = 'rgb(200, 200, 200)'; // draw wave with canvas
      canvasCtx.fillRect(0, 0, WIDTH, HEIGHT);

      canvasCtx.lineWidth = 2;
      canvasCtx.strokeStyle = 'rgb(0, 0, 0)';

      canvasCtx.beginPath();

      var sliceWidth = WIDTH * 1.0 / bufferLength;
      var x = 0;

      for(var i = 0; i < bufferLength; i++) {

        var v = dataArray[i] / 128.0;
        var y = v * HEIGHT/2;

        if(i === 0) {
          canvasCtx.moveTo(x, y);
        } else {
          canvasCtx.lineTo(x, y);
        }

        x += sliceWidth;
      }

      canvasCtx.lineTo(canvas.width, canvas.height/2);
      canvasCtx.stroke();
    };

    draw();

  } else if(visualSetting == "off") {
    canvasCtx.clearRect(0, 0, WIDTH, HEIGHT);
    canvasCtx.fillStyle = "red";
    canvasCtx.fillRect(0, 0, WIDTH, HEIGHT);
  }

}

function voiceChange() {
  distortion.curve = new Float32Array;
  biquadFilter.gain.value = 0; // reset the effects each time the voiceChange function is run

  var voiceSetting = voiceSelect.value;
  console.log(voiceSetting);

  if(voiceSetting == "distortion") {
    distortion.curve = makeDistortionCurve(400); // apply distortion to sound using waveshaper node
  } else if(voiceSetting == "biquad") {
    biquadFilter.type = "lowshelf";
    biquadFilter.frequency.value = 1000;
    biquadFilter.gain.value = 25; // apply lowshelf filter to sounds using biquad
  } else if(voiceSetting == "off") {
    console.log("Voice settings turned off"); // do nothing, as off option was chosen
  }

}

// event listeners to change visualize and voice settings

visualSelect.onchange = function() {
  window.cancelAnimationFrame(drawVisual);
  visualize(stream);
}

voiceSelect.onchange = function() {
  voiceChange();
}

mute.onclick = voiceMute;

function voiceMute() { // toggle to mute and unmute sound
  if(mute.id == "") {
    gainNode.gain.value = 0; // gain set to 0 to mute sound
    mute.id = "activated";
    mute.innerHTML = "Unmute";
  } else {
    gainNode.gain.value = 1; // gain set to 1 to unmute sound
    mute.id = "";    
    mute.innerHTML = "Mute";
  }
}

规范

Specification Status Comment
Web Audio API Working Draft  

浏览器兼容性

Feature Chrome Firefox (Gecko) Internet Explorer Opera Safari (WebKit)
Basic support 14 webkit 23 未实现 15 webkit
22 (unprefixed)
6 webkit
Feature Android Chrome Firefox Mobile (Gecko) Firefox OS IE Phone Opera Mobile Safari Mobile
Basic support 未实现 28 webkit 25 1.2 未实现 未实现 webkit

相关链接

文档标签和贡献者

 此页面的贡献者: smilewalker, SoAanyip, z__, wwj, luojia, sunwulovelove1
 最后编辑者: smilewalker,