Media Capabilities API

The Media Capabilities API allows developers to determine decoding and encoding abilities of the device, exposing information such as whether media is supported and whether playback should be smooth and power efficient, with real time feedback about playback to better enable adaptive streaming, and access to display property information.

Examples

Detect audio file support and expected performance

This example defines a audio configuration then checks to see if the user agent supports decoding that media configuration, and whether it will perform well in terms of smoothness and power efficiency.

if ('mediaCapabilities' in navigator) {
  const audioFileConfiguration = {
    type : 'file', 
    audio : {
        contentType: "audio/mp3",
        channels: 2,  
        bitrate: 132700, 
        samplerate: 5200 
    } 
  }; 

  navigator.mediaCapabilities.decodingInfo(audioFileConfiguration).then(result => { 
    console.log('This configuration is ' +                     
        (result.supported ? '' : 'not ') + 'supported, ' +           
        (result.smooth ? '' : 'not ') + 'smooth, and ' +            
        (result.powerEfficient ? '' : 'not ') + 'power efficient.') 
    })
    .catch(() => {
      console.log("decodingInfo error: " + contentType)
    });
}

Media Capabilities API concepts and usage

There are a myriad of video and audio codecs. Different browsers support different media types and new media types are always being developed. With the Media Capabilities API, developers can ensure each user is getting the best bitrate and storage savings for their browser, device, and OS capabilities.

Whether a device uses hardware or software decoding impacts how smooth and power efficient the video decoding is and how efficient the playback will be. The Media Capabilities API enables determining which codecs are supported and how performant a media file will be both in terms of smoothness and power efficiency.

The Media Capabilities API provide more powerful features than say MediaRecorder.isTypeSupported() or HTMLMediaElement.canPlayType(), which only address general browser support, not performance. The API also provides abilities to access display property information such as  supported color gamut, dynamic range abilities, and real-time feedback about the playback.

To test support, smoothness and power efficiency of a video or audio file, you define the media configuration you want to test, and then pass the audio or video configuration as the parameter of the MediaCapabilities interface's encodingInfo() and decodingInfo() methods.

Media capabilities information enables websites to enable adaptative streaming to alter the quality of content based on actual user-perceived quality, and react to a pick of CPU/GPU usage in real time.

Media Capabilities Interfaces

MediaCapabilities
Provides information about the decoding abilities of the device, system and browser based on codecs, profile, resolution, and bitrates. The  information can be used to serve optimal media streams to the user and determine if playback should be smooth and power efficient . 
MediaCapabilitiesInfo
The interface of the promise returned by  the mediaCapabilities's  encodingInfo() and decodingInfo() methods; returning whether the media configuration tested is  supported, smooth, and powerEfficient.
ScreenColorGamut
Will describe the color gamut, or the range of color, the screen can display (not currently supported anywhere).
ScreenLuminance
Will describe the known luminance characteristics of the screen (not currently supported anywhere).

Media Capabilities dictionaries

MediaConfiguration
Describes how video and audio configuration dictionaries must be configured, or defined, to be passed as a parameter of the MediaCapabilities.encodingInfo() and MediaCapabilities.decodingInfo() methods. It is inherited by the MediaDecodingConfiguration and MediaEncodingConfiguration dictionaries.
MediaDecodingConfiguration
Defines the valid values for allowed types of media when the media configuration is submitted as the parameter for MediaCapabilities.decodingInfo(). Consists of a media decoding type and a VideoConfiguration or AudioConfiguration dictionary.
MediaEncodingConfiguration
Defines the valid values for allowed types of media when the media configuration is submitted as the parameter for mediaCapabilities.encodingInfo(). Consists of a media encoding type and a VideoConfiguration or AudioConfiguration dictionary.
VideoConfiguration
Describes the required video properties that must be configured when querying a video configuration to be passed as part of a MediaConfiguration parameter to the MediaCapabilities.encodingInfo() and MediaCapabilities.decodingInfo() methods.
AudioConfiguration
Describes the required audio properties that must be configured when querying an audio configuration to be passed as part of a MediaConfiguration parameter to the MediaCapabilities.encodingInfo() and MediaCapabilities.decodingInfo() methods.

Specifications

Specification Status Comment
Media Capabilities Draft Initial definition

Browser compatibility

Update compatibility data on GitHub
DesktopMobile
ChromeEdgeFirefoxInternet ExplorerOperaSafariAndroid webviewChrome for AndroidEdge MobileFirefox for AndroidOpera for AndroidiOS SafariSamsung Internet
Basic support
Non-standard
Chrome Full support 66Edge ? Firefox Full support 63IE ? Opera Full support 55Safari ? WebView Android Full support 66Chrome Android Full support 66Edge Mobile ? Firefox Android ? Opera Android Full support 55Safari iOS ? Samsung Internet Android ?
encodingInfo
Non-standard
Chrome Full support 67
Disabled
Full support 67
Disabled
Disabled From version 67: this feature is behind the chrome://flags/#enable-experimental-web-platform-features preference (needs to be set to enabled). To change preferences in Chrome, visit chrome://flags.
Edge ? Firefox Full support 63IE ? Opera ? Safari ? WebView Android Full support 67
Disabled
Full support 67
Disabled
Disabled From version 67: this feature is behind the chrome://flags/#enable-experimental-web-platform-features preference (needs to be set to enabled).
Chrome Android Full support 67
Disabled
Full support 67
Disabled
Disabled From version 67: this feature is behind the chrome://flags/#enable-experimental-web-platform-features preference (needs to be set to enabled). To change preferences in Chrome, visit chrome://flags.
Edge Mobile ? Firefox Android ? Opera Android ? Safari iOS ? Samsung Internet Android ?
decodingInfo
Non-standard
Chrome Full support 66Edge ? Firefox Full support 63IE ? Opera Full support 55Safari ? WebView Android Full support 66Chrome Android Full support 66Edge Mobile ? Firefox Android ? Opera Android Full support 55Safari iOS ? Samsung Internet Android ?

Legend

Full support  
Full support
Compatibility unknown  
Compatibility unknown
Non-standard. Expect poor cross-browser support.
Non-standard. Expect poor cross-browser support.
User must explicitly enable this feature.
User must explicitly enable this feature.

See also

Document Tags and Contributors

Contributors to this page: ExE-Boss, sideshowbarker, estelle, chrisdavidmills
Last updated by: ExE-Boss,