Supporting alternate audio tracks in HTTP Streaming videos

OSMF 1.6 supports multiple language tracks for HTTP video streams, without requiring duplication and repackaging of the video for each audio track. This “late” binding of an audio track to a video stream allows content publishers to easily provide multiple language tracks for a given video asset, at any time before or after the asset’s initial packaging.

The initial packaging of the video asset can include an audio track, and the publisher can still choose to provide one or more additional audio tracks for the video. OSMF then provides the support that allows you to give viewers the option of switching to an alternate audio track either before or during playback.

For your player, OSMF signals that multiple audio streams are available. It also provides a mechanism to iterate through the available alternate streams.

OSMF detects the presence of alternate audio tracks from the F4M manifest. If an alternate audio track is selected, OSMF obtains the fragments for that track, and replaces the audio packets from the main video stream with the alternate ones. OSMF applies (multiplexes) the selected alternate track before passing the video stream to your player.

Late-binding audio supports:

  • Changing audio tracks both before the beginning of playback and during playback.

  • Live and VOD (recorded) content.

  • Playback of multibitrate (MBR) video streams with alternate (non-MBR) audio streams. Current MBR behavior and logic should work as expected and the selected alternate tracks are maintained during the playback and not affected by bit-rate switching.

  • Playback of DVR video streams with alternate audio streams. Current DVR behavior and logic should work as expected and the selected alternate tracks are maintained during playback and unaffected by going to live or seeking back into the DVR content.

  • Attribute tags for alternate audio tracks: lang (for the language of the track) and label (for additional descriptions, such as “director’s commentary” or “audio descriptions for the visually impaired”).

  • Gap resilience. If a selected alternative audio track becomes unavailable during playback, playback continues without audio. Thereafter, at the beginning of each fragment OSMF attempts to reactivate the selected audio track. OSMF also dispatches an NetStream.Play.StreamNotFound event containing the URL for the audio track, so that you can have your player respond to the failure.

The following limitations with late-binding audio currently apply:

  • Multi-bitrate (MBR) audio tracks are unsupported.

  • Switching between audio tracks is not typically instantaneous; there may be a short delay of a second or more.

The following snippet is taken from the OSMF sample HelloWorld8.as, which demonstrates simple event handling for a video with alternate audio streams:

// Assuming the media container and media factory exist, and that the media 
// element for this video has been created from an F4M file, we create the media 
// player and add event listeners to handle alternative audio stream changes. 
 
_player = new MediaPlayer(); _player.addEventListener(MediaPlayerStateChangeEvent.MEDIA_PLAYER_STATE_CHANGE, onPlayerStateChange); 
_player.addEventListener(AlternativeAudioEvent.NUM_ALTERNATIVE_AUDIO_STREAMS_CHANGE, onNumAlternativeAudioStreamsChange); 
_player.addEventListener(AlternativeAudioEvent.AUDIO_SWITCHING_CHANGE, onAlternateAudioSwitchingChange); 
// Listen for relevant player state change events, such as READY. 
private function onPlayerStateChange(event:MediaPlayerStateChangeEvent):void 
{    switch (event.state) 
    { 
        // When the media is ready, you can see what alternative audio tracks are present. 
        case MediaPlayerState.READY: 
            if (!_playbackInitiated) 
            { _playbackInitiated = true; trace("Alternative languages", 
                _player.hasAlternativeAudio ? "available" : " not available" ); 
                if (_player.hasAlternativeAudio) 
                {     for (var index:int=0; index<_player.numAlternativeAudioStreams; 
                        index++) 
                    { var item:StreamingItem=_player.getAlternativeAudioItemAt(index); 
                     trace("[LBA] [", item.info.language, "]", item.info.label); 
                    } 
                    // This is a good point to allow the user to select a language. 
                    // Here we pass 0, which plays the first alternative track. 
                    // Passing -1 plays the video stream's embedded audio track. 
                     _player.switchAlternativeAudioIndex(0); 
                }     
                if (_player.canPlay) 
                { 
                    _player.play(); 
                } 
            } 
            break; 
        // Handle other player state change events (such as PLAYING, etc.) here. 
    } 
} 
// Listen for a change in the number of alternative streams associated with this video. 
private function onNumAlternativeAudioStreamsChange(event:AlternativeAudioEvent):void 
{ if (_player.hasAlternativeAudio) 
    { trace("Number of alternative audio streams = ", _player.numAlternativeAudioStreams); } 
} 
 
// Listen for when an audio stream switch is in progress or has been completed. 
private function onAlternateAudioSwitchingChange(event:AlternativeAudioEvent):void 
{ if (event.switching) 
    { trace("Alternative audio stream switch in progress"); } 
    else 
    { trace("Alternate audio stream switch completed"); trace("Current alternate audio index is [" + _player.currentAlternativeAudioStreamIndex + "]."); } 
}