Troubleshooting VOD Streaming

Determining the issue 

If you experience a problem with your stream, there are a few steps you can take to determine what the exact issue is that you are experiencing. Sometimes, going through these steps will be enough to solve your problem. Or it might become clear that your issue if one of the more common problems that are described further down below, along with their solutions. And otherwise, these steps will give you the necessary info to open a well-defined ticket in our support portal, so that we can help you as quickly as possible:

Check license key
Check publishing point's state (Live only)
Check server's log files
Check encoder settings
Check release notes
Check your setup

All six steps are described in more detail below.

License key

The first thing is to check is whether the license is properly installed: Installation Check. Also check that there are no Error Messages.

Publishing point's state

This step applies to streaming Live only. To check whether a publishing point is properly set up, you can check its state. A working publishing point will return the following from a /state REST call, with the actual state differing based on whether a stream has been ingested or not ((see Retrieving additional publishing point information):

<?xml version="1.0" encoding="utf-8"?>
<!-- Created with Unified Streaming Platform(version=1.x.x) -->
<smil
  xmlns="http://www.w3.org/2001/SMIL20/Language">
  <head>
    <meta
      name="state"
      content="idle">
    </meta>
  </head>
</smil>

If a publishing point doesn't return a state, it might be either that it has not been set up at all, or that there are other problems with your setup (such as access rights). If the state returned by the publishing point is not as expected, the most obvious cause would be a problem with your ingest, either because of problems with the content itself (i.e. encoder settings) or because the webserver isn't correctly set up to handle the ingest.

Log files

The webserver's log files are the most valuable source for determining the exact nature of the problem that you are experiencing with your stream. These logs allow you to determine whether content is properly ingested (in the case of Live) and whether requests for content are handled correctly (both for Live and VOD). For Apache on Ubuntu, three log files are of importance:

The general error log (/var/log/apache2/error.log)

The virtual host access log (configured in the virtual host config)

The virtual host error log (configured in the virtual host config)

In the virtual host's access log, you can check how the ingest and requests for content are being handled. For every problem with either the ingest or a request, an error is written to the virtual host's error log.

Encoder settings

Adaptive bitrate video streaming is a delicate process that requires well formed content that has to adhere to many requirements. If those requirements are not met, a wide variety of issues can occur. Check Recommendations for Live and Recommendations for VOD to determine whether you have configured your encoder correctly. Do note that even with the right settings, there might still be problems with the output of your encoder. Determining whether this is or is not the case requires a detailed inspection of the content itself, with specialized tools (e.g., ffprobe). For streaming Live, make sure that your encoder is supported by checking the overview in our Factsheet.

Release notes

If you are currently not using the latest version of our software, please check the Release notes to determine whether we introduced a fix for your problem in a newer version.

Setup

A detailed check of your installation is advised. Please make sure that your webserver is installed and configured as detailed in the installation documentation that we provide for Apache:

How to Configure (Unified Origin)

FMP4_403 No virtual path specified 

This error is returned when you make a request for the server manifest (.ism(l)), instead of a playout format. Direct requests for the server manifest are prohibited for security reasons. You should add a virtual path to the URL of the server manifest, specifying the playout format. For more information about the URLs of the different playout formats, please consult Player URLs.

FMP4_403 origin: no policy for streaming 

This error typically indicates that your license key is not being loaded correctly. If you are using Apache, you can check if this is the case in the general error log of the webserver, by looking for the following error:

failed to initialize! (Invalid base64 character)

Common causes for this error are the addition of a newline or an erroneous character that has crept in when copying and pasting the license key.

If the error log does not indicate any problems with the license key, your license might not cover the use case that you are trying to achieve.

Further information about installing the license key can be found here: Installation Check. Additional info about any related errors can be found here: Error Messages.

Playback stops right after it starts 

When packaging MP4 into fragmented MP4 the resulting .ismv-file may not play on certain players: the player stops playback right after it starts.

If this happens, check the fragmented MP4 (.ismv) to see whether the duration in the "moov.trak.tkhd" box is zero.

Reading the ISO BMFF specification, it may seem that setting this box to zero goes against the spec. The same applies to the "mdhd" box, about which the spec says:

Duration is an integer that indicates the duration of this track (in the timescale indicated in the Movie Header Box). The value of this field is equal to the sum of the durations of all of the track's edits. If there is no edit list, then the duration is the sum of the sample durations, converted into the timescale in the Movie Header Box. If the duration of this track cannot be determined then duration is set to all 1s (32-bit maxint).

However, the duration should be zero. In a Technical Corrigendum this is clarified. The duration of the entire movie (including the fragments) should be set in another box and the player should be using that info:

The boxes that have duration fields that need to be correctly populated are:

"mvhd" (ISO BMFF spec section 8.3.3)

"tkhd" (ISO BMFF spec section 8.5.3)

"mdhd" (ISO BMFF spec section 8.8.3)

According to the spec the above boxes should hold the duration without the fragments, which equals 0 in the case of fragmented MP4. Please see:

INTERNATIONAL STANDARD ISO/IEC 14496-12:2012 TECHNICAL CORRIGENDUM 1 Published 2013-09-01

Add the following to 8.3.2.1, at the end:

The duration field here does not include the duration of following movie fragments, if any, but only of the media in the enclosing Movie Box. The Movie Extends Header box may be used to document the duration including movie fragments, when desired and possible.

As a work around for incompatible players, you can force the Track Header Box to contain an actual duration by using brand=piff as part of your command line when packaging:

#!/bin/bash

mp4split -o example.ismv --brand=piff \
  example.mp4

Audio is longer than video 

When an audio and video track do not have the same duration, and there is for instance 1 audio fragment more than there are video fragments, USP signals this with a "lmsg" (last media segment) in the case of MPEG-DASH.

However, the player needs to support this. When it does not it will request a non-existing fragment. This may result in the player going into an 'error-state' where in fact it should handle the "lmsg" signaling.

The solution is to contact the player vendor for support.

S3 and latency 

We have seen longer response times on S3 using progressive MP4. If the files are un-fragmented, Unified Origin has to do extra scanning to access index information and build timelines for the fragments. In combination with the latency that is inherent to S3, this can be the cause of the longer response times.

Ideally if using S3, fragment your files as described in Packaging for Unified Origin.

FMP4_500 precondition failed 'tmp_ism.tracks_.size() == 1'

This error indicates that the server manifest file doesn't match the referenced content. For example, when the TrackIDs of the audio and video are swapped. It can also be caused by tracks that are missing in the middle of the fMP4 or by the content being changed after generating the manifest, as well as other, similar problems.

FMP4_500 precondition failed 'base_media_decode_time + duration == time'

This error is specific to Microsoft Smooth Streaming. It is typically caused by a problem with the signaling of the content stream and indicates a timestamp discontinuity.

We verify that the duration in the "traf" box matches the duration in the "tfxd" box of the content. Although thje difference may be very small, a mismatch triggers a discontinuity. This is noticed in playlist generation and eventually by a player implementation that triggers a decoder reset (introducing ticks/clicks/pops) whenever it encounters a timestamp discontinuity.

Audio sync issues 

Audio sync problems can have a variety of causes, such as problems with the source media, wrongly packaged content, player related issues and so on.

'415 unsupported media type'

You might experience '415 unsupported media type' errors when a multi-bitrate stream is not keyframe-aligned.

In the error log of the webserver you will find 'FMP4_415 No samples for fragment', because the webserver tried to create a fragment and could not find samples to do so, as the streams were not keyframe aligned.

If the files are generated in-house then the encoder settings should be adjusted. Otherwise the party that generated the files should be notified. The requirements for ABR streaming are listed on the Content Preparation page.

In short, the GOPs of the different bitrate streams must start on the exact same frame. That is: the content must be GOP aligned across bitrates. When encountering problems like non-GOP aligned multi-bitrate streams, you should perform in-depth stream analysis on your files to find the issue, as explained below.

Checking for non-GOP aligned content 

GOP analysis based on the manifest

For DASH and Microsoft Smooth Streaming, you can check the client manifest file to make sure that the GOPs are aligned. To do so, please check the manifest and whether or not each <Representation> block includes a timeline:

For example:

<Representation
  id="video=7999946"
  bandwidth="7999946"
  width="1920"
  height="1080"
  codecs="avc1.640028"
  scanType="progressive">
  <SegmentTemplate
    timescale="90000"
    duration="360000"
    initialization="YourMediaFile-$RepresentationID$.dash"
    media="YourMediaFile-$RepresentationID$-$Number$.m4s">
  </SegmentTemplate>
</Representation>

In the example above we can see that we have a <SegmentTemplate inside the representation. If you have the same for all the <Representation> blocks, then it is a sign that the GOPs are not aligned.

Basically, when all representations have their own timeline then it is a sign that the media is not GOP aligned. This might result in playout problems.

GOP analysis using MP4Box

MP4Box is a tool for stream analysis that can be used to check if all the keyframes for all stream layers are interleaved with each other.

You can use this tool to compare all the SyncSampleEntry entries for all the (fragmented) MP4 files.

The following command will analyze the file and write the results to file:

#!/bin/bash

MP4Box -std -diso 400k.ismv | grep SyncSampleEntry > 400k_info.txt
MP4Box -std -diso 800k.ismv | grep SyncSampleEntry > 800k_info.txt

# for all bitrates ...

The contents of the text file will look something like this:

<SyncSampleEntry sampleNumber="1"/>
<SyncSampleEntry sampleNumber="241"/>
<SyncSampleEntry sampleNumber="480"/>
<SyncSampleEntry sampleNumber="719"/>
<SyncSampleEntry sampleNumber="959"/>
<SyncSampleEntry sampleNumber="1199"/>
<SyncSampleEntry sampleNumber="1439"/>
<SyncSampleEntry sampleNumber="1500"/>
<SyncSampleEntry sampleNumber="1548"/>
<SyncSampleEntry sampleNumber="1788"/>
<SyncSampleEntry sampleNumber="1915"/>
<SyncSampleEntry sampleNumber="2006"/>
<SyncSampleEntry sampleNumber="2245"/>
<SyncSampleEntry sampleNumber="2484"/>
<SyncSampleEntry sampleNumber="2636"/>
<SyncSampleEntry sampleNumber="2716"/>
<SyncSampleEntry sampleNumber="2954"/>
<SyncSampleEntry sampleNumber="3194"/>

You can compare the output text files with diff (e.g. vimdiff) to see if there is any inconsistency in the "SyncSampleEntry" sampleNumber.