Media Conditioning For Ad Insertion

Unified Remix can optionally transcode H264 encoded media segments in order to ensure that a keyframe is present at splice points. This generally affects only one segment per track for each splice point; all other segments are referenced as is.

Transcoding is done using the Intel Media SDK.

When transcoding is required the original GOP containing the splice point will be split into two GOPs with an additional keyframe at the time of the splice point. If the presentation time of the event does not match up exactly with the keyframe's presentation time because the event's time falls within the duration of the keyframe, the presentation time of the event will be adjusted to match that of the keyframe.

The transcoded media is included in the remixed MP4 created by Unified Remix.

As this means the final video will have been encoded by two different encoders some decoding parameters may vary. This is why the resultant video will be signaled as 'avc3' rather than 'avc1'. This may cause issues with players that do not support it.

In SMIL playlists for Unified Remix you define conditioning information either using the @clipMode="gop|sample" attribute that specifies how the @clipBegin and @clipEnd values are to be interpreted relative to the media, or using the syntax defined by the OpenCable Event Signaling and Management API specification.

Specifying conditioning info using 'clipMode'

The @clipMode="gop|sample" attribute specifies how @clipBegin and @clipEnd values are interpreted relative to the media:

@clipMode

Description

gop

Aligns to the keyframe using the time given and searching backwards (default).

sample

Condition the media by inserting a keyframe when not present at the given time.

In an example SMIL playlist:

<video
  src="http://sample-content/tears-of-steel.mp4"
  clipBegin="wallclock(1970-01-01T00:00:15.000Z)"
  clipEnd="wallclock(1970-01-01T00:00:45.000Z)"
  clipMode="sample" />

When @clipMode is set to gop, Unified Remix adjusts the provided start and end times if necessary, so that the selected timespan begins and ends with a keyframe (on the media's timeline, Remix will scan towards the left in search of a keyframe, starting from the provided start or end time).

When @clipMode is set to sample, Unified Remix reconditions the content if necessary, so that the start and end of the specified timespan both coincide with a keyframe.

Note that conditioning applies to both audio and video tracks (it's not possible on add @clip to only video). They can be grouped with the <par/> element. See the SMIL Examples

Specifying conditioning info using CableLabs' Event Signaling and Management API

You can also use the the syntax defined by the OpenCable Event Signaling and Management API specification to specify the exact points where content needs to be conditioned. To do this, you need to add a separate element to your SMIL playlist:

<ConditioningInfo xmlns="urn:cablelabs:iptvservices:esam:xsd:signal:1"
  startOffset="PT1S"
  duration="PT6S" />

The @startOffset is relative to the begin of the surrounding seq and signals where Unified Remix needs to create a keyframe for a seamless splice out.

The @duration signals the duration of the break/pod/avail where Unified Remix creates a keyframe for a seamless splice return.

The @presentationTime of the corresponding DASH Event Message must match @startOffset.

The example below shows how to specify ConditioningInfo in a SMIL playlist. It also signals the event that the conditioning information is associated with. As you can see on the highlighted lines, the event and the conditioning info share the same @duration, and the @presentationTime of the event matches the @startOffset of the specified conditioning info:

 1<?xml version="1.0" encoding="UTF-8"?>
 2<smil xmlns="http://www.w3.org/2001/SMIL20/Language">
 3  <head>
 4  </head>
 5  <body>
 6    <seq>
 7      <par
 8        clipBegin="wallclock(1970-01-01T00:00:00.000Z)"
 9        clipEnd="wallclock(1970-01-01T00:00:16.000Z)" >
10        <audio
11          src="tears-of-steel-aac-128k.mp4" />
12        <video
13          src="tears-of-steel-avc1-400k.mp4" />
14        <video
15          src="tears-of-steel-avc1-750k.mp4" />
16        <video
17          src="tears-of-steel-avc1-1500k.mp4" />
18        <video
19          src="tears-of-steel-en.ismt" />
20        <EventStream xmlns="urn:mpeg:dash:schema:mpd:2011"
21          schemeIdUri="urn:scte:scte35:2013:xml"
22          timescale="90000">
23          <Event
24            presentationTime="90000"
25            duration="540000"
26            id="1">
27            <Signal xmlns="http://www.scte.org/schemas/35/2016">
28              <SpliceInfoSection>
29                <SpliceInsert
30                  spliceEventId="1234"
31                  outOfNetworkIndicator="1"
32                  spliceImmediateFlag="1"
33                  availNum="0"
34                  availsExpected="0">
35                  <Program />
36                  <BreakDuration
37                    autoReturn="1"
38                    duration="540000" />
39                </SpliceInsert>
40              </SpliceInfoSection>
41            </Signal>
42          </Event>
43        </EventStream>
44        <ConditioningInfo xmlns="urn:cablelabs:iptvservices:esam:xsd:signal:1"
45          startOffset="PT1S"
46          duration="PT6S" />
47      </par>
48    </seq>
49  </body>
50</smil>

Ensuring a continuous media timeline

To ensure a continuous media timeline on which gaps in audio, video and subtitles are minimized as much as possible, Remix will insert empty subtitle cues and silent audio where necessary. However, do note that support for insertion of silent audio is limited to AAC-LC, Dolby's AC-3 and EC-3, and DTS and DTS:X (e.g., AAC-HC is not supported).

Also, when Remix decides where in the source media's timeline it should make a cut, it treats video as leading. And to make sure audio and video end at the exact same time, it may lengthen the last video sample slightly.

Track order in and selection of track from a remixed MP4

The order in which Remix puts tracks in a remixed MP4 is considered an implementation detail that should not be relied upon for selection of tracks from a remixed MP4.

This means that to select tracks from a remixed MP4, --track_id should not be used. Instead, rely on --track_type and --track_filter (which may be combined).

Having said that the general order of tracks in a remixed MP4 will be as follows:

  • Audio tracks first sorted by bitrate ascending

  • Video tracks sorted by bitrate ascending

  • Text tracks (TTML / WebVTT subtitles)

However, note that other track properties like language and track kind will play a role as well.

Output remixed content with out-of-band instead of in-band parameter sets

When using a remixed MP4 as source, Unified Origin automatically signals DASH AVC content with a codec ID of avc3 (i.e., using in-band parameter sets).

If this does not meet your requirements the --no_inband_parameter_sets option can be used to switch the codec ID to avc1 (i.e., using out-of-band parameter sets).

This is often a requirement for SSAI workflows or legacy playback devices where the handling or creation of media with in-band codec parameters is not fully supported.

Performance considerations when storing remixed MP4s on remote storage

Overall, storing remixed MP4s on remote storage should provide sufficient performance, although caching them might be worth considering: Cloud Storage Reducing Latency. However, when playlists from which remixed MP4s are generated are very long (several hours), resulting remixed MP4s become too large for Origin to perform well when fetching them from remote storage (as Origin needs to read the index from the remixed MP4 for each request it serves out). In such cases, store remixed MP4s locally or on NFS.