Enhance ad experiences with HLS interstitials

Hi everyone. I'm Julian, an AVFoundation engineer. In recent years, we introduced HLS interstitials as a simple way to insert ads into your main content. This year we are offering new ways to present and navigate with interstitials. So if you're interested in including ads or just want to learn how to incorporate HLS interstitials in new ways, you tuned into the right session.

In this talk, we will start with a recap of how HLS interstitial works today.

We will follow this with discussing our new Integrated Timeline API which provides a data model for building your app's UI experience.

Lastly, we will discuss what's new with Interstitial SharePlay.

Lets quickly review HLS interstitials.

With HLS interstitials, ads or interstitials are separate assets that can be scheduled on the main content timeline. This is your typical media playlist. If we want to schedule an interstitial at 5s into the content, we simple specify the start date as 5s from content's Program-Date-Time. The content will then play primary content up until 5 and then start playing the interstitial URL. Finally, we will resume playback of primary content after the interstitial completes.

HLS interstitials provides many benefits in comparison to direct insertion of ads into the main content. Interstitials allows for late binding ad decisioning and does not require ads to be conditioned with the primary content. While HLS interstitials has primarily been used for ad insertion, we would like to encourage its usage for presenting other auxiliary content such as show promos, studio banners and program recap segments as a drop in replacement to content served as discontinuities. This is scratching the surface of inserting HLS interstitials in your content. If you'd like to learn more please checkout "Explore dynamic pre-rolls and mid-rolls in HLS".

This year we are offering a new AVFoundation API called Integrated Timeline. This API introduces a data model for timing and sequence of playback of content with interstitials. You'll be able to utilize this data model to build new UI experiences and enable your user to seamlessly navigate in and out of interstitials.

We will start off with an overview of different ways interstitials can be presented on the integrated timeline.

Afterwards, we will move onto understanding the API and how you can use it.

We will then take a look at an example app using integrated timeline for building its UI. Lastly, we will discuss new HLS syntax for describing how an interstitial should behave in integrated timeline.

Lets jump in and look at some of the ways interstitials can be presented on a timeline.

A common use case when inserting ads with VOD content is marking the interstitials as a point on the transport bar.

When playback reaches the point, the playhead stops as we play the interstitial. After the interstitial ends, primary content resumes and the playhead proceeds onwards.

Another use case with interstitials is representing the interstitials as a range in the transport bar such that interstitial's duration is included in the overall transport bar duration.

This would be commonly used with ads in broadcast style live streams or anywhere you replace burned-in content with interstitials.

This last example is an extension to the previous example. In the previous example, we clearly denoted the interstitial in the UI with a yellow highlight. In this case, the interstitial to be incorporated as indistinguishable from the primary content. This is useful when using HLS interstitials with rating bumpers, pre-rolls such as a show recap or post-rolls for dub cards. In particular for cases where the content is closely related to the primary content.

Now that we looked at the different cases that integrated timeline will model, lets learn the additions to AVPlayerInterstitalEvent API for signaling each of these use-cases.

For signaling a point on the timeline, we can create an interstitial event. And set a new property called timelineOccupancy with the value as singlePoint.

This time for signaling a range on the timeline, we can set the event's timelineOccupancy to fill. However, since interstitial events are loaded dynamically as the playhead approaches, the actual duration of the event may be discovered late. We can provide a plannedDuration as a fallback for the timeline to use until the actual duration is known. It is best practice to set this property for range interstitials.

For the final case where the range interstitial should be indistinguishable in the UI, we can set supplementsPrimaryContent to true.

Now that we looked at how we can signal interstitials behavior on the timeline, lets look at the Integrated Timeline API.

For observing timeline progression, IntegratedTimeline provides an API for obtaining the currentTime.

When the playhead reaches the PointEvent, currentTime stops progressing while the event plays.

After the event finishes playing, currentTime progresses again. And in this case when we reach the next interstitial event, currentTime continues to progress as we play the event.

Integrated Timeline also provides the ability to seek into an interstitial. Suppose, the client would like to seek forward by 10s. This would land in the middle of this FillEvent. Without this API, it would be difficult to accomplish this seek. In particular because the interstitial event is neither loaded nor enqueued on AVPlayer at this moment. However, this API makes it easy to accomplish seeking arbitrarily to any location on the timeline irrespective whether its the main content or an interstitial.

As part of Integrated Timeline it vends out a new object called AVPlayerItemSegment. Each region on the timeline is a segment where each segment provides key details about the content and time it occupies on the timeline. Even ranges of the primary content are considered as segments.

One of the biggest benefits of the integrated timeline is that it makes it easy to draw your playback UI. But to do so can be difficult as the timeline is constantly changing. In order to provide a self-consistent state, we introduced a snapshot object which the timeline can vend out. The snapshot represents an entire set of self-consistent properties that are needed to draw your UI.

Now as playback progresses and the timeline changes, the snapshot on the bottom stays static and does not mutate.

With an AVPlayerItemIntegratedSnapshot, we can obtain a list of all the AVPlayerItemSegments in that snapshot. In this example, I pulled out a couple of segments so we can take a look at their values.

For example, the first Primary segment in the snapshot would mark itself as primary and occupy a range from 0 to 5 on the timeline.

The next segment is the interstitial, PointEvent. Here the segment indicates it occupies 5 to 5, a zero duration. It will also provide access to the interstitial event, which would also contain its timelineOccupancy as singlePoint.

Lastly, the final segment selected is the FillEvent. This segment occupies 20 to 30 on the timeline with a timeline occupancy of fill.

Now that we understand all of the key objects, lets look how we can build a simple UI with Integrated Timeline. We will start off by creating our primary AVPlayerItem. We then obtain the integratedTimeline from our AVPlayerItem. Now we can grab a snapshot from the timeline to draw the current state.

Lets assume we have a routine to draw a UISlider that just needs the start, end and currentPosition. We can obtain all these values from the snapshot. Our start will be zero, our duration is simply the snapshot's duration and the currentPosition is the snapshot's currentTime. At this point, we have a simple transport bar that would include the fill interstitials as part of the overall slider. Now we want to start drawing single points in the timeline. In order to obtain the locations for the point events, we can filter on the snapshot segments and only include segments that have a type of interstitial and a timelineOccupancy of singlePoint.

Now, for each of these segments, we will draw a point using the segment's timeMapping target start as its position on the overall timeline.

Lastly, lets highlight the fill interstitials. Here we will filter the snapshot's segments and include segments representing interstitials with an occupancy of fill. Since supplement primary content indicates that the event should not be distinguished in the UI, we will ignore any interstitials with that set. Once we have the segments, we simply use the segment's timeMapping target and highlight that region on the UI slider.

The timeline can be very dynamic and can change often as interstitials get resolved or is frequently updating in the case of live streams. As such, it is necessary to listen to snapshotsOutOfSyncNotification from the timeline to update our UI when this occurs. When the notification closure is called, we can examine the reason in the userInfo. For example, if the reason is segmentsChanged, we may want to redraw our transport bar using a new snapshot. However, when the reason is currentSegmentChanged, we may want to update playerControls or other UI elements if we are transitioning into an interstitial or out of one.

Now that we look at the API and some sample code, lets look at an example app attached to this talk.

In this sample app, we have included different examples. Lets take a look at example called Fill Interstitial. This example includes a range interstitial scheduled at 10s into the content.

The transport bar at the bottom shows the interstitial with a yellow range starting at 10s. As we play, playback transition to the interstitial at the expected time.

Now we can try some seeks and seek back into primary content. Finally, we performed a seek back into the interstitial.

Awesome! Earlier I showed you how to construct interstitials in your app, but you can also specify them in your primary playlists. Here's how you can describe their appearance on the timeline from the server.

Starting with the point example, here is a DateRange tag for a HLS interstitial. In order for the timeline to consider this a POINT, we have introduced a new attribute, X-TIMELINE-OCCUPIES where the value is set as POINT. We also include a new attribute in DateRange tag, X-TIMELINE-STYLE with a value of HIGHLIGHT. This denotes that client should mark this point in their UI presentation.

For the range case, the new X-TIMELINE-OCCUPIES attribute is set with the value RANGE. The X-TIMELINE-STYLE remains the same as the previous example.

Lastly, for the case of presenting the interstitial indistinguishable in the UI. X-TIMELINE-OCCUPIES remains as RANGE. X-TIMELINE-SYTLE is set to PRIMARY to indicate that the app's UI should not mark this uniquely.

Since Integrated Timeline would allow for interstitials to be presented part of the primary content in the UI, it would make sense that these interstitials can also be coordinated during a SharePlay session. So this year, we are adding support for interstitials to be coordinated.

Lets take a look at how Interstitial SharePlay will work and how you can enable it. So, here we have two players in a coordinated SharePlay session. The idea is that both Player1 and Player2 would continue to synchronize playback during playback of Event1.

For seeks, suppose Player1 wants to seek out of Event1 back into the primary content. This seek will get proposed to Player2 and both players will continue to synchronize playback from this new point. One of the key requirements for enabling SharePlay support with interstitials is requiring the interstitial event to be common with all participants. In other words, Event1 in this example needs to be common for both players and must be the same content. If the interstitials are not common, the player should not coordinate playback during the interstitials. Lets see how we can signal this requirement.

One of the new additions to AVPlayerInterstitialEvent is a boolean to specify whether the content will vary across participants or playback sessions. In this example, setting contentMayVary to false will signal that this interstitial is static or common and allow this event to be coordinated.

Lastly, we can also signal the same behavior in the DateRange tag using a new attribute, X-CONTENT-MAY-VARY and setting that to NO.

Okay, lets now take look at our sample app which supports SharePlay. We have two phones setup in a FaceTime call. For this demo, we will choose the Fill (Supplements Primary) example that includes an interstitial at 10s into the content.

Notice that both phones start in sync and when we transition to the interstitial they continue in sync. We then issue a pause, followed by a seek backwards and lastly we seek forward.

Notice how all of these commands work seamlessly with both devices. To wrap up, you can now use Integrated Timeline to build your UI experience with interstitials. You can customize the interstitial experience by specifying the behavior either in the API or using the new DateRange attributes. You can now also enable SharePlay support for interstitials assets by marking the interstitial as not variable in the API.

For additional related talks, please see "Explore dynamic pre-rolls and mid-rolls in HLS" for learning more about HLS interstitials. For learning more about SharePlay, see "Coordinate media experiences with Group Activities". Thank you for watching.