Simulating Adaptive Video Streaming Using Sabre

Author: Kevin Spiteri Affiliation: College of Information of Computer Sciences, University of Massachusetts, Amherst; Editors: Marco Bertini and Mathias Lux

URL: https://github.com/umass-lids/sabre

Sabre [1] is an open source framework for simulating adaptive bitrate (ABR) video-player algorithms. ABR is used when delivering online video to different devices with different connectivity characteristics. The delivery bitrate is adapted dynamically to match the network connection, with the design of the ABR algorithms being a very active research area [2]. Sabre is a useful tool for researchers to simulate performance of their ABR algorithms and to fairly compare performance of different algorithms.

Background

To use adaptive video streaming, a content provider partitions a video into short segments a few seconds long and encodes each segment at several bitrates. The provider then serves all segments at all different bitrate levels on the server side. On the client side, the video player downloads each segment and renders the video to the user, downloading each segment at only one bitrate.

Figure 1. Adaptive video streaming: the server provides each segment at all bitrates, but the user only downloads each segment at one bitrate.

Figure 1. Adaptive video streaming: the server provides each segment at all bitrates, but the user only downloads each segment at one bitrate.

The video player uses an ABR algorithm to decide which bitrate to use for each segment. Downloading each segment at the highest available bitrate gives the best picture quality. However, if the player continuously selects a bitrate higher than the available network throughput, downloading a segment will take longer than playing it. This leads to video stalling, also known as rebuffering, which impacts the user’s quality of experience (QoE). The ABR algorithm will attempt to maximize the QoE by choosing the highest bitrate that can be supported without rebuffering. QoE is also affected by bitrate oscillations that can happen if the ABR algorithm frequently switches between a lower bitrate and a higher bitrate.

Designing a new ABR algorithm comes with several challenges. Testing the algorithm and making sure it performs well on different network conditions is a significant challenge. One path to test the algorithm involves implementing it in a video player and streaming video over the Internet. However, this approach has a number of difficulties such as:

  1. Implementing a new ABR algorithm in a video player has a steep learning curve.
  2. Network conditions vary, and it is difficult to distinguish between the algorithm’s impact and the network’s impact on QoE.
  3. Video quality as judged by a viewer can be subjective.
  4. Video streaming is time consuming.

Sabre Overview

Sabre mitigates the above difficulties and enables designers to test their algorithms in a simulation environment. This involves implementing the algorithm using a simple Python interface, and then testing it using network traces. Sabre then generates a set of QoE metrics, and the simulation results are repeatable. Since this is done as a simulation, a video session longer than one hour can be simulated in a time shorter than one second.

Sabre takes a number of inputs, simulates a video session, and outputs QoE metrics.

Figure 2. Sabre takes a number of inputs, simulates a video session, and outputs QoE metrics.

To simulate a streaming session, Sabre needs three inputs.

  1. Network Trace
    A network trace contains information about how throughput and latency vary over time. Traces can be captured by measuring network performance on a real network or they can be generated synthetically. The Sabre project includes a number of traces than can be used.
  2. Video Manifest
    A video manifest contains information about the video such as total length, segment length, number of bitrate encoding levels, and the size in bits of each particular segment at each particular bitrate encoding. The Sabre project includes a number of manifests that can be used.
  3. ABR Algorithm
    Finally Sabre needs an ABR algorithm to control the simulated session. The Sabre project comes with a number of ABR algorithms implemented.

After simulating a session, Sabre provides a number of metrics than can be used to determine the session QoE. The metrics include the average bitrate, the number of rebuffering events, the total rebuffering time, and the degree of bitrate oscillations. Sabre can also provide a session log including information such as the bitrate chosen for each segment and the time when each segment download starts and ends.

The Sabre approach to testing ABR algorithms has several advantages:

  1. Implementing an algorithm within the Sabre framework is much easier than implementing it within a production video player.
  2. Each simulation is repeatable. An algorithm can be adjusted, and the effect of the adjustment can be measured accurately without having to account for variable test conditions. Different algorithms can also be compared for the same network conditions.
  3. The metrics provide a subjective measure of QoE. Sabre can also be updated to provide additional metrics.
  4. The simulation time for a session is much shorter than the video length being simulated.

The goal of designing a new ABR algorithm is its use in a video player. While Sabre is intended to make the initial design more efficient and effective, it does not replace testing the algorithm in the player itself over the network. The internal architecture of Sabre is designed to be similar to video players used in production such as dash.js [3], thus making the simulation results more relevant and the transition from simulation to production easier.

Installation and Use of Sabre

Sabre is a Python tool licensed under the Simplified BSD License. With Python 3 installed one can clone the repository with:

git clone https://github.com/UMass-LIDS/sabre.git

Sabre then is found in the src directory and can be run from the command line or called from another Python script. Examples for the input files (in JSON format) and how to extend Sabre with new ABR algorithms is found in [1]. An example for creating plots as an output is given in the mmsys18 directory.

Conclusion

Sabre is an open source framework for simulating ABR algorithms, and is distributed under the BSD license. Sabre helps developers quickly simulate algorithms for different network conditions and provides useful QoE metrics to guide the algorithm development.

References

[1] SPITERI K., SITARAMAN R., SPARACIO D. From theory to practice: improving bitrate adaptation in the DASH reference player. In Proceedings of the 9th ACM Multimedia Systems Conference, 2018.
[2] KUA J., ARMITAGE G., BRANCH P. A survey of rate adaptation techniques for dynamic adaptive streaming over HTTP. IEEE Communications Surveys & Tutorials, 2017.
[3] DASH INDUSTRY FORUM. dash.js. https://github.com/Dash-Industry-Forum/dash.js/wiki

Bookmark the permalink.