Parametric Assessment of Video Quality in Content Distribution Networks
Supervisor(s) and Committee member(s): Advisor(s): Andreas Mauthe (Supervisor), Ralf Steinmetz(Examiner)
IP-based packet-switched networks have become one of the main content distribution platforms for emerging multimedia services such as IPTV, thanks to the rapidly growing bandwidth and exclusive inter-networking and interactivity features of IP-based networks. Meanwhile, high quality video content services are becoming particularly popular within content delivery networks (CDN). During content distribution, packets of encoded video streams can be delayed, corrupted or dropped due to network impairments in packet-switched networks. This leads to perceptual quality degradations of the delivered content at receiver. Although network impairments are rare in commercial managed networks, any distortion caused by impairments can be highly detrimental to end users’ experience. Consequently the ability to meet customer expectations on video quality has become a critical service differentiator. Quality of Experience (QoE) that was merely recognised as a value-added service of traditional content distribution services is now one of the fundamental requirements and challenges of providing high quality video services. In order to maintain a high level of user experience throughout the life-cycle of a video service, a service quality measurement and management service must be established.
The thesis first outlines the problem space of video content distribution in packet-switched networks and motivates the in-service quality assessment model to evaluate quality degradation caused by the content loss effect. This is followed by the background research which explores the key elements and mechanisms of video content distribution systems including video codecs, content encapsulation, packet-switched networks and content presentation in conjunction with discussion of human vision and end user expectations. The potential causes of quality degradation and factors that could influence the perceptual impact of degradation in distribution systems are also discussed.
A detailed analysis of existing objective evaluation methodologies with regard to the performance in accessing user experience and the feasibility of implementing them in video content distribution networks is also carried out. It is concluded that quality of delivery (QoD) is the critical part of the overall acceptability of a video service and should be assessed in-service. Further, packet analysis is the ideal method of providing such an in-service assessment in distribution networks due to its operational nature.
In order to investigate the origin, appearance and perception of content loss in the distribution network, initial subjective experiments are necessary. One of the prerequisites of these experiments is the use of a testbed system that processes the source video content and produces test sequences for the experiment according to the test plan. The testbed system is also an essential tool for designing and validating the objective QoE model. The LA2 testbed system has been developed for the development and evaluation of objective quality assessment models. Under the framework of the LA2 system, operational functions (such as deep packet inspection and header analysis) of the proposed model have been realised. The LA2 system facilitates the efficient generation of test sequences for subjective experiments according to designated test plans with multiple functional modules such as the network impairment emulator (NIE) and configuration tools. The testbed system has been used to f
acilitate early exploratory data analysis which leads to the formulation of relevant scientific questions and helps to establish further quantitative data analysis. Test results prove that the perceivable impact of content loss on video content is determined by the joint influence of multiple impact factors.
With the results from analytical study and a number of exploratory tests, a discrete parametric assessment model to provide high performance in-service video quality measurements is introduced. A distinctive discrete network analysis methodology is outlined followed by details of three key functions: packet inspection, perceptual impact assessment and impact aggregation. Content factors, error factors, system factors, and user factors are identified as the essential impact factors from the modelling. Each impact factor is composed by one or multiple impact indices reflecting different aspects of the impact effect. Quantitative metrics are also defined for each impact index.
The thesis then elaborates on the subjective experiment and modelling efforts to realise the perceptual impact assessment model. Source content selection, test condition design, test environment establishment, and test procedures are introduced. After an overview of the observed user opinion scores obtained in the subjective, the analysis of the statistical inference process, including model specification, estimation of model parameters and estimation of precision, which are relevant to the modelling of assessment functions are introduced. Two assessment functions have been derived to evaluate the dichotomous and polytomous visibility of content loss respectively. Both functions provide high performance estimations according to their fitness to the subjective data.