An H.264/AVC Receiver Aware Video Encoder for Adaptation Applications
Supervisor(s) and Committee member(s): Mahmoud-Reza Hashemi (Supervisor), Shervin Shirmohammadi (Advisor)
With recent advances in computing and communication technologies, ubiquitous access to high quality multimedia content is a fact of our daily life. However, resource consumption is still a major concern, specially battery consumptions for mobile devices. Today, any user is able to produce and consume multimedia contents using a smartphone or a tablet. In many applications, a multimedia content has to be sent to a vast range of receivers with different characteristics. To do so, a simple and affordable method is to generate a single layer bitstream and to adapt it, through a set of middle nodes, to meet each receiver’s characteristics and its resource limitations. In addition, with the advances in computing technologies, a consumer (decoder) of video content is potentially a producer (encoder) of this content in many occasions. As a result, constraints such as power consumption, computational power and memory limitations should be considered for most multimedia devices during the encoding phase, as well.
In this thesis, we start by highlighting the necessity of having a resource aware encoder. Then, this general problem is formulated in terms of an abstract method, customized and solved for the special cases of receiver aware encoder, receiver aware encoder for adaptation applications, and self resource aware encoder. In a receiver aware encoder, the resource limitations of the receiver side are taken into account during the encoding phase at the server side. This will ensure that the generated bitstream will satisfy the receiver’s resource limitations during decoding. A receiver aware encoder for adaptation applications is an encoder which considers not only the receiver’s resource limitations but also the characteristics of the adaptation node and its operation during the encoding phase. Hence, it generates a bitstream that meets the decoder’s requirements even after adaptation. Finally, a self resource aware encoder refers to an encoder which is running on a device with limited resources, such as a video encoder on a smartphone. Such an encoder is able to encode video while optimizing its operations according to the available resources.
Our proposed abstract model is extensible, since it can be used for various combinations of resource limitations. In addition, it follows a methodology for extracting the proper subset of encoding parameters. In order to show the efficiency of the proposed approach, a case study is implemented for the reference software of H.264/AVC codec, for each of the three mentioned scenarios. In these case studies, the computational complexity of decoder/encoder is considered as the limited resource. Finally, we also propose a methodology for estimating resource consumption. This methodology was used for the implementation and evaluation of the case studies of our first contribution. In this context, and in order to evaluate the performance of the proposed methodology, we have modeled and estimated the complexity of an H.264/AVC decoder and encoder. Simulations results show the applicability and the performance of our proposed abstract level resource aware encoder as well as the resource estimation methodology. Both of the proposed approaches in this thesis are generic and can be used for a wide range of applications. For instance, the proposed abstract level resource aware encoder can be customized, for any application which produces or consumes video content and has limited resources, in order to design a resource aware encoder. Similarly, for any application with a clearly specified algorithm and a distinctive resource consumption metric, a resource estimation model can be derived using the proposed resource estimation methodology.
While the thesis is in Farsi (Persian) , the following papers from the thesis are in English:
• M. Semsarzadeh, A. Lotfi, M.R. Hashemi, and S. Shirmohammadi, “A Fine-Grain Distortion and Complexity Aware Parameter Tuning Model for the H.264/AVC Encoder”, Signal Processing: Image Communication, Elsevier, Vol. 28, Issue 5, May 2013, pp. 441–457.
• M. Semsarzadeh, M. Jamali, M.R. Hashemi, and S. Shirmohammadi, “Complexity Modeling of the Motion Compensation Process of the H.264/AVC Video Coding Standard”, Proc. IEEE International Conference on Multimedia & Expo, Melbourne, Australia, July 9-13 2012, pp. 925-930.
Multimedia Processing Laboratory, University of Tehran, and DISCOVER Lab, University of Ottawa
The Multimedia Processing Laboratory (MPL) at the University of Tehran hosts research projects in Multimedia Systems and Networking, specifically:
• Receiver aware video encoding and adaptation
• Scalable multi-view video coding
• Cloud media and Cloud gaming
• Dynamic mapping of multimedia applications on a cloud of MPSoCs
• Reconfigurable hardware architectures for multimedia processing
• Hardware implementation of multimedia applications.
Research at University of Ottawa's DISCOVER Lab is directed towards the enhancement of next generation human-human communication through advanced multimedia technology and virtual environments. Projects typically fall into the following categories:
• Networked Games and Collaborative Virtual Environments, including cloud gaming and mobile gaming
• Multimedia Systems and Applications, including 3D video, Ultra HD video, and mobile video
• 3D Physical Modelling and Animation
• Intelligent Sensor Networks and Ubiquitous Computing
• Haptics and Teleoperation
• Multimedia-Assisted Rehabilitation Engineering