A Content-aware Bitrate Controller for Cloud Gaming
Supervisor(s) and Committee member(s): Mahmoud Reza Hashemi (supervisor), Shervin Shirmohammadi (co-supervisor)
Fulfilling cloud gaming’s (CG) ultimate goal; i.e., playing video games wherever, whenever and on every devices, requires reduction of its high bandwidth demand in a way that doesn’t adversely affect the players’ quality of experience. One way to do so is to reduce the bitrate of the regions in the scene that the player pays less attention to. This signals the need for a game-specific visual attention model. Developing such models for cloud gaming is very challenging because of unique content of video games, such as their fantasy characters, object placements and complex game logic and design for the sake of immersion and satisfaction. Difference among players’ skill levels, playing habits and strategies is another challenge of developing such models. Therefore, the first step of employing perceptual coding in bandwidth reduction of cloud gaming is to develop models which can overcome the aforementioned challenges. It also needs a dataset including recorded players’ gaze locations and other data during their gameplay. Since there is not an appropriate game-specific dataset or model yet, in this thesis we work on them. Doing so, two datasets and two perceptual models are proposed.
The first dataset includes a variety of video games and their objects. The model built on this dataset is grounded on visual attention mechanism and predicts the player’s gaze location based on a combination of low level signal properties and game object prioritization. Experimental results show that this model decreases the required bit rate by nearly 25% on average, while maintaining a relatively high user quality of experience.
The second perceptual model, addresses the difference among attention patterns of the players. To develop this model, the recorded eye-tracking data is first clustered. Then, the correlation of clusters and skill levels are shown via statistical and experimental methods. Our analyses show that this model decreases the bandwidth by up to 15% based on the player’s skill. The second step is to incorporate the perceptual models into the video encoder by means of perceptual rate-distortion models to assign bits to each region of the video according to its importance to HVS. Since current attention-based bit allocation algorithms do not take other HVS properties into account, in some cases the amount of distortion in less important areas distracts the players and consequently lowers the user perceived quality. Therefore, a new model is proposed which controls the amount of attention in each region based on its distance to important areas by considering both attention and fovea mechanisms. This model results in better user perceived quality (20% increase in mean opinion score).
The final contribution of this thesis is the development of a cloud gaming testbed to boost further researches pertaining to cloud gaming.
Although the thesis is in Farsi (Persian), its main ideas (except the RD model which is under review at the time of this writing) can be found in the following publications:
Ahmadi, S. Zad Tootaghaj, M.R. Hashemi, and S. Shirmohammadi, “A Game Attention Model for Efficient Bitrate Allocation in Cloud Gaming”, Multimedia Systems, Springer, Vol. 20, Issue 5, October 2014, pp. 485-501. DOI: 10.1007/s00530-014-0381-1
Ahmadi, S. Zad Tootaghaj, S. Mowlaei, M.R. Hashemi, and S. Shirmohammadi, “GSET Somi: A Game-Specific Eye Tracking Dataset for Somi”, Proc. ACM Multimedia Systems, Klagenfurt am Wörthersee, Austria, May 10-13 2016, 6 pages.
Ahmadi, M.R. Hashemi, and S. Shirmohammadi, “An Open Source Cloud Gaming Testbed Using DirectShow”, Proc. IEEE Int. Conference on Cloud Computing Technology and Science, Vancouver, Canada, November 30 – December 3 2015, pp. 606 – 610. DOI: 10.1109/CloudCom.2015.53
Multimedia Processing Laboratory (MPL), and Distributed and Collaborative Virtual Environment Research Laboratory (DISCOVER Lab)
The Multimedia Processing Laboratory (MPL) at the University of Tehran hosts research projects in Multimedia Systems and Networking, specifically:
- Receiver aware video encoding and adaptation
- Scalable multi-view video coding
- Cloud media and Cloud gaming
- Dynamic mapping of multimedia applications on a cloud of MPSoCs
- Reconfigurable hardware architectures for multimedia processing
- Hardware implementation of multimedia applications.
Research at the DISCOVER Lab (Distributed and Collaborative Virtual Environment Research Laboratory) is directed towards the enhancement of next generation human-human and human-information communication and interaction through advanced multimedia technology and virtual environments. Through our many projects we are developing new ideas and technology that will make easy-to-use multimedia environments and systems a reality. Research projects at the DISCOVER lab typically fall into the following categories:
- Networked Games, and Collaborative Virtual Environments
- Multimedia Systems and Applications
- 3D Physical Modelling
- Ambient Intelligent Multimedia Environments
- Intelligent Sensor Networks and Ubiquitous Computing
- Haptics and Teleoperation
- Multimedia-Assisted Healthcare