Multimedia Quality of Experience (QoE): Current Status and Future Direction

Abstract

Quality of Experience (QoE) is becoming an important factor of User-Centred Design (UCD). The deployment of pure technical measures such as Quality of Service (QoS) parameters to assess the quality of multimedia applications is phasing out due to the failure of those methods to quantify true user satisfaction. Though significant research results and several deployments have occurred and been realized over the last few years, focusing on QoE-based multimedia technologies, several issues both of theoretical and practical importance remain open. Accordingly, the papers of this Special Issue are significant contribution samples within the general ecosystem highlighted above, ranging from QoE in the capture, processing and consumption of next-generation multimedia applications. In particular, a total of five excellent articles have been accepted, following a rigorous review process, which address many of the aforementioned challenges and beyond.

1. Introduction

Quality of Experience (QoE) can be defined as the overall delight or annoyance of the user of an application or service. Typically, multimedia QoE is expected to offer improved user satisfaction in the audio–visual channels with which the users interact. Even though a significant number of novel multimedia applications are emerging, it is a challenge to provide better multimedia QoE for the end user. As a result, emerging multimedia applications fail to provide a minimum level of QoE required for end users to keep them engaged with the content. This is mainly due to the complexity of the content representation, processing, communication, security and visualisation. Therefore, in order to design QoE-proofed multimedia applications, researchers and developers need to overcome these challenging five pillars. The advancement of technology and multi-disciplinary research have enabled some multimedia applications to provide end-user satisfaction by overcoming technological and social barriers [1,2]. It is interesting to explore how well novel multimedia applications meet end-user expectations or satisfy user QoE where the use of objective approaches of measuring quality (e.g., Quality of Service (QoS)) is no longer acceptable.

The use of Artificial Intelligence (AI) and Machine Learning (ML) for improved multimedia QoE is on the rise [3]. Most of the current solutions for multimedia problems are based AI or ML. QoE can have a major role in these multimedia applications to measure end-user satisfaction. Soon QoE will be the decisive factor of User-Centred Design (UCD) approaches to maximise overall user satisfaction, rather than objective measures such as Delay, Jitter, Packet-Loss, Peak Signal-to-Noise Ratio (PSNR) and Structure SIMilarity (SSIM). In addition, User Quality of Security (QoSec) is an important aspect of QoE. The multimedia content should be original (i.e., authentic) and users should feel safe when consuming multimedia content. With the increase in face swapping and deepfakes, the authenticity of multimedia content is being compromised more than ever [4,5]. Therefore, future multimedia applications should deploy solutions to measure their authenticity and, in turn, satisfy end users’ delight or overall satisfaction (i.e., QoE).

With the challenges in ensuring high levels of user experience in emerging multimedia applications and making future applications fool-proof against security threats, we have put together this Special Issue on multimedia Quality of Experience (QoE). We have aimed to summarise the current activities in this field and the future projections.

2. Contributions

The first paper [6], written by Tho Nguyen Duc, Chanh Minh Tran, Phan Xuan Tan and Eiji Kamioka, identifies the human-related factors which have significant influences on QoE. Furthermore, this study inspects the combined impact of perceptual factors, memory effect and the degree of interest. Based on this investigation, a novel QoE model is proposed that effectively incorporates those factors to reflect the user’s cumulative perception.

Buddhiprabha Erabadda, Thanuja Mallikarachchi, Chaminda Hewage and Anil Fernando are the authors of the second paper [7], which explores the resource requirements for the real-time operation of High Efficiency video Coding (HEVC), which has become a contributing factor towards the QoE of the end users of emerging multimedia and future internet applications. In this context, this study proposes a content-adaptive Coding Unit (CU) size selection algorithm for HEVC intra-prediction. The performance evaluation conducted in the paper demonstrates than the perceptual visual quality of the reconstructed videos of the proposed algorithm is minimally affected compared to state-of-the-art approaches.

The third paper [8], written by Thanuja Mallikarachchi, Dumidu Talagala, Hemantha Kodikara Arachchi, Chaminda Hewage and Anil Fernando, identifies the importance of addressing the control of decoding complexity and energy use when decoding video to improve user QoE. As a solution, this study proposes an encoding framework that is capable of generating video bit streams with arbitrary bit rate and decoding complexity levels using a decoding complexity rate distortion model.

Safak Dogan, Nasser Haddad, Erhan Ekmekcioglu and Ahmet M. Kondoz are the authors of the fourth paper [9], which focuses on multi-view 3D content with dense depth map information. In this paper, the authors present a no-reference depth map quality evaluation model based on a novel depth map edge confidence measurement technique, in order to assist with the estimation of the quality of virtual views in multi-view video content.

Finally, the last paper [10], written by Jesenka Pibernik, Jurica Dolic, Hrvoje Abraham Milicevic and Bojan Kanizaj, looks into the quality of experience (QoE) of static and animated floating action buttons (FABs) that are used in the user interfaces of web pages and mobile applications. The authors experimentally evaluate the perceived Quality of Experience of FAB functionality and compare it to the more traditional toolbar alternative.