Search Results (5)

An attention-aware patch-based deep-learning model for a blind 360-degree image quality assessment (360-IQA) is introduced in this paper. It employs spatial attention mechanisms to focus on spatially significant features, in addition to short skip connections to align them. A long skip connection is adopted to allow features from the earliest layers to be used at the final level. Patches are properly sampled on the sphere to correspond to the viewports displayed to the user using head-mounted displays. The sampling incorporates the relevance of patches by considering (i) the exploration behavior and (ii) a latitude-based selection. An adaptive strategy is applied to improve the pooling of local patch qualities to global image quality. This includes an outlier score rejection step relying on the standard deviation of the obtained scores to consider the agreement, as well as a saliency to weigh them based on their visual significance. Experiments on available 360-IQA databases show that our model outperforms the state of the art in terms of accuracy and generalization ability. This is valid for general deep-learning-based models, multichannel models, and natural scene statistic-based models. Furthermore, when compared to multichannel models, the computational complexity is significantly reduced. Finally, an extensive ablation study gives insights into the efficacy of each component of the proposed model. Full article

Virtual reality (VR) panoramic video is more expressive and experiential than traditional video. With the accelerated deployment of 5G networks, VR panoramic video has experienced explosive development. The large data volume and multi-viewport characteristics of VR panoramic videos make it more difficult to cache and transcode them in advance. Therefore, VR panoramic video services urgently need to provide powerful caching and computing power over the edge network. To address this problem, this paper establishes a hierarchical clustered mobile edge computing (MEC) network and develops a data perception-driven clustered-edge transmission model to meet the edge computing and caching capabilities required for VR panoramic video services. The joint optimization problem of caching and bitrate adaptation can be formulated as a Markov Decision Process (MDP). The federated deep reinforcement learning (FDRL) algorithm is proposed to solve the problem of caching and bitrate adaptation (called FDRL-CBA) for VR panoramic video services. The simulation results show that FDRL-CBA can outperform existing DRL-based methods in the same scenarios in terms of cache hit rate and quality of experience (QoE). In conclusion, this work developed a FDRL-CBA algorithm based on a data perception-driven clustered-edge transmission model, called Hierarchical Clustered MEC Networks. The proposed method can improve the performance of VR panoramic video services. Full article

Recently, the usage of 360-degree videos has prevailed in various sectors such as education, real estate, medical, entertainment and more. The development of the Virtual World “Metaverse” demanded a Virtual Reality (VR) environment with high immersion and a smooth user experience. However, various challenges are faced to provide real-time streaming due to the nature of high-resolution 360-degree videos such as high bandwidth requirement, high computing power and low delay tolerance. To overcome these challenges, streaming methods such as Dynamic Adaptive Streaming over HTTP (DASH), Tiling, Viewport-Adaptive and Machine Learning (ML) are discussed. Moreover, the superiorities of the development of 5G and 6G networks, Mobile Edge Computing (MEC) and Caching and the Information-Centric Network (ICN) approaches to optimize the 360-degree video streaming are elaborated. All of these methods strike to improve the Quality of Experience (QoE) and Quality of Service (QoS) of VR services. Next, the challenges faced in QoE modeling and the existing objective and subjective QoE assessment methods of 360-degree video are presented. Lastly, potential future research that utilizes and further improves the existing methods substantially is discussed. With the efforts of various research studies and industries and the gradual development of the network in recent years, a deep fake virtual world, “Metaverse” with high immersion and conducive for daily life working, learning and socializing are around the corner. Full article

In three-dimensional (3D) digital Earth environment, there are many problems when using the existing methods to express the ocean current, such as uneven distribution of seed points, density leap in scale change and messy visualization. In this paper, a new dynamic visualization method of multi-hierarchy flow field based on particle system is proposed; Specifically, three typical spherical uniform algorithms are studied and compared, and the streamline becoming denser from the equator to the poles on globe is eliminated by placing seed points using Marsaglia polar method as the most efficient. In addition, a viewport-adaptive adjustment algorithm is proposed, which realizes that the density of particles is always suitable to any viewing distance during continuous zooming. To solve the visual representation deficiency, we design a new dynamic pattern to enhance the expression and perception of current, which makes up for the shortcoming of the arrow glyph and streamline methods. Finally, a prototype of GPU parallel and viewport coherence is achieved, whose feasibility and effectiveness are verified by a series of experiments. The results show that our method can not only represent ocean current data clearly and efficiently, but also has outstanding uniformity and hierarchy effect. Full article

Scrolling is a frequently used Graphical User Interface widget that enables users to interact with a large amount of data using a limited viewport. However, if excessive data is included in the scroll, users are required to spend a substantial amount of time and effort to find the required information. In this paper, we present adaptive kinetic scrolling (AKS), a technique based on kinetic scrolling by which users can access target information more rapidly on mobile devices. Based on the user’s behavior, AKS detects situations when the user intends to access certain information that may be distant from the current viewport. At this point, AKS amplifies the speed of kinetic scrolling. Furthermore, the scrolling speed adapts according to the size of the remaining data to be scrolled. The more data that the scrolling widget contains, the more rapidly it scrolls so that the user can quickly reach the target. Kinetic scrolling is frequently used in scrolling widgets, and with AKS, users can save time and energy wasted on repetitive meaningless scrolling. We conducted a user study and verified that the proposed scrolling technique enables users to access target information more rapidly, particularly when there is a large dataset to navigate. Full article