Virtual Worlds, Volume 3, Issue 1 (March 2024) – 7 articles

Motion capture is a fundamental technique in the development of video games and in film production to animate a virtual character based on the movements of an actor, creating more realistic animations in a short amount of time. One of the ways to obtain this movement from an actor is to capture the motion of the player through an optical sensor to interact with the virtual world. However, during movement some parts of the human body can be occluded by others and there can be noise caused by difficulties in sensor capture, reducing the user experience. This work presents a solution to correct the motion capture errors from the Microsoft Kinect sensor or similar through a deep neural network (DNN) trained with a pre-processed dataset of poses offered by Carnegie Mellon University (CMU) Graphics Lab. A temporal filter is implemented to smooth the movement, given by a set of poses returned by the deep neural network. This system is implemented in Python with the TensorFlow application programming interface (API), which supports the machine learning techniques and the Unity game engine to visualize and interact with the obtained skeletons. The results are evaluated using the mean absolute error (MAE) metric where ground truth is available and with the feedback of 12 participants through a questionnaire for the Kinect data. Full article

Diminished reality (DR) is a technology in which a background image is overwritten on a real object to make it appear as if the object has been removed from real space. This paper presents a real-time DR application that employs deep learning. A DR application can remove objects inside a 3D region defined by a user in images captured using a smartphone. By specifying the 3D region containing the target object to be removed, DR can be realized for targets with various shapes and sizes, and the specified target can be removed even if the viewpoint changes. To achieve fast and accurate DR, a suitable network was employed based on the experimental results. Additionally, the loss function during the training process was improved to enhance completion accuracy. Then, the operation of the DR application at 10 fps was verified using a smartphone and a laptop computer. Full article

In this contribution, we propose a hybrid interaction technique that integrates near-field and object-space interaction techniques for manipulating objects at a distance in virtual reality (VR). The objective of the hybrid interaction technique was to seamlessly leverage the strengths of both the near-field and object-space manipulation techniques. We employed bimanual near-field metaphor with scaled replica (BMSR) as our near-field interaction technique, which enabled us to perform multilevel degrees-of-freedom (DoF) separation transformations, such as 1~3DoF translation, 1~3DoF uniform and anchored scaling, 1DoF and 3DoF rotation, and 6DoF simultaneous translation and rotation, with enhanced depth perception and fine motor control provided by near-field manipulation techniques. The object-space interaction technique we utilized was the classic Scaled HOMER, which is known to be effective and appropriate for coarse transformations in distant object manipulation. In a repeated measures within-subjects evaluation, we empirically evaluated the three interaction techniques for their accuracy, efficiency, and economy of movement in pick-and-place, docking, and tunneling tasks in VR. Our findings revealed that the near-field BMSR technique outperformed the object space Scaled HOMER technique in terms of accuracy and economy of movement, but the participants performed more slowly overall with BMSR. Additionally, our results revealed that the participants preferred to use the hybrid interaction technique, as it allowed them to switch and transition seamlessly between the constituent BMSR and Scaled HOMER interaction techniques, depending on the level of accuracy, precision and efficiency required. Full article

Background: Given that VR is used in multiple domains, understanding the effects of cybersickness on human cognition and motor skills and the factors contributing to cybersickness is becoming increasing important. This study aimed to explore the predictors of cybersickness and its interplay with cognitive and motor skills. Methods: 30 participants, 20–45 years old, completed the MSSQ and the CSQ-VR, and were immersed in VR. During immersion, they were exposed to a roller coaster ride. Before and after the ride, participants responded to the CSQ-VR and performed VR-based cognitive and psychomotor tasks. After the VR session, participants completed the CSQ-VR again. Results: Motion sickness susceptibility, during adulthood, was the most prominent predictor of cybersickness. Pupil dilation emerged as a significant predictor of cybersickness. Experience with videogaming was a significant predictor of cybersickness and cognitive/motor functions. Cybersickness negatively affected visuospatial working memory and psychomotor skills. Overall the intensity of cybersickness’s nausea and vestibular symptoms significantly decreased after removing the VR headset. Conclusions: In order of importance, motion sickness susceptibility and gaming experience are significant predictors of cybersickness. Pupil dilation appears to be a cybersickness biomarker. Cybersickness affects visuospatial working memory and psychomotor skills. Concerning user experience, cybersickness and its effects on performance should be examined during and not after immersion. Full article

3D audio spatializers for Virtual Reality (VR) can use the acoustic properties of the surfaces of a visualised game space to calculate a matching reverb. However, this approach could lead to reverbs that impair the tasks performed in such a space, such as listening to speech-based audio. Sound designers would then have to alter the room’s acoustic properties independently of its visualisation to improve speech intelligibility, causing audio-visual incongruency. As user expectation of simulated room acoustics regarding speech intelligibility in VR has not been studied, this study asked participants to rate the congruency of reverbs and their visualisations in 6-DoF VR while listening to speech-based audio. The participants compared unaltered, matching reverbs with sound-designed, mismatching reverbs. The latter feature improved D50s and reduced RT60s at the cost of lower audio-visual congruency. Results suggest participants preferred improved reverbs only when the unaltered reverbs had comparatively low D50s or excessive ringing. Otherwise, too dry or too reverberant reverbs were disliked. The range of expected RT60s depended on the surface visualisation. Differences in timbre between the reverbs may not affect preferences as strongly as shorter RT60s. Therefore, sound designers can intervene and prioritise speech intelligibility over audio-visual congruency in acoustically challenging game spaces. Full article

Designing virtual characters that are capable of reflecting a sense of personality is a key goal in research and applications in virtual reality and computer graphics. More and more research efforts are dedicated to investigating approaches to construct a diverse, equitable, and inclusive metaverse by infusing expressive personalities and styles into virtual avatars. While most previous work focused on exploring variations in virtual characters’ dynamic behaviors, characters’ visual appearance plays a crucial role in affecting their perceived personalities. This paper presents a series of experiments evaluating the effect of virtual characters’ outfits on their perceived personality. Based on the related psychology research conducted in the real world, we determined a set of outfit factors likely to reflect personality in virtual characters: color, design, and type. As a framework for our study, we used the “Big Five” personality model for evaluating personality traits. To test our hypothesis, we conducted three perceptual experiments to evaluate the outfit parameters’ contributions to the characters’ personality. In our first experiment, we studied the color factor by varying color hue, saturation, and value; in the second experiment, we evaluated the impact of different neckline, waistline, and sleeve designs; and in our third experiment, we examined the personality perception of five outfit types: professional, casual, fashionable, outdoor, and indoor. Significant results offer guidance to avatar designers on how to create virtual characters with specific personality profiles. We further conducted a verification test to extend the application of our findings to animated virtual characters in augmented reality (AR) and virtual reality (VR) settings. Results confirmed that our findings can be broadly applied to both static and animated virtual characters in VR and AR environments that are commonly used in games, entertainment, and social networking scenarios. Full article

Design discussion is crucial in the architectural design process. To enhance the spatial understanding of 3D space and discussion effectiveness, recently, some systems have been proposed to support design discussion interactively in an immersive virtual environment. The entire design discussion can be archived and potentially become course materials for future learners. In this paper, we propose an asynchronous VR exploration system that aims to help learners explore content effectively and efficiently anywhere and at any time. To improve effectiveness and efficiency, we also propose a summarization-to-detail approach with the application space by which students can observe the visualization of spatial summarization of actions and participants’ dwell time or the temporal distribution of dialogues and then locate the important or interesting region or dialogue for further exploration. To further explore the discussion content, students can call the preview to see the time-lapse animation of the object operation to understand the change in models or playback to view the discussion details. We conducted an exploratory user study with 10 participants to evaluate user experience, user impression, and effectiveness of learning the design discussion course content using our asynchronous VR design discussion content exploration system. The results indicate that the interactive VR exploration system presented can help learners study the design discussion content effectively. Participants also provided some positive feedback and confirmed the usefulness and value of the system presented. Our applications and lessons learned have implications for future asynchronous VR exploration systems, not only for architectural design discussion content, but also for other applications, such as industrial visual inspections and educational visualizations of design discussions. Full article