Next Article in Journal
Comparing and Contrasting Near-Field, Object Space, and a Novel Hybrid Interaction Technique for Distant Object Manipulation in VR
Previous Article in Journal
Speech Intelligibility versus Congruency: User Preferences of the Acoustics of Virtual Reality Game Spaces
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Cybersickness in Virtual Reality: The Role of Individual Differences, Its Effects on Cognitive Functions and Motor Skills, and Intensity Differences during and after Immersion

by
Panagiotis Kourtesis
1,2,3,*,
Agapi Papadopoulou
1 and
Petros Roussos
1
1
Department of Psychology, National and Kapodistrian University of Athens, 15772 Athens, Greece
2
Department of Psychology, The American College of Greece, 15342 Athens, Greece
3
Department of Psychology, University of Edinburgh, EH8 9YL Edinburgh, UK
*
Author to whom correspondence should be addressed.
Virtual Worlds 2024, 3(1), 62-93; https://doi.org/10.3390/virtualworlds3010004
Submission received: 26 October 2023 / Revised: 9 January 2024 / Accepted: 26 January 2024 / Published: 2 February 2024

Abstract

:
Background: Given that VR is used in multiple domains, understanding the effects of cybersickness on human cognition and motor skills and the factors contributing to cybersickness is becoming increasing important. This study aimed to explore the predictors of cybersickness and its interplay with cognitive and motor skills. Methods: 30 participants, 20–45 years old, completed the MSSQ and the CSQ-VR, and were immersed in VR. During immersion, they were exposed to a roller coaster ride. Before and after the ride, participants responded to the CSQ-VR and performed VR-based cognitive and psychomotor tasks. After the VR session, participants completed the CSQ-VR again. Results: Motion sickness susceptibility, during adulthood, was the most prominent predictor of cybersickness. Pupil dilation emerged as a significant predictor of cybersickness. Experience with videogaming was a significant predictor of cybersickness and cognitive/motor functions. Cybersickness negatively affected visuospatial working memory and psychomotor skills. Overall the intensity of cybersickness’s nausea and vestibular symptoms significantly decreased after removing the VR headset. Conclusions: In order of importance, motion sickness susceptibility and gaming experience are significant predictors of cybersickness. Pupil dilation appears to be a cybersickness biomarker. Cybersickness affects visuospatial working memory and psychomotor skills. Concerning user experience, cybersickness and its effects on performance should be examined during and not after immersion.

1. Introduction

Immersive virtual reality (VR) represents one of the most remarkable technological advancements of the 21st century. A digital interface that promises full immersion into an alternate or simulated environment, VR’s potential has been tapped across an array of disciplines. Entertainment industries have been early adopters, providing audiences with experiences that were once relegated to the realms of imagination [1,2,3]. Concurrently, the education sector has witnessed a paradigm shift, with VR-enhanced pedagogical tools fostering enriched learning experiences [4,5,6].
Furthermore, the domain of professional training has embraced VR to craft realistic scenarios for a myriad of professionals, from manual labourers mastering their craft to surgeons simulating complex procedures [7,8,9]. The medical area is reaping the benefits too. Beyond conventional treatments and therapies, VR is emerging as a powerful adjunctive tool. Pain management, once reliant solely on pharmacological interventions, is now exploring the pain-distracting potential of VR [10]. Rehabilitation, be it physical or neurological, is experiencing a renaissance with VR-infused therapies [11,12]. Neuropsychology, in particular, has found a robust partner in VR, aiding in cognitive assessments [13,14], training [15], and targeted rehabilitation efforts [16,17,18].
Such broad applications further extend their arms to vulnerable populations. The elderly, often considered tech-averse, find solace and cognitive rejuvenation in VR experiences tailored to their needs [19,20]. Individuals with Mild Cognitive Impairment (MCI) and/or a type of dementia [21,22], or developmental challenges such as Attention-Deficit Hyperactivity Disorder (ADHD) [23,24,25]) and Autism Spectrum Disorder (ASD) [17,26,27] are not mere spectators. VR interventions, designed with sensitivity and precision, are being administered to provide these populations with therapeutic as well as recreational relief. Considering the importance of the aforementioned applications of VR and the fragility of some of the targeted populations, the effective implementation of VR becomes imperative.
While the promise of VR in transforming various domains is indisputable, VR also harbours an inherent limitation—cybersickness, a condition affecting a segment of its users [28]. Manifesting as a triad of nausea, disorientation, and oculomotor disturbances, cybersickness remains a significant concern. While there is a temptation to draw parallels between cybersickness and simulator sickness, the two display distinct characteristics [29]. Notably, cybersickness presents with heightened general discomfort, particularly intensified by nausea and disorientation [29]. Adding another layer to this complex tapestry, cybersickness also stands apart from motion sickness. The former arises primarily from visual cues in VR, whereas the latter emerges from actual physical movement [30].
Delving into the underlying causes of cybersickness, a comprehensive theoretical understanding is still needed. The sensory conflict theory has gained traction, shedding light on the root of the issue [28,30,31]. According to the neural mismatch theory, an extension of the sensory conflict theory, the symptoms of cybersickness primarily arise from discrepancies between expected and sensed motion, with a significant involvement of the central nervous system (CNS) in the generation of cybersickness [28,31]. This theory suggests that cybersickness is not merely a result of sensorial conflict between the visual and vestibular systems but also involves the CNS’s interpretation of these conflicting cues. Our sense of balance and spatial orientation relies on the integration of visual, vestibular, and proprioceptive feedback. In VR environments, these systems often receive conflicting cues, leading to a neural mismatch that causes sensory dissonance and contributes to the onset of cybersickness. For VR, this conflict can be attributed to vection—an illusionary perception of motion [32]. This illusion, particularly when accompanied by movements like linear and angular accelerations, has been pinpointed as a major instigator of cybersickness in VR [33,34]. As VR continues its ascendancy in the tech world, the quest to understand and alleviate cybersickness remains a pressing concern.
Cybersickness in VR appears to be a result of both hardware and software issues, and poses a significant challenge. Hardware problems like latency and refresh rate discrepancies, alongside software issues such as inconsistencies in visual–vestibular integration, contribute to a disorienting VR experience [35]. The industry is addressing these through advancements in head-mounted displays (HMDs), enhanced display resolutions, and software that aligns more closely with human physiology [35,36]. Strategies to mitigate cybersickness symptoms include acclimatization, the use of medications or natural remedies, and innovations in Human–Computer Interactions, although these methods have limitations such as high costs, time consumption, or an impact on immersion [37,38,39,40,41].
The measurement of cybersickness has predominantly relied on questionnaires like the Simulator Sickness Questionnaire (SSQ) [42] and the Virtual Reality Sickness Questionnaire (VRSQ) [43], which have been criticized for their limited effectiveness [44,45,46,47]. The Cybersickness in Virtual Reality Questionnaire (CSQ-VR) is recognized as a more effective tool, especially with its integration of eye-tracking technology [45]. Physiological metrics like electroencephalography (EEG), electrocardiogram (ECG), and galvanic skin response (GSR) have also been used to predict cybersickness, but their widespread use is limited by costs and ergonomic issues [48,49,50]. Individual differences also influence cybersickness experiences, with mixed findings regarding the role of gender and gaming experience [51,52,53,54]. Furthermore, cybersickness can significantly impair cognitive and motor functions, which are crucial in VR’s application in education, research, and training [35,52,55]. Studies have shown varying impacts of cybersickness on these functions, indicating the need for further research in this area [51,56,57,58,59]. Overall, this research highlights the ongoing need for innovation and thorough assessment in mitigating and measuring cybersickness in VR environments.
The current study aims to address several key gaps in the literature regarding the impact of cybersickness on virtual reality (VR) environments, particularly focusing on its effects on cognitive functions and individual differences. The outlined hypotheses contribute to the literature in the following ways:
  • Pupil Size as a Predictor of Cybersickness Intensity (H1): This hypothesis explores the relationship between physiological responses, specifically pupil size, and the intensity of cybersickness. If confirmed, this would add to the understanding of how physiological markers can be used as reliable predictors of cybersickness, offering a more objective and potentially real-time method for assessing cybersickness severity.
  • The Link between Motion Sickness Susceptibility and Cybersickness (H2): By investigating the connection between a person’s susceptibility to motion sickness and their experience of cybersickness, this study could provide insights into individual differences in VR experiences. This would contribute to a more personalized approach toVR design and usage, especially for populations that are more prone to motion sickness.
  • The Impact of Computer and Video Game Experience (H3): This hypothesis examines the role of prior experience with computers and video games in predicting cybersickness. Clarifying this relationship could have implications for understanding how a familiarity with digital interfaces and interactive environments can influence one’s adaptation to VR, potentially guiding training protocols and user onboarding processes in VR applications.
  • The Effect of Cybersickness on Cognitive and Motor Skills (H4): Investigating the impact of cybersickness on key cognitive functions like verbal and visuospatial working memory, and psychomotor skills, directly addresses the discrepancies noted in previous studies about the extent of cybersickness’s impact. This could significantly enhance our understanding of the cognitive costs of VR usage, informing the design and application of VR in education, training, and rehabilitation.
  • The Comparison of Cybersickness Intensity During and After VR Exposure (H5): By comparing the intensity of cybersickness during and after VR exposure, this study aims to elucidate the temporal dynamics of cybersickness. This could lead to a better understanding of how cybersickness evolves over time, which is crucial for designing VR sessions and interventions to mitigate its effects.
In this paper, we delve into various aspects of cybersickness and its impact on users during virtual reality (VR) experiences. Section 2 presents a comprehensive literature review, examining existing research on cybersickness, including its causes, mitigation strategies, and the role of various hardware and software factors in VR. Section 3, ‘Materials and Methods’, details the VR hardware and software used, the development process of the virtual environment, and the methodologies for assessing cognitive and motor skills, as well as cybersickness in VR. In Section 4, ‘Results’, we present the findings from our study, including descriptive statistics, regression analysis results, and comparisons of cybersickness intensity during different stages of VR exposure. Section 5, ‘Discussion’, integrates our findings with the broader body of research, discussing the implications of our results and how they align with or contrast the existing literature. Finally, Section 6 concludes the paper, summarizing the key insights and contributions of our study to the understanding of cybersickness in VR environments.

2. Literature Review and Current Research Aims

2.1. Mitigation of Cybersickness

Two primary catalysts for cybersickness are VR’s hardware and software characteristics [35]. Hardware issues such as latency, which refers to the delay between a user’s action and the corresponding change in the virtual environment, can be incredibly disorienting. Similarly, refresh rates that are not synchronized with a user’s natural perception can lead to a jarring experience [35]. On the software side, inconsistencies in visual–vestibular integration, where what one sees does not match what one feels, can throw off the body’s equilibrium [35,36]. Similarly, increased cognitive workload and confusion may also play a role in modulating cybersickness intensity [35,36,60].
However, the industry and the scientific community are not passive in the face of these challenges. With advancements in head-mounted displays (HMDs), many of these hardware-related issues are gradually being addressed [35]. Better display resolutions, faster refresh rates, and improved motion tracking reduce the disconnect users feel. Simultaneously, on the software end, industry and research software developers are now more attuned to creating experiences that align with human physiology. Guidelines tailored to specific scientific fields and targeted populations are emerging, ensuring a more holistic and comfortable VR experience for all [36,61,62,63,64,65,66].
Furthermore, various strategies have been employed to counteract the symptoms of cybersickness, each with its own set of challenges. One such strategy is “acclimatization”, which involves frequently exposing a person to elements that induce cybersickness to help them build resilience against it [37]. While this might be an effective solution, it is both time-intensive and costly, demanding significant effort and dedication. Additionally, there are recommendations for using specific medications or natural remedies, such as ginger, to tackle the effects [38,67]. However, these methods can be obtrusive and might introduce unintended consequences like fatigue or potential allergies [38].
In the realm of Human–Computer Interactions (HCIs), innovations such as adjusting the user’s perspective [39], introducing dynamic focus shifts [40], ensuring bodily equilibrium [68], and incorporating brief intervals [41] have been put forward. However, these methods can impede certain virtual interactions (e.g., adjusting the user perspective and maintaining balance) and might detract from the overall immersive experience (e.g., focus shifts and intervals). Similarly, joyful and calming music effectively alleviates cybersickness symptoms in VR [51]. However, again, using music is only suitable in some virtual environments, given that it may confound the purpose of a VR application. An efficient approach, which may be a universal solution, is to detect and prevent cybersickness. However, this universal solution should be adaptive to the user’s needs and requirements.

2.2. Questionnaires and Physiological Metrics of Cybersickness

One of the most common approaches to measure cybersickness is the administration of questionnaires, typically before and after exposure to VR [36,52]. The effectiveness of these tools in evaluating cybersickness in VR settings has been a topic of keen interest. The Simulator Sickness Questionnaire (SSQ) [42] has received criticism from several studies for its inability to adequately measure cybersickness in VR [44,45,46,47]. While the Virtual Reality Sickness Questionnaire (VRSQ) was conceived of as an improvement on the SSQ, it too exhibits considerable limitations, especially in terms of its structure and specificity [43,45].
On the other hand, the Cybersickness in Virtual Reality Questionnaire (CSQ-VR) emerges as a more reliable and comprehensive tool [45,51,69,70]. Given that intense cybersickness can significantly hamper a user’s cognitive and motor functions, especially in terms of reaction times [34,45,51,56,57], it is imperative for a tool to detect these declines effectively. The CSQ-VR, with its robust design and metrics, compared to both SSQ and VRSQ, was substantially more efficient in detecting cognitive and/or motor skill declines due to cybersickness. Furthermore, the CSQ-VR is offered in a 3D VR version, allowing for a repeated evaluation of cybersickness while the user is immersed in VR. This is essential considering that the intensity of cybersickness can vary throughout a VR session [32,45,51,71]. Additionally, considering that pupil size was a significant predictor of cybersickness intensity [45,51] the integration of eye-tracking in the CSQ-VR further augments its cybersickness assessment capabilities.
Beyond questionnaires, and the pupil size discussed above, other objective physiological metrics have been used to examine and predict cybersickness symptomatology [30,48]. In this direction, electroencephalography (EEG), which captures electrical activity in the brain’s cortical areas, is an efficient approach to detect cybersickness [49]. However, using an EEG in combination with the VR HMD is not feasible for widespread utilization due to the high cost of EEG, as well as that it being ergonomically problematic to have both cumbersome devices mounted on the head of the user. Other physiological metrics encompass the electrocardiogram (ECG), electrogastrogram (EGG), electrooculogram (EOG), photoplethysmogram via pulse oximeter (PPG), breathing rate, and galvanic skin response (GSR), which have been found to predict cybersickness symptomatology and intensity [48,50]. For example, rises in bradygastric activity, breathing rate [50,72], heart rate [57], and forehead skin conductance [73,74] offer reliable indicators of cybersickness. However, beyond PPG and CSR, the other devices are also costly and not ergonomically appropriate for effective use. On the other hand, PPG and GSR can be embedded in haptic gloves (e.g., see TeslaGloves) and are ergonomically efficient [75,76]. Nevertheless, these haptic devices are costly, so, beyond their applications in industry by enterprises, these devices are not affordable for the general population [75,76]. Finally, pupil size is not the only eye-tracking metric that may indicate cybersickness. Other eye-tracking metrics, such as fixation duration and the distance between the eye gaze and the targeted object, may assist in predicting cybersickness [77,78,79]. Therefore, except for the eye-tracking that is embedded in many VR HMDs, the use of the rest of the physiological metrics is either ergonomically and/or financially problematic.

2.3. Individual Differences and Cybersickness

The experience of cybersickness varies among individuals, with factors like gender playing a potential role [51,52]. Some studies suggest female users may experience more intense cybersickness than male users, but findings have been inconsistent [51,52]. For instance, Petri et al.’s study [80] found no significant gender differences in objective metrics like heart rate but did observe a difference in subjective experiences based on the SSQ. Meanwhile, Melo et al. [53] found no such gender differences. Similarly, Stanney et al.’s experiments [54,81] found gender insignificant in predicting cybersickness in certain conditions. A meta-analysis also supported the absence of significant gender differences when evaluating cybersickness in VR settings using the SSQ [52]. However, it is speculated that gaming and VR experience might play a role.
Few studies have delved into the impact of computing, VR, or gaming experience on cybersickness. While Stanney et al. [54] considered gaming experience, their methodology regarding its measurement was ambiguous, and their results, postulating an absent effect, cannot thus be considered reliable. Kourtesis et al., in a series of studies [13,82,83], found no significant effect of gaming or VR experience on cybersickness. However, the VR software implemented in these studies was thoroughly designed and developed to elicit minimal to no cybersickness symptoms. Conversely, Weech et al. [84] found that gaming experience might influence cybersickness, especially when paired with narratives. Likewise, in the recent study of Kourtesis et al. [51] gaming experience was found to affect the experiencing of cybersickness, where more gaming experience indicated a higher resilience. Also, the same study showed that gaming experience explained the gender differences in terms of cybersickness, as participants of the opposite sex but with the same gaming experience did not experience different cybersickness intensities [51].
Moreover, each individual may demonstrate a diverse level of susceptibility to experiencing cybersickness [85,86]. Previous studies, using the Motion Sickness Susceptibility Questionnaire (MSSQ), which measures susceptibility to experiencing intense symptoms of motion sickness [87], found susceptibility to motion sickness to be associated with personality traits and anxiety levels [88,89]. Similar to motion sickness susceptibility, visually induced motion sickness (i.e., cybersickness induced by vection) demonstrates similar patterns of susceptibility among individuals [85,86]. Taking these together, given the similarities between motion sickness and cybersickness elicited by vection, the MSSQ scores may indicate susceptibility to cybersickness. However, in previous studies, the MSSQ did not predict cybersickness’s intensity [45,51]. Nevertheless, the aforementioned studies also used the MSSQ to exclude participants with moderate-to-high motion sickness susceptibility. Thus, the MSSQ’s utility in predicting cybersickness has not yet been examined appropriately. Overall, while individual differences influence cybersickness, the exact factors and their interplay remain a complex topic of investigation.

2.4. Effects of Cybersickness on Cognitive and Motor Skills

Apart from impacting the quality of the user experience in VR, cybersickness can also detrimentally influence a user’s cognitive and motor functions. Considering VR’s applications in areas demanding unimpaired cognitive and motor skills, like education, research, clinical settings, and training, cybersickness poses significant challenges to VR’s successful integration. Several systematic reviews [35,52,90] have highlighted a considerable, though temporary, decline in cognitive and/or motor functions due to cybersickness in immersive VR. Dahlman et al. [55] theorized that motion sickness notably impairs users’ verbal working memory. Similarly, a study by Varmaghani et al. [58], with 47 participants split into VR and control groups, found that the VR group did not experience the expected enhancement in visuospatial skills seen in the control group, suggesting cybersickness’s potential effect on visuospatial learning.
Mittelstaedt et al.’s research [56] explored cybersickness’s influence on various cognitive areas before and after VR exposure. The study revealed that cybersickness resulted in delayed reaction speeds and hindered the anticipated boost in visual processing speed, indicating a negative impact on attention and reaction time. However, spatial and visuospatial memory skills seemed unaffected. Similarly, studies by Nalivaiko et al. [57] and Nesbitt et al. [34] reported slowed reaction times correlated with increasing cybersickness severity, suggesting a link between cybersickness intensity and potential cognitive/motor deterioration. It is important to note that while these studies underscore cybersickness’s negative effects on cognitive and/or motor abilities, they assessed cybersickness post VR exposure, not during the experience itself.
Since the users experience a readjustment to physical space while removing the VR HMD and transitioning from the virtual to the physical environment [59], examining cybersickness, cognition, and motor skills after exposure may confound the observations. In a recent study examining cybersickness, and cognitive and motor skills during immersion, cybersickness had a significant negative effect on verbal working memory and psychomotor skills, but not on visuospatial working memory [51]. However, the order of the tasks was not randomized and counterbalanced, and the researchers, using the MSSQ, excluded participants who showed a high susceptibility to experiencing motion sickness. Thus, based on the studies mentioned above, while the evidence postulates a negative impact of cybersickness on cognitive and motor skills, there are still discrepancies, especially regarding the size of these effects.

2.5. The Current Study’s Aims and Hypotheses

Based on the review of existing literature, it is clear that VR is a cutting-edge tool with extensive applications spanning from entertainment to research and rehabilitation. However, the applications of VR may be hindered by the onset of cybersickness. There are discrepancies about the impact of cybersickness on core cognitive functions, which are required in the VR applications in several fields (e.g., education, occupational training, and rehabilitation). Also, the role of individual differences (e.g., IT skills, sex, and susceptibility to cybersickness) in experiencing cybersickness has not been fully understood. Given this background, our research inquiries have been articulated in the following hypotheses:
H1: 
Pupil size will be a significant predictor of the intensity of cybersickness.
H2: 
Susceptibility to motion sickness will be a significant predictor of the intensity of cybersickness.
H3: 
Computer and/or videogame experience will predict the intensity of cybersickness symptomatology.
H4: 
Cybersickness symptomatology will have a negative effect on verbal working memory, visuospatial working memory, and/or psychomotor skills.
H5: 
The cybersickness intensity during and after VR exposure will be significantly different.

3. Materials and Methods

3.1. Virtual Reality Hardware and Software

An HTC Vive Pro Eye was used, which embeds an eye-tracker with a binocular gaze data output frequency of 120 Hz (i.e., refresh rate), a tracking accuracy of 0.5°–1.1°, a 5-point calibration, and a 110° trackable field of view. The HTC Vive Pro Eye substantially surpasses the minimum hardware criteria for alleviating and/or avoiding cybersickness [35]. Thus, beyond facilitating the collection of eye-tracking metrics, its utilization further ensures that the linear and angular accelerations will be inducing cybersickness (see description below) in the virtual environment and not hardware inadequacies. Likewise, the software’s development was performed in line with the guidelines for VR software for research and clinical settings, which have been found efficient in substantially mitigating cybersickness symptomatology [14,63]. This further ensured the avoidance or decrease in the effects of software characteristics on the expression or intensity of cybersickness.

3.2. Virtual Environment Development

The virtual environment was based on the one used in our previous studies on cybersickness [45,51]. The Unity3D game engine was utilized to develop a virtual environment. Also, the SteamVR SDK aided in the creation of interactions. Given the potential influence of gaming experience on task performance [14], the virtual hands/gloves feature of the SteamVR SDK was integrated to enable straightforward interactions. Crucially, these interactions were designed to be intuitive, initiated by touching the object for selection and maintaining touch for confirmation, eliminating the need for button presses. The virtual gloves offered by SteamVR were neutral, not hinting at any particular gender or race, which helped mitigate any biases tied to these factors [91].
For audio instructions, Amazon Polly was used to generate neutral, realistic voice clips. To ensure clarity and user comprehension, instructions were offered in video, audio, and text formats, promoting smooth task execution. The SteamAudio plugin was employed for spatial audio effects, particularly feedback sounds. Eye-tracking and pupillometry, including monitoring pupil size, fixation counts, fixation duration, and the distance of the object from the eye, were facilitated through the SRapinal SDK. The bmlTUX SDK [92] enabled the randomization of experimental sequences, data exportation into a CSV format, and overall experimental management.

3.3. Roller Coaster Ride: Linear and Angular Accelerations

The design of the roller coaster ride in this research was modelled after the rides used in our prior cybersickness studies [45,51]. A 12 min ride was designed, in which each participant had to undergo the experience of linear and angular accelerations throughout the ride. The trajectory was animated to represent the moving platform the user stood on (see Figure 1). The general direction of movement was forward, with an exception towards the end (see the reverse z-axis). Overall, the platform’s motions mimicked the dynamics of a roller coaster.
The designed route encompassed a specific sequence of accelerations: (1) linear on the z-axis, (2) angular, involving the z- and y-axes, (3) angular, encompassing the z-, x-, and y-axes, (4) angular on the roll axis, (5) heightened linear acceleration on the z-axis, (6) angular on the yaw axis, and (7) intense linear acceleration involving the y-axis followed by an inverse on the z-axis. The environment was kept minimalistic, predominantly in shades of black and white (as illustrated in Figure 1). This design was selected to minimize extraneous factors that could influence or elicit cybersickness symptoms. Moreover, the tiled pattern provided visual references, helping users discern changes in direction and altitude. For further information please also see the video abstract of this article, which is available in the Supplementary Materials.

3.4. Cognitive and Motor Skills Assessments in Virtual Reality

Due to the need for repeated evaluations of cybersickness, cognition, and motor abilities, while users were immersed in VR, immersive VR iterations of recognized tasks/tests were used (see Figure 2). These VR cognitive and motor skills tests have been used in our previous studies [45,51]. In designing these VR-based cognitive and psychomotor tasks, we adhered to the specific design principles and development guidelines for cognitive evaluations in immersive VR as outlined in [14,63]. Also, given that these tasks necessitate physical actions, their design was aligned with the ISO 9241-400:2007 standards from the International Organization for Standardization, which focus on the ergonomics of human–system interactions [93]. This approach, which includes considerations like personalizing object heights and the distance between the shoulder and the object, has been used in a prior study [94]. For further information please also see the video abstract of this article.

3.4.1. Verbal Short/Working Memory: Digit Span Test

A VR version of the Digit Span Test (DST) [95] was developed and used (see Figure 2). The test encompasses two tasks, the forward and backward recall tasks, which examine verbal short-term memory and verbal working memory, respectively [96]. The order of the administration of the tasks is standardized, where examinees perform first the forward recall task, and then the backward recall task [95,96]. In the DST, participants were presented with a sequence of numbers to listen to, which they were then required to recall in the same (i.e., forward recall task) and reverse order (i.e., backward recall task). For instance, if the given sequence was 2, 4, 3, they were to correspondingly respond with 2, 4, 3 (i.e., forward recall task); or 3, 4, 2 (i.e., backward recall task). Post listening, a virtual keypad materialized for the participants to input their answers. To select a digit, they touched the corresponding white box on the keypad (see Figure 2). This box turned blue upon touch. A continuous touch for more than a second provided confirmation of their selection. Correct selections turned the button green accompanied by an affirmative sound, while incorrect ones made it turn orange, with a disapproving sound. A trial concluded either upon an error or once the sequence was correctly inputted. With every other correct trial, the sequence length increased. The task terminated after two consecutive errors at the same sequence length (e.g., three digits) or after completing the maximum sequence length (two trials of 7 digits). The total score combined the count of correctly completed trials and the longest digit sequence (i.e., digit span). A video description of the test can be accessed via the following link: https://www.youtube.com/watch?v=1H8cqci-lFs accessed on 1 January 2024).

3.4.2. Visuospatial Short and Working Memory: The Corsi Block Test

A VR version of the Corsi Block Test (CBT) [97] was developed and employed (see Figure 2). Comparable to the DST described above, the CBT incorporates a forward recall task and a backward recall task, which evaluate visuospatial short-term memory and visuospatial working memory correspondingly [97,98]. The sequence in which tasks are performed is also standardized: participants first perform the forward recall task and then conclude with the backward recall task [97,98]. Each task displays 27 white cubes, positioned distinctively across the x, y, and z dimensions. However, 9 of these 27 cubes were randomly showcased to participants in each trial (see Figure 2). Each trial began with nine cubes. Based on the current sequence length, a subset of these cubes would light up in blue for a second each, accompanied by a bell sound. Once this sequence finished, participants needed to recall and select the cubes in the same (i.e., forward recall task) or reverse order (i.e., backward recall task). A cube was selected by touching it and turning it blue. Maintaining the touch for a second confirmed the choice: the cube turned green and sounded a positive tone if correct, or orange with a negative tone if wrong. The trial ended upon a mistake or once the full normal (i.e., forward recall task) or reversed (i.e., backward recall task) sequence was recalled correctly. Trials started with sequences of two cubes, increasing by one if at least one of the two attempts was correct. The task concluded after two errors at a specific sequence length or reaching the maximum sequence of seven cubes. Perfect performance meant achieving up to seven cubes without errors. The total score was the sum of the highest achieved sequence length (i.e., Corsi block span) and the total number of correctly recalled sequences. A video description of the test can be accessed via the following link: https://www.youtube.com/watch?v=MLilvkyMt-g (accessed on 1 January 2024).

3.4.3. Psychomotor Skills: Deary–Liewald Reaction Time Test

A VR version of the Deary–Liewald Reaction Time test (DLRT) [99] test was developed and used to evaluate psychomotor skills. The DLRT incorporates two tasks: a simple reaction time (SRT) task and a choice reaction time (CRT) task [99]. In the SRT task, participants watched a white box, which they had to quickly touch whenever it turned blue (see Figure 2). This was repeated for 20 trials. For the CRT task, any one of four horizontally aligned boxes would randomly turn blue, prompting participants to touch it as fast as they could (see Figure 2). This occurred over 40 trials. In both tasks, participants were directed to touch the highlighted boxes swiftly using either hand. Before starting, a practice round ensured that participants grasped the instructions. The SRT score was derived by averaging the reaction times across the 20 trials, as was the CRT score. However, the CRT provided three separate scores. Using eye-tracking, we gauged the time to notice the target (i.e., attentional time) and the time between noticing and touching the target (i.e., Motor Time). An overall reaction time, from target appearance to touch, was also recorded. Thus, three scores emerged:
  • Reaction Time (RT), reflecting overall psychomotor speed.
  • Attentional Time (AT), showing attention processing speed.
  • Motor Time (MT), representing movement speed.
A video description of the test can be accessed via the following link: https://www.youtube.com/watch?v=wXdrt0PjNsk (accessed on 1 January 2024).

3.5. Cybersickness in Virtual Reality Questionnaire (CSQ-VR)

To assess the symptoms and intensity of cybersickness, the CSQ-VR was used, which is a valid tool for assessing cybersickness and has shown superior psychometric properties to the SSQ and the VRSQ [45]. Also, originating from the VR-Induced Symptoms and Effects section of the VR Neuroscience Questionnaire, the CSQ-VR boasts strong structural and construct validity [36]. Its strengths include a concise format (just six questions) and the generation of comprehensible results [45,70]. Additionally, it captures various cybersickness subcategories, such as nausea, disorientation, and oculomotor disturbances. Each category has two questions, scaled on a 7-point Likert scale, with options ranging from “1—absent feeling” to “7—extreme feeling”—each option combines text and a number (see Figure 3). From the CSQ-VR, a total score and three subscores (i.e., one for each subcategory: nausea, disorientation, and oculomotor symptoms) can be derived. The total score is the summation of the three subscores. The questions and the scoring of the CSQ-VR can be seen on the actual questionnaire, which is available in the Supplementary Materials. The paper-and-pencil version of the CSQ-VR was administered twice, before and after exposure to VR.
Given the study’s goal of repeatedly measuring cybersickness during VR immersion, the 3D VR version of the CSQ-VR was implemented. The question appeared at the top of the designed user interface, with the chosen response (in red) situated centrally. Users could modify their answers using a slider, by selecting a number directly or sliding along the scale (see Figure 3). The VR version of the CSQ-VR also incorporates eye-tracking metrics. Invisible tracking markers were positioned ahead of the text, and their dimensions always matched the visible text per line (see Figure 3). This setup allows us to gauge the duration of fixation on the text as an indicator of reading rate. Additionally, continuous pupil measurement occurred while users engaged with the CSQ-VR, allowing us to determine the average pupil size (for both eyes) during interactions, which has been previously seen as a biomarker of cybersickness intensity [45,51]. A video description of the 3D VR version of the CSQ-VR can be accessed via the following link: https://www.youtube.com/watch?v=npW4NKNLXok (accessed on 1 January 2024).

3.6. Demographics and Motion Sickness Susceptibility Questionnaire (MSSQ)

A custom questionnaire was used to collect demographic information, such as gender, age, education, and computer and videogame skills. Smartphone/computing/gaming experiences were determined by summing up scores from two specific questions, each based on a 6-point Likert scale. The initial question gauged the participant’s proficiency or skill level in using smartphone/computer/games, with ratings like ‘5: highly skilled’. In contrast, the subsequent question focused on how often they engage with these platforms, with responses such as ‘4: once a week’. This custom questionnaire is the same one that has been used in previous studies (e.g., [45,51,94]). Also, the short form of the Motion Sickness Susceptibility Questionnaire (MSSQ) was used to measure participants’ predisposition to experience motion sickness [87]. The MSSQ serves as a diagnostic instrument designed to gauge an individual’s vulnerability to motion sickness and pinpoint specific triggers associated with the onset of the symptoms. It delineates experiences into two distinct categories:
  • Childhood Experience (prior to the age of 12): here, respondents specify the frequency with which they encountered sensations of sickness or nausea in different modes of transport or during specific entertainment scenarios.
  • Experience over the Last 10 Years: this section requires individuals to recount the number of times they felt symptoms of sickness or nausea under similar circumstances within the past decade.
Each section receives an independent score, and the cumulative result from both areas offers the raw MSSQ score. This raw score can be translated into a percentile through reference tables or a dedicated polynomial for enhanced interpretability. Thus, three scores were derived from the MSSQ: the MSA-Child, the MSB-Adult, and the MSSQ-Total. However, it is pivotal to understand that the term “sickness” within the MSSQ’s framework encapsulates feelings ranging from mere queasiness to outright nausea or even vomiting. Notably, the MSSQ has clinical significance as it can indicate inherent motion sickness susceptibility, which is especially valuable for those diagnosed with vestibular diseases, as well as examining susceptibility to visually induced cybersickness in VR [85,86,87,88,89].

3.7. Participants and Experimental Procedures

Participants were recruited through convenience sampling, utilizing the internal mailing lists of the National and Kapodistrian University of Athens, as well as promotions on social media platforms. The research received approval from the Ethics Committee of the Department of Psychology of the National and Kapodistrian University of Athens. The sample consisted of 30 participants, 17 women and 13 men, aged 20 to 45 years old. All participants had normal or corrected vision using contact lenses or glasses.
In our study, participants stood while wearing the VR headset, as standing was necessary to ensure they experienced the vestibular symptoms integral to our research. We took several measures to ensure participant safety during the experiment:
  • Supervision by an Experimenter: each participant was closely monitored by an experienced experimenter throughout the VR session to immediately address any signs of discomfort or loss of balance.
  • Design of the VR Area: The VR area was specifically designed with health and safety considerations in mind. We ensured that the space was clear of obstacles, had soft padding on the floor, and was of sufficient size to allow free movement without the risk of collisions.
  • Use of Safety Equipment: we also used additional safety equipment, such as non-slip mats, to further minimize the risk of falls or other injuries.
Upon arrival, all participants were informed about the procedure and gave their informed consent in written form before proceeding with the experimental procedures. The process began with the completion of the demographic data questionnaire and the questions of the MSSQ questionnaire. Then, before using the VR HMD, the participants were asked to complete the CSQ-VR questionnaire in paper-and-pencil form. Only then did participants receive an induction to VR technology and how to use and wear the HTC Vive Pro Eye. The induction also described the cognitive and motor tasks they had to perform in VR. After the induction, under the guidance of the experimenter, the participants wore the VR HMD and performed the eye-tracking calibration offered by the HTC Vive in collaboration with SteamVR. Once in VR, they stood on a designated spot marked ‘X’ (see Figure 1). The session began with tutorials. For each task, a video tutorial, along with verbal and written guidelines, was provided.
During immersion, the first task participants had to perform was to provide their responses to the questions of the 3D VR version of the CSQ-VR. After that, the participants had to perform the VR versions of the cognitive (i.e., DST and CBT) and psychomotor skills (i.e., DLRT) tests (see Figure 2). After going through the tutorial of each task, where participants received audiovisual (i.e., audio, video, and label) information about the task, they then performed the corresponding task. This stage of going through the tutorials and performing all the tasks served as the baseline assessment, which lasted approximately 25 min. Note that the order of the tests was counterbalanced across participants. Given that the tests were DST, CBT, and DLRT, three different orders were used:
  • DST, CBT, and DLRT.
  • CBT, DLRT and DST.
  • DLRT, DST, and CBT.
Thus, the full circle of orders was complete for every three participants. Across the whole sample, the circle of orders was completed ten times (i.e., 30 participants divided by three orders).
The roller coaster ride followed, which lasted about 12 min, after this baseline assessment. After the ride, participants performed the same set of assessments as the baseline (i.e., CSQ-VR, DST, CBT, and DLRT). Note that the order of the administration of the tests was identical to the baseline assessment. Participants thus underwent one ride of 12 min and two assessment sessions (i.e., before and after the ride). The entire VR procedure spanned roughly 60 min for every participant. Finally, after removing the VR HMD (i.e., post VR exposure), participants completed a paper-and-pencil version of the CSQ-VR questionnaire. Then, refreshments rich in electrolytes were offered to the participants, and they were given a 10–15 min rest before departure. Before leaving, participants were advised against driving or operating heavy machinery for the rest of the day.

3.8. Statistical Analyses

All the statistical analyses were conducted using R language [100] in RStudio [101]. Furthermore, the psych (correlational analyses and t-tests) [102], the ggplot2 (plots) [103], and the lme4 (regression analyses) [104] R packages were used to perform their respective analyses. Descriptive statistical analysis was performed to provide an overall review of the sample. The paired-samples t-test examined the differences in the intensity of cybersickness during and after exposure to VR. Finally, multiple linear regression analyses were performed to examine the predictors of cybersickness symptomatology, and multiple mixed linear regression model analyses were performed to examine the predictors of performance on cognitive and psychomotor skills tasks. As the variables violated the condition of normality, the bestNormalize R package [105] was used to transform (e.g., into logarithms) and centre the data (i.e., converting them into z-scores). In this way, there was a normal distribution in the data to conduct parametric statistical analyses.

Regression Analyses

The assumption of normality for event- and time-based scores was evaluated using the Shapiro–Wilk Normality test. The Non-Constant Error Variance test was applied to verify the homoscedasticity assumption for the models. Multicollinearity was assessed by calculating the variance inflation factor for the predictors in each model. Linear regression analyses were utilized to explore the predictors of everyday PM functioning based on everyday cognitive functions. Model comparisons were made using analyses of variance. The comparison criteria among the models included Akaike’s information criterion (AIC), Bayesian criterion (BIC), the F statistic and its significance level, and the proportion of variance explained (i.e., R2).
In the analytical approach adopted, a wide array of variables was considered as potential predictors for the models. Specifically:
For the Linear Multiple Regression analyses, the following variables were considered: sex, education, age, computing experience, smartphone app experience, gaming experience, MSA-Child, MSB-Adult, MSSQ Total, pupil size while responding to the CSQ-VR, and pupil size during the ride.
For the Mixed Model Regression analyses (i.e., prediction of performance on cognitive and psychomotor skills tasks) the following variables were considered: CSQ-VR Total Score, CSQ-VR Nausea Score, CSQ-VR Vestibular Score, CSQ-VR Oculomotor Score, sex, education, age, computing experience, smartphone app experience, and gaming experience.
With these variables as the foundation, a systematic and incremental model development process was initiated:
Single-Predictor Models: Initially, separate models were developed. A single cognitive predictor from the list was incorporated into each of these models. When the performance of these models was compared, the individual variable that yielded the best preliminary results was identified.
Dyadic Predictor Models: A new set of models was crafted in the next phase, each containing two predictors. The top-performing variable from the first step was consistently used as one of the predictors in these models. The second predictor was drawn sequentially from the remaining list of variables. After the two-predictor models had been designed, their performances were critically evaluated. The top-performing two-predictor model was then juxtaposed with the best model from the first step.
Subsequent Iterations in the Incremental Approach: The methodology was characterized by its iterative nature. In each subsequent phase, the best predictors from the previous step were retained, and an additional predictor from the list was introduced. With each iteration, the models were rendered more complex. This step-by-step comparison continued until a point was reached where the introduction of more variables failed to enhance the model’s performance significantly. When a model from an earlier step outperformed a more complex model from a subsequent step, it indicated that the optimal combination of predictors had been identified, and the simpler model from the previous step was chosen as the final best model. This rigorous process ensured that the final model was robust and represented the best combination of the variables initially considered.

4. Results

Descriptive statistics of the data are presented in Table 1. Demographically, participants were relatively young adults with a wide range of educational backgrounds. Their technology engagement was evident from the experience they had with computing, smartphones, and gaming. This provides an interesting lens through which to understand the effects of VR, given their wide range of familiarity with digital tools. Regarding motion sickness, participants displayed varied susceptibility. Their scores from childhood to adulthood in motion sickness susceptibility showed a notable shift, suggesting that reactions to motion may evolve with age. When diving into the cybersickness, as assessed by the CSQ-VR, it seemed to intensify post VR exposure across all its subcategories: nausea, vestibular, and oculomotor symptoms.

4.1. Linear Regression Analyses: Prediction of Cybersickness Intensity

Linear regression analyses were conducted to detect and compare the significant predictors of overall, and per symptom category, cybersickness intensity. Table 2 elucidates how various individual factors predict overall cybersickness. In support of H1, the pupil size while responding to the CSQ-VR questionnaire and during the VR ride demonstrated significant cybersickness prediction. In agreement with H2, the most significant predictor is the motion sickness susceptibility score as an adult (MSB-Adult), with an impressive 39% variance (R2 = 0.39) accounted for. Finally, in line with H3, smartphone experience and gaming experience were notable for their significance in predicting cybersickness.
Table 3 provides insights specific to nausea symptoms. Once again, in line with H1, pupil size, both during CSQ-VR reading and the VR ride, was a significant predictor of nausea symptomatology. Notably, in agreement with H2, the MSB-Adult stands out as the most potent predictor. The child motion sickness score (MSA-Child) and the total motion sickness susceptibility (MSSQ Total) were also significant, hinting that one’s propensity to motion sickness earlier in life could have repercussions in a virtual environment. Finally, in support of H3, experience with using smartphones and playing videogames were significant predictors of the intensity of nausea symptoms.
In Table 4, the focus shifts to vestibular symptoms. While some predictors overlap with those for nausea, in disagreement with H1, the pupil size did not significantly predict vestibular symptoms. However, the MSB-Adult was found to be a robust predictor of nausea, further confirming H2. Finally, it is intriguing to see that smartphone and gaming experience are significant predictors of vestibular symptomatology, which agrees with H3 and implies a potential link between technological experience and experiencing vestibular symptoms in a VR.
Table 5 centres on oculomotor symptoms. Unlike previous findings and discrepantly to H1–H3, no single predictor emerges as significant. The table largely communicates that more conventional metrics (age, education, tech experience) do not have significant associations with oculomotor symptoms in VR. This highlights the complexities of predicting these specific symptoms.
Table 6 aggregates the best models for predicting various aspects of cybersickness. This table reemphasizes the overarching role of the MSB-Adult in determining cybersickness, nausea, and vestibular symptoms. This connotes that the susceptibility to experiencing motion sickness as an adult is robustly associated with experiencing visually induced cybersickness symptoms in VR. Interestingly, no predictors were identified for oculomotor symptoms, indicating a potential gap in our understanding or the need for more refined measures.
In essence, these results underscore the multifaceted nature of cybersickness and its determinants. While some predictors like the MSB-Adult consistently emerge as influential, others show symptom-specific associations. Furthermore, they accentuate the importance of not only considering the user’s history and demographics but also real-time metrics like pupil size in understanding their VR experience.

4.2. Mixed Model Regression Analyses: Effects on Cognitive and Motor Performance

Mixed linear regression model analyses were carried out to determine and evaluate the significant predictors of performance on cognitive and psychomotor skills tasks. Table 7 and Table 8 delve into the predictors for verbal short-term and working memory, respectively. In disagreement with H4, the critical observation here is that none of the predictors were significant predictors of verbal short-term and working memory. This might suggest that verbal memory is less susceptible to variations in these predictors, or that other unmeasured factors may have a more substantial impact.
Table 9 elucidates the predictors of visuospatial short-term memory. In contrast with H4, none of the cybersickness measurements was found to be a significant predictor of performance. However, it reveals some compelling findings. Sex, computing experience, and gaming experience were shown as significant predictors of visuospatial short-term memory. Gaming experience, in particular, accounts for a substantial 19% of the variance.
Table 10 examines predictors for visuospatial working memory. Supporting H4, the vestibular symptoms’ intensity was found to be a significant predictor of visuospatial working memory, although other cybersickness metrics did not substantially predict it. Furthermore, sex and gaming experience stand out as significant predictors, indicating that gaming might shape how individuals process and manipulate visual–spatial information.
Table 11, on attentional time, displays that the CSQ-VR—specifically the oculomotor component—has a notable relationship with attentional time. The positive β coefficient suggests a direct correlation, meaning that as cybersickness symptoms increase, attentional time might also increase.
Table 12 targets motor time. Here, gaming experience and the CSQ-VR–Nausea are prominent predictors. This implies that one’s gaming experience might influence motor response times, and individuals experiencing nausea-related cybersickness symptoms might exhibit changes in their motor time.
In Table 13, the focus is on reaction time (i.e., psychomotor skills). In full support of H4, every cybersickness metric was a significant predictor of psychomotor skills, postulating that overall cybersickness, nausea, vestibular, and oculomotor symptoms play a role in determining reaction times. Interestingly, gaming experience also emerges as a significant negative predictor, likely hinting that frequent gamers might have quicker reaction times.
Lastly, Table 14 consolidates the most impactful predictors for various cognitive and motor skills. A recurring theme here is the influence of gaming experience on cognitive and motor skills, emphasizing its potential cognitive benefits or the development of specific skills associated with gaming. Interestingly, overall cybersickness and its vestibular and oculomotor components were also included in the best models, postulating the negative effects of cybersickness on cognitive functioning and psychomotor skills.
In summary, these results underscore the potential cognitive and motor influences of digital experiences like gaming. They also hint at the intertwined nature of cybersickness symptoms and cognitive/motor skills, suggesting that our experience in virtual environments can have multifaceted impacts on our cognitive functioning.

4.3. Comparison of Cybersickness during and after Exposure to Virtual Reality

Paired-sample t-test analyses were performed to examine whether there was a difference in terms of cybersickness symptomatology during VR immersion and after exposure to VR (i.e., immediately after the removal of VR equipment). Figure 4 illustrates the z-scores, representing the intensity of the overall and various cybersickness symptoms experienced by participants both during their immersion in VR and after their exposure to VR (i.e., after the removal of the VR headset). In line with H5, the overall cybersickness intensity was found to have a significant and large decrease after the removal of the VR headset (i.e., re-adaptation to the physical world), t(29) = 3.59, p < 0.001, Hedges g = 0.64. Furthermore, other cybersickness symptoms showed significant decreases after removing the VR headset and transitioning from the virtual to the physical environment, further supporting H5.
For vestibular symptoms, after exposure to VR (Post-VR), a noticeable shift towards negative z-scores was identified, indicating that the experience of vestibular symptoms was less intense. This difference between the two stages was confirmed to be statistically significant and large, t(29) = 2.74, p = 0.010, Hedges g = 0.49. Regarding nausea, post-VR, a slightly broader spread of z-scores, predominantly on the negative side, was detected. This decrease in the nausea symptoms’ intensity was deemed significant and large, t(29) = 3.12, p < 0.001, Hedges g = 0.56. Finally, for oculomotor symptoms, after exposure to VR, the distribution pattern of the z-scores remained largely unchanged, t(29) = 1.22, p = 0.230, Hedges g = 0.22, indicating that the removal of the headset and re-adaptation to the physical world had a non-significant effect on the oculomotor symptoms’ intensity. To summarize, of the three cybersickness symptom categories examined, the vestibular and nausea symptoms exhibited a statistically significant decrease after headset removal (i.e., post VR exposure). It was determined, however, that the oculomotor symptoms’ intensity remained relatively stable after exposure to VR.

5. Discussion

Considering that VR is implemented in educational, professional, research, and clinical settings, the present study aimed to examine cybersickness symptoms during immersion in VR. The study explored the role of several factors pertinent to individual differences, such as motion sickness susceptibility, experience with playing videogames, experience using computers, sex, and age, in the experiencing of cybersickness symptomatology and its intensity. For predicting cybersickness overall, and the symptoms’ intensities, eye-tracking metrics, such as pupil dilation, were also considered. Furthermore, the study aimed to determine the effects of cybersickness symptoms on cognitive functions and motor skills. Finally, given that users experience a readjustment to physical space while removing the VR HMD and transitioning from the virtual to the physical environment, the current study also examined the differences in cybersickness intensity during and after immersion. A comprehensive discussion is offered by integrating findings from the current study with the insights from the reviewed literature.

5.1. Pupil Dilation as a Biomarker of Cybersickness

Pupil dilation has been previously seen as a biomarker of an affective state, where a bigger size indicates a positive (e.g., joy) and a smaller size indicates a negative affective state (e.g., fear) [106]. Our results demonstrated a comparable pattern in cybersickness, where more intense cybersickness induced a smaller diameter (i.e., more negative affective state), and less intense cybersickness a bigger diameter of the pupils (i.e., more positive affective state). In VR, pupil size has been previously incorporated into a deep fusion model to predict cybersickness [79]. However, this previous study did not assess pupil size’s association, predictive capacity, or role within this model, making it challenging to determine whether pupil size acts as a biomarker for cybersickness. However, in our previous studies, evidence was offered that pupil dilation while reading the questions of the CSQ-VR substantially predicts cybersickness intensity [45,51]. In line with all the studies mentioned above, pupil dilation was found to be a significant predictor of cybersickness in this study. However, the pupil dilation was measured during reading of the CSQ-VR questions, as well as during the ride. The pupil size during the ride was deemed a significant predictor of overall cybersickness and nausea symptoms, while the pupil size while responding to the CSQ-VR significantly predicted only overall cybersickness. Also, for overall cybersickness intensity, pupil dilation during the ride was a substantially better predictor than pupil dilation during the CSQ-VR. This suggests that pupil size is a more reliable biomarker during the triggering and experience of cybersickness than post exposure to stimuli eliciting symptomatology.

5.2. Modulators of Cybersickness: Sex, Smartphone Experience, and Videogame Experience

None of the demographics appeared to be a significant predictor of cybersickness intensity. Notably, age was not a significant predictor of cybersickness. Given the younger demographic of the participants, it is worth investigating these effects in a broader age range. Also, experience using computers failed to predict cybersickness. Finally, in disagreement with previous studies [54,107], sex (i.e., male, female) did not predict cybersickness’s intensity or symptoms, postulating that cybersickness is not more frequent or intense in either sex. Interestingly, Stanney et al. [54] proposed that the variations stemmed from the VR headset’s InterPupillary Distance (IPD). In their subsequent experiment, no disparities between the two sexes were observed when participants deemed the IPD agreeable. In our research, the HTC Vive Pro Eye was employed, known for its universal comfort. Each participant underwent an eye-tracking calibration to fine-tune the IPD, ensuring its appropriateness and comfort. Also, in our previous study [51], the differences between sexes in terms of cybersickness were eliminated when we controlled for gaming experience (i.e., male and female users with the same experience playing videogames). In this study, a balanced sample was attempted, with both female and male participants with a comparable level of (e.g., high/low) gaming experience. Therefore, the calibration of the IPD via eye-tracking, as well as the balance in terms of gaming experience among participants, may explain the non-significant effect of sex on experiencing cybersickness. In combination, the findings of this study and our previous study [51], along with the findings of Stanney et al., [54] suggest that the sex/gender of the participant does not modulate the intensity of their cybersickness symptomatology.
On the contrary, experience with playing videogames and experience using apps on a smartphone, for improving everyday functionality, significantly predicted the intensity of the perceived cybersickness symptoms. Notably, our study is the first to show that experience with smartphones may be a predictor of cybersickness. Specifically, more experience with smartphones was associated with a substantially lower intensity of cybersickness symptomatology. Interestingly, the usage of smartphones for performing various tasks (e.g., surfing on the internet, sending emails, writing documents, editing photos and videos, etc.), which traditionally were performed on a computer, is currently significantly higher compared to the usage of computers/laptops for performing the same tasks [108]. Note that visually induced cybersickness can be elicited by exposure to any screen, including smartphones [109,110,111]. Also, exposure to and experience with tech mediums with screens is associated with cultivating tolerance towards experiencing cybersickness symptomatology due to the user adapting to their spatial requirements and motion [32]. Thus, the finding of this study regarding smartphone experience for developing a resilience to cybersickness is aligned with the relevant literature.
In the same direction, the current study showed that more experience with playing videogames predicts a higher resilience to cybersickness. This finding corroborates with the findings of our previous studies [45,51] and a large-scale study by Weech et al., [84], as well as other studies on cybersickness [112] and simulator sickness [113]. Therefore, a more significant gaming background seems to act as a protective factor against cybersickness, whereas a limited one might heighten vulnerability. Notably, similar to our previous studies on cybersickness [45,51], in this study, experience with playing videogames considered both proficiency and frequency, which appears to be a refined measure of gaming experience. However, videogames are clustered under several diverse genres (e.g., action, first-person shooting, role-playing, etc.), and each one may modulate the physiological and biochemical state [114], as well as cognitive functioning and brain structure [115], in a significantly different way. Thus, further research is required for examining the effects of gaming on cybersickness, through considering experience with playing games of each genre as distinct metrics.

5.3. Cybersickness and Susceptibility to Motion Sickness

Susceptibility to motion sickness during adulthood (i.e., the MSB-Adult score of the MSSQ) emerged as the best predictor of cybersickness intensity. Except for oculomotor symptoms, the MSB-Adult was a significant predictor of overall cybersickness, nausea symptoms, and vestibular symptoms. Notably, the single-predictor models with the MSB-Adult (i.e., with only the MSB-Adult as a predictor) were found to be the best regression models for predicting the respective scores of the CSQ-VR. Considering that motion sickness and cybersickness induced by vection share common characteristics, such as motion cues acting as elicitors of sickness, it comes with no surprise that the findings of previous studies postulated that visually induced cybersickness and motion sickness demonstrate similar patterns of susceptibility among individuals [85,86]. The findings of this study agree with the findings of the aforementioned studies, since the MSB-Adult was the most prominent predictor of overall cybersickness, as well as nausea and vestibular symptoms.
However, in our previous study, the MSSQ scores failed to predict cybersickness’s intensity and symptoms [51]. Nonetheless, in this previous study, the MSSQ was also used to exclude participants who demonstrated a high susceptibility to motion sickness. This may explain why the MSSQ scores were not identified as significant predictors in our previous study. Finally, taken together, the MSSQ scores, especially the MSB-Adult, may be used to identify individuals who are prone to experience a vection-induced cybersickness. However, the Visual Induced MSSQ (VIMSSQ) has been recently developed and validated, which is specific to exposure to screens (e.g., computers, tablets, and VR HMDs) [111,116]. Also, it should be noted that vection is only one of the reasons for experiencing cybersickness in VR [31,35,36,45]. Thus, further research is required to examine whether MSSQ and/or VIMSSQ scores may predict cybersickness induced by other factors such as low latency or latency fluctuations, non-ergonomic navigation, and low-quality graphics.

5.4. Cybersickness’s Effects on Verbal Short-Term Memory and Working Memory

The study of Dahlman et al. [55] posited a direct negative effect of motion sickness on verbal working memory. Also, significant negative effects of cybersickness on verbal working memory were observed in our previous study [51]. Discrepantly to these previous studies, cybersickness was seen to significantly affect neither verbal short-term memory nor verbal working memory. Regarding short-term and working memory, there is consensus that they are two different cognitive constructs stemming from the activation of diverse brain regions, where the former requires substantially fewer cognitive resources than the latter [117,118]. Hence, short-term memory may not appear to decrease due to low difficulty. Furthermore, the study of Dahlman et al. [55] was on motion sickness. While motion sickness and cybersickness share some similarities, they are substantially different in terms of symptoms’ frequency and intensity [29,30]. Thus, this difference between cybersickness and motion sickness may explain the disagreement between the findings of this study and the findings of Dahlman et al.’ study.
Furthermore, in our previous study, only working memory was examined [51]. Also, the order of the tasks was not counterbalanced, as the verbal working memory task was always the first to be performed after exposure to linear and angular accelerations. Finally, the size effect of the performance decrease during the verbal working memory task was small [51]. These limitations of our previous study may thus explain the discrepancy between the findings of the two studies. Nonetheless, given that cybersickness effects are transient and of relatively short duration [44], if performing a task immediately after exposure to stimuli inducing cybersickness does indeed have an impact, then this poses a severe methodological consideration that should be further explored in future studies. However, in this study, in line with the design of the original tests (see [95,97]), short-term memory tasks always preceded working memory tasks. Thus, if the order had a significant impact, the cybersickness effects should have also been observed during short-term memory tasks.

5.5. Cybersickness’s Effects on Visuospatial Short-Term Memory and Working Memory

Cybersickness was found to have a significant negative effect on visuospatial working memory. This finding aligns with the findings of the study of Mittelstaedt et al. [56], where performance on the visuospatial working memory task was substantially decreased. The two studies hence postulate that cybersickness does indeed have a negative impact on visuospatial working memory. However, again, there was no effect on visuospatial short-term memory. Considering that the administration of the CBT tasks, the forward (short-term memory) and backward (working memory) recall tasks, should be in this order (i.e., forward recall and then backward recall) [97,98], the absence of an effect on short-term memory or a significant effect on working memory further supports that order of the tasks does not have an impact on observing the effects of cybersickness on cognitive performance. Furthermore, similar to verbal short-term and working memory, it is widely agreed that visuospatial short-term memory and working memory are distinct cognitive processes facilitated by different brain structures, with the former requiring fewer cognitive resources than the latter [118,119]. This difference between the two explains why visuospatial short-term memory was left intact by cybersickness while visuospatial working memory was substantially decreased.
Moreover, in this study, vestibular symptomatology, which implies a transient dysfunction of the vestibular system, was found to have a significant negative impact on visuospatial working memory. The functioning of visuospatial working memory pertains to the processing of visuospatial information [118,119]. The vestibular system has been suggested to have an important implication in visuospatial cognitive functioning [120]. Thus, the decrease in visuospatial working memory by predominantly vestibular symptomatology that was observed in this study further supports the importance of the vestibular system to visuospatial information processing. Nevertheless, the best model for predicting visuospatial working memory also included gaming experience, which revealed a positive effect on working memory. Notably, gaming experience was also included in the best model of visuospatial short-term memory. These findings are in line with the relevant literature, which suggests that gamers have enhanced short-term and working memory abilities [121]. In the investigation of cybersickness, these findings indicate that gaming experience should always be considered (e.g., as a covariate or an additional factor) when examining the effects of cybersickness on cognition.

5.6. Cybersickness’s Effects on Psychomotor Skills: Reaction, Attention, and Motor Speed

Psychomotor skills, such as attentional speed, motor speed, and overall reaction time were found to be substantially negatively affected by cybersickness’s symptomatology and intensity. In this study, the VR version of the DLRT was implemented, which, in contrast with the traditional version that produces a single score (i.e., reaction time), produces three metrics corresponding to attentional speed, motor speed, and overall reaction time. Since the previous studies on cybersickness (e.g., [34,56,57]) used the traditional version, and assessed participants after immersion, the current study may further decipher the effects of cybersickness on psychomotor skills. The observed significant deceleration of overall reaction speed is in line with our [45,51] and other previous studies [34,56,57]. Additionally, the current and previous findings thus offer robust evidence that cybersickness substantially compromised psychomotor skills. However, experience with playing videogames was also included in the best model, where a significant positive effect on reaction time (i.e., acceleration of reaction speed) was detected. This finding aligns with the previous literature pertaining to the effects of gaming experience on psychomotor speed [122,123,124,125]. In the context of cybersickness, this outcome indicates that the gaming experience has to be considered (e.g., as a covariate) to effectively examine the effects of cybersickness on psychomotor skills.
While the effects on overall reaction speed are well established by the current and previous findings, the cybersickness’s effects on the components of psychomotor skills (i.e., attentional and motor speed) still need to be investigated in depth. In this study, the attentional speed was found to be significantly decelerated by the oculomotor symptoms’ intensity. This outcome is in agreement with previous studies that revealed a significant negative effect on attentional processing speed [56,58]. However, these previous studies attributed the deceleration of attentional speed to overall cybersickness. In the current study, while both overall cybersickness intensity and the oculomotor symptoms’ intensity were deemed significant predictors of attentional speed, only the oculomotor symptomatology was incorporated in the respective best model. This outcome is in line with the established understanding that the oculomotor system is essential for facilitating visual attention functioning, especially for orienting attention [126,127]. Hence, our findings connote that a transient dysfunction of the oculomotor system (e.g., eye fatigue or strain) substantially compromises attentional speed.
However, the deceleration of the motor speed was found to be predominantly attributed to nausea symptomatology. This finding is aligned with the current understanding of the negative effects of nausea on motor coordination and skills due to a modulation of the activation of sensorimotor brain regions [128,129]. Nevertheless, gaming experience was also included in the best predictive model of motor speed, where a significant acceleration of motor speed was observed due to more gaming experience. This aligns with the relevant literature that postulates that gaming experience promotes an enhanced motor speed, especially for fine motor functions [130,131]. Thus, this finding suggests that gaming experience should be considered in the examination of cybersickness effects on motor speed. In summary, gaming experience appears central to enhancing psychomotor speed, while cybersickness substantially decelerates overall reaction speed. Regarding the components of psychomotor skills, the intensity of oculomotor and nausea symptoms significantly decelerates attentional and motor speed, respectively.

5.7. Cybersickness Symptoms and Their Intensity during and after Immersion

The intensities of overall cybersickness, nausea symptoms, and vestibular symptoms were substantially decreased after immersion (i.e., immediately after removing the headset). However, oculomotor symptoms did not decrease, which suggests that they may have a more lasting impact. This appears to be similar to simulator sickness, where oculomotor symptoms like eye strain persist long after the exposure to the simulator [132]. In contrast, vestibular and nausea symptoms are the most frequent and predominant in cybersickness [29,71,133]. Thus, these findings draw a stark contrast between the experience of cybersickness during immersion in a virtual environment and after immersion. This agrees with the suggestion that the human body and mind strive to readjust to the physical environment immediately after the removal of the VR headset [59].
Based on our results, after the exposure to stimuli inducing cybersickness, and during this transitory period of readapting to the physical space and body, the cybersickness’s intensity and symptoms substantially fade away. This also agrees with the current predominant understanding that cybersickness symptoms and effects are transient [44]. However, there is no consensus concerning how long cybersickness symptoms may persist, with some previous reviews suggesting that they may last for up to 12 h after exposure to VR [31,134]. However, our results suggest that the period of experiencing a substantial alleviation of cybersickness intensity commences during immersion (i.e., after exposure to stimuli inducing cybersickness, such as linear and angular accelerations) and that users attained a significant decrease after immersion, during their readaptation to the physical body and environment. Hence, this natural process counteracts or alleviates some of the dissonances experienced in the virtual environment during immersion. However, the cybersickness was measured after the ride (i.e., not during the exposure to the stimuli inducing vection and cybersickness), and not during the ride, which allows room for further research.
Nevertheless, our findings posit fundamental ramifications for the methodology used in VR research on cybersickness. To the best of our knowledge, except for the current and our previous studies [45,51], the studies on cybersickness (e.g., [28,39,40,41,43,68]) or cybersickness’s effects on human physiology or cognitive functioning or psychomotor skills (e.g., [34,50,56,57,58,72,73,133]) measure cybersickness’s intensity, symptomatology, and/or its effects, after immersion. Our findings indicate that this approach may only capture a part of the overall picture, or even worse, a substantially distorted picture.
If cybersickness symptoms subside or change in intensity immediately after VR exposure, then solely post-immersion evaluations could lead to unreliable conclusions. For instance, evaluations performed post immersion might underestimate the true intensity of the symptoms experienced during VR exposure, as well as the effects of cybersickness on cognition and motor skills. From a research design standpoint, these insights underscore the importance of adopting a more temporally accurate approach. Hence, in studies attempting to attain a holistic view of cybersickness, researchers should endeavour to capture data at multiple points—before, during, and after VR exposure. On the other hand, studies attempting to examine cybersickness aftereffects or the persistence of symptomatology may perform their examination after immersion. Lastly, it is crucial for studies striving to assess cybersickness’s intensity and symptomatology, as well as its effects on physiology (e.g., autonomic responses such as heart rate and temperature), cognition, and motor skills, to perform their examination during the VR immersion.

5.8. Limitations and Future Studies

This study also has some limitations that should be considered. While allowing for the conduction of the required statistical analyses, the sample size was relatively small. Also, the sample consisted predominantly of young adults aged 20–45 years old. A future study should incorporate a larger and/or more age-diverse sample (e.g., considering adolescents and/or older adults). Moreover, this study utilized the MSSQ and did not consider the VIMSSQ, which is specific to the vection elicited by screen-based mediums. Using both in a future study may assist in deciphering whether generic motion sickness susceptibility or a susceptibility that is specific to vection is an indicator for experiencing cybersickness’s symptomatology and intensity. Also, overall gaming experience was used for investigating gaming experience’s effects on cybersickness, cognitive functioning, and psychomotor skills. Since each genre of videogames offers diverse content that may stimulate distinct physiological and cognitive aspects, future studies should attempt to examine the effects of gaming experience by genre.
Moreover, this study explored the cybersickness intensity and symptoms in VR that were induced by vection. While vection is indeed one of the main reasons for cybersickness, several factors (e.g., low latency or latency fluctuations, non-ergonomic navigation, and low-quality graphics) may induce cybersickness. Future research should explore the intensity and symptomatology of cybersickness induced by each factor. The current study included several cognitive and psychomotor tasks. Given that a task’s characteristics may modulate cybersickness [135], future studies should either consider a single task or examine each task’s effects on cybersickness. Finally, considering that significant differences were found during and post immersion, future attempts should consider multiple assessment points of cybersickness to effectively scrutinize the cybersickness’s symptoms and intensity before, during (i.e., with several assessments), and after immersion.

6. Conclusions

This study comprehensively explored the modulators, symptomatology, and effects of cybersickness on an array of cognitive, physiological, and psychomotor functions. Notably, in line with our previous studies [45,51], it was found that pupil dilation was a valuable biomarker for cybersickness, reflecting the intensity of the experienced symptoms. Interestingly, demographic factors like sex and age did not significantly predict cybersickness. However, experience with videogames and smartphones emerged as protective factors, suggesting that familiarity with screen-based interactions could mitigate cybersickness symptoms. A particularly significant insight was the relationship between motion sickness susceptibility in adulthood and cybersickness intensity, suggesting an intertwined susceptibility pattern between these two phenomena. Furthermore, the research identified a selective impact of cybersickness when dissecting its effects on cognitive domains. While visuospatial working memory suffered from cybersickness, verbal short-term and working memory remained largely untouched. This distinction underscores the complex interplay between neural mechanisms and their susceptibility to cybersickness. One of the most crucial takeaways from this study was the marked difference in the experience and intensity of cybersickness during VR immersion compared to the post-immersion phase. Cybersickness symptomatology showed substantial changes post-immersion. This temporal difference poses significant methodological implications: if cybersickness evaluations are exclusively conducted post immersion, they might not offer reliable results and conclusions. Therefore, for a more accurate understanding of cybersickness, future research endeavours should consider assessment points during immersion and/or post-immersion phases, respective to their research aims.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/virtualworlds3010004/s1, Video S1: Video Abstract.

Author Contributions

Conceptualization, P.K.; methodology, P.K.; software, P.K.; validation, A.P., P.R. and P.K; formal analysis, P.K.; investigation, A.P.; resources, P.R. and P.K.; data curation, A.P. and P.K.; writing—original draft preparation, A.P. and P.K.; writing—review and editing, A.P., P.R. and P.K.; visualization, P.K.; supervision, P.K.; project administration, P.R. and P.K.; funding acquisition, A.P. and P.R. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

This study was conducted in accordance with the Declaration of Helsinki and approved by the Ethics Committee of the Department of Psychology of the National and Kapodistrian University of Athens (796-14/07/2023).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The data presented in this study are available on request from the corresponding author. The data are not publicly available due to ethical approval requirements.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Cruz-Neira, C.; Fernández, M.; Portalés, C. Virtual Reality and Games. Multimodal Technol. Interact. 2018, 2, 8. [Google Scholar] [CrossRef]
  2. Hartmann, T.; Fox, J. 717C36Entertainment in Virtual Reality and Beyond: The Influence of Embodiment, Co-Location, and Cognitive Distancing on Users’ Entertainment Experience. In The Oxford Handbook of Entertainment Theory; Vorderer, P., Klimmt, C., Eds.; Oxford University Press: Oxford, UK, 2021; ISBN 978-0-19-007221-6. [Google Scholar] [CrossRef]
  3. McGill, M.; Brewster, S. Virtual Reality Passenger Experiences. In Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications: Adjunct Proceedings (AutomotiveUI ’19), Utrecht, The Netherlands, 21–25 September 2019; Association for Computing Machinery: New York, NY, USA, 2019; pp. 434–441. [Google Scholar]
  4. Radianti, J.; Majchrzak, T.A.; Fromm, J.; Wohlgenannt, I. A systematic review of immersive virtual reality applications for higher education: Design elements, lessons learned, and research agenda. Comput. Educ. 2020, 147, 103778. [Google Scholar] [CrossRef]
  5. Rojas-Sánchez, M.A.; Palos-Sánchez, P.R.; Folgado-Fernández, J.A. Systematic literature review and bibliometric analysis on virtual reality and education. Educ. Inf. Technol. 2023, 28, 155–192. [Google Scholar] [CrossRef] [PubMed]
  6. Villena-Taranilla, R.; Tirado-Olivares, S.; Cózar-Gutiérrez, R.; González-Calero, J.A. Effects of virtual reality on learning outcomes in K-6 education: A meta-analysis. Educ. Res. Rev. 2022, 35, 100434. [Google Scholar] [CrossRef]
  7. Barteit, S.; Lanfermann, L.; Bärnighausen, T.; Neuhann, F.; Beiersmann, C. Augmented, Mixed, and Virtual Reality-Based Head-Mounted Devices for Medical Education: Systematic Review. JMIR Serious Games 2021, 9, e29080. [Google Scholar] [CrossRef] [PubMed]
  8. De Ponti, R.; Marazzato, J.; Maresca, A.M.; Rovera, F.; Carcano, G.; Ferrario, M.M. Pre-graduation medical training including virtual reality during COVID-19 pandemic: A report on students’ perception. BMC Med. Educ. 2020, 20, 332. [Google Scholar] [CrossRef]
  9. Xie, B.; Liu, H.; Alghofaili, R.; Zhang, Y.; Jiang, Y.; Lobo, F.D.; Li, C.; Li, W.; Huang, H.; Akdere, M.; et al. A Review on Virtual Reality Skill Training Applications. Front. Virtual Real. 2021, 2, 645153. [Google Scholar] [CrossRef]
  10. Pourmand, A.; Davis, S.; Marchak, A.; Whiteside, T.; Sikka, N. Virtual Reality as a Clinical Tool for Pain Management. Curr. Pain Headache Rep. 2018, 22, 53. [Google Scholar] [CrossRef]
  11. de Araújo, A.V.L.; de Oliveira Neiva, J.F.; de Mello Monteiro, C.B.; Magalhães, F.H. Efficacy of Virtual Reality Rehabilitation after Spinal Cord Injury: A Systematic Review. BioMed Res. Int. 2019, 2019, 7106951. [Google Scholar] [CrossRef]
  12. Lei, C.; Sunzi, K.; Dai, F.; Liu, X.; Wang, Y.; Zhang, B.; He, L.; Ju, M. Effects of virtual reality rehabilitation training on gait and balance in patients with Parkinson’s disease: A systematic review. PLoS ONE 2019, 14, e0224819. [Google Scholar] [CrossRef]
  13. Kourtesis, P.; Collina, S.; Doumas, L.A.A.; MacPherson, S.E. Validation of the Virtual Reality Everyday Assessment Lab (VR-EAL): An Immersive Virtual Reality Neuropsychological Battery with Enhanced Ecological Validity. J. Int. Neuropsychol. Soc. 2021, 27, 181–196. [Google Scholar] [CrossRef]
  14. Kourtesis, P.; MacPherson, S.E. How immersive virtual reality methods may meet the criteria of the National Academy of Neuropsychology and American Academy of Clinical Neuropsychology: A software review of the Virtual Reality Everyday Assessment Lab (VR-EAL). Comput. Hum. Behav. Rep. 2021, 4, 100151. [Google Scholar] [CrossRef]
  15. Liao, Y.-Y.; Chen, I.-H.; Lin, Y.-J.; Chen, Y.; Hsu, W.-C. Effects of Virtual Reality-Based Physical and Cognitive Training on Executive Function and Dual-Task Gait Performance in Older Adults With Mild Cognitive Impairment: A Randomized Control Trial. Front. Aging Neurosci. 2019, 11, 162. [Google Scholar] [CrossRef]
  16. Chatterjee, K.; Buchanan; Cottrell, K.; Hughes, S.; Day, T.W.; John, N.W. Immersive Virtual Reality for the Cognitive Rehabilitation of Stroke Survivors. IEEE Trans. Neural Syst. Rehabil. Eng. 2022, 30, 719–728. [Google Scholar] [CrossRef]
  17. Kourtesis, P.; Kouklari, E.-C.; Roussos, P.; Mantas, V.; Papanikolaou, K.; Skaloumbakas, C.; Pehlivanidis, A. Virtual Reality Training of Social Skills in Adults with Autism Spectrum Disorder: An Examination of Acceptability, Usability, User Experience, Social Skills, and Executive Functions. Behav. Sci. 2023, 13, 336. [Google Scholar] [CrossRef]
  18. Shahmoradi, L.; Rezayi, S. Cognitive rehabilitation in people with autism spectrum disorder: A systematic review of emerging virtual reality-based approaches. J. Neuroeng. Rehabil. 2022, 19, 91. [Google Scholar] [CrossRef]
  19. Bauer, A.C.M.; Andringa, G. The Potential of Immersive Virtual Reality for Cognitive Training in Elderly. Gerontology 2020, 66, 614–623. [Google Scholar] [CrossRef] [PubMed]
  20. Piech, J.; Czernicki, K. Virtual Reality Rehabilitation and Exergames—Physical and Psychological Impact on Fall Prevention among the Elderly—A Literature Review. Appl. Sci. 2021, 11, 4098. [Google Scholar] [CrossRef]
  21. Clay, F.; Howett, D.; FitzGerald, J.; Fletcher, P.; Chan, D.; Price, A. Use of Immersive Virtual Reality in the Assessment and Treatment of Alzheimer’s Disease: A Systematic Review. J. Alzheimer’s Dis. 2020, 75, 23–43. [Google Scholar] [CrossRef] [PubMed]
  22. Riaz, W.; Khan, Z.Y.; Jawaid, A.; Shahid, S. Virtual Reality (VR)-Based Environmental Enrichment in Older Adults with Mild Cognitive Impairment (MCI) and Mild Dementia. Brain Sci. 2021, 11, 1103. [Google Scholar] [CrossRef] [PubMed]
  23. Corrigan, N.; Păsărelu, C.-R.; Voinescu, A. Immersive virtual reality for improving cognitive deficits in children with ADHD: A systematic review and meta-analysis. Virtual Real. 2023, 18, 1–20. [Google Scholar] [CrossRef]
  24. Parsons, T.D.; Duffield, T.; Asbee, J. A Comparison of Virtual Reality Classroom Continuous Performance Tests to Traditional Continuous Performance Tests in Delineating ADHD: A Meta-Analysis. Neuropsychol. Rev. 2019, 29, 338–356. [Google Scholar] [CrossRef] [PubMed]
  25. Romero-Ayuso, D.; Toledano-González, A.; Rodríguez-Martínez, M.D.; Arroyo-Castillo, P.; Triviño-Juárez, J.M.; González, P.; Ariza-Vega, P.; Del Pino González, A.; Segura-Fragoso, A. Effectiveness of Virtual Reality-Based Interventions for Children and Adolescents with ADHD: A Systematic Review and Meta-Analysis. Children 2021, 8, 70. [Google Scholar] [CrossRef] [PubMed]
  26. Glaser, N.; Schmidt, M. Systematic Literature Review of Virtual Reality Intervention Design Patterns for Individuals with Autism Spectrum Disorders. Int. J. Hum.-Comput. Interact. 2022, 38, 753–788. [Google Scholar] [CrossRef]
  27. Zhang, M.; Ding, H.; Naumceska, M.; Zhang, Y. Virtual Reality Technology as an Educational and Intervention Tool for Children with Autism Spectrum Disorder: Current Perspectives and Future Directions. Behav. Sci. 2022, 12, 138. [Google Scholar] [CrossRef]
  28. Rebenitsch, L.; Owen, C. Estimating Cybersickness from Virtual Reality Applications. Virtual Real. 2021, 25, 165–174. [Google Scholar] [CrossRef]
  29. Stanney, K.M.; Kennedy, R.S.; Drexler, J.M. Cybersickness is Not Simulator Sickness. Proc. Hum. Factors Ergon. Soc. Annu. Meet. 1997, 41, 1138–1142. [Google Scholar] [CrossRef]
  30. Davis, S.; Nesbitt, K.; Nalivaiko, E. A Systematic Review of Cybersickness. In Proceedings of the 2014 Conference on Interactive Entertainment, Newcastle, NSW, Australia, 2–3 December 2014; Association for Computing Machinery: New York, NY, USA, 2014; pp. 1–9. [Google Scholar]
  31. LaViola, J.J. A discussion of cybersickness in virtual environments. SIGCHI Bull. 2000, 32, 47–56. [Google Scholar] [CrossRef]
  32. Stanney, K.; Lawson, B.; Rokers, B.; Dennison, M.; Fidopiastis, C.; Stoffregen, T.; Weech, S.; Fulvio, J. Identifying Causes of and Solutions for Cybersickness in Immersive Technology: Reformulation of a Research and Development Agenda. Int. J. Hum.-Comput. Interact. 2020, 36, 1783–1803. [Google Scholar] [CrossRef]
  33. Kim, J.; Palmisano, S.; Luu, W.; Iwasaki, S. Effects of Linear Visual-Vestibular Conflict on Presence, Perceived Scene Stability and Cybersickness in the Oculus Go and Oculus Quest. Front. Virtual Real. 2021, 2, 582156. [Google Scholar] [CrossRef]
  34. Nesbitt, K.; Davis, S.; Blackmore, K.; Nalivaiko, E. Correlating reaction time and nausea measures with traditional measures of cybersickness. Displays 2017, 48, 1–8. [Google Scholar] [CrossRef]
  35. Kourtesis, P.; Collina, S.; Doumas, L.A.A.; MacPherson, S.E. Technological Competence Is a Pre-condition for Effective Implementation of Virtual Reality Head Mounted Displays in Human Neuroscience: A Technological Review and Meta-Analysis. Front. Hum. Neurosci. 2019, 13, 342. [Google Scholar] [CrossRef] [PubMed]
  36. Kourtesis, P.; Collina, S.; Doumas, L.A.A.; MacPherson, S.E. Validation of the Virtual Reality Neuroscience Questionnaire: Maximum Duration of Immersive Virtual Reality Sessions without the Presence of Pertinent Adverse Symptomatology. Front. Hum. Neurosci. 2019, 13, 417. [Google Scholar] [CrossRef]
  37. Cheung, B.; Hofer, K. Desensitization to Strong Vestibular Stimuli Improves Tolerance to Simulated Aircraft Motion. Aviat. Space Environ. Med. 2005, 76, 1099–1104. [Google Scholar]
  38. Golding, J.F.; Gresty, M.A. Pathophysiology and treatment of motion sickness. Curr. Opin. Neurol. 2015, 28, 83–88. [Google Scholar] [CrossRef]
  39. Farmani, Y.; Teather, R.J. Evaluating Discrete Viewpoint Control to Reduce Cybersickness in Virtual Reality. Virtual Real. 2020, 24, 645–664. [Google Scholar] [CrossRef]
  40. Nie, G.-Y.; Duh, H.B.-L.; Liu, Y.; Wang, Y. Analysis on Mitigation of Visually Induced Motion Sickness by Applying Dynamical Blurring on a User’s Retina. IEEE Trans. Vis. Comput. Graph. 2020, 26, 2535–2545. [Google Scholar] [CrossRef] [PubMed]
  41. Szpak, A.; Michalski, S.C.; Saredakis, D.; Chen, C.S.; Loetscher, T. Beyond Feeling Sick: The Visual and Cognitive Aftereffects of Virtual Reality. IEEE Access 2019, 7, 130883–130892. [Google Scholar] [CrossRef]
  42. Kennedy, R.S.; Lane, N.E.; Berbaum, K.S.; Lilienthal, M.G. Simulator Sickness Questionnaire: An Enhanced Method for Quantifying Simulator Sickness. Int. J. Aviat. Psychol. 1993, 3, 203–220. [Google Scholar] [CrossRef]
  43. Kim, H.K.; Park, J.; Choi, Y.; Choe, M. Virtual reality sickness questionnaire (VRSQ): Motion sickness measurement index in a virtual reality environment. Appl. Ergon. 2018, 69, 66–73. [Google Scholar] [CrossRef]
  44. Bouchard, S.; Berthiaume, M.; Robillard, G.; Forget, H.; Daudelin-Peltier, C.; Renaud, P.; Blais, C.; Fiset, D. Arguing in Favor of Revising the Simulator Sickness Questionnaire Factor Structure When Assessing Side Effects Induced by Immersions in Virtual Reality. Front. Psychiatry 2021, 12, 739742. [Google Scholar] [CrossRef]
  45. Kourtesis, P.; Linnell, J.; Amir, R.; Argelaguet, F.; MacPherson, S.E. Cybersickness in Virtual Reality Questionnaire (CSQ-VR): A Validation and Comparison against SSQ and VRSQ. Virtual Worlds 2023, 2, 16–35. [Google Scholar] [CrossRef]
  46. Sevinc, V.; Berkman, M.I. Psychometric evaluation of Simulator Sickness Questionnaire and its variants as a measure of cybersickness in consumer virtual environments. Appl. Ergon. 2020, 82, 102958. [Google Scholar] [CrossRef] [PubMed]
  47. Stone Iii, W.B. Psychometric Evaluation of the Simulator Sickness Questionnaire as a Measure of Cybersickness. Ph.D. Thesis, Iowa State University, Digital Repository, Ames, IA, USA, 2017; p. 11054762. [Google Scholar]
  48. Weech, S.; Kenny, S.; Barnett-Cowan, M. Presence and Cybersickness in Virtual Reality Are Negatively Related: A Review. Front. Psychol. 2019, 10, 158. [Google Scholar] [CrossRef] [PubMed]
  49. Chang, E.; Billinghurst, M.; Yoo, B. Brain activity during cybersickness: A scoping review. Virtual Real. 2023, 27, 2073–2097. [Google Scholar] [CrossRef]
  50. Dennison, M.S.; Wisti, A.Z.; D’Zmura, M. Use of physiological signals to predict cybersickness. Displays 2016, 44, 42–52. [Google Scholar] [CrossRef]
  51. Kourtesis, P.; Amir, R.; Linnell, J.; Argelaguet, F.; MacPherson, S.E. Cybersickness, Cognition, & Motor Skills: The Effects of Music, Gender, and Gaming Experience. IEEE Trans. Vis. Comput. Graph. 2023, 29, 2326–2336. [Google Scholar] [CrossRef]
  52. Saredakis, D.; Szpak, A.; Birckhead, B.; Keage, H.A.D.; Rizzo, A.; Loetscher, T. Factors Associated With Virtual Reality Sickness in Head-Mounted Displays: A Systematic Review and Meta-Analysis. Front. Hum. Neurosci. 2020, 14, 96. [Google Scholar] [CrossRef]
  53. Melo, M.; Vasconcelos-Raposo, J.; Bessa, M. Presence and cybersickness in immersive content: Effects of content type, exposure time and gender. Comput. Graph. 2018, 71, 159–165. [Google Scholar] [CrossRef]
  54. Stanney, K.; Fidopiastis, C.; Foster, L. Virtual Reality Is Sexist: But It Does Not Have to Be. Front. Robot. AI 2020, 7, 4. [Google Scholar] [CrossRef] [PubMed]
  55. Dahlman, J.; Sjörs, A.; Lindström, J.; Ledin, T.; Falkmer, T. Performance and Autonomic Responses During Motion Sickness. Hum. Factors 2009, 51, 56–66. [Google Scholar] [CrossRef]
  56. Mittelstaedt, J.M.; Wacker, J.; Stelling, D. VR Aftereffect and the Relation of Cybersickness and Cognitive Performance. Virtual Real. 2019, 23, 143–154. [Google Scholar] [CrossRef]
  57. Nalivaiko, E.; Davis, S.L.; Blackmore, K.L.; Vakulin, A.; Nesbitt, K.V. Cybersickness provoked by head-mounted display affects cutaneous vascular tone, heart rate and reaction time. Physiol. Behav. 2015, 151, 583–590. [Google Scholar] [CrossRef] [PubMed]
  58. Varmaghani, S.; Abbasi, Z.; Weech, S.; Rasti, J. Spatial and Attentional Aftereffects of Virtual Reality and Relations to Cybersickness. Virtual Real. 2022, 26, 659–668. [Google Scholar] [CrossRef]
  59. Knibbe, J.; Schjerlund, J.; Petraeus, M.; Hornbæk, K. The Dream is Collapsing: The Experience of Exiting VR. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI ’18), Montreal, QC, Canada, 21–26 April 2018; Association for Computing Machinery: New York, NY, USA, 2018; pp. 1–13. [Google Scholar]
  60. Sepich, N.C.; Jasper, A.; Fieffer, S.; Gilbert, S.B.; Dorneich, M.C.; Kelly, J.W. The impact of task workload on cybersickness. Front. Virtual Real. 2022, 3, 943409. [Google Scholar] [CrossRef]
  61. Abeele, V.V.; Schraepen, B.; Huygelier, H.; Gillebert, C.; Gerling, K.; Van Ee, R. Immersive Virtual Reality for Older Adults: Empirically Grounded Design Guidelines. ACM Trans. Access. Comput. 2021, 14, 14. [Google Scholar] [CrossRef]
  62. Garcia Fracaro, S.; Chan, P.; Gallagher, T.; Tehreem, Y.; Toyoda, R.; Bernaerts, K.; Glassey, J.; Pfeiffer, T.; Slof, B.; Wachsmuth, S.; et al. Towards design guidelines for virtual reality training for the chemical industry. Educ. Chem. Eng. 2021, 36, 12–23. [Google Scholar] [CrossRef]
  63. Kourtesis, P.; Korre, D.; Collina, S.; Doumas, L.A.A.; MacPherson, S.E. Guidelines for the Development of Immersive Virtual Reality Software for Cognitive Neuroscience and Neuropsychology: The Development of Virtual Reality Everyday Assessment Lab (VR-EAL), a Neuropsychological Test Battery in Immersive Virtual Reality. Front. Comput. Sci. 2020, 1, 12. [Google Scholar] [CrossRef]
  64. Monge Pereira, E.; Molina Rueda, F.; Alguacil Diego, I.M.; Cano De La Cuerda, R.; De Mauro, A.; Miangolarra Page, J.C. Use of virtual reality systems as proprioception method in cerebral palsy: Clinical practice guideline. Neurología 2014, 29, 550–559. [Google Scholar] [CrossRef] [PubMed]
  65. O’Connor, E.A.; Domingo, J. A Practical Guide, With Theoretical Underpinnings, for Creating Effective Virtual Reality Learning Environments. J. Educ. Technol. Syst. 2017, 45, 343–364. [Google Scholar] [CrossRef]
  66. Monteiro, D.; Liang, H.-N.; Wang, J.; Chen, H.; Baghaei, N. An In-Depth Exploration of the Effect of 2D/3D Views and Controller Types on First Person Shooter Games in Virtual Reality. In Proceedings of the 2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Porto de Galinhas, Brazil, 9–13 November 2020; pp. 713–724. [Google Scholar]
  67. Wood, C.D.; Manno, J.E.; Wood, M.J.; Manno, B.R.; Mims, M.E. Comparison of Efficacy of Ginger with Various Antimotion Sickness Drugs. Clin. Res. Pract. Drug Regul. Aff. 1988, 6, 129–136. [Google Scholar] [CrossRef]
  68. Risi, D.; Palmisano, S. Effects of postural stability, active control, exposure duration and repeated exposures on HMD induced cybersickness. Displays 2019, 60, 9–17. [Google Scholar] [CrossRef]
  69. Sokołowska, B. Impact of Virtual Reality Cognitive and Motor Exercises on Brain Health. Int. J. Environ. Res. Public Health 2023, 20, 4150. [Google Scholar] [CrossRef]
  70. Somrak, A.; Pogačnik, M.; Guna, J. Suitability and Comparison of Questionnaires Assessing Virtual Reality-Induced Symptoms and Effects and User Experience in Virtual Environments. Sensors 2021, 21, 1185. [Google Scholar] [CrossRef]
  71. Sharples, S.; Cobb, S.; Moody, A.; Wilson, J.R. Virtual reality induced symptoms and effects (VRISE): Comparison of head mounted display (HMD), desktop and projection display systems. Displays 2008, 29, 58–69. [Google Scholar] [CrossRef]
  72. Kim, Y.Y.; Kim, H.J.; Kim, E.N.; Ko, H.D.; Kim, H.T. Characteristic changes in the physiological components of cybersickness. Psychophysiology 2005, 42, 616–625. [Google Scholar] [CrossRef] [PubMed]
  73. Gavgani, A.M.; Nesbitt, K.V.; Blackmore, K.L.; Nalivaiko, E. Profiling subjective symptoms and autonomic changes associated with cybersickness. Auton. Neurosci. 2017, 203, 41–50. [Google Scholar] [CrossRef]
  74. Golding, J.F. Phasic skin conductance activity and motion sickness. Aviat. Space Environ. Med. 1992, 63, 165–171. [Google Scholar]
  75. Pacchierotti, C.; Sinclair, S.; Solazzi, M.; Frisoli, A.; Hayward, V.; Prattichizzo, D. Wearable Haptic Systems for the Fingertip and the Hand: Taxonomy, Review, and Perspectives. IEEE Trans. Haptics 2017, 10, 580–600. [Google Scholar] [CrossRef]
  76. Kourtesis, P.; Argelaguet, F.; Vizcay, S.; Marchal, M.; Pacchierotti, C. Electrotactile Feedback Applications for Hand and Arm Interactions: A Systematic Review, Meta-Analysis, and Future Directions. IEEE Trans. Haptics 2022, 15, 479–496. [Google Scholar] [CrossRef]
  77. Chang, E.; Kim, H.T.; Yoo, B. Predicting cybersickness based on user’s gaze behaviors in HMD-based virtual reality. J. Comput. Des. Eng. 2021, 8, 728–739. [Google Scholar] [CrossRef]
  78. Wang, J.; Liang, H.-N.; Monteiro, D.; Xu, W.; Xiao, J. Real-Time Prediction of Simulator Sickness in Virtual Reality Games. IEEE Trans. Games 2023, 15, 252–261. [Google Scholar] [CrossRef]
  79. Islam, R.; Desai, K.; Quarles, J. Cybersickness Prediction from Integrated HMD’s Sensors: A Multimodal Deep Fusion Approach using Eye-tracking and Head-tracking Data. In Proceedings of the 2021 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Bari, Italy, 4–8 October 2021; pp. 31–40. [Google Scholar]
  80. Petri, K.; Feuerstein, K.; Folster, S.; Bariszlovich, F.; Witte, K. Effects of Age, Gender, Familiarity with the Content, and Exposure Time on Cybersickness in Immersive Head-mounted Display Based Virtual Reality. Am. J. Biomed. Sci. 2020, 12, 107–121. [Google Scholar] [CrossRef]
  81. Stanney, K.M.; Hale, K.S.; Nahmens, I.; Kennedy, R.S. What to Expect from Immersive Virtual Environment Exposure: Influences of Gender, Body Mass Index, and Past Experience. Hum. Factors 2003, 45, 504–520. [Google Scholar] [CrossRef]
  82. Kourtesis, P.; Collina, S.; Doumas, L.A.A.; MacPherson, S.E. An ecologically valid examination of event-based and time-based prospective memory using immersive virtual reality: The effects of delay and task type on everyday prospective memory. Memory 2021, 29, 486–506. [Google Scholar] [CrossRef]
  83. Kourtesis, P.; MacPherson, S.E. An ecologically valid examination of event-based and time-based prospective memory using immersive virtual reality: The influence of attention, memory, and executive function processes on real-world prospective memory. Neuropsychol. Rehabil. 2023, 33, 255–280. [Google Scholar] [CrossRef]
  84. Weech, S.; Kenny, S.; Lenizky, M.; Barnett-Cowan, M. Narrative and gaming experience interact to affect presence and cybersickness in virtual reality. Int. J. Hum.-Comput. Stud. 2020, 138, 102398. [Google Scholar] [CrossRef]
  85. Golding, J.F.; Rafiq, A.; Keshavarz, B. Predicting Individual Susceptibility to Visually Induced Motion Sickness by Questionnaire. Front. Virtual Real. 2021, 2, 576871. [Google Scholar] [CrossRef]
  86. Lukacova, I.; Keshavarz, B.; Golding, J.F. Measuring the susceptibility to visually induced motion sickness and its relationship with vertigo, dizziness, migraine, syncope and personality traits. Exp. Brain Res. 2023, 241, 1381–1391. [Google Scholar] [CrossRef]
  87. Golding, J.F. Motion sickness susceptibility questionnaire revised and its relationship to other forms of sickness. Brain Res. Bull. 1998, 47, 507–516. [Google Scholar] [CrossRef] [PubMed]
  88. Golding, J.F. Predicting individual differences in motion sickness susceptibility by questionnaire. Personal. Individ. Differ. 2006, 41, 237–248. [Google Scholar] [CrossRef]
  89. Paillard, A.C.; Quarck, G.; Paolino, F.; Denise, P.; Paolino, M.; Golding, J.F.; Ghulyan-Bedikian, V. Motion sickness susceptibility in healthy subjects and vestibular patients: Effects of gender, age and trait-anxiety. J. Vestib. Res. 2013, 23, 203–209. [Google Scholar] [CrossRef]
  90. Conner, N.O.; Freeman, H.R.; Jones, J.A.; Luczak, T.; Carruth, D.; Knight, A.C.; Chander, H. Virtual Reality Induced Symptoms and Effects: Concerns, Causes, Assessment & Mitigation. Virtual Worlds 2022, 1, 130–146. [Google Scholar] [CrossRef]
  91. Schwind, V.; Knierim, P.; Tasci, C.; Franczak, P.; Haas, N.; Henze, N. “These Are Not My Hands!”: Effect of Gender on the Perception of Avatar Hands in Virtual Reality. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, CO, USA, 6–11 May 2017; Association for Computing Machinery: New York, NY, USA, 2017; pp. 1577–1582, ISBN 978-1-4503-4655-9. [Google Scholar]
  92. Bebko, A.O.; Troje, N.F. bmlTUX: Design and Control of Experiments in Virtual Reality and Beyond. i-Perception 2020, 11, 2041669520938400. [Google Scholar] [CrossRef]
  93. Standard ISO 9241-400:2007; Ergonomics of Human—System Interaction—Part 400: Principles and Requirements for Physical Input Devices. International Organization for Standardization: Geneva, Switzerland, 2007. Available online: https://www.iso.org/standard/38896.html (accessed on 1 January 2024).
  94. Kourtesis, P.; Vizcay, S.; Marchal, M.; Pacchierotti, C.; Argelaguet, F. Action-Specific Perception & Performance on a Fitts’s Law Task in Virtual Reality: The Role of Haptic Feedback. IEEE Trans. Vis. Comput. Graph. 2022, 28, 3715–3726. [Google Scholar] [CrossRef]
  95. Wechsler, D. WAIS-IV: Wechsler Adult Intelligence Scale; Psychological Corporation: New York, NY, USA, 2008. [Google Scholar]
  96. Donolato, E.; Giofrè, D.; Mammarella, I.C. Differences in Verbal and Visuospatial Forward and Backward Order Recall: A Review of the Literature. Front. Psychol. 2017, 8, 663. [Google Scholar] [CrossRef]
  97. Corsi, P.M. Human Memory and the Medial Temporal Region of the Brain; ProQuest Information & Learning: Ann Arbor, MI, USA, 1973; Volume 34, p. 891. [Google Scholar]
  98. Kessels, R.P.C.; van den Berg, E.; Ruis, C.; Brands, A.M.A. The backward span of the Corsi Block-Tapping Task and its association with the WAIS-III Digit Span. Assessment 2008, 15, 426–434. [Google Scholar] [CrossRef]
  99. Deary, I.J.; Liewald, D.; Nissan, J. A free, easy-to-use, computer-based simple and four-choice reaction time programme: The Deary-Liewald reaction time task. Behav. Res. 2011, 43, 258–268. [Google Scholar] [CrossRef]
  100. R Core Team. R: A Language and Environment for Statistical Computing; R Foundation for Statistical Computing: Vienna, Austria, 2022; Available online: https://www.R-project.org/ (accessed on 1 January 2024).
  101. RStudio Team. RStudio: Integrated Development Environment for R; RStudio, PBC: Boston, MA, USA, 2022; Available online: http://www.rstudio.com/ (accessed on 1 January 2024).
  102. Revelle, W. psych: Procedures for Psychological, Psychometric, and Personality Research; Northwestern University: Evanston, IL, USA, 2022; Available online: https://CRAN.R-project.org/package=psych (accessed on 1 January 2024).
  103. Wickham, H. ggplot2: Elegant Graphics for Data Analysis; Springer: New York, NY, USA, 2016; ISBN 978-3-319-24277-4. Available online: https://ggplot2.tidyverse.org (accessed on 1 January 2024).
  104. Bates, D.; Mächler, M.; Bolker, B.; Walker, S. Fitting Linear Mixed-Effects Models Using lme4. J. Stat. Softw. 2015, 67, 1–48. [Google Scholar] [CrossRef]
  105. Peterson, R.A.; Cavanaugh, J.E. Ordered quantile normalization: A semiparametric transformation built for the cross-validation era. J. Appl. Stat. 2020, 47, 2312–2327. [Google Scholar] [CrossRef]
  106. Partala, T.; Surakka, V. Pupil size variation as an indication of affective processing. Int. J. Hum.-Comput. Stud. 2003, 59, 185–198. [Google Scholar] [CrossRef]
  107. Chattha, U.A.; Janjua, U.I.; Anwar, F.; Madni, T.M.; Cheema, M.F.; Janjua, S.I. Motion Sickness in Virtual Reality: An Empirical Evaluation. IEEE Access 2020, 8, 130486–130499. [Google Scholar] [CrossRef]
  108. Bouchrika, I. Mobile vs Desktop Usage Statistics for 2023. Available online: https://research.com/software/mobile-vs-desktop-usage (accessed on 21 October 2023).
  109. Soewardi, H.; Izzuddin, M.N. Study of Cybersickness on Non-Immersive Virtual Reality Using Smartphone. Malays. J. Public Health Med. 2020, 20, 88–93. [Google Scholar] [CrossRef]
  110. Kemeny, A.; Chardonnet, J.-R.; Colombet, F. Visualization and Motion Systems. In Getting Rid of Cybersickness: In Virtual Reality, Augmented Reality, and Simulators; Kemeny, A., Chardonnet, J.-R., Colombet, F., Eds.; Springer International Publishing: Cham, Switzerland, 2020; pp. 63–91. ISBN 978-3-030-59342-1. [Google Scholar]
  111. Keshavarz, B.; Saryazdi, R.; Campos, J.L.; Golding, J.F. Introducing the VIMSSQ: Measuring susceptibility to visually induced motion sickness. Proc. Hum. Factors Ergon. Soc. Annu. Meet. 2019, 63, 2267–2271. [Google Scholar] [CrossRef]
  112. Grassini, S.; Laumann, K.; Luzi, A.K. Association of Individual Factors with Simulator Sickness and Sense of Presence in Virtual Reality Mediated by Head-Mounted Displays (HMDs). Multimodal Technol. Interact. 2021, 5, 7. [Google Scholar] [CrossRef]
  113. Himmels, C.; Czupala, A.R.; Fahmüller, T.; Fuchs, B.; Hauf, L.; Heidler, L.; Riener, A. The Influence of Gaming Experience and Optic Flow on Simulator Sickness: Insights from a Driving Simulator Study. In Proceedings of the Adjunct Proceedings of the 14th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI ’22), Seoul, Republic of Korea, 17–20 September 2022; Association for Computing Machinery: New York, NY, USA, 2022; pp. 149–152. [Google Scholar]
  114. Krarup, K.B.; Krarup, H.B. The physiological and biochemical effects of gaming: A review. Environ. Res. 2020, 184, 109344. [Google Scholar] [CrossRef]
  115. Kühn, S.; Gallinat, J.; Mascherek, A. Effects of computer gaming on cognition, brain structure, and function: A critical reflection on existing literature. Dialogues Clin. Neurosci. 2019, 21, 319–330. [Google Scholar] [CrossRef]
  116. Keshavarz, B.; Murovec, B.; Mohanathas, N.; Golding, J.F. The Visually Induced Motion Sickness Susceptibility Questionnaire (VIMSSQ): Estimating Individual Susceptibility to Motion Sickness-Like Symptoms When Using Visual Devices. Hum. Factors 2023, 65, 107–124. [Google Scholar] [CrossRef]
  117. Cantor, J.; Engle, R.W.; Hamilton, G. Short-term memory, working memory, and verbal abilities: How do they relate? Intelligence 1991, 15, 229–246. [Google Scholar] [CrossRef]
  118. Kail, R.; Hall, L.K. Distinguishing short-term memory from working memory. Mem. Cogn. 2001, 29, 1–9. [Google Scholar] [CrossRef]
  119. Logie, R.H. The Functional Organization and Capacity Limits of Working Memory. Curr. Dir. Psychol. Sci. 2011, 20, 240–245. [Google Scholar] [CrossRef]
  120. Bigelow, R.T.; Agrawal, Y. Vestibular involvement in cognition: Visuospatial ability, attention, executive function, and memory. J. Vestib. Res. 2015, 25, 73–89. [Google Scholar] [CrossRef] [PubMed]
  121. Waris, O.; Jaeggi, S.M.; Seitz, A.R.; Lehtonen, M.; Soveri, A.; Lukasik, K.M.; Söderström, U.; Hoffing, R.A.C.; Laine, M. Video gaming and working memory: A large-scale cross-sectional correlative study. Comput. Hum. Behav. 2019, 97, 94–103. [Google Scholar] [CrossRef] [PubMed]
  122. Kazimoglu, C.; Bacon, L. An Analysis of a Video Game on Cognitive Abilities: A Study to Enhance Psychomotor Skills via Game-Play. IEEE Access 2020, 8, 110495–110510. [Google Scholar] [CrossRef]
  123. Dye, M.W.G.; Green, C.S.; Bavelier, D. Increasing Speed of Processing With Action Video Games. Curr. Dir. Psychol. Sci. 2009, 18, 321–326. [Google Scholar] [CrossRef]
  124. Kennedy, A.M.; Boyle, E.M.; Traynor, O.; Walsh, T.; Hill, A.D.K. Video Gaming Enhances Psychomotor Skills But Not Visuospatial and Perceptual Abilities in Surgical Trainees. J. Surg. Educ. 2011, 68, 414–420. [Google Scholar] [CrossRef]
  125. Kowal, M.; Toth, A.J.; Exton, C.; Campbell, M.J. Different cognitive abilities displayed by action video gamers and non-gamers. Comput. Hum. Behav. 2018, 88, 255–262. [Google Scholar] [CrossRef]
  126. Bisley, J.W. The neural basis of visual attention. J. Physiol. 2011, 589, 49–57. [Google Scholar] [CrossRef]
  127. Klein, R.; Kingstone, A.; Pontefract, A. Orienting of Visual Attention. In Eye Movements and Visual Cognition: Scene Perception and Reading; Rayner, K., Ed.; Springer: New York, NY, USA, 1992; pp. 46–65. ISBN 978-1-4612-2852-3. [Google Scholar]
  128. Napadow, V.; Sheehan, J.D.; Kim, J.; LaCount, L.T.; Park, K.; Kaptchuk, T.J.; Rosen, B.R.; Kuo, B. The Brain Circuitry Underlying the Temporal Evolution of Nausea in Humans. Cereb. Cortex 2013, 23, 806–813. [Google Scholar] [CrossRef]
  129. Singh, P.; Yoon, S.S.; Kuo, B. Nausea: A review of pathophysiology and therapeutics. Ther. Adv. Gastroenterol. 2016, 9, 98–112. [Google Scholar] [CrossRef]
  130. Borecki, L.; Tolstych, K.; Pokorski, M. Computer Games and Fine Motor Skills. In Respiratory Regulation—Clinical Advances; Pokorski, M., Ed.; Springer: Dordrecht, The Netherlands, 2013; pp. 343–348. [Google Scholar]
  131. Gianferrara, P.G.; Betts, S.; Anderson, J.R. Cognitive & motor skill transfer across speeds: A video game study. PLoS ONE 2021, 16, e0258242. [Google Scholar] [CrossRef]
  132. Geyer, D.J.; Biggs, A.T. The Persistent Issue of Simulator Sickness in Naval Aviation Training. Aerosp. Med. Hum. Perform. 2018, 89, 396–405. [Google Scholar] [CrossRef] [PubMed]
  133. Palmisano, S.; Mursic, R.; Kim, J. Vection and cybersickness generated by head-and-display motion in the Oculus Rift. Displays 2017, 46, 1–8. [Google Scholar] [CrossRef]
  134. Rebenitsch, L.; Owen, C. Review on cybersickness in applications and visual displays. Virtual Real. 2016, 20, 101–125. [Google Scholar] [CrossRef]
  135. Jasper, A.; Sepich, N.C.; Gilbert, S.B.; Kelly, J.W.; Dorneich, M.C. Predicting cybersickness using individual and task characteristics. Comput. Hum. Behav. 2023, 146, 107800. [Google Scholar] [CrossRef]
Figure 1. Examples of linear and angular accelerations during the ride.
Figure 1. Examples of linear and angular accelerations during the ride.
Virtualworlds 03 00004 g001
Figure 2. Digit Span Test (upper left), Corsi Block Test (upper right), and Deary–Liewald Reaction Time test (bottom).
Figure 2. Digit Span Test (upper left), Corsi Block Test (upper right), and Deary–Liewald Reaction Time test (bottom).
Virtualworlds 03 00004 g002
Figure 3. CSQ-VR user interface and eye-tracking (ET) targets. Note: eye-tracking targets were not visible to the user.
Figure 3. CSQ-VR user interface and eye-tracking (ET) targets. Note: eye-tracking targets were not visible to the user.
Virtualworlds 03 00004 g003
Figure 4. Comparison of cybersickness’s intensity during and after exposure to virtual reality. Note: CSQ-VR = Cybersickness in Virtual reality Questionnaire; In VR = during immersion; Post-VR = after exposure to VR; ns = non-significant, * p ≤ 0.05, *** p ≤ 0.001.
Figure 4. Comparison of cybersickness’s intensity during and after exposure to virtual reality. Note: CSQ-VR = Cybersickness in Virtual reality Questionnaire; In VR = during immersion; Post-VR = after exposure to VR; ns = non-significant, * p ≤ 0.05, *** p ≤ 0.001.
Virtualworlds 03 00004 g004
Table 1. Descriptive statistics.
Table 1. Descriptive statistics.
StageMean (SD)Range
Age29.17 (4.17)22–38
Education16.27 (2.92)12–25
Computing XP10.30 (1.05)8–12
Smartphone XP10.70 (0.75)9–12
Gaming XP37.23 (14.64)22–72
MSA–Child6.02 (4.38)0–14.14
MSB–Adult3.61 (3.18)0–12.60
MSSQ Total9.63 (6.73)0–26.74
CSQ-VR–TotalBaseline8.73 (3.06)6–19
Post-Ride14.78 (7.84)6–34
CSQ-VR–NauseaBaseline2.50 (0.90)2–6
Post-Ride5.20 (3.22)2–14
CSQ-VR–VestibularBaseline2.63 (0.89)2–5
Post-Ride4.63 (3.18)2–14
CSQ-VR–OculomotorBaseline3.60 (1.70)2–8
Post-Ride4.93 (2.87)2–12
DST–ForwardBaseline16.63 (2.58)12–20
Post-Ride16.90 (2.63)11–20
CBT–ForwardBaseline14.63 (2.72)10–20
Post-Ride15.17 (3.05)5–20
DST–BackwardBaseline14.83 (3.58)8–20
Post-Ride15.63 (3.64)7–20
CBT–BackwardBaseline15.10 (2.00)11–19
Post-Ride14.43 (2.89)8–20
DLRT–RTBaseline0.57 (0.10)0.36–0.81
Post-Ride0.61 (0.16)0.38–1.13
DLRT–ATBaseline0.30 (0.09)0–0.42
Post-Ride0.34 (1.44)0–0.87
DLRT–MTBaseline0.24 (0.10)0–0.45
Post-Ride0.27 (0.17)0–0.83
XP = experience; MS = motion sickness; MSSQ = Motion Sickness Susceptibility Questionnaire; CSQ-VR = Cybersickness in Virtual reality Questionnaire; DST = Digit Span Test; CBT = Corsi Block Test; DLRT = Deary–Liewald Reaction Time test; RT = reaction time; AT = Attention Time; MT = Motor Time.
Table 2. Single-predictor models for overall cybersickness.
Table 2. Single-predictor models for overall cybersickness.
Predictorβ Coefficientp-ValueR2
Sex (Male)−0.71 &0.0530.13
Education0.140.4700.02
Age−0.080.6570.01
Computing XP−0.350.0560.12
Smartphone XP−0.400.027 *0.16
Gaming XP−0.440.024 *0.17
MSA-Child0.400.031 *0.16
MSB-Adult0.66<0.001 ***0.39
MSSQ Total0.470.010 **0.22
Pupil–CSQ-VR−0.350.0560.15
Pupil–Ride−0.380.036 *0.16
XP = experience; MS = motion sickness; MSSQ = Motion Sickness Susceptibility Questionnaire; Pupil–CSQ-VR = Pupil Size While Reading CSQ-VR Questions; Pupil Ride = Pupil Size During Ride; * p ≤ 0.05, ** p ≤ 0.01, *** p ≤ 0.001; & unstandardized beta.
Table 3. Single-predictor models for nausea symptoms.
Table 3. Single-predictor models for nausea symptoms.
Predictorβ Coefficientp-ValueR2
Sex (Male)−0.65 &0.0760.11
Education0.130.4840.02
Age0.040.8510.00
Computing XP−0.340.0640.12
Smartphone XP−0.370.045 *014
Gaming XP−0.380.041 *0.14
MSA-Child0.440.014 *0.20
MSB-Adult0.63<0.001 ***0.39
MSSQ Total0.540.002 **0.29
Pupil–CSQ-VR−0.380.041 *0.14
Pupil–Ride−0.360.043 *0.14
XP = experience; MS = motion sickness; MSSQ = Motion Sickness Susceptibility Questionnaire; Pupil–CSQ-VR = Pupil Size While Reading CSQ-VR Questions; Pupil Ride = Pupil Size During Ride; * p ≤ 0.05, ** p ≤ 0.01, *** p ≤ 0.001; & unstandardized beta.
Table 4. Single-predictor models for vestibular symptoms.
Table 4. Single-predictor models for vestibular symptoms.
Predictorβ Coefficientp-ValueR2
Sex (Male)−0.61 &0.1000.09
Education0.240.2030.06
Age0.000.9800.00
Computing XP−0.220.2380.05
Smartphone XP−0.310.047 *0.13
Gaming XP−0.400.029 *0.16
MSA-Child0.200.2900.04
MSB-Adult0.530.003 **0.28
MSSQ Total0.280.1370.08
Pupil–CSQ-VR−0.250.1820.06
Pupil–Ride−0.240.2080.06
XP = experience; MS = motion sickness; MSSQ = Motion Sickness Susceptibility Questionnaire; Pupil–CSQ-VR = Pupil Size While Reading CSQ-VR Questions; Pupil Ride = Pupil Size During Ride; * p ≤ 0.05, ** p ≤ 0.01, & unstandardized beta.
Table 5. Single-predictor models for oculomotor symptoms.
Table 5. Single-predictor models for oculomotor symptoms.
Predictorβ Coefficientp-ValueR2
Sex (Male)−0.26 &0.4860.02
Education−0.140.4680.02
Age−0.230.2130.06
Computing XP−0.250.1790.06
Smartphone XP−0.220.2490.05
Gaming XP−0.180.3360.03
MSA-Child0.210.2650.04
MSB-Adult0.270.1520.07
MSSQ Total0.230.2210.05
Pupil–CSQ-VR−0.110.5690.01
Pupil–Ride−0.280.1380.08
XP = experience; MS = motion sickness; MSSQ = Motion Sickness Susceptibility Questionnaire; Pupil–CSQ-VR = Pupil Size While Reading CSQ-VR Questions; Pupil Ride = Pupil Size During Ride; & unstandardized beta.
Table 6. Best models for predicting cybersickness.
Table 6. Best models for predicting cybersickness.
PredictedPredictorsβ Coefficientp-Value (β)R2
CSQ-VR–TotalMSB-Adult0.66<0.001 ***0.39
CSQ-VR–NauseaMSB-Adult0.63<0.001 ***0.39
CSQ-VR–VestibularMSB-Adult0.530.003 **0.28
CSQ-VR–OculomotorNull Model
CSQ-VR = Cybersickness in Virtual reality Questionnaire; MS = motion sickness; ** p ≤ 0.01, *** p ≤ 0.001.
Table 7. Single-predictor models for verbal short-term memory.
Table 7. Single-predictor models for verbal short-term memory.
Predictorβ Coefficientp-ValueR2 (Fixed Effects/Overall)
Sex (Male)0.35 &0.2490.03/0.52
Education−0.130.4110.02/0.52
Age0.090.5340.01/0.52
Computing XP0.100.4960.01/0.52
Smartphone XP0.290.1530.05/0.52
Gaming XP0.100.5250.01/0.52
CSQ-VR–Total−0.120.2440.02/0.51
CSQ-VR–Nausea−0.010.9650.01/0.50
CSQ-VR–Vestibular−0.170.1050.03/0.52
CSQ-VR–Oculomotor−0.150.1730.03/0.54
CSQ-VR = Cybersickness in Virtual reality Questionnaire; XP = experience; & unstandardized beta.
Table 8. Single-predictor models for verbal working memory.
Table 8. Single-predictor models for verbal working memory.
Predictorβ Coefficientp-ValueR2 (Fixed Effects/Overall)
Sex (Male)0.27 &0.4040.02/0.54
Education0.090.5920.01/0.54
Age0.150.3510.02/0.54
Computing XP−0.020.9110.01/0.54
Smartphone XP0.100.6680.01/0.54
Gaming XP0.070.6790.01/0.54
CSQ-VR–Total−0.030.7980.01/0.53
CSQ-VR–Nausea−0.020.8620.01/0.52
CSQ-VR–Vestibular−0.050.6690.01/0.53
CSQ-VR–Oculomotor−0.010.9410.01/0.53
CSQ-VR = Cybersickness in Virtual reality Questionnaire; XP = experience; & unstandardized beta.
Table 9. Single-predictor models for visuospatial short-term memory.
Table 9. Single-predictor models for visuospatial short-term memory.
Predictorβ Coefficientp-ValueR2 (Fixed Effects/Overall)
Sex (Male)0.83 &0.006 **0.17/0.56
Education−0.130.4680.07/0.57
Age0.150.3630.02/0.57
Computing XP0.340.029 *0.11/0.57
Smartphone XP0.120.4760.01/0.57
Gaming XP0.460.003 **0.19/0.56
CSQ-VR–Total−0.070.5380.01/0.54
CSQ-VR–Nausea−0.060.5410.01/0.54
CSQ-VR–Vestibular−0.040.6940.01/0.54
CSQ-VR–Oculomotor−0.040.7600.01/0.55
CSQ-VR = Cybersickness in Virtual reality Questionnaire; XP = experience; * p ≤ 0.05, ** p ≤ 0.01; & unstandardized beta.
Table 10. Single-predictor models for visuospatial working memory.
Table 10. Single-predictor models for visuospatial working memory.
Predictorβ Coefficientp-ValueR2 (Fixed Effects/Overall)
Sex (Male)0.82 &0.013 *0.14/0.65
Education−0.120.5190.01/0.65
Age0.200.2330.04/0.65
Computing XP0.210.2210.04/0.65
Smartphone XP0.300.1860.05/0.65
Gaming XP0.400.015 *0.14/0.65
CSQ-VR–Total−0.190.0560.04/0.63
CSQ-VR–Nausea−0.120.2090.02/0.63
CSQ-VR–Vestibular−0.230.022 *0.05/0.66
CSQ-VR–Oculomotor−0.160.1410.03/0.63
CSQ-VR = Cybersickness in Virtual reality Questionnaire; XP = experience; * p ≤ 0.05,; & unstandardized beta.
Table 11. Single-predictor models for attentional time.
Table 11. Single-predictor models for attentional time.
Predictorβ Coefficientp-ValueR2 (Fixed Effects/Overall)
Sex (Male)−0.06 &0.8370.01/0.33
Education−0.140.3670.02/0.33
Age−0.160.2760.03/0.33
Computing XP−0.280.0600.07/0.33
Smartphone XP−0.170.2490.03/0.33
Gaming XP−0.250.0930.06/0.33
CSQ-VR–Total0.250.044 *0.06/0.34
CSQ-VR–Nausea0.210.0780.05/0.32
CSQ-VR–Vestibular0.160.1920.03/0.32
CSQ-VR–Oculomotor0.280.026 *0.08/0.36
CSQ-VR = Cybersickness in Virtual reality Questionnaire; XP = experience; * p ≤ 0.05; & unstandardized beta.
Table 12. Single-predictor models for motor time.
Table 12. Single-predictor models for motor time.
Predictorβ Coefficientp-ValueR2 (Fixed Effects/Overall)
Sex (Male)−0.59 &0.0750.08/0.69
Education0.180.2980.03/0.69
Age−0.030.8740.01/0.69
Computing XP−0.140.4170.02/0.69
Smartphone XP−0.140.4100.02/0.69
Gaming XP−0.380.018 *0.14/0.69
CSQ-VR–Total0.180.0670.03/0.67
CSQ-VR–Nausea0.200.030 *0.04/0.67
CSQ-VR–Vestibular0.140.1450.02/0.67
CSQ-VR–Oculomotor0.090.3680.01/0.67
CSQ-VR = Cybersickness in Virtual reality Questionnaire; XP = the Experience; * p ≤ 0.05; & unstandardized beta.
Table 13. Single-predictor models for reaction time.
Table 13. Single-predictor models for reaction time.
Predictorβ Coefficientp-ValueR2 (Fixed Effects/Overall)
Sex (Male)−0.65 &0.0540.10/0.75
Education0.100.5900.01/0.75
Age−0.070.6910.01/0.75
Computing XP−0.240.1570.06/0.76
Smartphone XP−0.200.2390.04/0.75
Gaming XP−0.410.013 *0.16/0.75
CSQ-VR–Total0.250.004 **0.06/0.76
CSQ-VR–Nausea0.220.009 **0.04/0.74
CSQ-VR–Vestibular0.190.028 *0.04/0.75
CSQ-VR–Oculomotor0.220.020 *0.05/0.76
CSQ-VR = Cybersickness in Virtual reality Questionnaire; XP = experience; * p ≤ 0.05, ** p ≤ 0.01; & unstandardized beta.
Table 14. Best models for predicting cognitive and motor skills.
Table 14. Best models for predicting cognitive and motor skills.
PredictedPredictorsβ Coefficientp-Value (β)R2 (Fixed Effects/Overall)
DST–ForwardNull Model
DST–BackwardNull Model
CBT–ForwardGaming XP0.460.003 **0.19/0.56
CBT–BackwardGaming XP0.360.034 *0.20/0.68
CSQ-VR–Vestibular−0.210.025 *
DLRT–ATCSQ-VR–Oculomotor0.280.026 *0.08/0.36
DLRT–MTGaming XP−0.360.023 *0.18/0.69
CSQ-VR–Nausea0.190.049 *
DLRT–RTGaming XP−0.350.028 *0.21/0.77
CSQ-VR–Total0.230.008 **
CSQ-VR = Cybersickness in Virtual reality Questionnaire; XP = experience; DST = Digit Span Test; CBT = Corsi Block Test; DLRT = Deary–Liewald Reaction Time test; RT = reaction time; AT = Attention Time; MT = Motor Time; * p ≤ 0.05, ** p ≤ 0.01.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Kourtesis, P.; Papadopoulou, A.; Roussos, P. Cybersickness in Virtual Reality: The Role of Individual Differences, Its Effects on Cognitive Functions and Motor Skills, and Intensity Differences during and after Immersion. Virtual Worlds 2024, 3, 62-93. https://doi.org/10.3390/virtualworlds3010004

AMA Style

Kourtesis P, Papadopoulou A, Roussos P. Cybersickness in Virtual Reality: The Role of Individual Differences, Its Effects on Cognitive Functions and Motor Skills, and Intensity Differences during and after Immersion. Virtual Worlds. 2024; 3(1):62-93. https://doi.org/10.3390/virtualworlds3010004

Chicago/Turabian Style

Kourtesis, Panagiotis, Agapi Papadopoulou, and Petros Roussos. 2024. "Cybersickness in Virtual Reality: The Role of Individual Differences, Its Effects on Cognitive Functions and Motor Skills, and Intensity Differences during and after Immersion" Virtual Worlds 3, no. 1: 62-93. https://doi.org/10.3390/virtualworlds3010004

Article Metrics

Back to TopTop