Cybersickness in Virtual Reality Questionnaire (CSQ-VR): A Validation and Comparison against SSQ and VRSQ
Abstract
:1. Introduction
1.1. Cybersickness, Cognition, and Motor Skills
1.2. Cybersickness Questionnaires
1.3. Research Aims
2. Materials and Methods
2.1. Virtual Environment Development
Linear and Angular Accelerations in VR
2.2. Cognitive and Psychomotor Skills’ Assessment
2.2.1. Verbal Working Memory
2.2.2. Visuospatial Working Memory
2.2.3. Psychomotor Skills
- (1)
- the reaction time (RT) to indicate overall psychomotor speed,
- (2)
- the attentional time (AT) to indicate attentional processing speed,
- (3)
- the motor time (MT) to indicate movement speed.
2.3. Cybersickness Questionnaires
- The SSQ is not specific to cybersickness, and the frequency and intensity of symptoms substantially differ between simulator sickness and cybersickness.
- The VRSQ does not consider nausea symptoms, and nausea symptoms are the second most frequent type of symptoms in cybersickness.
- VRSQ validation was performed in a study with a small sample size and a limited diversity of stimuli.
- Both the SSQ and VRSQ, being 4-point Likert scales, were not designed in line with the design guidelines for Likert scale questionnaires.
Cybersickness in VR Questionnaire
2.4. Participants and Procedures
2.5. Statistical Analyses
3. Results
3.1. Reliability and Validity
3.2. Detection of Temporary Decline due to Cybersickness
3.3. Mixed Model Regression Analysis
4. Discussion
4.1. Comparison of CSQ-VR, SSQ, and VRSQ
4.2. Limitations and Future Studies
5. Conclusions
Supplementary Materials
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Radianti, J.; Majchrzak, T.A.; Fromm, J.; Wohlgenannt, I. A systematic review of immersive virtual reality applications for higher education: Design elements, lessons learned, and research agenda. Comput. Educ. 2019, 147, 103778. [Google Scholar] [CrossRef]
- Xie, B.; Liu, H.; Alghofaili, R.; Zhang, Y.; Jiang, Y.; Lobo, F.D.; Li, C.; Li, W.; Huang, H.; Akdere, M.; et al. A Review on Virtual Reality Skill Training Applications. Front. Virtual Real. 2021, 2, 645153. [Google Scholar] [CrossRef]
- Kourtesis, P.; MacPherson, S.E. How immersive virtual reality methods may meet the criteria of the National Academy of Neuropsychology and American Academy of Clinical Neuropsychology: A software review of the Virtual Reality Everyday Assessment Lab (VR-EAL). Comput. Hum. Behav. Rep. 2021, 4, 100151. [Google Scholar] [CrossRef]
- Emmelkamp, P.M.; Meyerbröker, K. Virtual Reality Therapy in Mental Health. Annu. Rev. Clin. Psychol. 2021, 17, 495–519. [Google Scholar] [CrossRef] [PubMed]
- Cruz-Neira, C.; Fernández, M.; Portalés, C. Virtual Reality and Games. Multimodal Technol. Interact. 2018, 2, 8. [Google Scholar] [CrossRef] [Green Version]
- Rebenitsch, L.; Owen, C. Estimating cybersickness from virtual reality applications. Virtual Real. 2020, 25, 165–174. [Google Scholar] [CrossRef]
- Stanney, K.M.; Kennedy, R.S.; Drexler, J.M. Cybersickness is Not Simulator Sickness. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting; SAGE Publications: New York, NY, USA, 1997; Volume 41, pp. 1138–1142. [Google Scholar]
- Davis, S.; Nesbitt, K.; Nalivaiko, E. A Systematic Review of Cybersickness. In Proceedings of the 2014 Conference on Interactive Entertainment, Newcastle, Australia, 2 December 2014; Blackmore, K., Nesbitt, K., Smith, S.P., Eds.; ACM: New York, NY, USA, 2014. [Google Scholar] [CrossRef]
- LaViola, J.J. A discussion of cybersickness in virtual environments. ACM SIGCHI Bull. 2000, 32, 47–56. [Google Scholar] [CrossRef]
- Nesbitt, K.; Davis, S.; Blackmore, K.; Nalivaiko, E. Correlating reaction time and nausea measures with traditional measures of cybersickness. Displays 2017, 48, 1–8. [Google Scholar] [CrossRef]
- Kim, J.; Palmisano, S.; Luu, W.; Iwasaki, S. Effects of Linear Visual-Vestibular Conflict on Presence, Perceived Scene Stability and Cybersickness in the Oculus Go and Oculus Quest. Front. Virtual Real. 2021, 2, 582156. [Google Scholar] [CrossRef]
- Kourtesis, P.; Collina, S.; Doumas, L.A.A.; MacPherson, S.E. Technological Competence Is a Pre-condition for Effective Implementation of Virtual Reality Head Mounted Displays in Human Neuroscience: A Technological Review and Meta-Analysis. Front. Hum. Neurosci. 2019, 13, 342. [Google Scholar] [CrossRef] [Green Version]
- Saredakis, D.; Szpak, A.; Birckhead, B.; Keage, H.A.D.; Rizzo, A.; Loetscher, T. Factors Associated with Virtual Reality Sickness in Head-Mounted Displays: A Systematic Review and Meta-Analysis. Front. Hum. Neurosci. 2020, 14, 96. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Conner, N.O.; Freeman, H.R.; Jones, J.A.; Luczak, T.; Carruth, D.; Knight, A.C.; Chander, H. Virtual Reality Induced Symptoms and Effects: Concerns, Causes, Assessment & Mitigation. Virtual Worlds 2022, 1, 130–146. [Google Scholar] [CrossRef]
- Dahlman, J.; Sjörs, A.; Lindström, J.; Ledin, T.; Falkmer, T. Performance and Autonomic Responses during Motion Sickness. Hum. Factors: J. Hum. Factors Ergon. Soc. 2009, 51, 56–66. [Google Scholar] [CrossRef]
- Varmaghani, S.; Abbasi, Z.; Weech, S.; Rasti, J. Spatial and attentional after effects of virtual reality and relations to cybersickness. Virtual Real. 2021, 26, 659–668. [Google Scholar] [CrossRef]
- Mittelstaedt, J.M.; Wacker, J.; Stelling, D. VR aftereffect and the relation of cybersickness and cognitive performance. Virtual Real. 2018, 23, 143–154. [Google Scholar] [CrossRef]
- Nalivaiko, E.; Davis, S.L.; Blackmore, K.; Vakulin, A.; Nesbitt, K. Cybersickness provoked by head-mounted display affects cutaneous vascular tone, heart rate and reaction time. Physiol. Behav. 2015, 151, 583–590. [Google Scholar] [CrossRef]
- Kennedy, R.S.; Lane, N.E.; Berbaum, K.; Lilienthal, M.G. Simulator Sickness Questionnaire: An Enhanced Method for Quantifying Simulator Sickness. Int. J. Aviat. Psychol. 1993, 3, 203–220. [Google Scholar] [CrossRef]
- Sevinc, V.; Berkman, M.I. Psychometric evaluation of Simulator Sickness Questionnaire and its variants as a measure of cybersickness in consumer virtual environments. Appl. Ergon. 2019, 82, 102958. [Google Scholar] [CrossRef] [PubMed]
- Kim, H.K.; Park, J.; Choi, Y.; Choe, M. Virtual reality sickness questionnaire (VRSQ): Motion sickness measurement index in a virtual reality environment. Appl. Ergon. 2018, 69, 66–73. [Google Scholar] [CrossRef]
- Sharples, S.; Cobb, S.; Moody, A.; Wilson, J.R. Virtual reality induced symptoms and effects (VRISE): Comparison of head mounted display (HMD), desktop and projection display systems. Displays 2008, 29, 58–69. [Google Scholar] [CrossRef]
- Bohil, C.; Alicea, B.; Biocca, F.A. Virtual reality in neuroscience research and therapy. Nat. Rev. Neurosci. 2011, 12, 752–762. [Google Scholar] [CrossRef]
- Palmisano, S.; Mursic, R.; Kim, J. Vection and cybersickness generated by head-and-display motion in the Oculus Rift. Displays 2017, 46, 1–8. [Google Scholar] [CrossRef] [Green Version]
- Dawes, J. Do Data Characteristics Change According to the Number of Scale Points Used? An Experiment Using 5-Point, 7-Point and 10-Point Scales. Int. J. Mark. Res. 2008, 50, 61–104. [Google Scholar] [CrossRef]
- Joshi, A.; Kale, S.; Chandel, S.; Pal, D.K. Likert Scale: Explored and Explained. Br. J. Appl. Sci. Technol. 2015, 7, 396–403. [Google Scholar] [CrossRef]
- Taherdoost, H. What is the best response scale for survey and questionnaire design; review of different lengths of rating scale/attitude scale/Likert scale. Int. J. Acad. Res. Manag. 2019, 8, 1–10. [Google Scholar]
- Wakita, T.; Ueshima, N.; Noguchi, H. Psychological Distance between Categories in the Likert Scale. Educ. Psychol. Meas. 2012, 72, 533–546. [Google Scholar] [CrossRef]
- Kourtesis, P.; Collina, S.; Doumas, L.A.A.; MacPherson, S.E. Validation of the Virtual Reality Neuroscience Questionnaire: Maximum Duration of Immersive Virtual Reality Sessions without the Presence of Pertinent Adverse Symptomatology. Front. Hum. Neurosci. 2019, 13, 417. [Google Scholar] [CrossRef] [Green Version]
- Somrak, A.; Pogačnik, M.; Guna, J. Suitability and Comparison of Questionnaires Assessing Virtual Reality-Induced Symptoms and Effects and User Experience in Virtual Environments. Sensors 2021, 21, 1185. [Google Scholar] [CrossRef]
- Partala, T.; Surakka, V. Pupil size variation as an indication of affective processing. Int. J. Hum.-Comput. Stud. 2003, 59, 185–198. [Google Scholar] [CrossRef]
- Schwind, V.; Knierim, P.; Tasci, C.; Franczak, P.; Haas, N.; Henze, N. “These Are Not My Hands!”: Effect of Gender on the Perception of Avatar Hands in Virtual Reality. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, New York, NY, USA, 6–11 May 2017; pp. 1577–1582. [Google Scholar] [CrossRef]
- Bebko, A.O.; Troje, N.F. bmlTUX: Design and Control of Experiments in Virtual Reality and Beyond. I-Perception 2020, 11. [Google Scholar] [CrossRef]
- Bonato, F.; Bubka, A.; Palmisano, S. Combined Pitch and Roll and Cybersickness in a Virtual Environment. Aviat. Space Environ. Med. 2009, 80, 941–945. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Kourtesis, P.; Korre, D.; Collina, S.; Doumas, L.A.A.; MacPherson, S.E. Guidelines for the Development of Immersive Virtual Reality Software for Cognitive Neuroscience and Neuropsychology: The Development of Virtual Reality Everyday Assessment Lab (VR-EAL), a Neuropsychological Test Battery in Immersive Virtual Reality. Front. Comput. Sci. 2020, 1. [Google Scholar] [CrossRef] [Green Version]
- Wechsler, D. Wechsler Bellevue Adult Intelligence Scale; Williams & Wilkins: Baltimore, MD, USA, 1939. [Google Scholar]
- Corsi, P.M. Human memory and the medial temporal region of the brain. Diss. Abstr. Int. 1972, 34, 891. [Google Scholar]
- Deary, I.J.; Liewald, D.; Nissan, J. A free, easy-to-use, computer-based simple and four-choice reaction time programme: The Deary-Liewald reaction time task. Behav. Res. Methods 2010, 43, 258–268. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Golding, J.F. Predicting individual differences in motion sickness susceptibility by questionnaire. Pers. Individ. Differ. 2006, 41, 237–248. [Google Scholar] [CrossRef]
- Roth, H.L.; Lora, A.N.; Heilman, K.M. Effects of monocular viewing and eye dominance on spatial attention. Brain 2002, 125, 2023–2035. [Google Scholar] [CrossRef] [Green Version]
- Ursachi, G.; Horodnic, I.A.; Zait, A. How Reliable Are Measurement Scales? External Factors with Indirect Influence on Reliability Estimators. Procedia Econ. Financ. 2015, 20, 679–686. [Google Scholar] [CrossRef] [Green Version]
- Power, M.; Fell, G.; Wright, M. Principles for high-quality, high-value testing. Evid.-Base. Med. 2013, 18, 5–10. [Google Scholar] [CrossRef] [Green Version]
- Streiner, D.L.; Cairney, J. What’s under the ROC? An Introduction to Receiver Operating Characteristics Curves. Can. J. Psychiatry 2007, 52, 121–128. [Google Scholar] [CrossRef] [Green Version]
- Guilmette, T.J.; Sweet, J.J.; Hebben, N.; Koltai, D.; Mahone, E.M.; Spiegler, B.J.; Stucky, K.; Westerveld, M.; Participants, C. American Academy of Clinical Neuropsychology consensus conference statement on uniform labeling of performance test scores. Clin. Neuropsychol. 2020, 34, 437–453. [Google Scholar] [CrossRef]
- The Jamovi Project [Computer Software]. Available online: http://www.jamovi.org (accessed on 1 January 2020).
- R Core Team. A Language and Environment for Statistical Computing; R Foundation for Statistical Computing: Vienna, Austria, 2022; Available online: https://www.R-project.org/ (accessed on 12 December 2022).
- Peterson, R.A.; Cavanaugh, J.E. Ordered quantile normalization: A semiparametric transformation built for the cross-validation era. J. Appl. Stat. 2019, 47, 2312–2327. [Google Scholar] [CrossRef] [PubMed]
- Revelle, W. psych: Procedures for Psychological, Psychometric, and Personality Research; Northwestern University: Evanston, IL, USA, 2018; Available online: https://CRAN.R-project.org/package=psych (accessed on 12 December 2022).
- Wickham, H. ggplot2: Elegant Graphics for Data Analysis; Springer-Verlag: New York, NY, USA, 2016; Available online: https://ggplot2.tidyverse.org (accessed on 12 December 2022)ISBN 978-3-319-24277-4.
- Bates, D.; Mächler, M.; Bolker, B.; Walker, S. Fitting Linear Mixed-Effects Models Using lme4. J. Stat. Softw. 2015, 67, 48. [Google Scholar] [CrossRef]
- Bouchard, S.; Berthiaume, M.; Robillard, G.; Forget, H.; Daudelin-Peltier, C.; Renaud, P.; Blais, C.; Fiset, D. Arguing in Favor of Revising the Simulator Sickness Questionnaire Factor Structure When Assessing Side Effects Induced by Immersions in Virtual Reality. Front. Psychiatry 2021, 12, 739742. [Google Scholar] [CrossRef] [PubMed]
- Iii, W.B.S. Psychometric Evaluation of the Simulator Sickness Questionnaire as a Measure of Cybersickness. Ph.D. Thesis, Iowa State University, Ames, IA, USA, 2017. [Google Scholar] [CrossRef] [Green Version]
- Nunnally, J.C. Psychometric Theory—25 Years Ago and Now. Educ. Res. 1975, 4, 7–21. [Google Scholar] [CrossRef]
- Kourtesis, P.; Collina, S.; Doumas, L.A.; MacPherson, S.E. Validation of the Virtual Reality Everyday Assessment Lab (VR-EAL): An Immersive Virtual Reality Neuropsychological Battery with Enhanced Ecological Validity. J. Int. Neuropsychol. Soc. 2020, 27, 181–196. [Google Scholar] [CrossRef]
- Karray, F.; Alemzadeh, M.; Abou Saleh, J.; Arab, M.N. Human-Computer Interaction: Overview on State of the Art. Int. J. Smart Sens. Intell. Syst. 2008, 1, 137–159. [Google Scholar] [CrossRef] [Green Version]
- Stanney, K.; Lawson, B.D.; Rokers, B.; Dennison, M.; Fidopiastis, C.; Stoffregen, T.; Weech, S.; Fulvio, J.M. Identifying Causes of and Solutions for Cybersickness in Immersive Technology: Reformulation of a Research and Development Agenda. Int. J. Hum.–Comput. Interact. 2020, 36, 1783–1803. [Google Scholar] [CrossRef]
- Jeong, D.; Yoo, S.; Yun, J. Cybersickness Analysis with EEG Using Deep Learning Algorithms. In Proceedings of the 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), Osaka, Japan, 23–27 March 2019; pp. 827–835. [Google Scholar] [CrossRef]
- Krokos, E.; Varshney, A. Quantifying VR cybersickness using EEG. Virtual Real. 2021, 26, 77–89. [Google Scholar] [CrossRef]
- Lopes, P.; Tian, N.; Boulic, R. Eye Thought You Were Sick! Exploring Eye Behaviors for Cybersickness Detection in VR. In Proceedings of the 13th ACM SIGGRAPH Conference on Motion, Interaction and Games, New York, NY, USA, 16–18 October 2020. [Google Scholar] [CrossRef]
- Islam, R.; Desai, K.; Quarles, J. Cybersickness Prediction from Integrated HMD’s Sensors: A Multimodal Deep Fusion Approach using Eye-tracking and Head-tracking Data. In Proceedings of the 2021 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Bari, Italy, 4–8 October 2021; pp. 31–40. [Google Scholar] [CrossRef]
Mean (SD) | Range | Max. Score | |
---|---|---|---|
Sex (22F/17M) | - | - | - |
Age | 25.28 (3.22) | 22–36 | - |
Years of Education | 15.14 (5.18) | 13–20 | - |
VR Experience | 2.67 (0.92) | 2–6 | 14 |
Computing Experience | 10.36 (0.80) | 9–12 | 14 |
Gaming Experience | 5.54 (2.97) | 2–12 | 14 |
MSSQ Child Score | 4.69 (3.34) | 0–13.50 | 27 |
MSSQ Adult Score | 3.91 (3.20) | 0–11.25 | 27 |
MSSQ Total Score | 8.60 (5.23) | 0–20.13 | 54 |
Pupil Size (mm) | 5.37 (0.90) | 3.70–8.32 | - |
CSQ-VR (VR) Total Score * | 10.63 (4.97) | 6–28 | 42 |
CSQ-VR (VR) Nausea Score * | 3.18 (1.56) | 2–9 | 14 |
CSQ-VR (VR) Vestibular Score * | 3.66 (2.43) | 2–13 | 14 |
CSQ-VR (VR) Oculomotor Score * | 3.79 (1.70) | 2–9 | 14 |
CSQ-VR Total Score | 12.23 (4.96) | 6–27 | 42 |
CSQ-VR Nausea Score | 3.51 (1.68) | 2–9 | 14 |
CSQ-VR Vestibular Score | 3.97 (2.41) | 2–10 | 14 |
CSQ- VR Oculomotor Score | 4.74 (1.81) | 2–10 | 14 |
SSQ-Total Score | 67.24 (48.09) | 0–223.66 | 300 |
SSQ-Nausea Score | 24.22 (22.09) | 0–95.40 | 100 |
SSQ-Disorientation Score | 9.40 (9.98) | 0–44.88 | 100 |
SSQ-Oculomotor Score | 33.62 (21.84) | 0–83.38 | 100 |
VRSQ-Total Score | 19.17 (13.27) | 0–59.17 | 100 |
VRSQ-Disorientation Score | 11.62 (13.27) | 0–60.00 | 100 |
VRSQ-Oculomotor Score | 26.71 (15.63) | 0–58.33 | 100 |
Experimental Stage | CSQ-VR Scores * | Mean (SD) | Range | Max. Score |
---|---|---|---|---|
Baseline | Total Score | 7.59 (2.09) | 6–16 | 42 |
Nausea Score | 2.23 (0.54) | 2–4 | 14 | |
Vestibular Score | 2.38 (0.85) | 2–6 | 14 | |
Oculomotor Score | 2.79 (1.11) | 2–6 | 14 | |
Ride 1 | Total Score | 10.79 (4.35) | 6–24 | 42 |
Nausea Score | 3.41 (1.37) | 2–8 | 14 | |
Vestibular Score | 3.97 (2.47) | 2–12 | 14 | |
Oculomotor Score | 3.41 (1.41) | 2–8 | 14 | |
Ride 2 | Total Score | 11.87 (5.03) | 6–23 | 42 |
Nausea Score | 3.54 (1.57) | 2–8 | 14 | |
Vestibular Score | 4.13 (2.56) | 2–12 | 14 | |
Oculomotor Score | 4.21 (1.73) | 2–9 | 14 | |
Ride 3 | Total Score | 12.26 (6.19) | 6–28 | 42 |
Nausea Score | 3.54 (2.02) | 2–9 | 14 | |
Vestibular Score | 4.15 (2.91) | 2–13 | 14 | |
Oculomotor Score | 4.56 (2.00) | 2–9 | 14 |
Questionnaire | Scores | Cronbach’s α |
---|---|---|
CSQ-VR | Total Score | 0.865 |
Nausea | 0.792 | |
Vestibular | 0.934 | |
Oculomotor | 0.704 | |
SSQ | Total Score | 0.810 |
Nausea | 0.676 | |
Disorientation | 0.809 | |
Oculomotor | 0.744 | |
VRSQ | Total Score | 0.806 |
Disorientation | 0.718 | |
Oculomotor | 0.654 |
Correlation Pair | Pearson’s r | p-Value | |
---|---|---|---|
CSQ-VR–Total Score | VRSQ–Total Score | 0.77 | <0.001 |
CSQ-VR–Oculomotor | VRSQ–Oculomotor | 0.75 | <0.001 |
CSQ-VR–Vestibular | VRSQ–Disorientation | 0.55 | <0.001 |
CSQ-VR (VR)–Total Score | VRSQ–Total Score | 0.65 | <0.001 |
CSQ-VR (VR)–Oculomotor | VRSQ–Oculomotor | 0.62 | <0.001 |
CSQ-VR (VR)–Vestibular | VRSQ–Disorientation | 0.52 | <0.001 |
Cybersickness Score | Cut-Off | Sensitivity (%) | Specificity (%) | PPV (%) | NPV (%) | AUC (%) | Metric Score |
---|---|---|---|---|---|---|---|
CSQ-VR–Total Score | 10 | 100% | 75% | 15.15% | 100% | 87% | 1.75 |
CSQ-VR (VR)–Total Score | 9 | 100% | 75% | 15.15% | 100% | 86.5% | 1.75 |
SSQ–Total Score | 83.36 | 80% | 68.75% | 10.26% | 98.72% | 66.1% | 1.49 |
VRSQ–Total Score | 20 | 100% | 53.57% | 8.77% | 100% | 66.6% | 1.54 |
Cybersickness Score | Cut-off | Sensitivity (%) | Specificity (%) | PPV (%) | NPV (%) | AUC (%) | Metric Score |
---|---|---|---|---|---|---|---|
CSQ-VR–Total Score | 10 | 100% | 75.68% | 18.18% | 100% | 86.9% | 1.76 |
CSQ-VR (VR)–Total Score | 9 | 100% | 75.68% | 18.18% | 100% | 88% | 1.76 |
SSQ–Total Score | 83.36 | 83.33% | 69.37% | 12.82% | 98.72% | 68% | 1.53 |
VRSQ–Total Score | 20 | 100% | 54.05% | 10.53% | 100% | 67.53% | 1.54 |
Cybersickness Score | Cut-Off | Sensitivity (%) | Specificity (%) | PPV (%) | NPV (%) | AUC (%) | Metric Score |
---|---|---|---|---|---|---|---|
CSQ-VR–Nausea | 3 | 60% | 67.86% | 7.69% | 97.44% | 65.3% | 1.28 |
CSQ-VR–Vestibular | 5 | 100% | 77.68% | 16.67% | 100% | 92.6% | 1.78 |
CSQ-VR–Oculomotor | 7 | 40% | 93.75% | 22.22% | 97.22% | 65.8% | 1.34 |
CSQ-VR(VR)–Nausea | 3 | 100% | 66.96% | 11.09% | 100% | 83.6% | 1.67 |
CSQ-VR(VR)–Vestibular | 4 | 100% | 70.54% | 13.16% | 100% | 86.7% | 1.71 |
CSQ-VR (VR)–Oculomotor | 6 | 40% | 90.18% | 15.38% | 97.12% | 61.2% | 1.30 |
SSQ–Nausea | 47.7 | 40% | 88.39% | 13.33% | 97.06% | 60.04% | 1.28 |
SSQ–Disorientation | 11.22 | 100% | 64.29% | 11.11% | 100% | 70.1% | 1.64 |
SSQ–Oculomotor | 45.48 | 80% | 58.04% | 7.84% | 98.48% | 67.9% | 1.38 |
VRSQ–Disorientation | 20 | 80% | 74.01% | 12.12% | 98.81% | 73.06% | 1.54 |
VRSQ–Oculomotor | 33.33 | 100% | 53.57% | 8.77% | 100% | 63.4% | 1.54 |
Cybersickness Score | Cut-Off | Sensitivity (%) | Specificity (%) | PPV (%) | NPV (%) | AUC (%) | Metric Score |
---|---|---|---|---|---|---|---|
CSQ-VR–Nausea | 2 | 100% | 32.43% | 7.41% | 100% | 62.6% | 1.32 |
CSQ-VR–Vestibular | 5 | 100% | 78.38% | 20% | 100% | 94.4% | 1.78 |
CSQ-VR–Oculomotor | 7 | 33.33% | 93.69% | 22.22% | 96.3% | 61% | 1.27 |
CSQ-VR(VR)–Nausea | 3 | 100% | 67.57% | 14.29% | 100% | 85.1% | 1.68 |
CSQ-VR(VR)–Vestibular | 4 | 100% | 71.17% | 15.79% | 100% | 89.3% | 1.71 |
CSQ-VR (VR)–Oculomotor | 6 | 33.33% | 90.09% | 15.38% | 96.15% | 56.5% | 1.23 |
SSQ–Nausea | 47.7 | 50% | 89.19% | 20% | 97.06% | 65.08% | 1.39 |
SSQ–Disorientation | 11.22 | 100% | 64.86% | 13.33% | 100% | 70.3% | 1.65 |
SSQ–Oculomotor | 45.48 | 83.33% | 58.56% | 9.8% | 98.48% | 67.8% | 1.42 |
VRSQ–Disorientation | 20 | 82.3% | 74.77% | 15.15% | 98.81% | 75% | 1.58 |
VRSQ–Oculomotor | 33.33 | 100% | 54.05% | 10.53% | 100% | 63.5% | 1.54 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Kourtesis, P.; Linnell, J.; Amir, R.; Argelaguet, F.; MacPherson, S.E. Cybersickness in Virtual Reality Questionnaire (CSQ-VR): A Validation and Comparison against SSQ and VRSQ. Virtual Worlds 2023, 2, 16-35. https://doi.org/10.3390/virtualworlds2010002
Kourtesis P, Linnell J, Amir R, Argelaguet F, MacPherson SE. Cybersickness in Virtual Reality Questionnaire (CSQ-VR): A Validation and Comparison against SSQ and VRSQ. Virtual Worlds. 2023; 2(1):16-35. https://doi.org/10.3390/virtualworlds2010002
Chicago/Turabian StyleKourtesis, Panagiotis, Josie Linnell, Rayaan Amir, Ferran Argelaguet, and Sarah E. MacPherson. 2023. "Cybersickness in Virtual Reality Questionnaire (CSQ-VR): A Validation and Comparison against SSQ and VRSQ" Virtual Worlds 2, no. 1: 16-35. https://doi.org/10.3390/virtualworlds2010002