Combining Virtual Reality and Organizational Neuroscience for Leadership Assessment
Abstract
:Featured Application
Abstract
1. Introduction
2. Literature Review
2.1. Evidence-Centered Design for Serious Games
2.2. Biological Implicit Measures of Leadership Behavior
3. Materials and Methods
3.1. Participants
3.2. Leadership Assessment
3.3. Serious Game Task Modeling
3.4. Experimental Procedure
3.5. Eye-Tracking Measurement and Data Processing
3.6. Statistical Analysis
3.7. Machine Learning
4. Results
4.1. TOL and ROL Description
4.2. Statistical Significance of Leadership Styles in the Serious Game
4.3. Automatic Leadership Recognition Models
5. Discussion
5.1. High and Low TOL/ROL Differences between Measures
5.2. ML Methods for TOL/ROL Style Discrimination and Features that Better Discriminate between the Two Styles
5.3. Theoretical Implications
5.4. Practical Implications
5.5. Limitations and Future Directions
6. Conclusions
Supplementary Materials
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
Appendix A
Appendix B
References
- Bass, B.M.; Avolio, B.J. Developing transformational leadership: 1992 and beyond. J. Eur. Ind. Train. 1990. [Google Scholar] [CrossRef]
- Dvir, T.; Eden, D.; Avolio, B.J.; Shamir, B. Impact of transformational leadership on follower development and performance: A field experiment. Acad. Manag. J. 2002, 45, 735–744. [Google Scholar]
- Fiedler, F.E. A theory of leadership effectiveness. In Mcgraw-Hill Series in Management; McGraw Hill Text: New York, NY, USA, 1967. [Google Scholar]
- Hersey, P.; Blanchard, K.H.; Johnson, D.E. Management of Organizational Behavior; Prentice Hall: Hoboken, NJ, USA, 2007; Volume 9. [Google Scholar]
- Manktelow, J.; Carlson, A. Leadership Style. Mind Tools Retrieved 2012, 2. [Google Scholar]
- Yukl, G. Leadership in Organizations, 5th ed.; Prentice Hall Inc.: New Jersey, NJ, USA, 2002. [Google Scholar]
- Anzalone, C. Differences between task-oriented leaders & relational-oriented leaders. Demand Media. 2012. Available online: https://smallbusiness.chron.com/differences-between-taskoriented-leaders-relationaloriented-leaders-35998.html (accessed on 24 June 2021).
- Grimm, P. Social Desirability Bias. In Wiley International Encyclopedia of Marketing; Wiley: New York, NY, USA, 2010. [Google Scholar]
- Nederhof, A.J. Methods of coping with social desirability bias: A review. Eur. J. Soc. Psychol. 1985, 15, 263–280. [Google Scholar] [CrossRef]
- Schmuckler, M.A. What is ecological validity? A dimensional analysis. Infancy 2001, 2, 419–436. [Google Scholar] [CrossRef] [PubMed]
- Kato, P.M.; de Klerk, S. Serious games for assessment: Welcome to the Jungle. J. Appl. Test. Technol. 2017, 18, 1–6. [Google Scholar]
- Bellotti, F.; Kapralos, B.; Lee, K.; Moreno-Ger, P.; Berta, R. Assessment in and of Serious Games: An Overview. Adv. Human-COMPUT. Interact. 2013, 2013, 1–11. [Google Scholar] [CrossRef] [Green Version]
- Pasquier, P.; Mérat, S.; Malgras, B.; Petit, L.; Queran, X.; Bay, C.; Boutonnet, M.; Jault, P.; Ausset, S.; Auroy, Y.; et al. A Serious Game for Massive Training and Assessment of French Soldiers Involved in Forward Combat Casualty Care (3D-SC1): Development and Deployment. JMIR Serious Games 2016, 4, e5. [Google Scholar] [CrossRef] [PubMed]
- Shute, V.J.; Wang, L.; Greiff, S.; Zhao, W.; Moore, G. Measuring problem solving skills via stealth assessment in an engaging video game. Comput. Hum. Behav. 2016, 63, 106–117. [Google Scholar] [CrossRef]
- Chatziantoniou, N.; Politopoulos, N.; Stylianidis, P. Designing and Developing an Educational Game for Leadership Assessment and Soft Skill Optimization. In Interactive Mobile Communication, Technologies and Learning; Springer: Cham, Switzerland, 2017; pp. 258–265. [Google Scholar]
- Vos, L. Simulation games in business and marketing education: How educators assess student learning from simulations. Int. J. Manag. Educ. 2015, 13, 57–74. [Google Scholar] [CrossRef]
- Giglioli, I.A.C.; Ripoll, C.D.J.; Parra, E.; Raya, M.A. Are 3D virtual environments better than 2D interfaces in serious games performance? An explorative study for the assessment of executive functions. Appl. Neuropsychol. Adult 2021, 28, 148–157. [Google Scholar] [CrossRef] [PubMed]
- Mislevy, R.J.; Almond, R.G.; Lukas, J.F. A brief introduction to evidence-centered design. In ETS Research Report Series; American Psychological Association (APA): Colombia, WA, USA, 2003; Volume 1, p. 29. [Google Scholar]
- Alcañiz, M.; Parra, E.; Giglioli, I.A.C. Virtual Reality as an Emerging Methodology for Leadership Assessment and Training. Front. Psychol. 2018, 9, 1658. [Google Scholar] [CrossRef] [PubMed]
- Giglioli, I.; Carrasco-Ribelles, L.; Parra, E.; Marín-Morales, J.; Raya, M.A. An Immersive Serious Game for the Behavioral Assessment of Psychological Needs. Appl. Sci. 2021, 11, 1971. [Google Scholar] [CrossRef]
- De-Juan-Ripoll, C.; Llanes-Jurado, J.; Giglioli, I.A.C.; Marín-Morales, J.; Alcañiz, M. An Immersive Virtual Reality Game for Predicting Risk Taking through the Use of Implicit Measures. Appl. Sci. 2021, 11, 825. [Google Scholar] [CrossRef]
- Becker, W.J.; Menges, J.I. Biological implicit measures in HRM and OB: A question of how not if. Hum. Resour. Manag. Rev. 2013, 23, 219–228. [Google Scholar] [CrossRef]
- Gerpott, F.H.; Lehmann-Willenbrock, N.; Silvis, J.D.; Van Vugt, M. In the eye of the beholder? An eye-tracking experiment on emergent leadership in team interactions. Leadersh. Q. 2018, 29, 523–532. [Google Scholar] [CrossRef]
- Balthazard, P.A.; Waldman, D.A.; Thatcher, R.W.; Hannah, S.T. Differentiating transformational and non-transformational leaders on the basis of neurological imaging. Leadersh. Q. 2012, 23, 244–258. [Google Scholar] [CrossRef] [Green Version]
- Hannah, S.T.; Balthazard, P.A.; Waldman, D.A.; Jennings, P.L.; Thatcher, R.W. The psychological and neurological bases of leader self-complexity and effects on adaptive decision-making. J. Appl. Psychol. 2013, 98, 393–411. [Google Scholar] [CrossRef]
- Rowe, A.J.; Boulgarides, J.D. The Decision Maker. Managerial Decision Making: A Guide to Successful Business Decisions; John Wiley and Sons: Hoboken, NJ, USA, 1992; pp. 21–43. [Google Scholar]
- Leonard, N.H.; Scholl, R.W.; Kowalski, K.B. Information processing style and decision making. J. Organ. Behav. 1999, 20, 407–420. [Google Scholar] [CrossRef]
- Martinsons, M.G.; Davison, R.M. Strategic decision making and support systems: Comparing American, Japanese and Chinese management. Decis. Support Syst. 2007, 43, 284–300. [Google Scholar] [CrossRef]
- Torres, P.; Augusto, M. The impact of experiential learning on managers’ strategic competencies and decision style. J. Innov. Knowl. 2017, 2, 10–14. [Google Scholar] [CrossRef] [Green Version]
- Halpin, A.W.; Winer, B.J. The observed leader behavior and ideal leader behavior of aircraft commanders and school superintendents. In Leader Behavior: Its Description and Measurement; Stogdill, R.M., Coons, A.E., Eds.; Ohio State University, Bureau of Business Research: Columbus, OH, USA, 1957; pp. 65–68. [Google Scholar]
- Meißner, M.; Oll, J. The promise of eye-tracking methodology in organizational research: A taxonomy, review, and future avenues. Organ. Res. Methods 2019, 22, 590–617. [Google Scholar] [CrossRef]
- Frischen, A.; Bayliss, A.P.; Tipper, S.P. Gaze cueing of attention: Visual attention, social cognition, and individual differences. Psychol. Bull. 2007, 133, 694. [Google Scholar] [CrossRef] [PubMed]
- Tskhay, K.O.; Zhu, R.; Rule, N.O. Perceptions of charisma from thin slices of behavior predict leadership prototypicality judgments. Leadersh. Q. 2017, 28, 555–562. [Google Scholar] [CrossRef]
- Eckstein, M.K.; Guerra-Carrillo, B.; Singley, A.T.M.; Bunge, S.A. Beyond eye gaze: What else can eyetracking reveal about cognition and cognitive development? Dev. Cogn. Neurosci. 2017, 25, 69–91. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Blake, R.; Mouton, J. The Managerial Grid: The Key to Leadership Excellence; Gulf Publishing Co: Houston, TX, USA, 1964; p. 350. [Google Scholar]
- Gibaldi, A.; Vanegas, M.; Bex, P.J.; Maiello, G. Evaluation of the Tobii EyeX Eye tracking controller and Matlab toolkit for research. Behav. Res. Methods 2017, 49, 923–946. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Filzmoser, P.; Viertl, R. Testing hypotheses with fuzzy data: The fuzzy p-value. Metrika 2004, 59, 21–29. [Google Scholar] [CrossRef]
- Doak, J. Cse-92-18-an Evaluation of Feature Selection Methodsand Their Application to Computer Security. UC Davis. 1992. Available online: https://escholarship.org/uc/item/2jf918dh (accessed on 26 June 2021).
- Iederan, O.C.; Curşeu, P.L.; Vermeulen, P.A.M. Effective Decision-Making: The Role of Cognitive Complexity in Strategic Decisions; Radboud University: Nijmegen, The Netherlands, 2009. [Google Scholar]
- Lindebaum, D.; Jordan, P. A critique on neuroscientific methodologies in organizational behavior and management studies. J. Organ. Behav. 2014, 35, 898–908. [Google Scholar] [CrossRef] [Green Version]
- Madera, J.M.; Hebl, M.R. Discrimination against facially stigmatized applicants in interviews: An eye-tracking and face-to-face investigation. J. Appl. Psychol. 2012, 97, 317–330. [Google Scholar] [CrossRef] [PubMed] [Green Version]
Group of Variables | Definition | Number of Variables | |
---|---|---|---|
Situations | Decision-making | 29 | |
Eye Tracking | Role of the virtual agent the participant was looking at in that moment | Mean time (s) | 9 |
Mean number of fixations (n) | 9 | ||
Mean fixation time (s) | 9 | ||
Mean time to first fixation (s) | 9 | ||
Virtual agent the participant was looking at | Mean time (s) | 5 | |
Mean number of fixations (n) | 5 | ||
Mean fixation time (s) | 5 | ||
Mean time to first fixation (s) | 5 |
Metric | Definition |
---|---|
Mean time (s) | Mean time the participant looked at the defined virtual agent/role during the whole experience. |
Mean number of fixations (s) | Mean number of times the participant fixed the sight on the particular virtual agent/role during the whole experience. |
Mean fixation time (s) | Mean time that each fixation in a virtual agent/role lasted during the whole experience. |
Mean time to first fixation (s) | Time to first fixation is the time that elapses from the beginning of the experiment until the first time the eyes are fixed on the virtual agent/role. This metric defines the mean time to first fixation after the whole experience. |
Algorithm | Parameter | Values |
---|---|---|
GLM | Alpha | (0, 1) |
SVM | C | 2^(−10, 10) |
Sigma | 2^(−10, 10) | |
kNN | K | (3, 5, 7, 9, 11) |
Subscale | Level | N | Mean | Standard Deviation | p-Value |
---|---|---|---|---|---|
Relationship-oriented Leadership (ROL) | High | 28 | 11.8 | 0.92 | <0.001 |
Low | 23 | 7.26 | 1.76 | ||
Task-oriented Leadership (TOL) | High | 25 | 8.2 | 1.12 | <0.001 |
Low | 26 | 3.07 | 0.41 |
Variable | Subscale | Test | p-Value | High Level | Low Level |
---|---|---|---|---|---|
S5: Communication | ROL | Chi-square | 0.040 | +: 7%, −: 7%, None: 86% | +: 13%, −: 26%, None: 61% |
Passive Role: Mean Time | ROL | Wilcoxon | 0.036 | 9.66 ± 1.06 | 9.02 ± 1.05 |
AOI Martina: Mean Time | ROL | Wilcoxon | 0.047 | 1.02 ± 1.01 | 9.57 ± 1.09 |
Supportive Role: Mean Fixation Time | TOL | t-test | 0.048 | 0.99 ± 0.40 | 0.80 ± 0.22 |
Subscale | Algorithm | Features | Accuracy | Kappa | TPR | TNR | ||
---|---|---|---|---|---|---|---|---|
Decisions | ET: Role | ET: Virtual Agent | ||||||
ROL | kNN | 9 | 3 | 2 | 0.79 | 0.51 | 0.71 | 0.82 |
TOL | GLM | 2 | 7 | 1 | 0.76 | 0.52 | 0.77 | 0.76 |
ROL Features | ||
---|---|---|
Decisions | 9 | S4: communication S4: communication S5: communication S5: communication S9: execution S9: execution S1: execution S3: execution S7: giving orders |
ET: Role | 3 | Role: Mean Time Empathy Role: First Fixation Mean Time Background Role: Mean Time |
ET: AOI | 2 | AOIMARTINA: Mean Time (organizer) AOISUSANA: Mean Time (strategist) |
TOL Features | ||
---|---|---|
Decisions | 2 | S4: communication S5: execution |
ET: Role | 7 | Supportive Role: Mean Time Supportive Role: Mean Number of Fixations Decision-making Role: Mean Time Decision-making Role: First Fixation Mean Time Informative Role: Mean Time Passive Role: Mean Fixation Time Background Role: Mean Number of Fixations |
ET: AOI | 1 | AOI MARCOS: Mean Time (Communicative) |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Parra, E.; Chicchi Giglioli, I.A.; Philip, J.; Carrasco-Ribelles, L.A.; Marín-Morales, J.; Alcañiz Raya, M. Combining Virtual Reality and Organizational Neuroscience for Leadership Assessment. Appl. Sci. 2021, 11, 5956. https://doi.org/10.3390/app11135956
Parra E, Chicchi Giglioli IA, Philip J, Carrasco-Ribelles LA, Marín-Morales J, Alcañiz Raya M. Combining Virtual Reality and Organizational Neuroscience for Leadership Assessment. Applied Sciences. 2021; 11(13):5956. https://doi.org/10.3390/app11135956
Chicago/Turabian StyleParra, Elena, Irene Alice Chicchi Giglioli, Jestine Philip, Lucia Amalia Carrasco-Ribelles, Javier Marín-Morales, and Mariano Alcañiz Raya. 2021. "Combining Virtual Reality and Organizational Neuroscience for Leadership Assessment" Applied Sciences 11, no. 13: 5956. https://doi.org/10.3390/app11135956