The aviation industry has a high degree of standardization and control. Safety is the top priority, which has made flying the safest means of transportation. Pilots flying complex aircraft in commercial air traffic require specified levels of qualifications. Apart from the initial license, the pilot needs an advanced, aircraft type-specific training, which is endorsed in his/her license. Regular check flights are required to maintain the rights associated with a type rating (TR) license. During TR training, pilots learn the specifics of certain types of aircraft. The training consists of a theoretical and a practical part, which usually takes place on a certified full-flight simulator that is specific for the aircraft type. These devices are expensive and, depending on the aircraft type, rare, especially in business aviation, depending on the worldwide aircraft fleet size.
Since particular flight simulators are spread at different locations across the world, pilots must travel long distances for the TR training. Travel times of twelve hours or more are usual. The overall TR course duration can reach 30 or 45 days. Depending on the aircraft type, a TR training can cost up to EUR 80,000 per pilot, which is a high investment for an aircraft operator or for a pilot.
Anticipating the technological changes in aviation, research on improving the quality of TR training and the learning conditions is important for the industry.
Because aeronautical education is highly standardized, the training providers must comply with standards and receive approval for their courses from the competent aviation authority. As a consequence, the courses offered on the market are similar in both content and extent. Methods for differentiation from the competition are customer service, availability of courses, use of modern equipment, and pricing.
The practical part of pilot training is mainly conducted in modern full-flight simulators as these replicate the aircraft in detail and provide a safe and efficient environment to practice normal and emergency procedures. Theoretical knowledge is acquired using computer-based training (CBT), lectures, self-study and physical devices, representing cockpit instrumentation. The latter area is showing an increased use of new technologies, such as virtual reality trainers or other advanced digital solutions. New technologies will prevail, if they add extra value to the education or replace existing applications in terms of cost and learning success. In the course of this, training providers are facing the challenges arising from the interaction between production costs versus profitability, and the provision of high-quality training. Actual market trends show that virtual reality applications are finding their way into pilot training. The European Union Aviation Safety Agency (EASA) approved the first virtual-reality-based flight simulator in April 2021. This represents an evolution and is likely to make more cost-effective training devices available to complement full-flight simulators. In this regard, large pilot training providers have an advantage because they have the market power and resources to develop their own applications, which they can distribute to a large number of users.
As the TR course is comprehensive and complex, priorities need to be set for its future development. For a learner-centered and inclusive TR training, various social and economic factors need to be considered. Because women are underrepresented in the pilot population worldwide [1
], flying an aircraft is stereotypically assigned to men. Thus, technological developments in support of advanced pilot training can be prone to gender bias [3
]. As the aviation industry is interested in attracting talented women [4
], it is important to address gender aspects in shaping the future of TR training.
Objective and Structure
The present study aimed to explore the preferences and difficulties experienced by pilots during the advanced TR training. In addition, preferences for the development of AR applications were investigated. By considering gender-specific preferences and difficulties in the development of future AR-based learning aids, which were aimed at in this study, the advanced pilot training should become more inclusive and learner-centered.
The rest of the article is structured as follows: In related work (Section 2
), more information is given on the contents this study is based on. In methods (Section 3
), the participants, the survey, and the data-processing methods are presented. Section 4
is dedicated to the results, followed by discussion (Section 5
), conclusions (Section 6
), and recommendations for future research (Section 7
2. Related Work
2.1. Gender Aspects Related to Flight Training
Gender differences in flight training and barriers experienced by women who may have different starting conditions and experiences than men have been reported [5
]. Research of cognitive processing has shown that men were better in visuospatial tasks that require mental rotation of objects, and women were better in verbal tasks [6
]. However, research confirms that specific types of training can help both women and men to overcome initial gender differences and to reach comparable levels of performance. In an experiment, Neubauer et al. [7
] found that although men performed better than women in two-dimensional visuospatial, mental rotation tasks, training had a positive effect on the performance of both genders, and as a result the gender difference decreased after training. Interestingly, gender differences were absent when tasks were performed in three-dimensional, as compared to two-dimensional, settings [7
Research into flight training confirms these findings, showing that women and men can reach a similar level of flight performance after the same training periods, especially when the training concept used an inclusive design. Bauer et al. [8
] found that despite initial gender differences in the pre-test phase, both women and men significantly improved their performance of upset recovery and reached similar levels of performance during post-test, after the same number of simulator exercises. In addition, Koglbauer and Braunstingl [9
] investigated the effects of training traffic separation and airport procedures in a naturalistic flight simulation environment. They did not find gender differences in performance but in the subjective workload related to specific flight tasks that involved processing of both visuospatial and verbal information [9
]. Ideally, the future developments in TR training should be made with full knowledge about gender diversity and should address the needs and preferences of diverse learners.
2.2. Theoretical Instruction
Theoretical flight instruction can be performed in the classroom or remotely by means, for example, of CBT, distance learning, or online instruction. Synchronous learning requires the trainees and the instructor to participate at the same time in a real or a virtual classroom. Asynchronous learning allows trainees more flexibility, as they can learn from pre-recorded material (e.g., texts, animations, and explanatory videos) at their own pace. Research from various domains shows that students can achieve similar levels of performance when attending a class synchronously or asynchronously [10
]. In addition, research shows that blended online synchronous and asynchronous learning could improve student engagement with class activities and their experience of connectivity with their peers and instructors [11
]. Trainee communication, interaction, and collaboration are considered to be more straightforward in synchronous learning (e.g., video, audio, or web conferencing; live chat; white boarding; and application sharing) [12
2.3. Practical Instruction
Flight skills can be acquired and maintained by practice not only in the aircraft but also in flight simulators with various degrees of fidelity. Research has investigated the effectiveness of simulator training for pilots. For example, Hays et al. [13
] found the combination of simulated and real flight more effective than training with one single training method only. Research into this shows a significant improvement in pilot procedural memory and generalization of skills acquired by pilots in a flight simulator and positive transfer to a new situation [14
]. Furthermore, real flight performance can be significantly improved by training in a flight simulator [8
2.4. Augmented Reality
The concepts of virtual reality (VR) and augmented reality (AR) are best described with the reality–virtuality continuum by Milgram et al. [20
]. Here the space between reality and virtual reality is seen as a continuum referred to as mixed reality, which includes AR [20
]. In the last decades, AR and VR were increasingly present in a multitude of use cases. Vergara et al. [21
] mentions a number of examples for using virtual environments in education to attract and inspire students from different disciplines, e.g., electrical installations, hydraulics, or medicine. Among others, AR training means have been explored in virtual laboratories and 3D visualizations [21
], educational gaming [24
], interactive manipulation of holographic learning materials [25
], and enrichment for the content of a book with pop-ups including 3D or animated AR content [26
]. Wang et al. [28
] analyzed AR in the area of education, showing the utilization for different pedagogical approaches. They concluded that AR technology represents an enormous potential for innovate learning and teaching. A similar conclusion was reached by Yuen et al. [27
] where various educational applications were explored. In AR books the publication is extended with virtual information using AR, or in skill training AR facilitates the practice of certain tasks or procedures [27
]. Virtual environments foster the transfer of theoretical knowledge into practice and allow the risk-free usage of expensive devices [22
]. Additionally, AR can improve the learning experience and collaboration among student peers or between instructors and students [29
]. AR was interesting from the start for the aviation industry, e.g., the term AR was coined by Caudell and Mizell [30
] while working for Boeing. AR can be used in both synchronous and asynchronous learning. In aviation AR applications have been explored especially in relation to training materials [31
]. In addition, the AR benefit to enable flight instructors the monitoring of pilot’s visual scan during simulated flight was highlighted by Vlasblom et al. [36
2.5. Study in the Context of Sustainability
In addition to the primary goal of this study to explore direct benefits for the pilots from the use of AR in TR, we also discuss implications of this study in the wider context of sustainability. In the course of this we follow [37
] and thereby base our considerations on the concept of sustainability as defined in the ”Report of the World Commission on Environment and Development: Our Common Future” [38
] (p. 39) to, “[…
] seek to meet the needs and aspirations of the present without compromising the ability to meet those of the future”. This general concept and goal of sustainability has been further developed and has influenced various research initiatives. In the context of business model innovation for example the research field “New Business Models (NBM)” [39
] evolved, which investigates the development and consequences of (strongly) sustainable business models [42
] and thereby aims to create business models that are “[…
] simultaneously creating financial rewards, social benefits, and environmental regeneration” [42
] (p. 17).
Sustainability has emerged as an important topic for the recovery of aviation after the COVID-19 pandemic. During the pandemic, some pilot training providers experienced increased competition and a decline in the number of customers (e.g., individuals and air-operators) due to the grounding of fleets, aircraft retirements, and reductions in crews. The forecasts for the post-pandemic recovery predict a return of the industry to operational levels close to those of pre-pandemic volumes over the next two years. How quickly and broadly the desired recovery will relieve the industry remains unclear. Concerns about pollution and lack of sustainability, however, are indicators of change: New inventions and developments, such as biofuel and alternative energy, will drive the industry in coming decades, while new pilot skills and specific training will also be required.
The focus of this study was especially on the social benefit and increased gender diversity in the advanced pilot education. The economic benefits of introducing AR in TR training, although expected, are not addressed here.
A survey was conducted with TR pilots and TR instructors to explore the difficulties of TR training and the potential of AR to improve the delivery of contents and learning conditions. The survey consisted of three parts. The first part addressed biographical aspects such as age, gender, pilot/instructor ratings, and activity.
The second part included items for assessment of TR training contents and conditions:
Did the learning conditions make it difficult for some students to complete learning activities?
Did the assessment conditions make it difficult for some students to complete assessment activities?
Suggest support mechanisms for improving the learning outcomes of trainees.
Name parts of the training perceived as most difficult.
Select from the list of TR contents the items that were most difficult to learn.
Name parts of the training pilots enjoy the most.
Select from the list of TR contents the items that were easiest to learn.
The third part began with a video demonstration of typical AR features and applications. Three different AR use cases were introduced by presenting short videos to the participants to give them insights in the use of this AR technology. The shown use cases were not related to the aviation industry in order not to influence in specific directions. Subsequently, the participants were asked to select from a list of TR contents the items that could benefit from AR. The list included generic TR contents that were specified in the European aviation standards. Table 1
presents a part of the syllabus “theoretical knowledge”.
The instruction program for TR has two main parts: the theoretical and the practical one. Theory aims to provide knowledge about complex aircraft, i.e., technical systems such as the fuel or electrical system, the operation of the systems, malfunctions, and limitations. Furthermore, knowledge about performance, flight planning, and monitoring is provided, which includes route planning and performance calculations such as take-off run and distance. Load and balancing is another important part of the theoretical instruction, as well as emergency procedures.
The practical part consists of eight training sessions and a final license skill test, all performed on a full flight simulator, representing the aircraft type for which the training is performed. The duration of each training session is usually four hours, in which each pilot performs two hours as the pilot flying and two hours as the pilot monitoring. The duty of the pilot flying is to control the path of the aircraft and order configuration changes (e.g., extend/retract landing gear). The pilot monitoring assists the pilot flying in respect of maintaining situational awareness and communication with air traffic control.
The training sessions start with a one-hour-long briefing, covering the learning objectives and expected program. The first sessions cover basic flying techniques and also focus on transferring the theoretical knowledge into the cockpit environment. Simple abnormals (e.g., failed back-up systems) are introduced before more severe situations (e.g., engine failures and fires and loss of main electrics) are thought and trained. Besides technical knowledge, interpersonal decision making and collaboration tools are also part of the course. The final goal is to get the student ready to successfully perform the license skill test. The program investigated here is defined by EASA and covers among others operation with a failed engine and different types of approaches.
3.1. Data Analysis
The results are presented descriptively in percentages and relative frequencies. Data obtained by selecting from a list of training elements were grouped in 15 topic categories, which are shown in Table 2
. The number of responses per topic category were weighted to counterbalance the varying number of items in different categories by dividing the number of answers per category by the number of items of the category. In addition, the responses were weighted to counterbalance differences in group sizes (e.g., women versus men or pilots versus instructors). The number of responses per group in the context of a specific category was divided by the total number of responses provided by the individuals included in the respective group (e.g., women, men, or instructors). In summary, the data are presented as percentages and weighted relative frequencies.
Responses to open-ended questions were separately analyzed for pilots and instructors and men and women. Nominations of specific contents were counted.
For obtaining pilots’ views, 31 pilots with a TR were surveyed. In total, 24 male pilots aged between 22 and 57 years (mean = 36.67, median = 35, SD = 9.04) and 7 female pilots aged between 29 and 48 years (mean = 36.29, mMedian = 37, SD = 7.57) participated in the study. The pilots held TR from different types of aircraft such as Boeing 767; MD-11; Cessna Citation types C525, C560XLS, and C750; Airbus A320 and A350; and Junkers Ju 52. For obtaining instructors’ views, 22 type rating instructor (TRI) were surveyed. All instructors were men aged between 29 and 65 years (mean = 45.90, median = 43, SD = 10.25).
The results are structured in two parts: assessment of the course content and aspects related to learning and assessment conditions.
4.1. Type Rating Contents
The course content was analyzed with respect to the following four questions: course contents, which are the most difficult to learn followed by the contents, which are the easiest to learn. The third section examines contents, which participants enjoyed the most. The forth concludes with the contents, which are considered to benefit the most from AR.
4.1.1. The Most Difficult TR Contents to Learn
Participants could select from a list of course elements those that they experienced as being most difficult to learn. The contents were grouped in 15 categories for data analysis. Figure 1
shows that women assessed missed approach procedures as the most difficult content category, followed by landings, the flight management system, abnormal and emergency flight procedures, and instrument flight procedures. A number of female pilots rated contents from the categories of aircraft structure and equipment, flight maneuvers, instrument approaches, take-offs, normal flight procedures, and limitations as being the most difficult to learn. There were no contents in the categories of performance, emergency procedures, special requirements, and flight preparation; however, those were rated by women as being difficult. Similarly, male pilots rated missed approach procedures as the most difficult content, followed by the flight management system, landings, take-offs, and abnormal and emergency flight procedures. A number of men rated contents from the categories of instrument approaches, special requirements, flight maneuvers, instrument flight procedures, normal flight procedures, aircraft structure and equipment, limitations, and flight preparation as difficult. None of the contents in the categories performance and emergency procedures were rated by men as being difficult. The overall scores of the pilots including both genders indicate that the missed approach procedures were the most difficult content, followed by the flight management system, landings, abnormal and emergency flight procedures, and take-offs. By comparison, the TR instructors identified take-offs as the most difficult content, followed by the flight management system, missed approach procedures, aircraft structure and equipment, and emergency procedures. The contents missed approach procedures and the flight management system appeared in the top five for each group. Interestingly, the flight instructors did not rate course contents in the categories limitations, special requirements and instrument approaches as being the most difficult for their trainees. Table 3
gives an overview of the most difficult items for each category rated by type rating pilots (TRPs) and TRIs.
In addition to the selection from the list, there was an open-ended question asking pilots “What part of your TR training was the most difficult? Why?” The question for instructors was “What part of your TR training was the most difficult for your trainees? Why?” The responses address both training contents and learning conditions such as stress. Six female pilots, 21 male pilots, and 20 instructors responded to this question. One woman mentioned language barriers because all the lessons were held in English. Two others identified the theoretical learning of aircraft systems as being most difficult. One woman replied that “the long theory days” were difficult. Three men pointed out the stress caused by the very short course duration and the sheer volume of information to assimilate. Seven male pilots brought up the learning of theoretical systems’ contents and six mentioned the practical parts as being most difficult. Twelve TRIs stated issues in the practical part, and three emphasized malfunctions during practical training. Furthermore, six TRIs referred to theoretical contents; two mentioned that candidates are nervous before tests; and two noted issues with multi-crew operations. One instructor mentioned the short time and the huge quantity of information contained in the advanced pilot training due to the complexity of aircraft systems.
4.1.2. Easiest to Learn TR Contents
Pilots were asked to indicate the easiest contents of the course by selecting items from the list. Women selected contents in the categories limitations, emergency procedures, take-offs, landings, and normal flight procedures as being the easiest to learn, as shown in Figure 2
. Flight maneuvers were the easiest for men, followed by take-offs, instrument flight procedures, landings, and aircraft structure and equipment. For pilots of both genders, overall, the easiest contents were in the categories: flight maneuvers, limitations, take-offs, landings, and flight preparation. TRIs rated the easiest contents as follows: limitations, special requirements, performance, flight preparation, and emergency procedures. The easiest content mentioned in all groups was in the category of limitations. Interestingly, take-off and landing contents were selected in both the most difficult and easiest to learn contents, an indicator of the diversity of students’ experiences with the same material.
4.1.3. The TR Contents Participants Enjoyed the Most
Another open-ended question asked pilots “What part of your TR training did you enjoy the most? Why?” The question for instructors was “What part of your TR training do your trainees enjoy the most? Why?” These questions were answered by six female pilots, 23 male pilots, and 21 instructors. Five women said that they enjoyed simulator training. Two women explained that they like hands-on training, and two said that they like learning new things. Fourteen men mentioned either the simulator or practical training as the most enjoyable part of the training. Fourteen instructors noted that their trainees enjoyed simulator training the most.
4.1.4. The TR Contents That Were Considered to Benefit from AR
After a demonstration of typical AR use cases, all pilots and instructors were asked to select course contents that could benefit from AR to improve students’ development of mental models/knowledge during training. As illustrated in Figure 3
, women rated special requirements as the category to benefit most from AR, followed by emergency procedures, the flight management system, aircraft structure and equipment, and instrument approaches. The flight management system was selected most often by men. Other contents among the top five selected by men were: emergency procedures, aircraft structure and equipment, flight preparation, and instrument flight procedures. This order also corresponds to the order of overall pilot scores. In addition, the instructors considered the category aircraft structure and equipment to benefit most from AR, followed by emergency procedures, flight preparation, performance, and abnormal and emergency flight procedures. The potential benefit of AR for learning emergency procedures and aircraft structure and equipment was rated highly by all groups. More information on the specific items of each category which can benefit from AR is given in Table 3
4.2. Learning and Assessment Conditions
The learning conditions for pilots participating in TR courses are characterized by long travel times, because they often come from abroad. The course duration depends on the content and costs of the course.
(left) indicates that most of the male and female pilots as well as most of the instructors did not consider that the learning conditions created difficulties for some students to complete the course. None of the female pilots reported such difficulties. There were five responses from male pilots and four from instructors. They mentioned stress and time pressure, differences in the previous knowledge of the learners, and problems with self-study.
In addition, the participants were asked if the conditions made it difficult for some students to complete the assessment. As can be seen from Figure 4
(right), only between 0 % and 16 % of the participants thought that this was the case. Two male pilots and three instructors mentioned time pressure, the need to manage the pressure on students, and the need to better prepare for the assessment. Difficulties in learning were also reflected in difficulties with evaluation.
Moreover, participants were asked “Can you suggest any support mechanisms to improve the performance/ results of the students?” A female pilot, seven male pilots, and nine instructors responded to this question. The woman had no suggestion but said the course was very well done. Men suggested having more classical classroom instructions but also mentioned additional CBTs to improve performance and results. Furthermore, they suggested allocating more time to difficult topics. One pilot suggested deviating from the course plan in the event of the pilots not being able to achieve the learning performance expected from them as planned. TRIs proposed giving more time for learning, standardizing instructions, using other media (e.g., videos), and using a mock-up or procedure trainer.
The aim of this study was to identify gender-specific preferences and difficulties in TR training that can be addressed by future development of AR-based training means. Thus, the advanced pilot training could become more inclusive and learner-centered, and this would bring both social and economic benefits [1
]. Despite many similarities, the results of this study confirm that there are gender-specific needs and preferences in advanced flight training, as revealed in previous research [5
]. Due to the underrepresentation of women in the pilot population, their learning needs and preferences can easily vanish in the pool of the collected data. The results also show that both trainees and instructor views are required to obtain a comprehensive assessment of the learning contents and conditions. For future development of TR training means in the context of theoretical and practical content, a variety of AR applications are listed in Table 4
In the following two categories AR applications can be developed for both theoretical and practical training: aircraft structure and equipment and flight preparation. Possible methods for teaching aircraft structure and equipment, which can be used individually or combined are:
Synchronous and asynchronous training materials [31
AR training means 3D visualizations [23
Educational gaming [24
Interactive manipulation of holographic learning materials [25
Book content with pop-ups including 3D or animated AR content [26
For addressing the learning conditions, which play an important role for a successful completion of the course, AR could be integrated in future applications to support:
Collaboration among peer students or between instructors and students [29
Skill training with AR-facilitated practice of certain (part) tasks or procedures [27
Flight instructors monitoring of the pilot’s visual scan during simulated flight as highlighted by Vlasblom et al. [36
As research shows, the proposed AR solutions are seen as complementary to the classroom and full-flight simulator training. We propose the use of AR for the initial part of the training in blended online synchronous and asynchronous settings that have been shown to improve student commitment and the feeling of connectivity with peers and instructors [11
]. This could mitigate the stress and time pressure factor related to the conditions of TR training that were highlighted by a number of pilots and instructors. Future AR applications for synchronous learning in TR training need to consider the richness of media for communication, interaction, and collaboration [12
] such as application sharing and remote consultation that is enriched with virtual annotations and synchronous video sharing [45
]. Asynchronous interactions [12
] could be implemented in AR (for example, video recording of a practice session and guidance videos enriched with holographic on-site animations [46
] has shown that students liked the interactivity enabled by virtual laboratories, because the “learning process is more pleasant” and it “enhances their comprehension of the concepts and ideas developed in theoretical classes”. The tools used enhanced learning and led to better performance of the students. [22
] This effect was also shown when using computer simulations [47
]. According to [48
], it is important to consider individual and social perspectives in the learning process and thus to shape it in line with group and individual activities.
The positive impact of AR on learning gains and the increase in motivation was also confirmed by a meta-analysis from [49
]. With respect to the level of education, they found that “AR has a large effect on Bachelor’s or equivalent level and a medium to large effect on short-cycle tertiary education” and in the field of education context “engineering, manufacturing and construction” records the largest effect [50
]. In a recent review, [51
] showed that AR enables “learning by doing” (kinesthetic learning), and the integration of AR in pedagogical approaches supports the creation of “human-centered learning environments”. More efficient visualization, enhancing of collaborative learning, and boosting motivation and engagement are, among other points, some of the key benefits of using AR. However, technical issues, information/cognitive overload are some of the key challenges identified [51
]. The positive impact of AR on student motivation was also reported [52
], showing an increase in attention, satisfaction, and confidence. Research [53
] has identified educational benefits of AR such as fun, dynamics, facilitation, interactivity, and entertainment. However, the main limitations of the device should be mentioned among a number of others. The integration of AR in learning concepts facilitates more interactivity. In e-learning environments AR offers more possibilities to discover technical concepts by using 3D-visualizations, which help to lower the barriers and at the same time to increase the motivation, both of which lead to better performance results.
There are also several examples for the use of VR in teaching in various disciplines to improve the learning process and understanding of it by the students. The usage of virtual laboratorys (VLs) has various benefits such as enrichment of the teaching–learning process with interactive elements, which in turn affects student motivation, brings in new possibilities for visualization (e.g., see through components), and gives more autonomy to students in the matter of how often and when they use the learning tool. In addition, VLs mitigates the risk of damage to a real machine/product or of students endangering themselves when they are inexperienced in handling them, and VLs are significantly less investment intensive than “real” devices. VLs also allows training with devices that would not be available in reality because of the (extremely) high costs involved [21
]. The latter is of particular relevance to TR courses, because a real aircraft is often not available during the course.
Social sustainability aspects such as gender diversity and inclusivity can be addressed both in design [3
] and evaluation of AR-based training means. Specific methods and techniques for gender research in aviation are detailed in the recent literature [3
Sustainability aspects in terms of AR engineering issues and costs could be considered in future research. Ref. [54
] mentions that if the application is not on the latest technological level, the user motivation and interactivity decreases until the application is updated again. This factor has a major influence on the operational life of the AR application, its maintenance, and the resulting costs [54
Social sustainability aspects such as gender diversity and inclusivity were addressed in the evaluation of the TR course. This study with pilots and flight instructors identified gender-specific preferences and difficulties related to the content of TR training but also related to training and assessment conditions. Pilots of both genders and flight instructors noted that simulator training was the most enjoyable part of the course. The most difficult content categories for female pilots were missed approach procedures, followed by landings, the flight management system, abnormal and emergency flight procedures, and instrument flight procedures. The most difficult content categories for male pilots were missed approach procedures, followed by the flight management system, landings, take-offs, and abnormal and emergency flight procedures. Flight instructors and a number of male pilots mentioned difficult training conditions such as stress, time pressure, differences in the previous knowledge of the learners, problems with self-study, and the need to better prepare for the assessment.
The results show a variety of TR training elements that could potentially benefit from AR-based training means. The potential benefit of AR for learning emergency procedures, aircraft structure and equipment was rated highly by pilots of both genders and by flight instructors. In addition, female pilots rated special requirements, the flight management system, and instrument approaches as course contents to benefit most from AR-based training means. Besides the flight management system, flight preparation and instrument flight procedures were supplementary categories highly rated by male pilots. Further categories highly rated by flight instructors were flight preparation, performance, abnormal and emergency flight procedures. In addition, AR features were specified that could be used to address a number of learning conditions by facilitating the remote training of particular elements and collaboration among peer students or between instructors and students. The proposed AR solutions are seen as complementary to the classroom and full-flight simulator training. The results are used to inform the future development of AR-based training means for pilot training.
7. Future Research
Flexibility for TR trainees to learn and practice remotely and asynchronously using wearable AR devices could support their acquisition and maintenance of aircraft knowledge and flight skills. Controlled experiments [8
] could be used to assess the effect of AR-based training means on pilot’s procedural memory, acquisition, and generalization of skills and transfer of training in a gender-sensitive manner.
In addition, various aspects of economic and environmental sustainability could be pursued in future research. Economic benefits for flight schools may be generated by novel AR-enhanced CBTs with an increased level of automation of the teaching and assessment processes.
CBT and possible new forms of virtual training fostered by AR may also have positive environmental impacts such as reduced travel demands for the students and instructors. These environmental sustainability effects of AR-based training could be investigated in dedicated case studies, field experiments, surveys, and long-term studies.
Conceptualization, B.M., H.S., W.V., I.V.K., T.H., R.B. and M.H.; methodology, B.M., I.V.K., H.S., R.B. and W.V.; data analysis, B.M., R.B., H.S., W.V. and I.V.K.; original draft preparation, B.M., H.S., W.V., T.H. and M.H.; review and editing, W.V., I.V.K. and R.B. All authors have read and agreed to the published version of the manuscript.
This research was funded by the Austrian Federal Ministry for Climate Action, Environment, Energy, Mobility, Innovation, and Technology and the Austrian Research Promotion Agency, FEMtech Program “Talent”, grant number 866702.
Institutional Review Board Statement
Informed Consent Statement
Informed consent was obtained from all subjects involved in the study.
Data Availability Statement
The authors gratefully acknowledge all the pilots and instructors who completed the survey. We thank Robert Novak and Eva-Maria Herzog for their support with the data collection and their feedback on a previous version. Furthermore, the authors thank Open Access Funding by the Graz University of Technology.
Conflicts of Interest
The authors declare no conflict of interest. The funders had no role in the design, the collection, analysis, interpretation of data, in writing of the manuscript, nor in the decision to publish the results.
The following abbreviations are used in this manuscript:
|APU||auxiliary power unit|
|ATC||air traffic control|
|CAT II/III||category II/III operation (precision instrument approach and landing)|
|EASA||European Union Aviation Safety Agency|
|EFIS||electronic flight instrument system|
|MAPt||missed approach point|
|MDH||minimum descent height|
|RVR||runway visual range|
|TRI||type rating instructor|
|TRP||type rating pilot|
|V1||take-off decision speed|
|V2||take-off safety speed|
- Ferla, M.; Graham, A. Women slowly taking off: An investigation into female underrepresentation in commercial aviation. Res. Transp. Bus. Manag. 2019, 31, 100378. [Google Scholar] [CrossRef]
- Mitchell, J.; Kristovics, A.; Vermeulen, L.; Wilson, J.; Martinussen, M. How Pink is the Sky?: A Cross-national Study of the Gendered Occupation of Pilot. Employ. Relations Rec. 2005, 5, 43–60. [Google Scholar]
- Koglbauer, I.V. Forschungsmethoden in der Verbindung Gender und Technik: Research Methods Linking Gender and Technology. Psychol. Österreich 2017, 37, 354–359. [Google Scholar]
- Bailey, T. Keynote address. In Women in Aviation International Conference; Convention News Company, Incorporated: Cebu City, Philippines, 2001. [Google Scholar]
- Hamilton, P.R. Chapter The teaching women to fly research project. In Absent Aviators; Bridges, D., Neal-Smith, J., Mills, A.J., Eds.; Ashgate Publishing Ltd.: Surrey, UK, 2014; pp. 313–331. [Google Scholar]
- Neubauer, A.C.; Grabner, R.H.; Fink, A.; Neuper, C. Intelligence and neural efficiency: Further evidence of the influence of task content and sex on the brain–IQ relationship. Cogn. Brain Res. 2005, 25, 217–225. [Google Scholar] [CrossRef]
- Neubauer, A.C.; Bergner, S.; Schatz, M. Two- vs. three-dimensional presentation of mental rotation tasks: Sex differences and effects of training on performance and brain activation. Intelligence 2010, 38, 529–539. [Google Scholar] [CrossRef][Green Version]
- Bauer, S.; Braunstingl, R.; Riesel, M.; Koglbauer, I. Improving the method for upset recovery training of ab initio student pilots in simulated and real flight. In Proceedings of the 33rd Conference of the European Association for Aviation Psychology, Dubrovnik, Croatia, 24–28 September 2019; Schwarz, M., Lasry, J., Schnücker, G.N., Becherstorfer, H., Eds.; EAAP European Association for Aviation Psychology: Amsterdam, The Netherlands, 2019; pp. 167–179. [Google Scholar]
- Koglbauer, I.; Braunstingl, R. Ab initio pilot training for traffic separation and visual airport procedures in a naturalistic flight simulation environment. Transp. Res. Part F Traffic Psychol. Behav. 2018, 58, 1–10. [Google Scholar] [CrossRef]
- Nieuwoudt, J.E. Investigating synchronous and asynchronous class attendance as predictors of academic success in online education. Australas. J. Educ. Technol. 2020, 36, 15–25. [Google Scholar] [CrossRef]
- Yamagata-Lynch, L.C. Blending online asynchronous and synchronous learning. Int. Rev. Res. Open Distrib. Learn. 2014, 15, 189–212. [Google Scholar] [CrossRef][Green Version]
- Lim, F.P. An Analysis of Synchronous and Asynchronous Communication Tools in e-Learning. Adv. Sci. Technol. Lett. 2017, 143, 230–234. [Google Scholar] [CrossRef][Green Version]
- Hays, R.T.; Jacobs, J.W.; Prince, C.; Salas, E. Flight Simulator Training Effectiveness: A Meta-Analysis. Mil. Psychol. 1992, 4, 63–74. [Google Scholar] [CrossRef]
- Koglbauer, I. Simulator training improves pilots’ procedural memory and generalization of behavior in critical flight situations. Cogn. Brain Behav. 2016, 20, 357–366. [Google Scholar]
- Dennis, K.A.; Harris, D. Computer-Based Simulation as an Adjunct to Ab Initio Flight Training. Int. J. Aviat. Psychol. 1998, 8, 261–276. [Google Scholar] [CrossRef]
- Gawron, V.J.; Peer, J. Evaluation of Airplane Upset Recovery Training. Aviat. Psychol. Appl. Hum. Factors 2014, 4, 74–85. [Google Scholar] [CrossRef]
- Koglbauer, I.; Kallus, K.W.; Braunstingl, R.; Boucsein, W. Recovery Training in Simulator Improves Performance and Psychophysiological State of Pilots During Simulated and Real Visual Flight Rules Flight. Int. J. Aviat. Psychol. 2011, 21, 307–324. [Google Scholar] [CrossRef]
- Koglbauer, I.V.; Riesel, M.; Braunstingl, R. Positive effects of combined aircraft and simulator training on the acquisition of visual flight skills. Cogn. Brain Behav. Interdiscip. J. 2016, 20, 309–318. [Google Scholar]
- Roessingh, J.J.M. Transfer of Manual Flying Skills From PC-Based Simulation to Actual Flight-Comparison of In-Flight Measured Data and Instructor Ratings. Int. J. Aviat. Psychol. 2005, 15, 67–90. [Google Scholar] [CrossRef]
- Milgram, P.; Takemura, H.; Utsumi, A.; Kishino, F. Augmented reality: A class of displays on the reality-virtuality continuum. In Telemanipulator and Telepresence Technologies; Das, H., Ed.; International Society for Optics and Photonics, SPIE: Bellingham, WA USA, 1995; Volume 2351, pp. 282–292. [Google Scholar] [CrossRef]
- Vergara, D.; Extremera, J.; Rubio, M.P.; Dávila, L.P. The proliferation of virtual laboratories in educational fields. ADCAIJ Adv. Distrib. Comput. Artif. Intell. J. 2020, 9, 85–97. [Google Scholar] [CrossRef]
- Vergara, D.; Rubio, M.P.; Lorenzo, M. New approach for the teaching of concrete compression tests in large groups of engineering students. J. Prof. Issues Eng. Educ. Pract. 2017, 143, 05016009. [Google Scholar] [CrossRef]
- Extremera, J.; Vergara, D.; Dávila, L.P.; Rubio, M.P. Virtual and Augmented Reality Environments to Learn the Fundamentals of Crystallography. Crystals 2020, 10, 456. [Google Scholar] [CrossRef]
- Kidd, S.H.; Crompton, H. Augmented Learning with Augmented Reality. In Mobile Learning Design; Churchill, D., Lu, J., Chiu, T.K., Fox, B., Eds.; Lecture Notes in Educational Technology; Springer: Singapore, 2016; pp. 97–108. [Google Scholar] [CrossRef]
- Wu, H.K.; Lee, S.W.Y.; Chang, H.Y.; Liang, J.C. Current status, opportunities and challenges of augmented reality in education. Comput. Educ. 2013, 62, 41–49. [Google Scholar] [CrossRef]
- Billinghurst, M.; Kato, H.; Poupyrev, I. The magicbook-moving seamlessly between reality and virtuality. IEEE Comput. Graph. Appl. 2001, 21, 6–8. [Google Scholar] [CrossRef]
- Yuen, S.C.Y.; Yaoyuneyong, G.; Johnson, E. Augmented Reality: An Overview and Five Directions for AR in Education. J. Educ. Technol. 2011, 4, 119–140. [Google Scholar] [CrossRef]
- Wang, M.; Callaghan, V.; Bernhardt, J.; White, K.; Peña-Rios, A. Augmented reality in education and training: Pedagogical approaches and illustrative case studies. J Ambient Intell Hum. Comput. 2018, 9, 1391–1402. [Google Scholar] [CrossRef]
- Billinghurst, M.; Kato, H. Collaborative augmented reality. Commun. ACM 2002, 45, 64–70. [Google Scholar] [CrossRef]
- Caudell, T.; Mizell, D. Augmented reality: An application of heads-up display technology to manual manufacturing processes. In Proceedings of the Twenty-Fifth Hawaii International Conference on System Sciences, Kauai, HI, USA, 7–10 January 1992; Volume 2, pp. 659–669. [Google Scholar] [CrossRef]
- Brown, L. The Next Generation Classroom: Transforming Aviation Training with Augmented Reality. In Proceedings of the National Training Aircraft Symposium (NTAS), Daytona Beach, FL, USA, 14–16 August 2017. [Google Scholar]
- Brown, L. Holographic Micro-simulations to Enhance Aviation Training with Mixed Reality. In Proceedings of the National Training Aircraft Symposium (NTAS), Daytona Beach, FL, USA, 13–15 August 2018. [Google Scholar]
- Brown, L. Augmented Reality in International Pilot Training to Meet Training Demands. In Proceedings of the Fall Convocation 2019, Kalamazoo, MI, USA, 20 September 2019. [Google Scholar]
- Schaffernak, H.; Moesl, B.; Vorraber, W.; Koglbauer, I.V. Potential Augmented Reality Application Areas for Pilot Education: An Exploratory Study. Educ. Sci. 2020, 10, 86. [Google Scholar] [CrossRef][Green Version]
- Schaffernak, H.; Moesl, B.; Vorraber, W.; Braunstingl, R.; Herrele, T.; Koglbauer, I. Design and Evaluation of an Augmented Reality Application for Landing Training. In Human Interaction, Emerging Technologies and Future Applications IV; Advances in Intelligent Systems and Computing; Ahram, T., Taiar, R., Groff, F., Eds.; Springer International Publishing: Cham, Switzerland, 2021; Volume 1378, pp. 107–114. [Google Scholar] [CrossRef]
- Vlasblom, J.; vd Pal, J.; Sewnath, G. Making the invisible visible—Increasing pilot training effectiveness by visualizing scan patterns of trainees through AR. In Proceedings of the International Training Technology Exhibition & Conference (IT2EC), Bangkok, Thailand, 1–2 March 2019; pp. 1–3. [Google Scholar]
- Pavie, X.; Carthy, D.; Scholten, V. Responsible Innovation: From Concept to Practice; World Scientific Pub. Co: Singapore; Hackensack, NJ, USA, 2014. [Google Scholar]
- UN; Secretary-General; World Commission on Environment and Development. Report of the World Commission on Environment and Development; UN: London, UK, 1987. [Google Scholar]
- Dentchev, N.; Baumgartner, R.; Dieleman, H.; Jóhannsdóttir, L.; Jonker, J.; Nyberg, T.; Rauter, R.; Rosano, M.; Snihur, Y.; Tang, X.; et al. Embracing the variety of sustainable business models: Social entrepreneurship, corporate intrapreneurship, creativity, innovation, and other approaches to sustainability challenges. J. Clean. Prod. 2016, 113, 1–4. [Google Scholar] [CrossRef]
- Rauter, R.; Jonker, J.; Baumgartner, R.J. Going one’s own way: Drivers in developing business models for sustainability. J. Clean. Prod. 2017, 140, 144–154. [Google Scholar] [CrossRef]
- Schaltegger, S.; Freund, F.L.; Hansen, E.G. Business cases for sustainability: The role of business model innovation for corporate sustainability. Int. J. Innov. Sustain. Dev. 2012, 6, 95–119. [Google Scholar] [CrossRef]
- Upward, A.; Jones, P. An Ontology for Strongly Sustainable Business Models. Organ. Environ. 2016, 29, 97–123. [Google Scholar] [CrossRef][Green Version]
- European Aviation Safety Agency. Acceptable Means of Compliance and Guidance Material to Part-FCL. 2021. Available online: https://www.easa.europa.eu/downloads/1533/en (accessed on 15 December 2011).
- Vergara, D.; Extremera, J.; Rubio, M.P.; Dávila, L.P. Meaningful Learning Through Virtual Reality Learning Environments: A Case Study in Materials Engineering. Appl. Sci. 2019, 9, 4625. [Google Scholar] [CrossRef][Green Version]
- Vorraber, W.; Gasser, J.; Webb, H.; Neubacher, D.; Url, P. Assessing augmented reality in production: Remote-assisted maintenance with HoloLens. Procedia CIRP 2020, 88, 139–144. [Google Scholar] [CrossRef]
- Url, P.; Vorraber, W.; Gasser, J. Practical Insights On Augmented Reality Support for Shop-Floor Tasks. In Proceedings of the 25th International Conference on Production Research Manufacturing Innovation: Cyber Physical Manufacturing, Chicago, IL, USA, 9–14 August 2019. [Google Scholar] [CrossRef]
- Forcael, E.; Glagola, C.R.; González, V. Incorporation of computer simulations into teaching linear scheduling techniques. J. Prof. Iss Eng. Ed. Prac. 2012, 138, 21–30. [Google Scholar] [CrossRef]
- Anderson, J.R.; Greeno, J.G.; Reder, L.M.; Simon, H.A. Perspectives on Learning, Thinking, and Activity. Educ. Res. 2000, 29, 11–13. [Google Scholar] [CrossRef]
- Garzón, J.; Pavón, J.; Baldiris, S. Systematic review and meta-analysis of augmented reality in educational settings. Virtual Real. 2019, 23, 447–459. [Google Scholar] [CrossRef]
- Garzón, J.; Acevedo, J. Meta-analysis of the impact of Augmented Reality on students’ learning gains. Educ. Res. Rev. 2019, 27, 244–260. [Google Scholar] [CrossRef]
- Alzahrani, N.M. Augmented Reality: A systematic review of its benefits and challenges in e-learning contexts. Appl. Sci. 2020, 10, 5660. [Google Scholar] [CrossRef]
- Khan, T.; Johnston, K.; Ophoff, J. The impact of an augmented reality application on learning motivation of students. Adv. Hum.-Comput. Interact. 2019, 2019, 7208494. [Google Scholar] [CrossRef][Green Version]
- Gómez-Galán, J.; Vázquez-Cano, E.; Luque de la Rosa, A.; López-Meneses, E. Socio-educational impact of Augmented Reality (AR) in sustainable learning ecologies: A semantic modeling approach. Sustainability 2020, 12, 9116. [Google Scholar] [CrossRef]
- Vergara, D.; Extremera, J.; Rubio, M.P.; Dávila, L.P. The Technological Obsolescence of Virtual Reality Learning Environments. Appl. Sci. 2020, 10, 915. [Google Scholar] [CrossRef][Green Version]
- Lehman, M.M. Laws of Software Evolution Revisited. In Proceedings of the 5th European Workshop on Software Process Technology; EWSPT ’96; Springer: Berlin/Heidelberg, Germany, 1996; pp. 108–124. [Google Scholar]
- Lehman, M.M.; Ramil, J.F. Software Evolution: Background, Theory, Practice. Inf. Process. Lett. 2003, 88, 33–44. [Google Scholar] [CrossRef]
TR content categories considered to be the most difficult to learn.
TR content categories considered to be the most difficult to learn.
TR content categories considered to be the easiest to learn.
TR content categories considered to be the easiest to learn.
TR content categories considered to benefit from AR-based training means.
TR content categories considered to benefit from AR-based training means.
Conditions that made it difficult for some students to complete the course.
Conditions that made it difficult for some students to complete the course.
Extract of the syllabus of theoretical knowledge for class or type ratings [43
Extract of the syllabus of theoretical knowledge for class or type ratings [43
|Detailed listing for airplane structure and equipment, normal operation of systems, and malfunctions|
|engine including auxiliary power unit|
|pressurisation and air conditioning|
|ice and rain protection, windshield wipers, and rain repellent|
|flight controls and high lift devices|
|electrical power supply|
|flight instruments, communication, radar and navigation equipment, autoflight, and flight data recorders|
|cockpit, cabin, and cargo compartment|
|emergency equipment operation and correct application of the following emergency equipment in the airplane|
Categories and assigned training elements.
Categories and assigned training elements.
|Category||No. of Elements||Excerpt of Training Elements Assigned to This Category|
|Aircraft structure and equipment||13||Fuel system, hydraulic system, landing gear, electrical power supply, etc.|
|Limitations||4||Engine limitations, systems limitations, minimum equipment list, etc.|
|Performance||4||Flight monitoring, load and balance, etc.|
|Emergency procedures||2||Actions according to the approved abnormal and emergency checklist|
|Special requirements||2||For extension of a TR, for “glass cockpit”|
|Flight management system||1|
|Flight preparation||6||Performance calculation, cockpit inspection, etc.|
|Take-offs||8||Normal take-offs with different flap settings, crosswind take-off, etc.|
|Flight maneuvers||3||Turns with and without spoilers, etc.|
|Normal flight procedures||15||Engine, pressurization and air-conditioning, etc.|
|Abnormal and emergency flight procedures||12||Fire drills, engine failures, etc.|
|Instrument flight procedures||5||Adherence to departure and arrival routes and ATC instructions, etc.|
|Missed approach procedures||3||Go-around with all engines operating, etc.|
|Landings||4||Normal landings, landing with critical engine simulated inoperative, etc.|
|Instrument approaches||4||Rejected take-off at minimum authorized RVR, etc.|
Categories: contents most difficult to learn and most promising for using AR from pilots’ and instructors’ perspective.
Categories: contents most difficult to learn and most promising for using AR from pilots’ and instructors’ perspective.
|Category||Most difficult item||Content That Could Mostly Benefit from AR|
|Aircraft structure and equipment||Electrical power supply (TRP); fuel system, pressurization and air conditioning, ice and rain protection, windshield wipers and rain repellent, landing gear, pneumatic system (TRI)||Cockpit, cabin and cargo compartment (TRP); dimensions: minimum required runway width for 180° turn (TRI)|
|Limitations||General limitations (TRP)||General and engine limitations
(TRP); general limitations (TRI)|
|Performance||Flight monitoring, on ground, servicing connections (TRI)||On ground, servicing connections (TRP); flight planning for normal and abnormal conditions, on ground, servicing connections (TRI)|
|Emergency procedures||Emergency procedures actions according to the approved abnormal and emergency checklist (TRI)||Emergency procedures (TRI, TRP), emergency procedures actions according to the approved abnormal and emergency checklist (TRI)|
|Special requirements||Special requirements for “glass cockpit” aircrafts with EFIS (TRP)||Special requirements for extension of a TR for instrument approaches down to decision heights of less than 200 ft (60 m) (TRP), special requirements for “glass cockpit” aircrafts with EFIS (TRP, TRI)|
|Flight management system||Flight management system (TRP, TRI)||Flight management system (TRP, TRI)|
|Flight preparation||Aircraft external visual inspection; location of each item and purpose of inspection (TRP, TRI)||Aircraft external visual inspection; location of each item and purpose of inspection (TRP, TRI), Cockpit inspection (TRP)|
|Take-offs||Take-offs with simulated engine failure between V1 and V2 (TRP); take-offs with simulated engine failure (TRI)||Crosswind take-off (TRP); take-offs with simulated engine failure, shortly after reaching V2, between V1 and V2 (TRI)|
|Flight maneuvers||Tuck under and Mach buffets after reaching the critical Mach number, and other specific flight characteristics of the aircraft (e.g., Dutch Roll) (TRP, TRI), normal operation of systems and controls engineer’s panel (TRP)|
|Normal flight procedures||Electrical and hydraulical system (TRP); pitot/static system, landing gear and brake (TRI)||Engine (TRP); pitot/static system (TRI)|
|Abnormal and emergency flight procedures||Fire drills, e.g., engine, APU, cabin, cargo compartment, flight deck, wing, and electrical fires including evacuation, smoke control and removal, incapacitation of flight crew member (TRP); steep turns with 45° bank, 180° to 360° left and right (TRI)||Wind shear at takeoff/landing (TRP); fire drills e.g., engine, APU, cabin, cargo compartment, flight deck, wing and electrical fires including evacuation, smoke control and removal (TRI)|
|Instrument flight procedures||Circling approach (TRP, TRI)||Holding procedures (TRP); circling approach (TRI)|
|Missed approach procedures||Manual go-around with the critical engine simulated inoperative after an instrument approach on reaching DH, MDH, or MAPt (TRP); go-around with all engines operating during a 3D operation on reaching decision height; other missed approach procedures (TRI)||Manual go-around with the critical engine simulated inoperative after an instrument approach on reaching DH, MDH, or MAPt (TRP, TRI); go-around with all engines operating during a 3D operation on reaching decision height; other missed approach procedures; rejected landing at 15 m (50 ft) above runway threshold and go-around (TRI)|
|Landings||Landing with simulated jammed horizontal stabiliser in any out-of-trim position (TRP); crosswind landings (a/c, if practicable); traffic pattern and landing without extended or with partly extended flaps and slats (TRI)||Landing with simulated jammed horizontal stabiliser in any out-of-trim position (TRP), crosswind landings (a/c, if practicable); traffic pattern and landing without extended or with partly extended flaps and slats (TRP, TRI)|
|Instrument approaches||Rejected take-off at minimum authorised RVR, go-around after approaches on reaching DH (TRP)||Rejected take-off at minimum authorised RVR, go-around after approaches on reaching DH (TRP), CAT II/III approaches: in simulated instrument flight conditions down to the applicable DH, using flight guidance system (TRP, TRI)|
Potential AR applications for TR training.
Potential AR applications for TR training.
|Aircraft structure and equipment||✔||✔|
|Flight management system||✔|
|Normal flight procedures||✔|
|Abnormal and emergency flight procedures||✔|
|Instrument flight procedures||✔|
|Missed approach procedures||✔|
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).