Optimization of Air Traffic Control Training at the Federal Aviation Administration Academy

This paper investigates current and future uses of simulation in FAA Academy ATC training to identify potential improvement areas to the current training program in the areas of simulation technologies and course content. Once identified, recommendations for changes to the current training program are made. A thorough literature review of current training techniques used at the FAA Academy and training centers was conducted. In addition, this study utilized interviews and surveys to collect data regarding a variety of ATC training interest areas, such as virtual reality, current maintenance schedules, and simulator features. Finally, a thorough cost-benefit analysis was conducted to determine the potential improvement areas with the highest feasibility for implementation and highest potential to reduce training costs and/or time. The primary findings of this research revealed three feasible improvement areas to the current training process and simulation technologies: (1) reducing the dependence on instructors during simulation training, (2) utilizing web-based training methods, and (3) updating current simulator systems to include new features, such as recording and playback features. These changes were recommended to be implemented first, with voice recognition and virtual reality improvement areas being recommended for future studies and/or implementation.


Introduction
With advancements in computer simulation and visual graphics capabilities, the use of advanced flight simulation training devices in general aviation has increased significantly.A variety of simulation training devices and technologies are utilized in every phase of current pilot and air traffic control (ATC) training programs, ranging from low-fidelity desktop trainers to high-fidelity full-flight simulators.Simulation is one of the only methods currently available which gives trainees hands-on experience in learning such a high-skilled and detail-oriented occupation as ATC.However, as technology rapidly improves, current training devices can become outdated and less effective than when first implemented in a training program, leading to training inefficiencies and increased training times [1,2].To reduce the effects of outdated simulators and course content, current simulation training practices must be evaluated and analyzed to identify inefficiencies, areas of improvement, and potential applications for new or updated simulation technologies.
As pilot and ATC training programs constitute a large area of research, improvements to the Federal Aviation Administration (FAA) Academy's ATC training process is the primary focus of this study.In this paper, a brief background of the FAA ATC training process is presented in Section 2. In Section 3, a literature review introduces current simulation technologies available to the training community, as well as a presentation of current training techniques and areas of interest.Section 4 provides the methods and materials used in conducting a research study for the investigation of current training techniques and the resulting analysis, including an industry vendor survey and individual vendor interviews.Section 5 presents a cost-benefit analysis of topics and areas of interest identified via the literature review, survey, and interviews.Section 6 suggests recommendations for improving the ATC training process and simulation at the FAA Academy.Finally, in Section 7, an overview of the paper's contents and contributions to the field of ATC training is provided via a brief conclusion.

Background
To understand where simulation technologies can be applied or improved, the current FAA Academy training process and how it fits into the overall ATC training process must first be understood.These processes, explained in detail in a previous publication [3] by this research team, are presented in brevity here as an introduction.
ATC training occurs over two main phases: FAA Academy training and on-site facility training.All official training for air traffic control starts at the FAA Academy, which is located in Oklahoma City, Oklahoma.All new air traffic controller students hired by the FAA for training at the FAA Academy, known as developmental ATCs, are hired from one of three pools: previous controllers, Air Traffic-Collegiate Training Initiative (AT-CTI) developmentals, or the general public [4].Previous ATC training and knowledge varies in each of these three pools, resulting in a wide range of skill levels as developmental ATCs begin training at the FAA Academy.At the FAA Academy, ATC training consists of three main phases: air traffic academics, part-task training, and skills building.The individual courses in these phases are listed in the FAA Catalog of Training [5].Some courses are considered job jeopardy, or courses deemed essential to ATC, and failure of these courses results in a permanent dismissal from the FAA Academy and all further ATC training.To evaluate if a developmental ATC has successfully acquired the essential knowledge needed to be successful as an air traffic controller, all job jeopardy courses utilize a cumulative scoring process consisting of multiple performance assessments and written tests [2].A developmental ATC must achieve a score of 70 points out of 100 total points to pass the course and continue training at an on-site facility [3].
There are three major flight control divisions in ATC and ATC training: Tower, Terminal Radar Approach Control (TRACON), and En Route.Each division has a unique training track, with each track specifically designed to teach the required materials for only one division; however, all three tracks follow the aforementioned training phases.The Tower track focuses on air traffic management activities within a radius of a few miles of the airport and utilizes simulators to replicate real ATC towers and airport views.The TRACON track is comprised of the job jeopardy Terminal Basic Radar Training Course (RTC) for developmental proceeding to a standalone radar facility.This course incorporates classroom and simulation training focused on managing traffic outside the radius managed by Towers, generally extending only to a 40-mile radius from the primary airport.The En Route track is comprised of the most detailed courses at the FAA Academy and focuses on managing air traffic along defined routes.This track consists of classroom instruction, medium-fidelity skills practices utilizing interactive computer-based instructional systems, and high-fidelity En Route Automation Modernization (ERAM) simulation in an En Route lab [2,3].
After successful completion of the FAA Academy training program, developmental ATCs begin training at their assigned facilities.On-site facility training generally requires 1.5 to 3 years of instruction, and its content varies depending on the location and size of the facility [2].However, all on-site facility training consists of on-the-job (OTJ) training, classroom instruction, and simulation instruction.Unlike the FAA Academy, which trains all students on the same practice airspace [6], on-site facility training focuses only on the specific airspace controlled by the facility; this ensures that the developmental ATC has full knowledge of the airspace and regulations he or she will be responsible for once fully certified.After completion of on-site facility training, the developmental ATC is recognized as a Certified Professional Controller (CPC) and can begin working live air traffic [3].

Literature Review
In January 2013, the FAA conducted a review and evaluation of air traffic controller training at the FAA Academy and reported their findings, detailed in [7].The Air Traffic Division (AMA-500) was identified as delivering initial, advanced, and specialized air traffic training at the FAA Academy.Although divided into three primary areas of research: roles and responsibilities, communication of roles and responsibilities, and accommodation of developmental ATCs, only the accommodation of developmental ATCs proved to be particularly relevant to this research.This area focused on the ability of the FAA Academy to meet developmental ATC demands in the form of facility size, available instructors, and available training equipment, among other factors.It was found that the FAA Academy historically had no issues accommodating developmental ATC demands, as instructors could be contracted as needed and limited simulator space could be mitigated via a rotating class schedule.However, it was noted that the number of available simulators can eventually affect the total number of developmental ATCs that the FAA Academy can accommodate.High-fidelity simulation creates a very limited amount of learning positions, effectively creating a bottleneck for developmentals as they approach the end of their training.In addition, a lack of funding and financial support was found to make course maintenance and simulation updates more challenging to complete in a timely manner.As a result, it was recommended that new technologies be utilized in the development, maintenance, and delivery of training materials, as well as for training developmental ATCs more effectively.More courses which utilize web-based material and online training solutions were suggested.In addition, an analysis of ways to stabilize and improve the FAA Academy's funding was recommended [7].
In a 2006 study, Brudnicki, Chastain, and Ethier [8] provided an in-depth overview of the training technologies utilized in the FAA Academy and on-site facility training, as well as recommendations for future applications of technologies.High-fidelity, intelligent training systems technologies such as voice recognition and synthesis (VR&S), intelligent tutoring systems (ITSs), and instructor support capabilities were identified as potential areas of improvement in the FAA Academy training process.In addition, web-based instruction and location-independent, site-specific training was recommended.These technologies are explained in further detail below.
Although current implementations were deemed too immature for complex scenarios, VR&S has the potential to automate pseudo pilot and ghost controller capabilities in a variety of air traffic scenarios, which can improve the overall practice environment for skill development, support self-paced and independent learning, enforce the teaching and use of standard ATC phraseology, and reduce faculty costs [8].A similar study, detailed in [9], supports these findings and shows a vast potential for VR&S in ATC training if the drawbacks of VR&S, such as response time and accuracy, can be overcome.As this technology grows, these limitations can potentially be mitigated through improvements and/or alternatives in the design and approach of voice recognition systems, such as context-specific syntax pools [9].
An ITS is a set of automated capabilities combined with human performance models that provide for an objective assessment of learned skill levels, infer strengths and weaknesses of a student, and enable tailored instruction [8].ITSs have been proven effective in several domains of similar complexity to ATC, including U.S. Navy tactical decision-making simulations [10,11], Air Force technician troubleshooting simulations [12], and a Submarine ship handling simulation [13].This technology can reduce training times and increase instructional quality via accurate and objective feedback [8].In addition, ITSs facilitate student-driven learning by allowing for more independent operation, thus allowing a student to practice independently of an instructor and continually practice a problem area while still receiving informative feedback [1].
A well-known 1984 study by Bloom [14] demonstrated the advantages of one-on-one student learning versus group instruction, with 98% of tutored students achieving a higher score when given controlled tests than those taught via traditional classroom methods, such as group lectures.However, the study also recognized that individualized tutoring for every student in a training program is not a feasible solution for the training and education domains for a variety of reasons, including costs.As a result, the 2-sigma problem was discovered, with educators and training programs searching for techniques to train groups of students in ways that allow each student to attain the levels of achievement possible through individualized tutoring [14].Instructor support features, including updated simulator features such as the recording and playback of scenarios, reduce instructor workloads and improve teacher feedback to real-time student progress [8].This can bring ATC training one step closer to a more personalized training experience for each student in a large class by allowing instructors more freedom to move about the classroom and instruct each student individually.Improving the quality of ATC training through instructor support features is one solution to the 2-sigma problem and can potentially assist each student in achieving his or her maximum potential [14].
Distance-based learning can greatly reduce training time and costs when implemented effectively.Web-based instruction can replace or supplement standard lecture-based training courses [15], support self-paced learning in geographically separate locations, and test the aptitude of trainees before entry into the FAA Academy or a field facility [8].In addition, FAA Academy resources and on-site staff could be reduced.Training costs can also be reduced in a variety of areas, such as travel and housing expenses for trainees and contract employees [16].Networking and video teleconferencing technologies can also facilitate location-independent training in a variety of courses.This can greatly reduce training times and costs via reductions in instructor and developmental ATC travel requirements, and improve the quality of training by providing access to nationally and internationally recognized experts and educators, regardless of their locations [8,16].
A 2016 study by Knecht, Muehlethaler, and Ethier [17] analyzed training development, noting that training ATC nontechnical skills is an important aspect of ATC training that is better suited to practice-based environments.As a result, it was recommended that practice-based training be utilized over traditional frontal teaching methods when teaching nontechnical skills.Similarly, a study by Kang and Landry [18] explored using scanpaths as a learning method for air traffic conflict detection and showed significant performance increases and confidence boosts in students.A 2015 case study by Airservices Australia, detailed in [19], demonstrated the benefits of updated simulation systems with new features, such as recording, pausing, and replaying training sessions with both video and audio.Finally, Arminen, Koskela, and Palukka [20] analyzed the stimulus and response pairs of ATC training to identify activities and stimuli in the training process, leading to the observation that ATC trainees respond either to the scenario itself or to an instructor prompt.
Along with studies conducted in the field of ATC, related studies have also shown training benefits through the incorporation of new technology.Risukhin et al. [21] investigated the benefits of utilizing more affordable software and hardware low-fidelity simulators (LFSs) for improving the development of pilots' cognitive and behavioral skills, as well as acquiring research results from simulator use.Although LFSs are not as realistic in portraying what a student can expect to see in the workplace, the use of LFSs can be effective for training in normal flight operations if a high-quality training program supplements it.Therefore, LFS training can be best utilized in the early stages of the training process.LFS peer mentoring was found to be a beneficial training technique, with certified instructor pilots confirming that coupling student pilots with more proficient students in an informal way in an LFS accelerated the development of the student pilots' flight relevant actions.In addition, student pilot retention was improved when using an LFS as a replacement for lost flight training time due to weather or maintenance issues.When using an LFS, training costs and time were reduced by allowing for more training opportunities in a lower-cost simulator.This study also included an investigation into the communication breakdowns between pilot and ATC leading to fatal accidents and other safety hazards.The LFS software used in the study could simulate pilot-ATC communication, which was then used to better understand which language factors influence pilots' radio communication with English-as-a-second-language controllers.The ability to use technology to study and improve safety between pilots and controllers can provide valuable benefits to the training community, such as a better awareness of the consequences of unclear phraseology or speech on training performance [17].

Methods and Materials
After the literature review was completed, an investigation into the current state of simulation and training for the FAA Academy's ATC training process was conducted.A confirmation of the current state of simulation and training at the FAA Academy was needed before recommendations for improvements regarding the training process and use of simulators could be made.To better understand the current state of simulation and training in ATC, the research team contacted a variety of industry professionals and vendors of ATC simulation technology to the FAA and other training establishments.These vendors and professionals were contacted via two methods: (1) phone interviews, and (2) surveys.These contact methods allowed the research team to better understand the current successes and barriers in the current ATC training process, and, in combination with the knowledge gained from the literature review, allowed for recommendations to be made that are more applicable to the current state of the field than recommendations based only on the literature review.

Setup
While the literature review provided an overview regarding the history of the ATC training process and a large variety of simulation technologies, the current state of the field was still unknown.Recommendations made in previous studies needed to be investigated by the research team to determine if these changes were in fact implemented as suggested.The feedback gained from the industry professionals and vendors allowed the research team to cross-compare recommendations given by those contacted with the recommendations and observations found in the literature review.In addition, new ideas and recommendations could be formulated from the feedback of those vendors and professionals currently experiencing the drawbacks and limitations of current technology.

Approach
As the research topic continues to span a vast area of knowledge, only a select few topics were included in the survey and interviews.These topic areas were selected based upon the research goals, which focused on investigating and optimizing the current uses of ATC simulation technologies in the FAA Academy training program, communicated to the research team by the research sponsor, the Center of Excellence for Technical Training and Human Performance (COE HTTP).As a result, the primary research areas the team focused on investigating were: (1) the current types of simulation technologies being used for ATC training and the maintenance and upkeep of these technologies, and (2) how simulation technologies affect the training process, both positively and negatively.As a result, two major topic areas can be seen in the selected questions for the survey and interviews: (1) current simulation technologies, their maintenance, and potential future simulation technologies, and (2) how simulation affects the training process.These topics were selected to validate and confirm which technologies are currently being used in FAA training against the literature review, understand the current limitations of simulator technology features, and determine an average update and/or replacement time for outdated simulator technologies.In addition, questions targeting how simulation is currently being used in the training progress focus on improving how simulation is incorporated into a training program.

Methodology
The survey and interviews were formulated with a time restriction in mind.It was assumed that a shorter survey would encourage more responses from an already limited number of participants.Interviews were conducted prior to the survey primarily as a means of determining which topic areas the survey should focus on.Interviewees were initially selected from a list of industry vendors and professionals, available at [22], potentially interested in the research area, many of whom are industry partners with the COE HTTP sponsored by the FAA.The survey focused on collecting objective responses to some questions or topic areas discussed during the interview stage.In addition, open-ended questions allowed industry vendors and professionals to provide additional comments and ideas for future recommendations.
After the survey was distributed and responses were being collected, some responders opted to hold an informal interview in lieu of submitting a survey.This was primarily because some vendors did not have enough experience in all survey topic areas to complete the survey as intended, but had vast knowledge regarding a specific topic area that would benefit the research team.

Data Gathering
Interviews were conducted over the phone and typically lasted 30-45 minutes.The interviews were informal in nature and consisted of large, open-ended prompts aimed at promoting discussion in the following areas:

•
The relationship of the interviewee to the FAA and/or the FAA Academy; The types of technologies the interviewee provides to the FAA and/or the FAA Academy, if any;

•
The types of technologies the interviewee has prior experience using, developing, installing, and maintaining; • Any recommendations and/or directions the interviewee believes would benefit the FAA Academy the most, when considering the future of ATC training and technologies known by the interviewee.
During a review of the interview responses, potential research areas were determined and listed for further investigation.Survey topic areas and questions were formulated via the knowledge gained from the literature review and prior interviews.The surveys were designed to be completed within 20 min, and were distributed via email to those industry vendors and professionals that demonstrated knowledge or experience in air traffic simulation and/or the FAA Academy.In addition to the survey, a consent form was provided.The survey was comprised of 17 questions and included a mix of multiple choice, multiple selection, and short answer formats.The survey questions targeted the following areas: The availability and inclusion of advanced simulator features, including ITS and instructor support features.
Surveys and consent forms were completed and returned via email.Survey completion is considered anonymous and no vendor or professional will be specifically linked to a response without prior permission from the responder.After all surveys were collected, questions were consolidated and analyzed for patterns in responses, potential recommendations, and new research areas.

Results
A total of thirteen industry vendors and professionals were contacted to participate in a survey, an interview, or in some cases both.Nine vendors and professionals were contacted for an interview initially, with five focusing on a similar topic area to the research team.From the initial round of interviews, the following topic areas were identified as relevant to the current research: virtual reality (VR), adaptive training technologies, web-based training solutions, and VR&S.Of the thirteen industry vendors and professionals contacted to complete a survey, nine provided survey responses and one opted for a phone interview in lieu of completing a survey.Of the nine returned surveys, one was removed from analysis due to incomplete information, with only three of the seventeen questions having been answered.However, any information provided via this survey, when applicable to the selected topic areas, was still considered when deliberating on potential recommendations.
Due to the limited scope of this survey, which intended to assess the current state of simulation technologies at the FAA Academy only, the survey was distributed to a small group of participants from very similar domains.As a result, the implications of the survey results may not transfer well to other air traffic control training sectors, such as on-site facility training.However, this survey limitation was expected as the intent of the survey was to assess training at the FAA Academy specifically and was not intended to assess air traffic training and simulation technologies as a whole.
The results from the survey analysis supported many of the discoveries made during the literature review and interviews.When asked if simulation technologies enhance the ATC training process, respondents unanimously agreed that simulation is a great benefit in the training process, noting that training devices allow students to bridge the gap between training and reality and that training devices are cheaper, more available, and safer to use than traditional hands-on training methods.In addition, simulation allows for learning from mistakes, replicating obscure training situations, and building student confidence.When asked which simulation technologies are currently used the most for ATC training, desktop simulators and high-fidelity simulators topped the list at four responses each, with VR, web-based trainers, and VR&S falling behind, as shown in Figure 1.However, these technologies were found to have only a similar potential for reducing training costs and times when compared with the currently used technologies, as shown in Figure 2.These technologies allow for home training, self-paced learning, easy accessibility, and lower costs to prepare and maintain than the previously used high-fidelity simulators that are popular today.Note that for these questions, respondents selected multiple answers, creating a total greater than eight responses.When asked if current ATC simulators provide personalized feedback depending on each student's need, only three responders could confirm their simulator technologies did.This result suggests an area of potential for improving simulator effectiveness and reducing training times through personalized and effective feedback.
Aerospace 2017, 4, 49 7 of 12 applicable to the selected topic areas, was still considered when deliberating on potential recommendations.Due to the limited scope of this survey, which intended to assess the current state of simulation technologies at the FAA Academy only, the survey was distributed to a small group of participants from very similar domains.As a result, the implications of the survey results may not transfer well to other air traffic control training sectors, such as on-site facility training.However, this survey limitation was expected as the intent of the survey was to assess training at the FAA Academy specifically and was not intended to assess air traffic training and simulation technologies as a whole.
The results from the survey analysis supported many of the discoveries made during the literature review and interviews.When asked if simulation technologies enhance the ATC training process, respondents unanimously agreed that simulation is a great benefit in the training process, noting that training devices allow students to bridge the gap between training and reality and that training devices are cheaper, more available, and safer to use than traditional hands-on training methods.In addition, simulation allows for learning from mistakes, replicating obscure training situations, and building student confidence.When asked which simulation technologies are currently used the most for ATC training, desktop simulators and high-fidelity simulators topped the list at four responses each, with VR, web-based trainers, and VR&S falling behind, as shown in Figure 1.However, these technologies were found to have only a similar potential for reducing training costs and times when compared with the currently used technologies, as shown in Figure 2.These technologies allow for home training, self-paced learning, easy accessibility, and lower costs to prepare and maintain than the previously used high-fidelity simulators that are popular today.Note that for these questions, respondents selected multiple answers, creating a total greater than eight responses.When asked if current ATC simulators provide personalized feedback depending on each student's need, only three responders could confirm their simulator technologies did.This result suggests an area of potential for improving simulator effectiveness and reducing training times through personalized and effective feedback.Although many responses supported prior discoveries, some seemed to waver from previous assertions made.One example is seen in the updating and maintenance of simulation technologies responses.When asked if training simulators are current and up-to-date with present technologies available in the field, only two responders answered positively.However, later responses to more specific questions regarding the updates and maintenance of these same simulation technologies appear to claim the opposite.Although the responses did vary, update and maintenance schedules for air traffic simulators averaged between updating at least once a year and up to four times a year.However, in order to better quantify this frequency, the rate at which air traffic technologies are updated for use in the National Aerospace System (NAS) must be determined.This rate was not assessed within this survey.
To update and maintain an air traffic simulator, a software update must first be created.When asked how long, on average, it takes to create a simulator update, the responses averaged below three months.Finally, installing these updates was found to take less than thirty minutes on average.These findings seem to somewhat counter the previous assumption that most simulator software is out of date in that it suggests that the creation of updates for simulators, which accounts for the changing state of technology, would not take long to complete.However, a note must be made that these questions did not include potential hardware updates, which could still be lagging behind industry improvements, and the time in which a new technology could be released before the simulation community is made aware.
One of the most controversial topic areas was the use of VR.Although supported in the literature review as a potential area of improvement for ATC training, the initial round of interviews refuted this idea.However, the survey again refuted the interviews, with 88% of respondents answering positively when asked if incorporating VR components will improve simulator transfer of learning.However, while the responses were positive, they were also skeptical.Many responses included recommendations, such as using augmented reality instead of VR, and warnings, such as VR only being useful for experienced students and/or controllers needing refreshers on complicated scenarios and that, while potentially useful, VR has not yet been proven as an effective training method.However, many positives were also identified, such as VR providing a great potential for fourdimensional (4-D) learning, adding additional immersion and training realism, allowing students to experience a realistic training location and environment before arriving at a facility, and incorporating additional kinds of stress found on the job, such as facility noise and coordination.Although many responses supported prior discoveries, some seemed to waver from previous assertions made.One example is seen in the updating and maintenance of simulation technologies responses.When asked if training simulators are current and up-to-date with present technologies available in the field, only two responders answered positively.However, later responses to more specific questions regarding the updates and maintenance of these same simulation technologies appear to claim the opposite.Although the responses did vary, update and maintenance schedules for air traffic simulators averaged between updating at least once a year and up to four times a year.However, in order to better quantify this frequency, the rate at which air traffic technologies are updated for use in the National Aerospace System (NAS) must be determined.This rate was not assessed within this survey.
To update and maintain an air traffic simulator, a software update must first be created.When asked how long, on average, it takes to create a simulator update, the responses averaged below three months.Finally, installing these updates was found to take less than thirty minutes on average.These findings seem to somewhat counter the previous assumption that most simulator software is out of date in that it suggests that the creation of updates for simulators, which accounts for the changing state of technology, would not take long to complete.However, a note must be made that these questions did not include potential hardware updates, which could still be lagging behind industry improvements, and the time in which a new technology could be released before the simulation community is made aware.
One of the most controversial topic areas was the use of VR.Although supported in the literature review as a potential area of improvement for ATC training, the initial round of interviews refuted this idea.However, the survey again refuted the interviews, with 88% of respondents answering positively when asked if incorporating VR components will improve simulator transfer of learning.However, while the responses were positive, they were also skeptical.Many responses included recommendations, such as using augmented reality instead of VR, and warnings, such as VR only being useful for experienced students and/or controllers needing refreshers on complicated scenarios and that, while potentially useful, VR has not yet been proven as an effective training method.However, many positives were also identified, such as VR providing a great potential for four-dimensional (4-D) learning, adding additional immersion and training realism, allowing students to experience a realistic training location and environment before arriving at a facility, and incorporating additional kinds of stress found on the job, such as facility noise and coordination.

Cost-Benefit Analysis
After a careful analysis of the literature review suggestions, interview responses, and survey responses and recommendations, five potential improvement areas were selected for further analysis: (1) the replacement of live pseudo-pilots with simulated pseudo-pilots, (2) the use of VR, (3) updating current simulation systems' features, (4) reducing the dependence on instructors during training, and (5) utilizing web-based training solutions.These improvement areas were chosen after considering a variety of factors; however, many were selected due to popularity among responses, feasibility of implementation, and potential for improving the current training process.
To better compare these five improvement areas, a break-even, cost-benefit analysis was conducted by the research team, similarly to those analyses conducted in [23,24].A break-even analysis that finds the minimum amount of benefits needed to outweigh the costs for a particular improvement area was deemed practical for this application, as costs associated with simulation technologies vary vastly between different vendors, technology types, and technology maturity, and it is therefore difficult to quantify costs via monetary values.As a result, each improvement area is instead ranked in relation to how far below or above the break-even line, which is defined as the point at which the sum of all benefit-and cost-values produced a result of zero, the improvement area fell, when all identified benefits and costs were assigned.An improvement area that fell above the break-even line was considered more feasible than those that fell below the break-even line.
Each area was examined and potential benefits and costs/disadvantages were identified from the literature review, interview responses, and survey responses.Each benefit was given a benefit value between 1 and 10, with 1 indicating the benefit is of low value to improving the training process and 10 indicating the benefit is of high value to improving the training process.These values were chosen and assigned by the research team based on all previously-gained knowledge, correspondence with the FAA, correspondence with the Embry-Riddle Aeronautical University (ERAU) ATC department, and in relation to the other listed benefits.Each cost was given a cost value between 1 and 10, with 1 indicating the cost is of low value to improving the training process and 10 indicating the cost is of high value to improving the training process.These values were chosen following the same methods as the benefit values.In addition to the cost value, costs were also given a time cost, or cost-to-implement, value between 0 and 10.This value takes into consideration how long it would take to implement an identified cost and/or how expensive the cost could be to the organization implementing the change.A time cost of 0 is given when a cost does not require a change to be implemented, such as when a cost is a disadvantage of the improvement area and not a change in technology.A time cost of 10 is given when an identified cost would require substantial changes to the current training process or training technologies.
The overall estimated value/feasibility value is defined as: The overall estimated value creates a comparable value by which all five improvement areas can be ordered by feasibility, while also accounting for all costs and benefits.Finally, all five improvement areas were given an overall recommendation rank based on the overall estimated value.The overall estimated value and overall recommendation rank are shown in Table 1.An overall estimated value less than 0 indicates a potential improvement area evaluated with more costs than benefits.As a result, these improvement areas would not be as easy, cost-effective, or feasible to implement.An overall estimated value greater than 0 indicates a potential improvement area evaluated with more benefits than costs and would therefore be a better investment of time and/or resources.Simulated pseudo-pilots and VR each had eight identified benefits and five identified costs.Updated simulation systems had five identified benefits and four identified costs.Reducing instructor dependence during training had five identified benefits and two identified costs.Finally, web-based training solutions had ten identified benefits and three identified costs.As shown in Table 1, reducing instructor dependence during training was identified as the most feasible improvement area, with an overall estimated value of 22.This was due primarily to a multitude of potential benefits achieved through a reduction in instructor dependence, such as increased one-on-one time with trainees and allowing for larger class sizes and/or fewer instructors, while requiring very low implementation costs.This improvement area would focus primarily on improving the capabilities of simulation systems (with separate costs and benefits identified under updated simulation systems) and improving the teaching process in ATC training programs.

Recommendations
After considering the cost-benefit analysis results, it is recommended that the top three potential improvement areas be implemented.First, a focus should be made to reduce the dependence on instructors during training.This dependence can be reduced by incorporating more instructor support capabilities in the simulators, such as the recording and playback of scenarios, and through ITS systems, which provide automated, personal feedback to each student.A reduction on instructor dependence allows for larger class sizes and would require fewer instructors overall.In addition, instructors would have more opportunities to increase the quality and frequency of feedback during a training session as the number of tasks an instructor must manage could be significantly reduced with the aid of technologies such as VR&S.However, one downside of reducing instructor dependence would be the need to update current simulator features to support this change, leading to implementation and/or replacement costs.Second, web-based training technologies should be utilized more heavily during the training process.These technologies can reduce the reliance on long, in-person lectures, allow students to learn at their own pace, allow materials to be more easily accessible, and can allow students to prepare for in-person training on their own time.However, some disadvantages include the need to create and manage new web-based courses and potential security issues, as training materials would be hosted online.Third, current simulation systems should be updated to include additional features deemed beneficial to ATC training.These features include the ability to pause, rewind, record, share, and replay scenario sessions, the ability to provide personalized training feedback for each student, and the ability to display simulated trainer prompts that can notify trainees when an action that should have occurred was taken incorrectly or missed completely.Some disadvantages include the potentially large cost to implement these new features and higher maintenance costs of the updated simulators.
While the remaining two improvement areas are still being considered to be potential improvement areas, they are not being recommended at this time due to the potentially high implementation costs.These technologies must also mature before they can be deemed feasible for use in ATC training, as both technologies are afflicted by disadvantages and limitations in both hardware and software aspects.However, it is recommended that these potential improvement areas be considered again in future studies.

Conclusions
To reduce the effects of outdated simulators, current simulation training practices must be evaluated and analyzed to identify inefficiencies, areas of improvement, and potential applications for new or updated simulation technologies.To accomplish this task, current literature following the current ATC training process at the FAA Academy was identified and analyzed for potential improvement areas in both the training process and training simulation technologies.In addition to a literature review, this study conducted phone interviews and distributed a survey to industry professionals and vendors to obtain insight into the current state of simulation technology and training processes in ATC.From the literature review, interviews, and survey results, five potential improvement areas were identified: (1) the replacement of live pseudo-pilots with simulated pseudo-pilots, (2) the use of VR, (3) the updating of current simulation systems' features, (4) the reduction of the dependence on instructors during training, and (5) the utilization of web-based training solutions.Of these improvement areas, reducing the dependence on instructors during training was found to have the most benefits with the fewest costs and was recommended as the first area of improvement to be implemented in future training processes.In addition, utilizing web-based training solutions and updating current simulation systems' features followed as the second and third recommendations, respectively.The remaining two improvement areas were deemed not currently feasible; however, they are still being recommended for further analysis in future studies.

Figure 1 .
Figure 1.Response distribution for current technology usage.ATC: air traffic control; TRACON: Terminal Radar Approach Control.

Figure 1 .
Figure 1.Response distribution for current technology usage.ATC: air traffic control; TRACON: Terminal Radar Approach Control.

Figure 2 .
Figure 2. Technologies with the most potential for reducing training times and costs.

Figure 2 .
Figure 2. Technologies with the most potential for reducing training times and costs.

•
Demographic questions, including years worked in the ATC training and simulation domain and experience with listed technologies; • Simulation technologies and their relationship(s) to training, including which technologies were thought to provide the most potential for increasing trainee learning and understanding of material; • Simulation technologies used in training and their relationship(s) to the newest technologies available in the market, including if simulation technologies used in training are up-to-date and competitive with the current technologies available outside of training, such as NextGen; •