Core Competency Assessment Model for Entry-Level Air Traffic Controllers Based on International Civil Aviation Organization Document 10056

Abstract

With the increasing air traffic flow, the workload of air traffic controllers is also growing, and their proficiency directly impacts civil aviation safety and efficiency. To address the lack of clear training objectives and inconsistent evaluation methods in the initial controller training at the Southwest Air Traffic Management Bureau, this study aimed to develop and validate a core competency model for initial air traffic controllers. Referencing ICAO Document 10056, the study first defined core competencies. Subsequently, using job analysis, the behavioral event interview (BEI) method, and expert panels, a core competency model tailored to the training objectives of the Southwest ATMB was constructed. The key findings of this research include: first, the defined structure of the developed model, comprising seven competency dimensions, 21 elements, and 26 observable behaviors (OBs); second, the determination of combined weights for each dimension and indicator using questionnaire surveys, the Analytic Hierarchy Process (AHP), and the Entropy Weight Method; and third, the successful application and validation of the model. Specifically, in its application, the weighted TOPSIS method was employed to evaluate trainees in a specific group. This not only provided a ranking of trainee abilities but also facilitated in-depth analysis through radar charts of competency dimensions and box plots of OB items. These application results demonstrate the model’s effectiveness and practicality.

1. Introduction

With the rapid growth of global air traffic and the increasing complexity of air traffic management, air traffic controllers—being the core human resource in aviation safety operations—have a direct impact on airspace operational efficiency and flight safety. The International Civil Aviation Organization (ICAO) clearly stated in Doc 10056—Manual on Air Traffic Controller Competency-Based Training and Assessment—that constructing a competency-based training system is a key approach to addressing the standardization of controller capabilities [1]. In the coming years, the Federal Aviation Administration (FAA) will face a significant loss of air traffic controllers, with an estimated loss of 8595 controllers from 2021 to 2030, including retirement, promotion, transfer, and loss during training. To meet this challenge, the FAA plans to recruit approximately 8854 new controllers during this period to meet the needs of air traffic control [2].

Currently, the Chinese civil aviation controller training system divides training into separate subjects, which limits its flexibility; each subject must be taught and assessed independently. When faced with complex or rapidly changing situations, this approach may fall short, as trainees may only exhibit skills related to isolated tasks and fail to respond effectively to realistic operational scenarios. Moreover, qualitative analyses of individual trainees lack traceability throughout the training process and cannot provide scientific and reasonable guidance for future improvement. The Civil Aviation Administration of China (CAAC) has proposed an assessment model that emphasizes three key areas: core competencies, psychological capabilities, and professional work style. Within this model, core competencies are identified as the crucial determinant of an individual’s ability to successfully perform their designated tasks. It is further stipulated that the core competency framework forms the bedrock of the entire Evidence-Based Training (EBT) system, thereby providing a definitive basis for the establishment of training objectives. This approach ensures that personnel can comprehensively apply their knowledge, skills, and attitudes to effectively manage real and complex operational environments, particularly when encountering complex conditions or rapidly evolving situations, ultimately leading to enhanced safety and efficiency [3]. The main purpose of this study is to establish a core competency model that meets international standards and adapts to the localization needs of the Southwest Air Traffic Control Bureau through systematic research, and then quantify ATCO capabilities through instructors’ scoring of observable behaviors. Finally, quantitatively analyze ATCO capabilities and provide training program recommendations for the bureau.

In the field of competency research, Taylor argued that through time and motion studies, managers can distinguish between high-quality and high-performance work processes, identifying key contributing factors [4]. In 1973, McClelland from Harvard University first introduced the concept of “competency”, which has since become a research focus in fields such as organizational behavior and human resource management [5]. In 1991, Boyatzis further aligned competency indicators with specific industries and clarified the primary methods for collecting and analyzing competency characteristics [6]. In recent years, competency models have been widely used in recruitment and training. Examples include the 3M’s model for senior managers, modular modeling methods for multi-role functions, and the seven-step modeling process in the healthcare industry—all reflecting the diversified development of competency models. Scholars have also discussed the distinctions between “capability” and “competency” and evaluated model effectiveness [7,8,9,10,11,12]. In 2001, the Australian government defined five core competencies for public sector leadership: professional ethics and integrity, effective team communication, critical thinking, operational control, and competitive communication abilities [13]. Competency models have been extensively applied in talent assessment, training, and performance management, offering advantages such as strategic alignment and performance orientation. Studies have proposed best practices for modeling while pointing out the need for improved definitions and methodologies. Some models have demonstrated effectiveness in enterprises and educational contexts, and with the integration of information technology, their application efficiency and personalization levels have been enhanced [14,15,16,17,18]. These generic competency frameworks and their evolution provide the necessary context for understanding more specialized models, while these studies provide a broad perspective on competencies, their application to safety-critical and highly specialized roles, such as the aviation industry, requires a more context-specific approach.

The successful application of competency models in professional capability assessment has indeed yielded significant results in various fields. For instance, research in corporate HR has demonstrated that competency-based career management systems can improve employee performance by 20% and reduce turnover by 10% [19]. Particularly relevant to the aviation sector, Haiwen Xu et al. (2024) proposed behavioral indicators of core competencies for pilots, aiming to support pilot training and enhance flight safety [20]. This underscores the value of tailored competency frameworks in high-consequence professions.

Regarding the study of the air traffic controller competency model, in 2006, Esther Oprins conducted a study based on the Dutch air traffic control system and constructed a controller competency model based on competency assessment. The model classifies various competencies by importance, including situational awareness, decision making, emergency response, workload control, conflict resolution, multi-tasking, prioritization, coordination and communication, flexible planning, leadership, teamwork, and perseverance [21]. The International Civil Aviation Organization (ICAO), the Federal Aviation Administration (FAA) of the United States and the European Organization for the Safety of Air Navigation (EUROCONTROL) have also conducted research on the competencies of air traffic controllers and applied them to training and assessment. A key framework is the European Union Regulation (EU) 2015/340 [22], which specifies the common technical requirements and management procedures for air traffic controller licenses and certificates. Although the EU regulation specifies common core competencies for initial training, it does not specify how training organizations should define and use assessment criteria to assess competencies by observing the behavior of candidates. In 2017, ICAO published Doc 10056.

A. Luppo summarized assessment methods for controller competencies, including continuous assessment, targeted assessments, and hybrid approaches. Assessment content covers theoretical knowledge, radio communication and coordination, interpersonal communication, numerical sensitivity, traffic planning, special situation identification, response ability, situational awareness, application of minimum separation, flight crew requirements, fatigue recovery, and teamwork. His findings emphasize the need to evaluate not only controllers but also instructors and management personnel. In 2018, S. K. Soldatov identified the fundamental core skills of ATCOs as the perceptual abilities to search, detect, and identify targets, and the decision-making capabilities to resolve conflicts [23].

Although mature ATCO competency frameworks exist internationally, these frameworks do not fully consider China’s national conditions or the specific context of the Southwest Air Traffic Management Bureau. Differences in operational environments, regulatory standards, infrastructure, and management philosophies between Chinese and international ATC units lead to divergent competency requirements.

Domestically, early research into controller competency models began with Wang Peng, who suggested that introducing competency models can effectively improve training needs analysis and optimize training structures [24]. Xu Nuo (2007) examined the competency characteristics of Chinese ATC personnel and constructed a model using hypothetical methods, proposing a framework for evaluating controller performance to support selection and training [25]. In 2016, Xu Hongjia studied 746 controllers from the North and Southwest ATC bureaus and found a significant positive correlation between job stress and burnout, paving the way for incorporating psychological factors into competency models [26]. In 2018, Li Jianqiao and Zhong Fengwei developed a competency model for ATC supervisors using literature reviews, case studies, behavioral event interviews, expert panels, and surveys, thus advancing the refinement of competency research for specific ATC roles [27]. In 2020, Lai Guijin, Ma Xuepeng, and Wu Dingjie pointed out new challenges in ATCO training under evolving requirements and emphasized the utility of competency theory in guiding training and adapting to new demands [28]. In 2022, Liu Chengxue summarized current issues such as regional disparities in controller development, selection systems, professional quality, training, and exit mechanisms [29]. Liu Che emphasized that competency models support efficient training and provide reference frameworks for promotions and talent development [30].

Despite technological advances enhancing safety and efficiency in aviation systems, air traffic control remains heavily dependent on human cognitive abilities. With increasingly complex airspace and the integration of UAVs, establishing a scientific competency model for entry-level controllers is more crucial than ever [31]. C. Duan developed a competency model to calculate the position-fit scores of individual trainees [32]. However, most of these efforts rely on generic models and none of them build the core competency model for air traffic controllers according to the framework in ICAO Doc10056. The Southwest Air Traffic Management Bureau, in contrast, requires a more differentiated approach due to its controller career progression structure—ranging from entry-level controllers to solo controllers, instructor controllers, and team leaders. Additionally, solo, instructor, and team leader roles are further divided by control domains: tower, approach, and area control. These variations result in significant differences in required competencies across roles.

In December 2020, the Civil Aviation Administration of China released the Implementation Plan for the Construction of a Professionalized Full-Lifecycle Management System for Civil Aviation Transportation. Based on this, the present study conducts a preliminary exploration into the core competency model of controllers from a lifecycle perspective, tailored specifically for the Southwest Air Traffic Management Bureau. It aims to construct an entry-level controller competency model for the first time based on the ICAO 10056 framework. This study used literature analysis, expert interviews and empirical methods to establish a multidimensional core competency model covering indicators such as situational awareness, traffic and capacity management, communication, coordination, and their sub-elements and observable behaviors (OB). In addition, a set of comments that meet the requirements of ICAO Doc10056 evaluation standards was constructed for instructors to choose from through LLM. It then scientifically determines the weights of each dimension and indicator. Instructors score OBs to quantify each trainee’s competencies and generate recommendations for the bureau’s training programs.

This study is based on the guiding ideology of the “Implementation Plan for the Construction of the Lifecycle Management System of China’s Civil Aviation Transport Occupation” issued by the Civil Aviation Administration of China in December 2020, and provides a quantitative basis for the selection and assessment of air traffic controllers. The core goal of this study is to build a scientific and multi-dimensional core competency model for entry-level air traffic controllers for the Southwest Air Traffic Control Bureau and explore its application methods. To achieve this goal, this study will for the first time take the framework of the International Civil Aviation Organization (ICAO) Document 10056 as an important theoretical basis, and comprehensively use literature analysis, expert interviews, and empirical research methods. Specifically, the research will focus on establishing a core competency indicator system covering key dimensions such as situational awareness, traffic and capacity management, communication, coordination, and their subordinate elements and observable behaviors (OB); and then determine the combined weights of the competency indicators of each dimension through game theory combined with the subjective and objective weights obtained by AHP and EWM; and through standardized scoring of the instructor’s evaluation of the observable behavior of the trainees, to achieve a quantitative assessment of the trainees’ abilities, and provide suggestions for the optimization of the training program of the Southwest Air Traffic Control Bureau. The specific research roadmap is shown in Figure 1.

2. Construction of the Entry-Level Air Traffic Controller Core Competency Model

2.1. Concept of the Initial Air Traffic Controller Core Competency Model

Through extensive literature review and a detailed study of ICAO Doc 10056, a three-level dimensional framework for the core competency model—comprising Competency, Element, and Observable Behavior (OB)—was clearly established. The core competency model is defined as a job-requirement-based set of interrelated behavioral capabilities that describe how to perform tasks effectively and demonstrate proficient skill performance. This includes the competency title, description, and a list of behavioral indicators.

An Element is defined as an action within a task that has a triggering event and an ending event, with clear boundaries and observable outcomes. A Competency consists of a series of such elements and represents a distinct job function.

An Observable Behavior (OB) is defined as a concise evaluative statement about the expected outcome of a competency element, including a description of the criteria used to judge whether the required performance level has been met. In essence, it serves as a performance standard—a specific, observable behavioral manifestation used by instructors to assess trainees against OB items during operations.

2.2. Construction of the Competency Dimension Indicator

Core competencies include both technical and non-technical knowledge, skills, and attitudes. According to ICAO Doc 10056, the competency framework is universal and applicable to all training phases. It is necessary to understand the local environment and adjust accordingly to develop a curriculum suitable for the challenges of the training phase and operational environment. Therefore, the indicators of the competency dimensions directly adopt existing competencies from ICAO Doc 10056.

Through expert interviews, it was concluded that the Southwest Air Traffic Control Bureau does not require training for the competencies of self-management, workload management, and teamwork during the initial air traffic controller training phase, so these competencies were removed. The final competencies retained are: situational awareness, traffic and capacity management, separation and conflict resolution, communication, coordination, Management of non-routine situations, and problem solving and decision making.

2.3. Construction of the Element and Observable Behavior Dimensions

ICAO Doc 10056 does not provide references for the Element and Observable Behavior dimensions. Therefore, this section will combine the initial air traffic controller training needs of the Southwest Air Traffic Control Bureau and use a mixed research method to construct the Element and Observable Behavior parts of the core competency model. The specific process is divided into three stages: literature and standard analysis, expert panel discussion, and Behavioral Event Interviews (BEIs), ultimately extracting the competency elements and their associated observable behavior indicators.

2.3.1. Literature and Standard Analysis

First, a systematic review of the research findings on air traffic controller competency models from various literature and sources was conducted. The conclusions of some representative and well-established models are summarized and presented in

Table 1. Example of research on typical competency characteristics.

Source of Information	Dimensions	Specific Indicator Items
FAA Air Traffic Control Training/Assessment [33]	Separation Control	Safety Separation, Safety Alerts
	Coordination Ability	Transfer of Control, Coordination
	Control Awareness	Control Judgment, Priority Control, Proactive Control, Traffic Flow Control
	Procedures and Methods	Situational Awareness, Use of Progression Sheets, Instruction Issuance, Adherence to Protocols, Other Services, Emergency Handling, Attention Allocation
	Equipment Usage	Understanding of Equipment Operating Status, Equipment Usage
	Communication Skills	Coordination and Cooperation, Clarity of Speech, Standardized Communication, Shift Handover
Vuckovic A [34]	Personal Aspects	Attitude, Decision-Making, Emotional State, Attention, Motivation, Professional Experience, Personality, Perceptual Ability
	Organizational Aspects	Organizational Environment, System Support
	Psychological Aspects	Workplace Relationships, Crew Dynamics, External Situations
Lai Guijin [28]	Professional Competence	Learning Ability, English Proficiency, Communication Skills, Equipment Mastery
	Personal Qualities	Quick Reaction, Short-Term Memory, Energy Allocation, Stress Resistance
	Professional Qualities	Teamwork, Responsibility, Self-Control, Rule Awareness, Professional Interest

.

2.3.2. Expert Panel Discussion

The study selected a Delphi expert panel consisting of n = 12 experienced air traffic controllers who meet the following three important criteria:

• Minimum of 5 years of professional experience;
• Continuous involvement in the development of initial training courses in the past three years;
• Familiarity with the model framework of ICAO Doc 10056.

The study first collected 38 potential element indicators, and then employed a two-round iterative pre-discussion mechanism:

• The first round focused on analyzing the compatibility between the ICAO Doc 10056 standard and the operational characteristics of entry-level air traffic controllers, establishing an indicator mapping matrix:

  $$A_{m\times n}=\left[a_{ij}\right],a_{ij}=\left\{\begin{array}{cc}1\hfill & \mathrm{Element}i\mathrm{meets}\mathrm{the}\mathrm{standard}j;\hfill \\ 0\hfill & \mathrm{Element}i\mathrm{does}\mathrm{not}\mathrm{meet}\mathrm{the}\mathrm{standard}j\hfill \end{array}\right.$$

  (1)
  In which m represents the 12 experts, and n represents the 38 potential element indicators. The frequency of each indicator in the matrix $A_{m\ast n}$ was then counted, and those with a frequency of 6 or more were selected as the elements to be discussed in the second round.
• The second round involved establishing a three-dimensional structured discussion framework (Knowledge/Skills/Attitudes). Ultimately, the Critical Incident Technique (CIT) was used to extract 21 examples of typical work scenario elements. See Figure 1 below for details.

2.3.3. Behavioral Event Interviews (BEI)

Based on the Southwest Air Traffic Control Bureau’s air traffic controller qualification database, a stratified random sampling method was used to select 100 controllers of various levels as research subjects. A 5-day semi-structured in-depth interview was conducted, with a three-stage progressive questioning approach: “Success/Failure Critical Incident Description → Behavioral Detail Retrospection → Self-assessment of Competency Requirements”. The focus was on typical work scenarios related to the competencies (such as thunderstorm rerouting commands, special incident handling coordination, etc.). After the interview recordings were transcribed into text using a professional speech transcription system, thematic analysis was applied to extract high-frequency behavioral keywords, which were then refined into Observable Behaviors (OB). Finally, the results were cross-verified with the literature analysis and expert opinions from the initial training phase, leading to the identification of the Observable Behaviors (OB) for the Southwest Air Traffic Control Bureau’s entry-level controller training. See Figure 2 below for details.

2.4. Construction of a Collection of Optional Comments

Based on the content of ICAO Doc 10056, in order to provide a more detailed scoring system, corresponding comments need to be given to provide better feedback to both trainees and instructors. To achieve effective alignment between the ICAO Doc 10056 standards and the operational practices of the Southwest Air Traffic Management Bureau, and to address inconsistencies in wording and difficulties in evaluation arising from different instructors providing comments at different times and in various contexts, this section draws upon years of original comment records accumulated by the Southwest Air Traffic Management Bureau. By fine-tuning a pre-trained language model, the DeepSeek-v2 architecture is used as the base model. The DeepSeek-v2 summarizes, refines, and categorizes the comments used by instructors in various scenarios, allowing instructors to quickly select the needed comments after training. This ensures uniformity in the comments provided by different instructors, making it easier for both trainees and instructors to understand and quickly assess the training. And the specific construction of the initial air traffic controller core competency model is shown in

Table 2. Entry-level traffic controller core competency model.

Competency	Element	Observable Behavior
C1: Situational Awareness	E1: Monitoring	OB1: Detecting Abnormal Aircraft Conditions through Monitoring
		OB2: Maintaining Awareness of Flight Dynamics
	E2: Operational Information Integration and Analysis	OB3: Mastering All Available Sources of Information
	E3: Risk Identification	OB4: Identifying Potential Threats (e.g., high traffic volume, mountainous terrain, complex procedures)
	E4: Using Tools	OB5: Using Available Tools to Monitor, Understand, and Anticipate Operational Conditions
C2: Traffic and Capacity Management	E5: Using Procedures	OB6: Managing Traffic Using Established Procedures
	E6: Issuing Clearances and Instructions	OB7: Issuing Appropriate Clearances and Instructions
		OB8: Issuing Clearances and Instructions in a Timely Manner
	E7: Managing Risks and Hazards	OB9: Maintaining Continuous Attention Under Varying Levels of Traffic Complexity
	E8: Providing Alert and Intelligence Services	OB10: Providing Air Traffic Activity Information to the Crew in an Appropriate, Accurate, and Timely Manner
C3: Separation and Conflict Resolution	E9: Identifying Conflicts	OB11: Identifying Potential Traffic Conflicts
	E10: Resolving Conflicts	OB12: Issuing Clearances and Instructions to Resolve Conflicts
	E11: Maintaining Separation	OB13: Providing Appropriate Separation
		OB14: Adjusting Control Actions as Necessary to Maintain Separation
		OB15: Issuing Clearances and Instructions to Maintain Safe Separation
C4: Communication	E12: Communication Applications	OB16: Adjusting Communication Techniques (e.g., tone, pitch, repetition) According to the Situation
	E13: Communication Protocols	OB17: Speaking Clearly, Accurately, and Concisely
		OB18: Using Standard Radio Communication Phrases in Accordance with Regulations
	E14: Listening and Repeating	OB19: Verifying the Accuracy of the Repetition and Correcting It When Necessary
	E15: Data Communication	OB20: Verifying the Accuracy of Information Input and Making Necessary Corrections
C5: Coordination	E16: Coordinating Implementation	OB21: Determining the Necessity of Coordination and Timely Coordinating with Personnel from Other Positions and Relevant Parties
	E17: Coordination Phrases	OB22: Using Clear and Concise Language for Verbal Coordination
C6: Management of Non-Routine Situations	E18: Identifying Abnormal Situations	OB23: Identifying Potential Emergencies or Abnormal Situations Based on Available Information
	E19: Handling Abnormal Situations	OB24: Determining the Priority of Actions Based on the Urgency of the Situation and Executing Abnormal Situation Handling According to Procedures
C7: Problem-Solving and Decision-Making	E20: Decision-Making for Solutions	OB25: Considering Existing Rules and Operational Procedures, Formulating Appropriate Control Plans
	E21: Implementation of the Plan	OB26: Clarifying Task Priorities and Organizing Work Tasks Efficiently

.

3. Calculation of the Weight for Each Dimension of the Core Competency Model

The uniqueness and professionalism of the air traffic controller profession make its core competency model a comprehensive analysis problem with multiple indicators and levels. To accurately calculate the weights of each indicator in the model, this study adopts a combination of subjective and objective weighting methods for assigning weights to the indicators.

3.1. Calculation of Subjective Weights

To determine the subjective weights of the three dimensions—“Competency”, “Element”, and “Observable Behavior (OB)”—in the initial air traffic controller core competency model, this study employs the Analytic Hierarchy Process (AHP). AHP is commonly used for multi-objective decision analysis and comprehensive system evaluation. It allows for the modeling and quantification of multi-objective decision problems, representing them through a structured hierarchical framework. AHP then ranks the decision alternatives based on judgment, using pairwise comparisons to determine the relative importance of various factors through comprehensive human judgment.

3.1.1. Construction of the Hierarchical Model

Based on the competency theory framework, a three-level hierarchical structure is established:

$$\left\{\begin{array}{c}\mathrm{Goal}\mathrm{Layer}G:\mathrm{Entry}\text{-}\mathrm{Level}\mathrm{Controller}\mathrm{Core}\mathrm{Competency}\mathrm{Model}\mathrm{Weight}\mathrm{Allocation}\hfill \\ \mathrm{Criterion}\mathrm{Layer}C:C_1\left(\mathrm{Competency}\mathrm{Dimensions}\right),C_2\left(\mathrm{Element}\mathrm{Dimensions}\right)\hfill \\ \mathrm{Solution}\mathrm{Layer}S:\{S_k\mid k=1,2,\dots ,m\},m=26\left(\mathrm{Observable}\mathrm{Behaviors}\right)\hfill \end{array}\right.$$

In which the criteria layer and the alternative layer satisfy the mapping relationship:

$$C_1\to C_2\to S_k$$

3.1.2. Construction of the Group Judgment Matrix

A “Core Competency Indicator Weight Ranking Questionnaire for Entry-Level Air Traffic Controllers” was designed, and five experienced air traffic control experts (with an average of 7.6 years of professional experience) were invited to perform pairwise comparisons of the criteria and alternative layers using Saaty’s 1–9 scale method, constructing a reciprocal matrix.

$$A={\left[{\mathbf{a}}_{pq}\right]}_{m\times m},a_{pq}={\displaystyle \frac{1}{a_{qp}}},a_{pp}=1$$

(2)

The individual judgments of the experts were integrated into a group matrix using the geometric mean method.

$$\overline{A}={\left[{\overline{a}}_{pq}\right]}_{m\times m}$$

(3)

$${\overline{a}}_{pq}={\left(\prod _{i=1}^n{a}_{pq}^e\right)}^{1/n}$$

(4)

Here, $a_{pq}$ represents the comparison value between indicators p and q given by expert, m is the total number of indicators, $a_{pq}^e$ represents the score given by expert e for the comparison between indicators p and q, and n represents the total number of experts.

3.1.3. Consistency Check and Weight Synthesis

First, calculate the maximum eigenvalue ${\lambda }_{max}$:

$${\lambda }_{max}={\displaystyle \frac{1}{n}}\sum _{p=1}^n{\displaystyle \frac{\overline{A}{w}_p}{w_p}}$$

(5)

The consistency index (CI) and consistency ratio (CR) are calculated as follows:

$$CI={\displaystyle \frac{{\lambda }_{max}-m}{m-1}},CR={\displaystyle \frac{CI}{R{I}_{\left(m\right)}}}$$

(6)

When $CR<0.1$, the matrix consistency is accepted. Finally, the geometric mean method is used to calculate the weights. Taking the first-level indicators as an example, the judgment matrices for the first-level indicators from five experts are first aggregated using the geometric mean method to obtain the judgment matrix A, as shown in

Table 3. Competence judgment matrix.

	Situational Awareness	Traffic and Capacity Mgmt	Separation and Conflict Res	Communication	Coordination	Non-Routine Mgmt	Problem-Solving
Situational Awareness	1.00000	2.41307	0.46704	2.67801	3.91376	2.46446	1.24573
Traffic and Capacity Mgmt	0.41441	1.00000	0.20000	1.93707	1.24824	1.08447	0.33333
Separation and Conflict Res	2.14113	5.00000	1.00000	3.33608	4.09238	3.69493	1.93707
Communication	0.37341	0.51624	0.29975	1.00000	1.08447	0.87005	0.41524
Coordination	0.25551	0.80113	0.24436	0.92211	1.00000	0.51728	0.51728
Non-Routine Mgmt	0.40557	0.92211	0.27064	1.14870	1.93318	1.00000	0.80274
Problem-Solving	0.80574	3.00000	0.51624	2.40822	1.93318	1.24573	1.00000

.

To ensure the reliability of the decision-making results, a consistency check of the matrix is required. The first step is to calculate the maximum eigenvalue of the judgment matrix A.

$${\lambda }_{max}={\displaystyle \frac{1}{n}}\sum _{i=1}^n{\displaystyle \frac{{\left(AM\right)}_i}{m_i}}=7.16234$$

(7)

From

Table 4. RI standard value table.

The Order of the Matrix	1	2	3	4	5	6	7	8	9	10	11
RI	0	0	0.58	0.9	1.12	1.24	1.32	1.41	1.45	1.49	1.51

, it can be seen that when n = 7, the consistency index RI = 1.32. Then, the consistency ratio CR is calculated.

$$CI={\displaystyle \frac{{\lambda }_{max}-n}{n-1}}=0.02706$$

(8)

$$CR={\displaystyle \frac{CI}{RI}}=0.02050$$

(9)

Therefore, the consistency test result is $CR<0.1$, indicating that the judgment matrix has satisfactory consistency. Finally, the geometric mean method is used to calculate the weights, resulting in the weight vector for the first-level indicators as follows:

$$W=(w_1,w_2,w_3,w_4,w_5,w_6,w_7)=(0.2017,0.0830,0.3260,0.0703,0.0644,0.0945,0.1601)$$

Similarly, the subjective weights of each element and the subjective weights of each observable behavior are shown in

Table 5. Weights for each dimension.

Comp. 1	Elem. 2	Beh. 3	Subjective Weights			Objective Weights			Combined Weights
			Weights	Elem. Sub	Beh. Sub	Weights	Elem. Sub	Beh. Sub	Weights	Elem. Sub	Beh. Sub
C1	E1	OB1	0.2017	0.2987	0.2934	0.0723	0.2186	0.4954	0.1191	0.2841	0.5
		OB2			0.7066			0.5046			0.5
	E2	OB3		0.2270	1		0.5210	1		0.2806	1
	E3	OB4		0.3237	1		0.1214	1		0.2868	1
	E4	OB5		0.1506	1		0.1390	1		0.1485	1
C2	E5	OB6	0.0830	0.1414	1	0.1782	0.3662	1	0.1438	0.2631	1
	E6	OB7		0.3837	0.6081		0.1672	0.4717		0.2665	0.5
		OB8			0.3919			0.5283			0.5
	E7	OB9		0.2565	1		0.2066	1		0.2295	1
	E8	OB10		0.2184	1		0.2600	1		0.2409	1
C3	E9	OB11	0.3260	0.4337	1	0.0854	0.6560	1	0.1724	0.4385	1
	E10	OB12		0.1715	1		0.2224	1		0.1726	1
	E11	OB13		0.3949	0.3715		0.1216	0.3509		0.3889	0.3554
		OB14			0.2150			0.3553			0.3249
		OB15			0.4135			0.2938			0.3197
C4	E12	OB16	0.0703	0.1426	1	0.1122	0.2457	1	0.0971	0.1901	1
	E13	OB17		0.2928	0.4453		0.3062	0.3596		0.2990	0.5
		OB18			0.5547			0.6404			0.5
	E14	OB19		0.3634	1		0.0972	1		0.2407	1
	E15	OB20		0.2012	1		0.3509	1		0.2702	1
C5	E16	OB21	0.0644	0.6600	1	0.1954	0.7800	1	0.1480	0.5000	1
	E17	OB22		0.3400	1		0.2200	1		0.5000	1
C6	E18	OB23	0.0945	0.6406	1	0.2291	0.3625	1	0.1804	0.5000	1
	E19	OB24		0.3594	1		0.6375	1		0.5000	1
C7	E20	OB25	0.1601	0.5547	1	0.1274	0.5213	1	0.1392	0.5000	1
	E21	OB26		0.4453	1		0.4787	1		0.5000	1

¹ Comp. stands for Competency. ² Elem. stands for Element. ³ Beh. stands for Observable Behavior.

.

3.2. Objective Weight Calculation

To eliminate the influence of subjective preferences introduced by the Analytic Hierarchy Process (AHP), the objective weights of each dimension are determined using information entropy theory. The entropy weight method is an objective weighting approach based on information entropy, which assigns weights by calculating the degree of dispersion of indicator data. The core principle is that the smaller the information entropy of an indicator, the greater its variation, the more information it contains, and the more significant its influence on the overall evaluation—thus warranting a higher weight. The specific steps are as follows:

• Standardize the data to eliminate differences in units of measurement;
• Calculate the information entropy of each indicator using the formula: $e_j=-k{\sum }_{i=1}^n{f}_{ij}ln{f}_{ij}$. where $f_{ij}$ is the proportion of the ith sample under the jth indicator, and k is a normalization constant;
• Compute the weights based on the entropy values, where a lower entropy corresponds to a higher weight.

By identifying key elements with high data variability (such as “operational information integration analysis”), this method provides a data-driven basis for allocating training resources, thereby enhancing the interpretability and practical value of the model. The following section takes the competency dimension as an example.

Data Standardization: An evaluation data matrix $X={\left(x_{ij}\right)}_{n\times 7}$ is constructed by collecting data from n samples under the competency dimension. The range standardization method is applied as follows:

$$p_{ij}={\displaystyle \frac{x_{ij}-min{x}_j}{max{x}_j-min{x}_j}}$$

(10)

Information entropy calculation:

$$e_j=-k\sum _{i=1}^n{f}_{ij}ln{f}_{ij}$$

(11)

Entropy weight determination: The objective weights are calculated based on the information utility value $d_j=1-e_j$:

$$w_j=d_j/\sum _{j=1}^7{d}_j$$

(12)

3.3. Game-Theoretic Combination Weighting

In order to scientifically integrate the role of subjective and objective weighting methods in determining the weights of evaluation indicators and overcome the possible conflicts and limitations between the two, this study adopted a combined weighting model based on game theory. Traditional weight combination methods, such as simple averaging, may not be able to effectively deal with the significant differences between subjective and objective weights, and may even lead to suboptimal or biased final weight distribution due to the lack of a systematic coordination mechanism. Game theory provides an ideal framework, it regards subjective weights and objective weight vectors as participants in the game, and seeks an equilibrium strategy between the two (such as Nash equilibrium) to achieve a combined weight that can simultaneously minimize the deviation from each initial weight. This method ensures that the final weight not only takes into account the subjective considerations of decision makers and the objective information of indicator data, but is also obtained through a systematic process aimed at achieving a balance of “interests” (i.e., weight influence) of all parties, thereby improving the strategic nature of weight distribution and the fairness of the final evaluation, which is difficult to achieve with non-game theory combination methods.

Let the subjective weight vector be denoted as $W_s={(w_{s1},w_{s2},\dots ,w_{sn})}^T$, and the objective weight vector as $W_o={(w_{o1},w_{o2},\dots ,w_{on})}^T$. The combined weight vector is constructed as follows:

$$W=\alpha W_s+\beta W_o$$

(13)

where $\alpha$ and $\beta$ is the combination coefficient. An optimization objective function is established based on the game-theoretic approach:

$$min\parallel W-W_s{\parallel }^2+{\parallel W-W_o\parallel }^2$$

(14)

Substitute Equation (13) into Equation (14), and expand to obtain:

$$min\left[{(\alpha -1)}^2{W}_s^T{W}_s+2\alpha \beta W_s^T{W}_o+{\beta }^2{W}_o^T{W}_o\right]$$

(15)

Construct the Lagrangian function to find the extreme value:

$$L={(\alpha -1)}^2{W}_s^T{W}_s+2\alpha \beta W_s^T{W}_o+{\beta }^2{W}_o^T{W}_o+\lambda (\alpha +\beta -1)$$

(16)

Take the partial derivatives of $\alpha$, $\beta$ and $\lambda$ with respect to each variable, and set them equal to zero, resulting in the system of equations:

$$\left\{\begin{array}{c}2(\alpha -1)W_s^T{W}_s+2\beta W_s^T{W}_o+\lambda =0\hfill \\ 2\alpha W_s^T{W}_o+2\beta W_o^T{W}_o+\lambda =0\hfill \\ \alpha +\beta =1\hfill \end{array}\right.$$

(17)

Solving the system of Equations (17) yields the optimal combination coefficient:

$${\alpha }^{*}={\displaystyle \frac{W_o^T{W}_o-W_s^T{W}_o}{W_s^T{W}_s+W_o^T{W}_o-2W_s^T{W}_o}}$$

(18)

$${\beta }^{*}=1-{\alpha }^{*}$$

(19)

After normalizing the combination coefficient, the final combined weight is as follows:

$$W^{*}={\displaystyle \frac{{\alpha }^{*}}{{\alpha }^{*}+{\beta }^{*}}}{W}_s+{\displaystyle \frac{{\beta }^{*}}{{\alpha }^{*}+{\beta }^{*}}}{W}_o$$

(20)

The combined weight matrix obtained through Equation (20) retains the systematic expression of expert experience provided by the Analytic Hierarchy Process (AHP) while integrating the objective analysis of data characteristics from the entropy weight method, achieving a Pareto improvement in the weight allocation strategy. As shown in Table 5, the combined weight values lie between the subjective and objective weights and satisfy $\sum w_i=1$, validating the convergence and rationality of the model.

4. Model Application

According to ICAO Doc 10056, assessment and training evaluations can be divided into progress evaluations and summative evaluations. Summative evaluation refers to assessments conducted during specific periods of training and/or at the end of the training process. It provides a determination of “competent” or “not competent”, but it can also involve a more detailed scoring system, along with corresponding comments to offer better feedback for both trainees and instructors. This study will focus on summative evaluation as an example, proposing a comprehensive evaluation framework centered on the weight-TOPSIS method. Based on this framework, training will first be provided to each instructor to clarify the definitions and scoring criteria for each dimension, including

• 1—Causing unsafe consequences;
• 2—Failing to manage threats and errors, or actions leading to a reduction in safety margins;
• 3—Behavior meets the requirements and ensures safety;
• 4—Behavior meets the requirements and enhances safety margins;
• 5—Providing high-quality service and effectively improving safety margins.

Following this, in a particular team, an instructor will evaluate each trainee’s observable behaviors as presented in

Table 6. Observable behavior scores of each trainee.

	OB1	OB2	OB3	……	OB25	OB26	Instructors’ Selected Comments from the Comment Set
trainee 1	1	3	3	……	3	1	It is necessary to strengthen basic monitoring and the implementation of separation standards; the capability for emergency response is relatively good.
trainee 2	4	4	5	……	5	5	Core situational awareness is strong; however, systemic risks exist in separation management and multi-position coordination.
……	……	……	……	……	……	……	……
trainee 19	1	3	1	……	5	5	Response to threats in complex environments is outstanding; however, poor adherence to basic procedures increases the likelihood of operational compliance issues.
trainee 20	4	5	3	……	1	1	Separation control techniques are solid; however, there are significant deficiencies in global situational awareness and workflow management.

. By integrating multidimensional data, a comprehensive competency score will be generated for each trainee, leading to a summative assessment. The specific objectives of this application are listed in Table 6.

4.1. Ability Ranking

Based on the standardized scores of the trainees’ observable behaviors (OB), the proximity to the ideal solution is calculated by combining the combined weights. The closer the score is to 1, the more outstanding the trainee is. The ability ranking results for all trainees are presented, as shown in Figure 2.

Figure 2 presents the ability ranking results of trainees based on TOPSIS closeness coefficient scores. The color depth is negatively correlated with the score, visually reflecting the gradient differences in trainee abilities. The closeness coefficient scores of the 20 trainees range from 0.320 to 0.720. Among them, Trainee 17 ranks first with the highest score of 0.720, indicating that their overall ability is closest to the ideal optimal solution. Trainee 10, with the lowest score of 0.320, requires focused attention on their areas of weakness. The TOPSIS ranking results have been cross-verified by multiple experts, receiving unanimous approval and aligning with objective facts, demonstrating that the method can objectively reflect differences in trainee abilities. The red dashed line represents the overall mean score of 0.485. The scores of all trainees exhibit a right-skewed distribution (mean = 0.485, standard deviation = 0.1019), indicating that there are significantly fewer high-scoring trainees compared to those in the low-scoring range. This suggests that the competency of this team needs improvement, and instructors should intensify their training efforts. All scores are accurately displayed through data labels, ensuring the verifiability of the results.

4.2. Personalized Feedback and Training Effectiveness Diagnosis

The competency radar charts are generated based on the OB item scores of each trainee, visually presenting the competency scores and comparative gaps among the trainees. This aids in identifying individual weaknesses. Additionally, OB items with scores below three for a specific trainee are highlighted, providing data support for the design of personalized training plans.

By observing the radar charts of Trainees 1, 2, and 3 in Figure 3, it can be seen that Trainee 1 shows relatively balanced abilities across the six dimensions but has an overall low level, particularly in “Traffic and Capacity Management”, where there is a clear weakness that requires further improvement in overall competency. Trainee 2 excels in “Problem-solving and decision-making ” and “Situational Awareness”, with abilities significantly higher than those of the other trainees, but is weaker in “Traffic and Capacity Management” and needs targeted improvement in this area. Trainee 3 has a relatively average distribution of abilities, with a notable weakness in “Coordination”, requiring focused development in coordination and implementation skills.

As shown in

Table 7. Statistics of low-scoring behaviors of each student (score < 3).

Trainee	Number of Low-Score Behaviors	Specific Low-Score Behaviors (Behavior: Score)	Weakest Behaviors
trainee 17	6	Maintaining Awareness of Flight Dynamics (1); Using Available Tools to Monitor, Understand, and Anticipate Operational Conditions (1)……	Maintaining Awareness of Flight Dynamics (1 min); Using Available Tools to Monitor, Understand, and Anticipate Operational Conditions (1)……
trainee 19	11	Detecting Abnormal Aircraft Conditions through Monitoring (1); Providing Air Traffic Activity Information to the Crew in an Appropriate, Accurate, and Timely Manner (2)……	Detecting Abnormal Aircraft Conditions through Monitoring (1); Adjusting Control Actions as Necessary to Maintain Separation (1)……
trainee 2	8	Using Available Tools to Monitor, Understand, and Anticipate Operational Conditions (1); Managing Traffic Using Established Procedures (2)……	Using Available Tools to Monitor, Understand, and Anticipate Operational Conditions (1); Providing Air Traffic Activity Information to the Crew in an Appropriate, Accurate, and Timely Manner (1)…
	…	…	…

and Figure 4, through threshold screening (score < 3), the low-score OB items of each trainee are highlighted, which helps both the trainees and instructors focus on training specific low-score OB items. This approach enables trainees to quickly identify their areas of weakness and more effectively assist in their growth. For example, Trainee 17 can focus on OB items such as mastering flight dynamics and using available tools to monitor, understand, and anticipate operational conditions to improve more rapidly. Among all the low-score OB items, “Providing appropriate intervals” (13 times) and “Using available tools to monitor, understand, and anticipate operational conditions” (12 times) are the main weak points. For these high-frequency low-score behaviors, it is recommended to incorporate tools such as priority matrices into the training program.

4.3. Group Difference Analysis and Quality Control

The box plot statistical method is used to depict the score distribution characteristics of all trainees across each OB item. The interquartile range (IQR) is used to define outliers, outputting the top performers and outliers for each specific OB item. This reveals the dispersion trends of the group’s ability structure and potential risk points, providing a reference for optimizing air traffic control training courses. The output results of the trainees’ training evaluation are shown in Figure 5.

Figure 5 shows the box plot distribution of scores for each observable behavior. Overall, the median scores of all behaviors vary, indicating that the competency levels of trainees in this initial controller group are currently inconsistent. Among them, OB6 (using prescribed procedures to manage traffic) has a median score of 2.5 and an IQR of 1.75, showing the most optimal central tendency. In contrast, OB13 (providing appropriate intervals) has a median score of 2 and an IQR of 4, displaying the greatest individual variation. The Q1 is 1, and the median is 2, indicating that half of the group’s scores on this OB are below 2, with one-quarter of trainees scoring between 1 and 2. This suggests that the instructors have been insufficient in training the group on “Providing appropriate intervals”, and subsequent course training needs to be optimized to improve the group’s performance on OB13. No high or low outliers were detected in any of the OB items, suggesting that the group currently does not have any standout or underperforming trainees.

5. Conclusions

This study, based on the core framework of the ICAO Doc 10056, systematically developed an initial air traffic controller core competency model tailored to the operational characteristics of the Southwest Air Traffic Management Bureau of China. The research findings are summarized as follows:

(1)

At the model construction level, through a combination of literature analysis and Delphi expert interview methods, an indicator system was established that includes seven core competency dimensions such as “Situational Awareness”, “Traffic and Capacity Management”, “Separation and Conflict Resolution”, and their 21 elements and 26 observable behaviors. It not only complies with the international standards of the ICAO capability framework, but also integrates the operating environment and control culture characteristics of my country’s Southwest Air Traffic Control Bureau, achieving an effective integration of international norms and localized practices.

(2)

In the weight calculation stage, the AHP hierarchical analysis method and entropy weight method are used to perform subjective and objective weighting, breaking through the limitations of traditional subjective assignment. Then, game theory is used to calculate the combined weight of subjective and objective weights, accurately quantifying the priority relationship of each competency element. This study found that the weights of the seven competencies are not much different, and the weights of the elements “identifying conflicts”, “coordinating implementation”, and “scheme decision-making” are relatively large, which are considered to be important elements for initial air traffic controllers. This finding provides data support for the allocation of air traffic control training resources.

(3)

At the model application level, instructors score students’ performance on observable behaviors, and through the weighted-TOPSIS method, we can calculate the ability ranking of each student, which is convenient for the Air Traffic Control Bureau to intuitively understand the students’ abilities. At the same time, we also calculate the radar chart of each student’s competency dimension, and the box plot of the OB item is convenient for the Air Traffic Control Bureau to conduct further detailed analysis.

Future research can continue to explore two areas:

(1)

Continue to construct core competency models for other positions in conjunction with ICAO Doc 10056. The air traffic controller roles include Tower, Area, and Approach control, with further specifics such as Tower Clearance Position, Tower Ground Position, etc. The work and responsibilities of these different positions vary, leading to different competency requirements for controllers. Therefore, to provide more targeted training for specific controller positions, it is necessary to explore core competency models for other roles.

(2)

Building upon the Air Traffic Controller (ATC) core competency model established in this study, a significant future application involves exploring and establishing objective performance assessment standards. Specifically, we propose that future research could focus on defining two key performance benchmarks: a “Qualification Threshold” and a “Promotion Threshold”.

Considering the career progression path of ATCs, which includes stages from trainee to entry-level controllers and potentially to instructor, future research, grounded in the current competency model, could employ methods such as historical data quantile analysis or standard-setting procedures like the Angoff method. These methods can be used to exploratorily determine appropriate thresholds for different career stages. The potential utility of these thresholds lies in the following:

• Qualification Threshold: When an ATC’s (especially a trainee is or an individual undergoing recurrent assessment) comprehensive evaluation meets this threshold, it could serve as a preliminary indication that they possess the fundamental core competencies required for the role. Alternatively, in recurrent assessments, it could confirm they maintain the necessary skill proficiency to safely perform or continue their duties.
• Promotion Threshold: Reaching this threshold might provide a preliminary basis for an ATC’s eligibility to apply for promotion (e.g., from entry-level controller to clearance controllers, or to higher-level positions/instructor roles) or to participate in advanced professional competency assessments.

This study, as an active exploration of the localization of the ICAO competency framework, provides theoretical foundations and methodological support for the development of air traffic control talent. It must be pointed out that the findings of this study are primarily based on the specific knowledge, skills, and attitudes of the China Southwest Air Traffic Management Bureau, thus bearing the characteristics of a case study. This means that its specific dimensions, elements, and weights of observable behaviors should be treated with caution when generalized to other organizations with different backgrounds, and cannot simply be a case of “Learn Once Apply Anywhere”. However, the significant contribution of this research lies in the “methodology developed and applied to define competency dimensions, elements, and the weights of observable behaviors. This methodology offers other air traffic control organizations, and even related industries, a structured and operational framework that they can adapt and apply according to their own specific circumstances to build competency systems tailored to their needs. In the future, efforts will focus on combining the methodological experience and specific findings refined in this study to cautiously promote their transformation into industry standards or guidelines, and continuously improving the air traffic controller competency development system to meet the high-quality development needs of China’s civil aviation sector.