Artificial Intelligence Technology for Assessing the Practical Knowledge of Air Traffic Controller Students Based on Their Responses in Multitasking Situations

Abstract

The main goal of the research is to develop an artificial intelligence technology to assess the practical knowledge of air traffic controller (ATCo) students based on their responses in simulated multitasking situations using the proposed neuro-fuzzy model verified in experiments. An informational neuro-fuzzy model was developed and verified on 157,500 real data points. It illustrates an example of inferring the level of practical knowledge in selected ATCo students who were tested using a device measuring the reaction time and relative error rate in multiple-task tasks. The average error in the incorrect response was 7.7% of the experimental data. Data processing was performed using fuzzy set theory and intellectual knowledge analysis. These measurement results are useful for an individual approach to the student’s education to understand and master the correct solutions to achieve the desired educational results. Ensuring a personal approach to the student’s education is key to acquiring the necessary skills, knowledge, and competencies in the profile of the graduate. The developed technology will enable the integration of automated knowledge and skills assessment systems into the real educational process and the identification of problematic topics and tasks in the training of individuals. The result of the conducted research was used for the software design for the practical application in the flight training of ATCo students.

1. Introduction

The above information outlines the relevance of developing and implementing modern intelligent technologies for training ATCos. Modern technologies based on the automatic assessment of the ability of ATCos to respond to simultaneous tasks should analyze the speed and accuracy of their decisions as well as the effectiveness of managing aviation systems in stressful conditions. To assess the level of practical knowledge of ATCos based on their reactions in multitasking situations, it is advisable to use the tools of fuzzy set theory and neural network machine learning technologies to analyze their behavior in various situations. This will make it possible to objectively assess the level of ATCo training and improve the training and the training process, providing a personalized approach to training and correcting identified deficiencies. Adequate assessment will increase flight safety and optimize the time and resources for ATCo training.

The main goal of the research is to develop artificial intelligence technology for assessing the practical knowledge of air traffic controller students based on their reactions and answers in multitasking situations using the proposed neuro-fuzzy model verified within the experiments.

The following research questions are formulated from the above.

• Will the proposed artificial intelligence technology make it possible to identify problematic topics or simulated practical tasks to increase the quality of individual support for students in acquiring the necessary skills, knowledge, and competencies?
• Will the developed artificial intelligence technology make it possible to automate the process of objective assessment of future air traffic controllers during training?

These main results are useful for an individual approach to the students for understanding and mastering the correct solutions to achieve the desired educational results. The developed technology will enable the integration of automated knowledge and skills assessment systems into the real educational process and the identification of problematic topics and tasks in the training of individuals.

The rest of this article is organized as follows. Section 2 provides an overview of the current research on the issue under consideration. Section 3 describes the formal formulation of the problem and the artificial intelligence technology used to assess ATCos’ level of practical knowledge. In Section 4, the neuro-fuzzy model is verified and adjusted. An example of the evaluation of accurate data is given. Section 5 discusses the results of the conducted research, the advantages of the presented technology, and the research limitations. The first scientific results and ideas for future research are presented in Section 6, which concludes the study.

2. Overview of Selected Domestic and Foreign Research Studies

In recent years, we have observed the development of automation and artificial intelligence in control processes in civil aviation [1]. Within the air traffic management process, it is essential to monitor the ability of team members to anticipate and identify errors and adapt to unforeseen events [2]. With the increasingly severe problem of airspace congestion, the aviation industry faces enormous challenges, including a sharp increase in pressure on air traffic controllers, a significant increase in flight delays, and frequent conflicts [3]. Effective conflict detection and resolution technology is paramount to ensure flight safety, which is crucial for the complex, high-density airspace environment [4]. Research in this area is also relevant to preventing aircraft collisions, relieving pressure on air traffic controllers, and predicting the cognitive load on air traffic controllers [5]. People working in modern work systems, especially air traffic control, are increasingly required to oversee the automation of tasks [6]. Enhancing the generalizability and reliability of simulation models in air traffic control is challenging [7].

When predicting the workload of air traffic controllers, the tracking of reaction time in multitasking tasks can be considered and is the main monitored parameter [8]. Various methods can be used to measure reaction time. One of the methods is eye-tracking using a video camera, where the result is a comparison of the time difference in response to visual, acoustic, and audiovisual stimuli [9]. Previous research has found that air traffic controllers’ reactions to visual stimuli are more accurate but slower than their corresponding performance in reaction to auditory stimuli [10]. In our previous research, a device was developed to measure the reaction time of air traffic controllers that monitored their reaction speed in the handling of and response to visual and auditory cues. We evaluated the results based on fuzzy modeling, neural networks, and AI involvement [11]. Air traffic controllers are on duty 24 h daily based on multiple shifts [12]. They work between day and night and have different schedules for day and night shifts, which can affect biorhythms and sleep quality [13]. Increased fatigue decreases the ability to process information [14].

The problem of air traffic control automation is a significant problem that is constantly being investigated because it offers a concrete solution in the face of the exponential growth of air traffic, an increase that is estimated for the next 30 years to be an urgent problem that requires an early resolution [15]. Our research [16] demonstrates the use of AI in qualitative and quantitative research data analysis using neuro-fuzzy modeling. The research focused on optimizing personalized training methods, assessment, and feedback systems to improve the comprehensiveness and accuracy of aviation specialist training evaluations [17]. This study examines how advanced data analytics and artificial intelligence (AI) can work together to improve air traffic controllers’ decision-making processes [18]. As we navigate today’s data-driven environment, discovering synergies between these fields is critical, given the increasing complexity of data sets.

Personalized learning is based on individual interests, learning styles, and self-pacing. It provides focused and self-directed learning. The education system is advanced in various aspects. Hence, personalized learning with the help of AI provides adaptive learning [19]. A customized approach to the learning process using AI and fuzzy modeling helps learners succeed in training based on feedback analysis and the prediction of their progress [20]. AI brings unique capabilities to address complex challenges in training aviation personnel, emphasizing increased efficiency, task automation, enhanced quality and consistency of measured data, advanced analytics, and performance report generation [21]. Artificial intelligence (AI) has the potential to revolutionize the way we learn and teach, making it more personalized, engaging, and efficient [22]. Artificial intelligence in education uses technologies such as machine learning and natural language processing to enhance the learning experience [23]. Research involves using algorithms that analyze data, identify patterns, and make predictions, allowing instructors to customize learning for each student [24]. With the advent of AI, the scope of data analysis, performance prediction, and real-time feedback has expanded [25]. AI-assisted fuzzy modeling has revolutionized aviation personnel training practices. It offers researchers, including students at various academic levels, efficient and robust methods for collecting data, screening, and analyzing their results during practical training [26].

The primary contribution of this study lies in the development and application of AI technologies to assess and enhance the training of aviation personnel, particularly student pilots and air traffic controllers, under multitasking conditions. The study highlights significant limitations of previous research [4,16], notably the challenges in ensuring the reliability and generalizability of models simulating the work of air traffic controllers [1,3]. While earlier studies employed various methods to measure reaction times and evaluate controller workload, they lacked the flexibility and precision required for multitasking scenarios and personalized learning.

This research advances these approaches by incorporating neuro-fuzzy modeling and AI to develop adaptive learning methods [17] as well as tools for assessment and feedback. This integration significantly improves the accuracy and efficiency of the learning process [22,23]. The use of personalized learning strategies [17,19], which consider the individual characteristics of students [20], enhances their ability to make quick, effective decisions in high-pressure multitasking situations [21]. As a result, the training of aviation specialists is substantially improved, leading to higher operational efficiency in critical tasks, a reduction in human error, and a mitigation of risk factors in aviation.

3. Material and Methods

3.1. Formal Formulation of the Evaluation Problem

Students’ ATCo $P=\{p_1;p_2;\dots ;p_n\}$ is tested using a device to measure reaction time and relative error rate in multitasking tasks. The developed measuring device is designed to assess the efficiency of student pilots and air traffic controllers during multitasking tasks. It consists of hardware and software components that together provide data collection and analysis of students’ reactions to various stimuli during training sessions. The device allows measuring both qualitative and quantitative performance indicators, including reaction time to visual and acoustic signals as well as the number of errors during task performance. Specialized sensors and software are used to collect and process data. Measuring reaction time in multitasking conditions allows assessment of the speed and accuracy of students’ task performance, which is an important aspect of their training. Such a device was proposed by the authors and a useful model was obtained for it [27]. This device generates 50 instructions $I=\left\{I_1;I_2;\dots ;I_{50}\right\}$ from 6 problems $Z=\left\{Z_1;Z_2;\dots ;Z_6\right\}$ within one test. As a result, the student ATCo must answer 300 tasks focused on visual and acoustic perception. Tasks 1–3 focus on visual stimuli to which the subject responds in the manipulation zone. Their purpose is to assess the predisposition to the profession and to determine the speed and accuracy of reactions to visual stimuli under a standard load. Additionally, the ability to respond to visual stimuli with different wavelengths is assessed, which indicates the correction of color vision. Tasks 4–5 also focus on visual stimuli, to which the subject responds in the population zone. The main purpose is to determine the predisposition to the profession and to assess the speed and accuracy of reactions to visual stimuli under a standard load. Additionally, the ability to use both hemispheres of the brain is studied. Task 6 focuses on acoustic stimuli in the manipulation zone. The purpose is to assess the predisposition to the profession and to determine the speed and accuracy of reactions to acoustic stimuli. The ability to perform several types of tasks simultaneously is also tested. A detailed description of the functions is given in the work [16].

The construction of artificial intelligence technology to assess the level of practical knowledge of ATCo students is possible only with enough test data. Such data are needed for training neural networks, which make it possible to detect complex and non-linear dependencies in the data. To conduct the research, it is necessary to have data on the level of practical knowledge of student ATCo Ps, who are called the research respondents. This level is expressed in a hundred-point scale and is denoted by $D$.

The level of practical knowledge $D$ from the teacher is determined taking into account two aspects:

• Evaluation of reaction time and accuracy of responses: Quantification of the level of practical knowledge of ATCo student pilots is carried out by assessing their average reaction times to visual and acoustic signals as well as the number of errors in performing tasks. Reaction time and error rate in multitasking conditions show the effectiveness of task performance and the level of preparation for work in real situations.
• Multitasking and adaptability: This assesses students’ ability to perform multiple tasks simultaneously, including measuring the number of correct answers when multitasking and the level of errors under stress or changing circumstances. Also determined is the stability of task performance and the ability to adapt to change, which is important for assessing students’ practical knowledge in real-world work environments.

Also, the number of mistakes made ($e)$ and the reaction time (t) to perform the assigned task $Z=Z(e,t)$ are considered for each ATCo student. Testing for each student is carried out several times, denoted by $E$, and determines the repetition of the exercise within the testing. An information model for processing the data of ATCo student reactions in multitasking situations is proposed: $I_{FI}$ to process the input data. Based on the processed input data, a neuro-fuzzy model for assessing the level of practical knowledge of ATCo students, $M_{NT}$, is built.

Neural fuzzy models combine the advantages of neural networks and fuzzy logic, allowing you to work with inaccurate, incomplete, or contradictory data, which is important for assessing students’ cognitive and emotional reactions. This allows the model to adapt to different levels of task complexity and take into account the individual characteristics of students, which increases the accuracy and reliability of the results.

Definition 1.

Artificial intelligence technology is a system that uses machine learning algorithms and neural networks to analyze students’ reactions to multitasking situations, assessing their speed, accuracy, and efficiency. It allows you to automatically assess the level of practical knowledge of students based on their reactions to visual and acoustic stimuli as well as their ability to perform several tasks simultaneously.

The artificial intelligence technology for assessing the level of practical knowledge of ATCo students based on their reactions in multitasking situations is presented in the form of a theoretical multiple model system, as follows:

$$\{P,{D, Z, E, I}_{FI},M_{NT}|Y\}$$

(1)

Based on the input data, a technology is built that enables ATCo students, $\left\{p_{n+1};p_{n+2};\dots ;p_m\right\}$, after passing their testing on the device, to receive an initial quantitative assessment of their level of practical knowledge, Y.

For the development of artificial intelligence technology, the following subjects are defined: the respondents are ATCo students who will be trained in a neuro-fuzzy model based on the number of mistakes made, reaction time to perform the task, and assessment of the level of practical knowledge; a system analyst is a person who configures all the management processes of the artificial intelligence technology; the decision-making person (DM) is the instructor of the student ATCo, who is responsible for the level of training and, based on the initial evaluations, makes a decision to approve the received assessment of the level of practical knowledge of the ATCo based on their reactions in multitasking situations.

An illustration of the processes of artificial intelligence technology for assessing the level of practical knowledge of ATCo students is shown in Figure 1.

Figure 1 shows the structural scheme of the artificial intelligence technology for assessing the level of practical knowledge of ATCo students based on their reactions in multitasking situations. The input data are divided into two types: test input data, which is the input data of the student ATCo respondents ($p_1;p_2;\dots ;p_n$), and input data of the evaluated student pilots ($p_{n+1};p_{n+2};\dots ;p_m$). There are two blocks at the input: “Input Test Data” and “Input Data”. Student ATCos are tested using a device to measure the reaction time and relative error rate by generating 50 instructions ($I$) from 6 tasks ($Z$). This testing is repeated 15 times ($E)$. Also, the level of practical knowledge ($D)$ is considered for student ATCo respondents. The “Measuring Device” block is responsible for this. After that, the data on the reactions of the ATCo students in multitasking situations are processed using the information model—$I_{FI}$. The “Information Model—$I_{FI}$” block is responsible for this. The results of the data obtained from the pilot student respondents form the “database” block of the study. After that, the data are entered into the neuro-fuzzy model to assess students’ level of practical knowledge at ATCo—$M_{NT}$ (block “Neuro-fuzzy model—$M_{NT}$”). Based on the data from student ATCo respondents, the neural network is trained to determine the synaptic weights. The knowledge gained forms the “knowledge base” block of the study. The acquired knowledge forms the research knowledge base. Based on the input data, a student ATCo’s practical knowledge is evaluated based on their reactions in multitasking situations using the calculation of the neuro-fuzzy model $M_{NT}$. Next, the assessment of the level of practical knowledge is derived using the knowledge base of the study. After that, the final evaluation of the DM is determined. After that, the result goes to the block “Determination of the final assessment of the DM”.

3.2. An Information Model for Processing the Data of ATCo Student Reactions in Multitasking Situations—$I_{FI}$

Student ATCo $\mathrm{P}$ is tested using a device to measure reaction time and relative error rate in multitasking tasks. The number of mistakes made and the reaction time to perform the tasks, $Z_1(e_1,t_1)$, $Z_2(e_2,t_2)$… $Z_6(e_6,t_6)$, are considered for each task. As a result, a data set is obtained for each student $p_i$, $i=\overline{1,n}$:

$$\left\{\begin{array}{c}\begin{array}{c}\begin{array}{l}{I}_{i1}\left(Z_{i11}\left(e_{i11},t_{i11}\right), Z_{i12}\left(e_{i12},t_{i12}\right),\dots ,Z_{i16}\left(e_{i16},t_{i16}\right)\right),\\ I_{i2}\left(Z_{i21}\left(e_{i21},t_{i21}\right), Z_{i22}\left(e_{i22},t_{i22}\right),\dots ,Z_{i26}\left(e_{i26},t_{i26}\right)\right),\\ \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots ..\\ I_{ij}\left(Z_{ij1}\left(e_{ij1},t_{ij1}\right), Z_{ij2}\left(e_{ij2},t_{ij2}\right),\dots ,Z_{ij6}\left(e_{ij6},t_{ij6}\right)\right),\\ \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots ..\dots \dots \dots .\\ I_{i50}\left(Z_{i501}\left(e_{i501},t_{i501}\right), Z_{i502}\left(e_{i502},t_{i502}\right),\dots ,Z_{i506}\left(e_{i506},t_{i506}\right)\right).\end{array} \end{array} \end{array}\right.$$

(2)

In general, the set of data received from the device for measuring reaction time and relative error rate in multitasking tasks will look like $I_{ij}\left(Z_{ijk}\left(e_{ijk},t_{ijk}\right)\right)$, where $i$ is the student number $i=\overline{1,n}$; $j$ is the instruction number $j=\overline{1,50}$; and $k$ is the number of the given problem $k=\overline{1,6}.$ The information model for processing student ATCo reaction data in multitasking situations is presented as a step-by-step algorithm.

First step. In the first step, the error data and reaction time for the performance of the given task are fuzzified.

Since the obtained data take different values and have different semantic contents, it is suggested that the theory of fuzzy sets and one-dimensional membership functions be used to compare them. The greater the number of mistakes made in the task, the worse the level of skill acquisition, which indicates the need for additional training or retraining. Similarly, the longer the task time, the worse the response, which may indicate uncertainty in decision making. It may also indicate a lack of experience in dealing with similar situations. Long reaction times can be critical in stressful and multitasking environments, requiring additional training to improve decision-making speed and accuracy in real-world flight conditions. Given this, in both cases, the “small amount” type is uncertain. Membership functions of this type are described by Z-like membership functions: quadratic Z-spline, harmonic Z-spline, Z-sigmoid, and Z-linear. To construct the membership function, it is necessary to determine the parameters of the lower and upper limits. Membership functions are constructed based on accurate data from the students’ training [28]. For example, a quadratic Z-spline given by the following analytical formula is proposed for the number of errors:

$${\mu }_1(e_{ijk})=\left\{\begin{array}{cc}1,& e_{ijk}\le 0;\\ 1-{\displaystyle \frac{1}{450}}{\left(e_{ijk}\right)}^2,& 1<e_{ijk}\le 15;\\ {\displaystyle \frac{1}{450}}{\left(29-e_{ijk}\right)}^2,& 15<e_{ijk}<30,\\ 0,& e_{ijk}\ge 30.\end{array}\right.$$

(3)

${\mu }_1\left(e_{ijk}\right)\in [0;1]$, and the larger the value, the fewer errors the student makes. The quadratic Z-spline will have the form for the reaction time to perform the assigned task:

$${\mu }_2(t_{ijk})=\left\{\begin{array}{cc}1,& t_{ijk}\le 0;\\ 1-2{\left({\displaystyle \frac{t_{ijk}}{3}}\right)}^2,& 0<t_{ijk}\le 1,5;\\ 2{\left({\displaystyle \frac{{3-t}_{ijk}}{3}}\right)}^2,& \mathrm{1,5}<t_{ijk}<3,\\ 0,& t_{ijk}\ge 3.\end{array}\right.$$

(4)

${\mu }_2\left(e_{ijk}\right)\in [0;1]$, and the value goes to 1 when the reaction time is the minimum possible.

Thus, the obtained values ${\mu }_1\left(e_{ijk}\right)$, ${\mu }_2\left(t_{ijk}\right)$ are normalized and comparative, $i=\overline{1,n}$; $j=\overline{1,50}$; $k=\overline{1,6}.$

Second step. In the second step, the aggregated level of performance of the assigned task is obtained.

Aggregation of values ${\mu }_1\left(e_{ijk}\right)$, ${\mu }_2\left(t_{ijk}\right)$ is carried out within the limits of $Z_k$ problems by modeling the uncertainty of the “average value” form based on multidimensional membership functions. For example, such modeling can be naturally performed using a cone-shaped membership function. Putting the value of the center of the base of the cone as a unit vector, the scaling according to the coordinates of the vector $\left({\mu }_1\left(e_{ijk}\right);{\mu }_2\left(t_{ijk}\right)\right),$ will be $\left(2;2\right)$:

$${\lambda }_{ijk}=\left\{\begin{array}{c}1-g_{ijk}, \mathrm{if} g_{ijk}<1,\\ 0, \mathrm{otherwise}.\end{array}\right.$$

(5)

where $g_{ijk}={\displaystyle \frac{1}{2}}·\sqrt{{\left({\mu }_1\left(e_{ijk}\right)-1\right)}^2+{\left({\mu }_2\left(t_{ijk}\right)-1\right)}^2}$. $i=\overline{1,n}$; $j=\overline{1,50}$; $k=\overline{1,6}.$

The obtained values of ${\lambda }_{ijk}\in [0;1]$ characterize the aggregated level of performance of the assigned task by the student ATCo regarding the mistakes made and the reaction time to perform the assigned task. When the value goes to 1, it means the best result, accompanied by a quick reaction and error-free task execution.

Third step. In the third step, a generalized assessment of the performance of the assigned task is obtained within 50 instructions.

The transition to the generalized evaluation in terms of the set tasks $Z_k$ is calculated using the following formula:

$${∆}_{ik}={\displaystyle \frac{1}{50}}\sum _{j=1}^{50}{\lambda }_{ijk}$$

(6)

where ${∆}_{ik}$ is one normalized assessment of the performance of the given task $Z_k,k=\overline{1,6}$ within 50 instructions in a section by student $p_i$, $i=\overline{1,n}$. The obtained normalized scores ${∆}_{ik}\in \left[0;1\right]$ determine the level of performance of the given task.

Fourth step. This step considers the repetition of the performance exercise within the limits of testing.

It is estimated that each student ATCo must perform the test 15 times to assess their level of practical knowledge of reactions in multitasking situations. Of course, when the number of repetitions of the exercises within the testing framework is increased, the result in terms of reaction time and the number of errors improves. This needs to be considered, as the improvement in performance with time and the number of repetitions is a consequence of the learning effect. When a student ATCo repeats the exercise several times, they remember the sequence of actions better and begin to react faster and more accurately. Such a factor can give a false impression of the level of their basic skills since the improvement in results is due to habituation to a specific task and does not always reflect the ATCo’s real ability to adapt to new or unexpected situations. Such exercises are denoted by $E=\{E_1;E_2;\dots ;E_{15}\}$ and characterize the number of repetitions. Normalized values are required to consider the repeatability of exercises and compare data. From the above logic, it is proposed to use the Z-linear membership function:

$${\delta }_i(E_h)=\left\{\begin{array}{ccc}1,& 0<E_h\le 1;& \\ {\displaystyle \frac{15-E_h}{14}},& 1<E_h\le 15;& i=\overline{1,n}; h=\overline{1,15.}\\ 0,& E_h>15.& \end{array}\right.$$

(7)

The obtained values ${\delta }_i\left(E_h\right)\in [0;1]$ are called the coefficients of repetition of the exercise within the testing limits by the student ATCo. As the number of repetitions of the exercise increases, the coefficient will approach zero.

The instructor also evaluates the student ATCo and gives an assessment of the level of practical knowledge on a one-hundred-point scale $D=\{D_1;D_2;\dots ;D_n\}$.

Thus, based on the information model of data processing of student ATCo reactions in multitasking situations, a set of input data was obtained separately for exercises $E_h$ ($h=\overline{1,15}$),

Table 1. Input data of reactions of student ATCos in multitasking situations.

$\mathit{P}$	${\mathit{Z}}_1$	${\mathit{Z}}_2$	…	${\mathit{Z}}_6$	${\mathit{E}}_{\mathit{h}}$	$\mathit{D}$
$p_1$	${∆}_{11}$	${∆}_{12}$	$\dots$	${∆}_{16}$	${\delta }_1\left(E_h\right)$	$D_1$
$p_2$	${∆}_{21}$	${∆}_{22}$	$\dots$	${∆}_{26}$	${\delta }_2\left(E_h\right)$	$D_2$
$\dots$	$\dots$	$\dots$	$\dots$	$\dots$	$\dots$	$\dots$
$p_i$	${∆}_{i1}$	${∆}_{i2}$	$\dots$	${∆}_{i6}$	${\delta }_i\left(E_h\right)$	$D_i$
$\dots$	$\dots$	$\dots$	$\dots$	$\dots$	$\dots$	$\dots$
$p_n$	${∆}_{n1}$	${∆}_{n2}$	$\dots$	${∆}_{n6}$	${\delta }_n\left(E_h\right)$	$D_n$

.

Where ${∆}_{ik}$ is the normalized assessment of the performance of the assigned task $Z_k,k=\overline{1,6}$ by the student ATCo $p_i$; ${\delta }_i\left(E_h\right)$ is the coefficient of repetition of the exercise within the testing limits; and $D_i$ is the evaluation of the level of practical knowledge, $i=\overline{1,n}$.

3.3. A Neuro-Fuzzy Model for Assessing the Level of Practical Knowledge of Student ATCos, $M_{NT}$

Let the input of the neuro-fuzzy model be supplied with normalized assessments of the performance of the given task by the student (${∆}_{ik}$), the evaluation of the level of practical knowledge ($D_i$), and the coefficient of repetition of the exercise within the limits of testing (${\delta }_i\left(E_h\right)$). Then, the object of the form $Y=D\left(∆,\delta \right),$ д is considered, for which the relationship “inputs $(∆,\delta )$—output $Y$” is presented as production rules of a fuzzy knowledge base. Then, you need to build a base of vague knowledge, which consists of systems of logical statements: “If–Then, Else”. They associate the values of the input data $\left({∆}_1,{∆}_2,\dots ,{∆}_6,\delta \right)$ with one possible assessment of the level of practical knowledge $D$:

IF $Z_1={∆}_{i1}$ (with weight ${\alpha }_1$) and $Z_2={∆}_{i2}$ (with weight ${\alpha }_2$) … and $Z_6={∆}_{i6}$ (with weight ${\alpha }_6$) and $E={\delta }_i\left(E\right)$(with weight ${\alpha }_7$) THEN $Y=D_i$ ELSE …

$i$ is a student ATCo based on which the level of practical knowledge is determined, $i=\overline{1,m}$. ${\alpha }_1,{\alpha }_2,\dots ,{\alpha }_7$ are the synaptic weights of the neuro-fuzzy model. Synaptic weights are obtained in the process of learning a neuro-fuzzy network.

The derivation of levels of practical knowledge can be illustrated in the form of a four-layer neuro-fuzzy network (Figure 2). Where the following notations are used, the input is data on student $P$, for which an assessment is made on 6 tasks $\left\{Z_1;Z_2;\dots ;Z_6\right\}$ in exercises, which characterizes the number of repetitions $E$. Next is the calculation in terms of four layers: First layer: {${∆}_1, {∆}_2, \dots , {∆}_6\}$ is normalized assessments of the performance of the corresponding tasks $\left\{Z_1;Z_2;\dots ;Z_6\right\};$ $\delta \left(E\right)$ is the repetition coefficient of the exercise within the limits of data fuzzification testing. Second layer: $W_1,W_2,\dots ,W_7$ is the value of the excitation level of neurons that form input signals with synaptic weights. Third layer: $Q_1, Q_2,\dots , Q_7$ is the postsynaptic potential of neurons of the third layer. Fourth layer: $Y$ is the level of practical knowledge of the student.

Below are the formalized steps performed at each layer of the neuro-fuzzy model.

First layer: Fuzzification of input data

In the neurons of the first layer, a data fuzzification operation is performed to obtain normalized and comparative data. Fuzzification is carried out with the help of the developed information model of data processing of student ATCo reactions in multitasking situations, $I_{FI}$. The formalization of error data and the reaction time for the performance of the given task is carried out according to Formulas (3) and (4), after which the data are aggregated according to Formula (5), and a generalized assessment of the performance of the given task is obtained within 50 instructions, according to Formula (6). Further, Formula (7) considers the performance exercise’s repetition within the testing limits.

Second layer: Combining values of activation conditions

At the second layer, the functions of the postsynaptic potential are aggregated according to the tasks set by the student ATCo and the repetition of the performance exercise within the testing framework. The second layer contains seven neurons, and the synaptic weights ${\alpha }_1,{\alpha }_2,\dots ,{\alpha }_7$ and are obtained in the process of learning the neuro-fuzzy model. For this, the authors suggest, but are not limited to, the method of backpropagation of the error. It is said that input signals with synaptic weights form the values of the neurons’ excitation level $W_1,W_2,\dots ,W_7$. In this case, the functions of the postsynaptic potential of neurons of the second layer are determined by the following formula:

$$\left\{\begin{array}{c}{W}_1\left(p_i\right)={\alpha }_1\cdot {∆}_{i1};\\ W_2\left(p_i\right)={\alpha }_2\cdot {∆}_{i2} ;\\ \dots \dots \dots \dots \dots \dots \dots \dots .;\\ W_6\left(p_i\right)={\alpha }_6\cdot {∆}_{i6};\\ W_7\left(p_i\right)={\alpha }_7\cdot {\delta }_i\left(E\right).\end{array}\right.$$

(8)

As a result, the output neurons of the second layer $W_1$, $W_2$, …, $W_7$ are obtained, and this process is repeated separately for exercises $E_h$ ($h=\overline{1,15}$).

Third layer: Adjustment of neurons of the second layer

On the third layer, the neurons of the second layer are adjusted to the importance of a given task and the repetition of the exercise. For this, the DM enters the synaptic weights ${\beta }_1,{\beta }_2,\dots ,{\beta }_7$, respectively, from some interval $[1;10]$. Calculation of the postsynaptic potential of neurons of the third layer is carried out as follows:

$$\left\{\begin{array}{c}{Q}_1\left(p_i\right)={\displaystyle \frac{{\beta }_1}{{\beta }_1+{\beta }_2+\dots +{\beta }_7}}\cdot W_1\left(p_i\right);\\ Q_2\left(p_i\right)={\displaystyle \frac{{\beta }_1}{{\beta }_1+{\beta }_2+\dots +{\beta }_7}}\cdot W_2\left(p_i\right);\\ \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots ;\\ Q_7\left(p_i\right)={\displaystyle \frac{{\beta }_1}{{\beta }_1+{\beta }_2+\dots +{\beta }_7}}\cdot W_7\left(p_i\right).\end{array}\right.$$

(9)

Here, the output neurons of the third layer are $Q_1\left(p_i\right),Q_2\left(p_i\right),\dots ,Q_7\left(p_i\right)$ $\in \left[0;1\right],$ and $p_i$ is some student ATCo for whom the level of practical knowledge is determined, $i=\overline{1,n}$.

Fourth layer: Output layer

At the fourth layer, the data are defuzzified to derive the levels of practical knowledge $Y$. For this, the activation function is used in the output neuron:

$$Y=100·\left(Q_1\left(p_i\right)+Q_2\left(p_i\right)+\dots +Q_7\left(p_i\right)\right).$$

(10)

The obtained assessment of the level of practical knowledge is on a 100-point scale $Y\in \left[0;100\right]$, which is substituted according to the accepted classification levels [28]: “A” is excellent if 91–100; “B” is very good if 81–90; “C” is good if 71–80; “D” is satisfactory if 61–70; “E” is sufficient if 51–60; “FX” is not enough if 0–50. The student completes practical training on reactions in multitasking situations if his or her results are rated no lower than 75.

Synaptic weights (${\alpha }_1,{\alpha }_2,\dots ,{\alpha }_7$) in a neuro-fuzzy model determine the influence of each of the input parameters (${∆}_1,{∆}_2,\dots ,{∆}_6,\delta$) on the output result ($Y=D_i$, the level of practical knowledge of the pilot student). They provide the flexibility of the model and the accuracy of its estimates, adapting to the specifics and significance of each parameter. The role of the weights is to weigh the influence of the input data. That is, each parameter receives its weight, which determines its significance. Also, in the learning process, the synaptic weights are adjusted, which allows the model to adapt to the characteristics of each student ($p_i$) and consider different conditions. The weights adjust the “If–Then” rules, determining the influence of the variables (${∆}_1,{∆}_2,\dots ,{∆}_6,\delta$) on the result.

Also, in the study, the mechanism of synaptic weighting coefficients ${\beta }_1,{\beta }_2,\dots ,{\beta }_7$ is used to adjust the importance of the neurons in the third layer of the neuro-fuzzy network. These coefficients determine the relative influence of parameters such as the level of performance of the task or the repetition of the exercise on the calculation of the postsynaptic potential. The coefficients allow setting a different significance for each parameter depending on the conditions. Thus, the model achieves high accuracy, considering all aspects of practical knowledge.

The artificial intelligence technology for assessing the level of practical knowledge of student ATCos based on their reactions in multitasking situations is presented. The result is a standardized assessment of the level of practical knowledge based on testing with a device for measuring reaction time and the relative frequency of errors in multitasking tasks. The automation of the process of obtaining a standardized assessment was made possible owing to the available sample of real data on which the neuro-fuzzy model was trained. Of course, the training process of ATCo students cannot be completely automated. Therefore, the obtained assessment of the level of practical knowledge based on artificial intelligence technology is an auxiliary tool for the DM in determining the final evaluation. Such an assessment allows for a more objective assessment of the ATCo student’s reactions to the various stressful situations and multitasking conditions. Still, the final decision always requires the participation of an instructor or an expert who considers all other factors that can affect the training of the ATCo, including their psychological readiness and real experience.

4. Results

The research verified and tested the artificial intelligence technology for assessing the level of practical knowledge of ATCo students based on their reactions in multitasking situations on 157,500 points of real data of the Technical University of Košice (Slovakia) in the “PILOT” study program at the Faculty of Aeronautics [29]. Data collection was carried out systematically during the last academic year (2023/2024) of the practical training of air traffic control students. All students underwent a rigorous selection process consisting of a medical examination, a psychological examination, and a language proficiency test. Due to the strict entry requirements, the data collection took a long time.

Based on the data obtained from 35 students, many experiments were conducted to configure the artificial intelligence technology and train a neuro-fuzzy model to assess the level of practical knowledge of the students to obtain synaptic weights ${\alpha }_1,{\alpha }_2,\dots ,{\alpha }_7$. An example of technology verification was carried out on the following three ATCo students: n36, n37, n38. The group of students studied consisted mainly of bachelors aged around 19–21. The age homogeneity of the sample allowed us to evaluate the results of the study within a typical cohort group for this stage of education. The students underwent training in theoretical subjects intended for training air traffic controllers as well as practical classes on the Aviation Visual Information System simulation software (The ICZ LETVIS® SIM SW, OS LINUX SLED/SLES 12 AND HIGHER), which is used to practice radar control procedures and situations that arise in air traffic control work.

The experimental process in the study included several stages:

1.

Data collection—The data collection process was systematic and took place during the practical training of students in aviation air traffic control. The students underwent a careful selection process that included a medical examination, psychological testing, and language proficiency testing, which ensured a high level of sample homogeneity.

2.

Tuning the AI technology—Based on the collected data, numerous experiments were conducted to tune the AI technology and train the neuro-fuzzy model. The goal was to determine the synaptic weights (α₁, α₂…, α₇) used to assess the level of practical knowledge of the students. This stage required significant data processing and tuning of the model parameters to achieve high accuracy of assessments.

3.

Technology verification—As part of the technology verification, an experiment was conducted on data from three ATCo students (n36, n37, n38). This allowed us to test how the AI system assessed the level of practical knowledge in real multitasking conditions and how effectively the model coped with analyzing the students’ reactions.

In general, the experiment included the stages of data collection, tuning the artificial intelligence model, and testing it on real-world examples to confirm its effectiveness in assessing the practical knowledge of the ATCo students.

Based on the entire set of data obtained, several experiments were conducted to set up the artificial intelligence technology and train a neuro-fuzzy model for assessing the level of practical knowledge of ATCo students to obtain the synaptic weights ${\alpha }_1,{\alpha }_2,\dots ,{\alpha }_7$. To illustrate the step-by-step evaluation process, an example of the performance level derivation for a selected student, ATCo S, tested using a device measuring reaction time and relative error rate in multitasking tasks is illustrated.

The developed artificial intelligence technology consists of an information model of data processing ($I_{FI}$) and a neuro-fuzzy model of assessing the level of practical knowledge ($M_{NT})$.

In the first stage, testing takes place using a data acquisition device, and the data are processed using the information model $I_{FI}.$ All the data obtained after testing the student ATCo are given in [30], and fragments of the input data are presented in

Table 2. Snippets of input data device testing.

$\mathit{E}$	$\mathit{I}$	$\mathit{Z}$	$\mathit{e}$	$\mathit{t}$
$E_1$	$I_1$	$Z_4$	30	6
		$Z_3$	30	10
		$Z_6$	4.14913	2
		$Z_2$	2.94398	0
		$Z_5$	30	8
		$Z_1$	30	6
	$\dots$	$\dots$	$\dots$	$\dots$
	$I_{50}$	$Z_6$	2.466	1
		$Z_5$	1.26612	0
		$Z_4$	1.33315	0
		$Z_2$	1.6602	1
		$Z_1$	1.38387	0
		$Z_3$	0.76124	0
		$Z_2$	2.466	1
$\dots$	$\dots$	$\dots$	$\dots$	$\dots$
$E_{15}$	$I_1$	$Z_5$	1.11748	0
		$Z_3$	0.849132	0
		$Z_4$	1.47215	0
		$Z_6$	1.1362	0
		$Z_1$	1.22992	0
		$Z_2$	0.717265	0
	$\dots$	$\dots$	$\dots$	$\dots$
	$I_{50}$	$Z_6$	1.80702	1
		$Z_1$	1.06926	0
		$Z_4$	1.33634	0
		$Z_3$	0.997361	0
		$Z_2$	0.822338	0
		$Z_5$	0.800477	0

.

Further, the data processing of student ATCo reactions in multitasking situations is presented as a step-by-step algorithm.

In the first step, the error data and reaction time to perform the given task are fuzzified. For this purpose, the membership function of the quadratic Z-spline, according to Formulas (3) and (4), is used for the number of errors made and the reaction time for the task, respectively.

In the second step, the given task’s aggregate performance level is obtained using Formula (5). Fragments of the results of the calculations of the first and second steps are shown in

Table 3. Fragments of the results of calculations of the first and second steps.

$\mathit{E}$	$\mathit{I}$	$\mathit{Z}$	${\mathit{\mu }}_1$	${\mathit{\mu }}_2$	$\mathit{\lambda }$
$E_1$	$I_1$	$Z_4$	0.92	0	0.498
		$Z_3$	0.778	0	0.488
		$Z_6$	0.991	0	0.5
		$Z_2$	1	0.001	0.5
		$Z_5$	0.858	0	0.495
		$Z_1$	0.92	0	0.498
	$\dots$	$\dots$	$\dots$	$\dots$	$\dots$
	$I_{50}$	$Z_6$	0.998	0.063	0.532
		$Z_5$	1	0.644	0.822
		$Z_4$	1	0.605	0.803
		$Z_2$	0.998	0.399	0.699
		$Z_1$	1	0.574	0.787
		$Z_3$	1	0.871	0.936
		$Z_2$	0.998	0.063	0.532
$\dots$	$\dots$	$\dots$	$\dots$	$\dots$	$\dots$
$E_{15}$	$I_1$	$Z_5$	1	0.722	0.861
		$Z_3$	1	0.84	0.92
		$Z_4$	1	0.518	0.759
		$Z_6$	1	0.713	0.857
		$Z_1$	1	0.664	0.832
		$Z_2$	1	0.886	0.943
	$\dots$	$\dots$	$\dots$	$\dots$	$\dots$
	$I_{50}$	$Z_6$	0.998	0.316	0.658
		$Z_1$	1	0.746	0.873
		$Z_4$	1	0.603	0.802
		$Z_3$	1	0.779	0.889
		$Z_2$	1	0.85	0.925
		$Z_5$	1	0.858	0.929

.

In the third step, according to Formula (6), a generalized assessment of the performance of the given task is obtained within 50 instructions separately for repeated exercises $E$.

Then, in the fourth step, the repetition of the exercise is considered within the testing limits. To calculate the coefficients of the repetition of the exercise within the limits of testing by students, Formula (7) is used. The obtained results of the calculation of the third and fourth steps are given in

Table 4. Fragments of the results of calculations of the third and fourth steps.

$\mathit{E}$	${\mathit{Z}}_1$	${\mathit{Z}}_2$	${\mathit{Z}}_3$	${\mathit{Z}}_4$	${\mathit{Z}}_5$	${\mathit{Z}}_6$	$\mathit{\delta }$
$E_1$	0.799	0.838	0.785	0.732	0.702	0.57	1
$E_2$	0.794	0.871	0.818	0.812	0.778	0.645	0.929
$E_3$	0.833	0.848	0.819	0.828	0.811	0.561	0.857
$E_4$	0.854	0.901	0.852	0.853	0.847	0.774	0.786
$E_5$	0.847	0.895	0.876	0.84	0.864	0.697	0.714
$E_6$	0.844	0.903	0.859	0.845	0.87	0.828	0.643
$E_7$	0.857	0.926	0.909	0.855	0.883	0.835	0.571
$E_8$	0.893	0.924	0.916	0.861	0.897	0.844	0.5
$E_9$	0.853	0.922	0.894	0.888	0.888	0.811	0.429
$E_{10}$	0.861	0.918	0.895	0.87	0.864	0.816	0.357
$E_{11}$	0.868	0.919	0.911	0.887	0.88	0.828	0.286
$E_{12}$	0.855	0.899	0.889	0.87	0.912	0.827	0.214
$E_{13}$	0.862	0.907	0.909	0.872	0.89	0.848	0.143
$E_{14}$	0.845	0.898	0.907	0.888	0.904	0.82	0.071
$E_{15}$	0.848	0.917	0.893	0.852	0.884	0.818	0

.

In the second stage, a neuro-fuzzy model is used to derive the level of practical knowledge of student ATCos.

In the neurons of the first layer, a data fuzzification operation is performed to obtain normalized data (Table 4).

Then, on the second layer, the functions of the postsynaptic potential are aggregated according to the tasks set by the student and the repetition of the performance exercise within the limits of testing. Synaptic weights ${\alpha }_1,{\alpha }_2,\dots ,{\alpha }_7$ were obtained by the authors in the process of learning a neuro-fuzzy model using the method of backpropagation of errors on 157,500 accurate data obtained from the ATCo students [29]: ${\alpha }_1=1.81;$ ${\alpha }_2=1.56;$ ${\alpha }_3=1.12;$ ${\alpha }_4=0.74;$ ${\alpha }_5=0.97;$ ${\alpha }_6=1.47;$ ${\alpha }_7=0.02.$

Next, the functions of the postsynaptic potential of neurons of the second layer are determined by Formula (8),

Table 5. Calculated functions of the postsynaptic of neurons of the second layer.

$\mathit{E}$	${\mathit{W}}_1$	${\mathit{W}}_2$	${\mathit{W}}_3$	${\mathit{W}}_4$	${\mathit{W}}_5$	${\mathit{W}}_6$	${\mathit{W}}_7$
$E_1$	1.44	1.31	0.88	0.54	0.68	0.84	0.02
$E_2$	1.51	1.36	0.92	0.6	0.75	0.95	0.02
$E_3$	1.55	1.32	0.92	0.61	0.79	0.82	0.02
$E_4$	1.53	1.41	0.95	0.63	0.82	1.14	0.02
$E_5$	1.53	1.4	0.98	0.62	0.84	1.02	0.01
$E_6$	1.55	1.41	0.96	0.63	0.84	1.22	0.01
$E_7$	1.62	1.44	1.02	0.63	0.86	1.23	0.01
$E_8$	1.54	1.44	1.03	0.64	0.87	1.24	0.01
$E_9$	1.56	1.44	1	0.66	0.86	1.19	0.01
$E_{10}$	1.57	1.43	1	0.64	0.84	1.2	0.01
$E_{11}$	1.55	1.43	1.02	0.66	0.85	1.22	0.01
$E_{12}$	1.56	1.4	1	0.64	0.88	1.22	0
$E_{13}$	1.53	1.41	1.02	0.65	0.86	1.25	0
$E_{14}$	1.54	1.4	1.02	0.66	0.88	1.21	0
$E_{15}$	1.54	1.43	1	0.63	0.86	1.2	0

:

On the third layer, the neurons of the second layer are adjusted around the importance of a given task and the repetition of the exercise. Let DM consider the performed tasks equally important, with synaptic weights ${\beta }_1={\beta }_2=\dots ={\beta }_6=6$, and the synaptic weight for the repetition of the execution exercise ${\beta }_7=10$. Calculating the postsynaptic potential of neurons of the third layer is carried out according to Formula (9).

After that, on the fourth layer, the data are defuzzified to derive the levels of practical knowledge using the activation function according to Formula (10). The obtained results of the calculation of the third and fourth layers are given in

Table 6. Calculations of neurons of the third and fourth layers.

$\mathit{E}$	${\mathit{Q}}_1$	${\mathit{Q}}_2$	${\mathit{Q}}_3$	${\mathit{Q}}_4$	${\mathit{Q}}_5$	${\mathit{Q}}_6$	${\mathit{Q}}_7$	$\mathit{Y}$
$E_1$	0.1869	0.1699	0.1143	0.0704	0.0885	0.1089	0.0044	74
$E_2$	0.1961	0.1766	0.1191	0.0781	0.0981	0.1233	0.0041	80
$E_3$	0.2009	0.1720	0.1192	0.0797	0.1023	0.1072	0.0038	78
$E_4$	0.1994	0.1827	0.1241	0.0821	0.1068	0.1479	0.0035	85
$E_5$	0.1985	0.1815	0.1275	0.0808	0.109	0.1332	0.0031	83
$E_6$	0.2018	0.1831	0.1251	0.0813	0.1097	0.1582	0.0028	86
$E_7$	0.2101	0.1878	0.1324	0.0823	0.1113	0.1596	0.0025	89
$E_8$	0.2006	0.1874	0.1334	0.0828	0.1131	0.1613	0.0022	88
$E_9$	0.2026	0.1870	0.1302	0.0854	0.112	0.155	0.0019	87
$E_{10}$	0.2043	0.1862	0.1303	0.0837	0.109	0.1559	0.0016	87
$E_{11}$	0.2011	0.1864	0.1326	0.0853	0.111	0.1582	0.0013	88
$E_{12}$	0.2028	0.1823	0.1294	0.0837	0.115	0.158	0.0009	87
$E_{13}$	0.1989	0.1839	0.1324	0.0839	0.1122	0.1621	0.0006	87
$E_{14}$	0.1996	0.1821	0.1321	0.0854	0.114	0.1567	0.0003	87
$E_{15}$	0.1996	0.186	0.13	0.082	0.1115	0.1563	0	87

.

We obtained assessments of the levels of practical knowledge in terms of repetition of exercises. The DM analyzes the received data and can determine the final evaluation of the level of practical knowledge based on reactions in multitasking situations. For example, the final grade can be an arithmetic average. We obtained the evaluation at the “B” level, very good (81–90).

The technology was also example verified on three students (n36, n37, n38) [30]. For this, their levels of practical knowledge were calculated based on reactions in multitasking situations using the developed technology, and the levels were compared with the values obtained from the instructors. These levels were obtained in terms of repetition of exercises $E=\{E_1;E_2;\dots ;E_{15}\}$. The final score, which is the arithmetic mean, was calculated. Also, to compare the results, the estimation error was calculated. The calculation results are presented in

Table 7. Calculations of the level of practical knowledge for students $p_{36},p_{37},p_{38}$.

$\mathit{E}$	${\mathit{p}}_{36}$		${\mathit{p}}_{37}$		${\mathit{p}}_{38}$
	The Level from AI	The Level from the Instructor	The Level from AI	The Level from the Instructor	The Level from AI	The Level from the Instructor
$E_1$	71	95	88	100	89	100
$E_2$	89	100	91	100	90	100
$E_3$	90	89	96	95	97	98
$E_4$	81	89	90	95	98	99
$E_5$	82	96	82	96	84	98
$E_6$	93	97	90	94	94	99
$E_7$	87	97	87	98	91	100
$E_8$	87	98	88	97	84	93
$E_9$	82	94	78	85	90	98
$E_{10}$	77	88	85	96	84	85
$E_{11}$	85	88	97	100	87	90
$E_{12}$	89	100	81	90	87	99
$E_{13}$	77	87	91	88	92	100
$E_{14}$	84	96	87	98	98	100
$E_{15}$	88	100	95	100	96	100
Average	85	94	88	96	91	97
Error	9		8		6

.

The average error of the entire data set is 7.7%, which is a good result for automatically determining the level of practical knowledge of ATCo students, as this indicates the high accuracy of the artificial intelligence model. This means the system can accurately assess students’ skills, minimizing human factors and subjective assessments. Further improvement of the model may include increasing the volume of data for training the system as well as the integration of additional parameters, such as psychological state or reaction to stressful situations, to more accurately determine the students’ preparation level. This approach will allow instructors to receive even more detailed information about the weaknesses of each student, which will contribute to a more individualized approach to training and increase the quality of training.

5. Discussion

Using a neuro-fuzzy model, the paper developed an artificial intelligence technology for assessing the level of practical knowledge of ATCo students based on their reactions in multitasking situations. For this purpose, the information model was developed for processing the data of ATCo student reactions in multitasking situations; a neuro-fuzzy model was developed for assessing the level of practical knowledge of ATCo students; the developed artificial intelligence technology was verified on 157,500 accurate data points from ATCo students; and an example of deriving the level of practical knowledge for a selected student S who was tested using a device for measuring reaction time and relative error rate in multitasking tasks is illustrated. Also, the developed technology was tested on student ATCos, and an average error of 7.7% was obtained compared with the ratings received from the instructors.

The study logically links the availability of a test database that can be used for the training and analysis of artificial intelligence models with the possibility of assessing the level of practical knowledge of ATCo students. Based on their responses in multitasking situations, AI technology can perform an automated assessment that correlates with the instructors’ knowledge levels. This provides an additional level of objectivity and accuracy in evaluation.

The artificial intelligence technology is based on data obtained by testing with a device to measure the reaction time and relative error rate in multitasking tasks. The value of the technology is that based on the test data and the assessment of the level of practical knowledge received from the instructor, the training of a neuro-fuzzy network is carried out to create a research knowledge base; data processing is carried out using the theory of fuzzy sets and intellectual analysis of knowledge; and an evaluation of the level of practical knowledge of the evaluated students is carried out using the knowledge base of the study.

The mathematical apparatus of the theory of fuzzy sets, intellectual analysis of knowledge, and neural networks are used to formalize the data. Data processing from the testing device is based on the theory of fuzzy sets and membership functions, which are components of intellectual knowledge analysis. Based on their reactions in multitasking situations, a neuro-fuzzy model was used to determine students’ practical knowledge level, which was verified on accurate data. Such a mathematical toolkit increases the degree of validity of decision making and can reveal complex and non-linear dependencies in the data. The presented research is based on models driven by data and the knowledge of training instructors and has significant practical value.

The advantages of the developed technology are that the information model is based on accurate data obtained from the device and the expert assessment of the training instructor. The neuro-fuzzy model can change the setting of synaptic weights relative to the importance of a given task and the repetition of the exercise, and by training the network, the knowledge base can be supplemented.

A limitation of our study was the sample of ATCo students from our Technical University of Košice (Slovakia) for training the neuro-fuzzy model. To compare the results of the experiment, we need, in the future, to obtain data from other educational institutions as well. To overcome these limitations, future studies will involve more educational institutions, providing a greater diversity of data, which will increase the generalizability of the results. For this purpose, the following partner universities will be involved: Group 1 includes the University of Seville (Spain) and Haaga-Helia University of Applied Sciences (Finland), with which we are part of the Ulysseus European University consortium; Group 2 consists of long-term project partners, namely, the Silesian University of Technology (Poland), the University of Pardubice (Czech Republic), Óbuda University (Hungary), and the University of Public Service (Hungary); Group 3 includes universities from the PEGASUS Aerospace Engineering Universities Association. In addition to measuring reaction time and error rate, additional variables, such as student stress levels, can be integrated into the technology to improve its adaptability. Also, to improve the generalizability of the model, cross-validation should be carried out by testing the model on data from different institutions. Using machine learning methods to process data from different sources will allow the creation of a multifactorial model that takes into account differences between institutions. It is also important to develop a long-term strategy for data collection and for updating the model, which will contribute to its high efficiency and adaptation to changes in educational processes.

The rationality of the developed artificial intelligence technology has been proven experimentally. An adequate and well-founded mathematical apparatus provides the results’ reliability. For the application of artificial intelligence technology in other educational institutions, it is necessary to obtain student data using a device to measure reaction time and the relative frequency of errors in multitasking tasks and to assess the levels of practical knowledge acquired from instructors. The obtained data will serve to train a neuro-fuzzy model. In addition, the developed technology will allow the discovery of topics and/or simulated practical tasks to improve individual learning for students. For example:

• Incorporating practical examples. During the analysis, it was observed that some students consistently performed poorly on tasks that required simultaneous attention to visual and auditory stimuli. This understanding prompts the system to identify “multisensory task management” as a problem area for the student. As a result, the system should automatically suggest practical exercises that focus on sensory coordination, such as reaction exercises that involve simultaneous visual and auditory stimuli.
• Demonstrate feedback mechanisms. For students, the technology should not only identify specific areas of difficulty but also suggest targeted learning exercises. After completing these exercises, the technology should reassess the students’ performance, demonstrating a measurable improvement in reaction time and a reduction in error rates. This iterative feedback loop highlights how the technology facilitates continuous improvement and personalized learning.
• Explain the problem detection process. When processing student response data, the neuro-fuzzy model analyzes trends and deviations in performance indicators such as reaction time and error rate. If certain patterns emerge, such as consistently higher error rates on certain types of tasks, the model assigns a higher degree of membership in the “problematic” category for these tasks. This information can then be used to recommend targeted simulated practice tasks, ensuring that the learning process is individualized and focused on those areas where the student has deficiencies.

It is noted that this technology can be adapted for training other ATCo students. Since the tasks are focused on assessing the speed, accuracy of reactions, and the ability to multitask, they can be useful for developing the necessary skills in air traffic control. However, for the effective application of this method, it is necessary to take into account the specifics of the work of ATCo students, such as their ability to work under pressure and to process visual and acoustic signals simultaneously, and adapt the complexity of the tasks to the level of training of the students. In addition, the tasks can be modified to assess not only reactions to stimuli but also the ability to make quick decisions in conditions of high intensity and stress, which is an important part of the ATCo work.

6. Conclusions

The research’s primary goal was to develop artificial intelligence technology for assessing the level of practical knowledge of ATCo students based on their reactions and answers in multitasking situations using the proposed neuro-fuzzy model, which was verified within the experiments. The following results were obtained. For the first time, an information model was developed to process data on the reactions of student ATCos in multitasking situations. For this purpose, a step-by-step algorithm for processing ATCo student data in multitasking situations is proposed. The algorithm works on data obtained using a device to measure reaction time and relative error rate. Data processing is carried out with the help of intellectual analysis of knowledge. For the first time, a neuro-fuzzy model for evaluating students’ level of practical ATCo knowledge was developed. It consists of four layers, and the connection between the input and output data is presented as the production rules of a fuzzy knowledge base. At the output, an assessment of the level of practical knowledge of the evaluated student ATCo based on reactions in multitasking situations is derived. The neuro-fuzzy model was verified and trained on 157,500 accurate data of ATCo students. An example of the derivation of the level of practical knowledge for a selected ATCo student from their performance in training is illustrated. An innovative algorithm is designed in comparison with existing solutions.

The research that was conducted made it possible to obtain answers to key scientific questions. In particular, the proposed artificial intelligence technology contributed to identifying problematic topics and practical tasks to increase the individual support of students, helping them acquire the necessary knowledge, skills, and competencies. In addition, this technology makes it possible to automate the objective assessment of future air traffic controllers during training.

The importance of the developed technology lies in the fact that it will allow the integration of automated knowledge and skill assessment systems into the real educational process. This will provide a more accurate and objective assessment of students’ readiness to perform tasks in stressful and multitasking conditions, improve the quality of training, and promote the development of innovative pilot training methods. In addition, it will make it possible to provide an individual approach to the educational component to achieve the main learning result. Ensuring an individual approach to students’ education is key to their obtaining the necessary skills, knowledge, and competencies. The result of the conducted research serves the possibility of designing software for practical application of the developed technology in teaching ATCo students.