Analyzing Emerging Challenges for Data-Driven Predictive Aircraft Maintenance Using Agent-Based Modeling and Hazard Identification

Abstract

The increasing use of on-board sensor monitoring and data-driven algorithms has stimulated the recent shift to data-driven predictive maintenance for aircraft. This paper discusses emerging challenges for data-driven predictive aircraft maintenance. We identify new hazards associated with the introduction of data-driven technologies into aircraft maintenance using a structured brainstorming conducted with a panel of maintenance experts. This brainstorming is facilitated by a prior modeling of the aircraft maintenance process as an agent-based model. As a result, we identify 20 hazards associated with data-driven predictive aircraft maintenance. We validate these hazards in the context of maintenance-related aircraft incidents that occurred between 2008 and 2013. Based on our findings, the main challenges identified for data-driven predictive maintenance are: (i) improving the reliability of the condition monitoring systems and diagnostics/prognostics algorithms, (ii) ensuring timely and accurate communication between the agents, and (iii) building the stakeholders’ trust in the new data-driven technologies.

1. Introduction

New technologies and data-driven algorithms bring both opportunities and challenges for aircraft maintenance. Traditionally, the aircraft maintenance process consists of periodic tasks performed by mechanics at pre-determined, fixed time intervals, i.e., time-based maintenance (TBM) [1]. In the last years, however, aircraft maintenance has increasingly made use of on-board sensors, aircraft condition monitoring systems (ACMS), and data-driven predictive algorithms. These new technologies increase the level of automation of the aircraft maintenance process. For example, on-board sensors and ACMS are used to continuously monitor the health condition of aircraft systems. The data are used to make predictions about the degradation levels of the systems. For example, data-driven algorithms are developed to detect damages (diagnostics) and predict the remaining useful life (RUL) of aircraft systems (prognostics) [2,3]. Using such predictive algorithms, maintenance tasks are generated only when needed [4]. We refer this process of using sensor data and predictive algorithms to generate maintenance tasks as data-driven predictive aircraft maintenance (PdAM).

The use of data-driven technologies for aircraft maintenance poses novel challenges. For example, the retrieval, storage, processing, and utilization of sensor data involves risks such as data loss, data corruption, data transmission delays, and lack of accuracy of failure prediction algorithms. Furthermore, new experts handling the data and algorithms need to be involved in the traditional aircraft maintenance process. The manner in which these new experts interact with the existing maintenance teams may lead to new challenges. Thus, to safely implement data-driven PdAM, an analysis of emerging challenges is required.

To the best of our knowledge, emerging challenges of data-driven PdAM have not yet been identified and discussed. Existing studies mostly discuss challenges associated with the traditional aircraft maintenance process, TBM. In [5], the authors use an extensive safety questionnaire and show that the behavior of the maintenance personnel is a critical contributing factor to errors in aircraft maintenance. In [6], the authors show that the manner in which the maintenance personnel interact with each other and their use of hardware/software are the main contributing factors to human errors in aircraft maintenance. However, these studies are not considering the use of data-driven technologies for aircraft maintenance. Since 2018, when the EASA (European Union Aviation Safety Agency) integrated aircraft health monitoring (AHM) into the regulatory basis for aircraft maintenance [7], no studies have discussed emerging challenges of data-driven PdAM, taking into account the entire maintenance process and interactions between maintenance personnel and new data-driven technologies.

The aim of this paper is to discuss emerging challenges of the data-driven PdAM, based on the identification and analysis of new hazards associated with the new data-driven technologies. In general, a hazard implies the intrinsic ability of an agent or situation to cause adverse effects to a target [8]. Specifically, a hazard in aviation is defined as follows:

Definition 1 (Hazard).

A condition that could foreseeably cause or contribute to an aircraft accident [9]; any condition, event, or circumstance which could induce an accident [10].

In this paper, we consider hazards related to aircraft maintenance. We especially focus on the hazards associated with the adoption of new data-driven technologies, and the hazards related to the interactions between the maintenance personnel involved in new data-driven PdAM.

Traditional hazard identification methods, such as FMEA (failure mode and effects analysis) or HAZOP (hazard and operability study), look at individual process components. For each such component, potential failure modes, their causes and effects are identified [11]. However, these methods fail to capture the interactions between process components and the hazards associated with these interactions [12,13]. For the case of aircraft maintenance, the interactions between maintenance personnel and the manner in which the personnel interacts with the digital systems are important contributing factors to hazards [6]. Moreover, due to the only recent consideration of data-driven technologies for aircraft maintenance, there is a very limited amount of data and experience of data-driven PdAM.

To address the drawbacks of traditional methods and the lack of data and experience of data-driven PdAM, we apply a structured hazard identification brainstorming [13,14,15]. The brainstorming is especially suited to identifying emerging hazards associated with novel processes. For instance, the brainstorming is used to identify hazards associated with maintenance outsourcing [16] and future aviation concepts [17]. Furthermore, the brainstorming is a useful method to supplement the lack of data in hazard identification [18]. We facilitate this brainstorming using an agent-based model of the aircraft maintenance process [4], which provides an intuitive understanding of the interactions between agents. The identified hazards are validated in the context of maintenance-related aircraft accidents reported between 2008 and 2013. Finally, in the light of the identified hazards, we discuss emerging challenges for a safe implementation of data-driven PdAM.

The main contributions of this paper are as follows:

• We identify the agents and their interactions during the data-driven predictive aircraft maintenance process. This agent-based model illustrates how future aircraft maintenance will be changed when data-driven technologies and new experts are integrated into the traditional aircraft maintenance process.
• We identify emerging hazards associated with data-driven predictive aircraft maintenance through a structured brainstorming session of experts. Here, the agent-based model is used to facilitate the brainstorming. We validate the identified hazards based on the historical accident/incident related to aircraft maintenance.
• Based on the analysis of the hazards, we discuss three main challenges of data-driven predictive aircraft maintenance. These challenges suggest directions of future research and development in aircraft maintenance.

The remainder of this paper is organized as follows. Section 2 introduces an agent-based model showing the stakeholders, digital systems, and their interactions in the data-driven PdAM. Section 3 identifies and discusses the hazards associated with the data-driven PdAM. Section 4 validates the identified hazards in the context of past aircraft accidents related to maintenance. In Section 5, we discuss the emerging challenges of data-driven PdAM based on the identified hazards. Finally, we provide conclusions in Section 6.

2. Agent-Based Model of Data-Driven Predictive Aircraft Maintenance

In this section, we model a data-driven predictive aircraft maintenance process (PdAM) using an agent-based model [4]. Here, an agent is defined as an independent entity that makes decisions based on a set of rules, interacts with other agents, and has its own goals [19,20].

The purpose of modeling the PdAM process is to facilitate brainstorming for hazard identification. The agent-based model of data-driven PdAM is first presented to the experts participating in the brainstorming to provide a solid understanding of this new aircraft maintenance process, and to trigger ideas about emerging hazards.

Table 1. Agents of data-driven predictive aircraft maintenance (PdAM).

Agent Name	Acronym
Task Generating Team	TG
Task Planning Team	TP
Mechanics Team	ME
Data Management Team	DM
Flight Crew	CR

and Figure 1 show the main agents of the data-driven PdAM process and the interactions between them, respectively. In particular, we consider PdAM where a new data management team is introduced to the traditional aircraft maintenance process [4]. The main agents identified for PdAM are: (i) the task generating team (TG), (ii) the task planning team (TP), (iii) the mechanics team (ME), (iv) the flight crews (CR), and (v) the data management team (DM). Among them, four agents (TG, TP, ME, and CR) are involved in both the traditional aircraft maintenance process (TBM) and the new aircraft maintenance process (PdAM), while DM is a new agent specifically supporting PdAM.

Below we characterize the agents of the aircraft maintenance process by describing their roles and interactions with other agents. In particular, we first elaborate the role and interactions under traditional TBM, and then describe the changes under new PdAM. A detailed model for each agent is given in [4].

2.1. Task Generating Team (TG)

The role of the task generating team (TG) is to define the type, due date, and method used for a maintenance task. TG generates two types of tasks: periodic tasks and one-time tasks. The periodic tasks are generated based on the regulations introduced by air authorities such as EASA, the manuals provided by aircraft manufacturers, and the analysis of airlines’ operation data. TG integrates all this information and generates periodic tasks (type, due date, and method). Under TBM, these periodic tasks are extensively used as the primary measure to prevent failures. Apart from periodic tasks, one-time maintenance tasks are generated whenever TG receives complaints or findings from flight crews or mechanics. For example, if flight crews observe an abnormal performance of the aircraft during a flight, then they submit a complaint to TG. Similarly, during an inspection, if the mechanics observe an issue, then they submit a finding to TG. Finally, TG analyzes the submitted complaints and findings and generates necessary tasks to address these issues.

Under PdAM, TG receives additional input such as diagnostics and remaining useful life (RUL) prognostics based on DM’s data analytics with aircraft condition data. This input is verified and analyzed by TG. When needed, TG asks TP to plan necessary one-time tasks. For example, let the RUL prognostics of a brake indicate that the brake is expected to wear out within 50 flight cycles. If this is shorter than the remaining number of flight cycles before a planned periodic replacement for this brake (periodic task), then TG asks TP to reschedule the replacement of the brake earlier (data-driven one-time task). In this example, TG anticipates a maintenance issue before it happens, i.e., the maintenance tasks triggered by the prognostics are predictive.

2.2. Data Management Team (DM)

The data management team (DM) is a new agent specifically introduced to support the data-driven PdAM process. DM is responsible for handling the aircraft condition data and generating diagnostics and RUL prognostics. DM first collects the condition monitoring data from aircraft condition monitoring systems (ACMS), the sensors installed on board of the aircraft. Here, DM may also integrate external databases such as weather data, airport data, and/or data shared by other airlines or maintenance organizations [4]. Data processing and validation are also part of the role of DM. With such data, DM generates diagnostics and RUL prognostics for aircraft systems and structures. In this step, various data-driven algorithms are utilized to generate diagnostics and prognostics depending on the characteristics of the target system, the inspection/monitoring intervals, and the redundancy of the system [21,22,23]. Finally, DM transfers the diagnostics and prognostics information to TG.

During the entire process, DM uses a digitalized platform to collect, validate, analyze, and transfer the data and prognostics information. Such platforms to monitor condition data of an aircraft fleet are, for instance, Skywise of Airbus [24] and Airplane Health Management of Boeing [25].

2.3. Task Planning Team (TP)

The task planning team (TP) schedules in time for the execution of maintenance tasks. The tasks are given by TG (periodic and one-time tasks), as well as by mechanics (deferred tasks), in case additional issues are observed during inspections. TP finds available time slots when the aircraft can undergo maintenance, given the flight schedule of the aircraft, the due dates of each maintenance task, the availability of the mechanics, and the availability of necessary materials and resources. Ultimately, TG generates a schedule for the maintenance tasks. A scheduled task specifies the aircraft, the target system/structure, the maintenance tasks type, and the mechanics that need to execute the task.

Under PdAM, the role of TP does not change significantly since the tasks generated by TG using diagnostics and prognostics will be given to TP in a similar format as the non-data-driven tasks.

2.4. Mechanics Team (ME)

The mechanics team (ME) executes the scheduled tasks received from TP. Various types of maintenance tasks are executed, such as system/structure replacement, restoration, lubrication, and inspection [1]. During an inspection, ME may observe additional issues such as an unexpected level of degradation in aircraft structure. Based on the manuals, ME reports such findings. The necessary tasks addressing these findings are executed on-site (unscheduled tasks) or reported to TG for rescheduling in other maintenance slots (deferred tasks).

Similar to TP, the role of ME does not change significantly under PdAM since the task type and schedules are already specified by TG and TP.

2.5. Flight Crew (CR)

The flight crew (CR) includes pilots and cabin crews who actually operate the aircraft. During a flight, CR monitors the condition of the aircraft using on-board ACMS. CR reports a complaint to TG when any abnormality is noticed. The complaints reported by CR are analyzed by TG who may generate additional tasks to address these issues.

Given that the operation of the aircraft is not subject to changes under PdAM, the role of CR is not expected to change significantly under PdAM.

3. Hazard Identification for Data-Driven Predictive Aircraft Maintenance

In this section, we identify emerging hazards associated with the data-driven predictive aircraft maintenance process (PdAM) by means of a structured brainstorming conducted with the aircraft maintenance agents (see Section 2). Thereby, the hazards are identified from diverse perspectives of multiple agents. The obtained hazards are analyzed and clustered relative to the agents.

3.1. Methodology

3.1.1. Brainstorming for Hazard Identification

Inspired by [13,14,15], a structured hazard identification brainstorm was performed on 28 February 2019. A total of 10 experts in aircraft maintenance participated at the brainstorming session.

Table 2. Participants at the brainstorming session.

Role in Brainstorming	Expertise & Experience	Number of Attendees
Domain expert	Data management team	2
Domain expert	Task generating team	4
Domain expert	Task planning team	1
Domain expert	Flight crew (Pilot)	1
Domain expert	Safety manager	2
Moderator	Brainstorming method	1
Secretary		1

shows the expertise of the participants and their role in the session. Each participant had at least 2 years of experience in the indicated domain. During the brainstorming session, they represented one of the agents identified for data-driven PdAM (see Table 1 and Figure 1). For the mechanics, their point of view was delegated to the task generating team since their role is not expected to change significantly under PdAM, relative to the changes envisioned for the other agents. We note that two safety managers represented the overall safety point of view of the aircraft maintenance process. Finally, the session was conducted by a moderator with expertise in aviation safety and experience with the brainstorming methodology for hazard identification. During the session, notes were taken by a secretary.

At the beginning of the brainstorming session, the agent-based model of the aircraft maintenance process in Figure 1 is presented to and discussed with the participating domain experts. This ensures that each participant knows its own agent role as well as the agent roles of the other participants during the brainstorm. It is also verified with each participant if its agent role is correctly presented in the figure. If not, then the agent-based figure has to be adapted prior to starting the brainstorm. The validated agent-based model in Figure 1 is projected throughout the entire brainstorm. This allows participants to easily express their brainstorm inputs relative to this agent-based model.

The two main rules used for the brainstorming were: (i) to obtain as many hazards as possible, and (ii) criticism and analysis during the session are not allowed. These rules are motivated by cognitive science. The amount of ideas generated is regarded as more important than the quality of the ideas generated during brainstorming [17,26]. Criticism has been shown to have a negative impact on the open atmosphere necessary for productive brainstorming [17,26]. In order to avoid discussions about the validity of a hazard, prior to the start of the brainstorm, the participants were told that the brainstorm is about “wide-sense hazards”, i.e., anything that may influence the operation. This means that, in later safety analyses, some of the generated hazards may turn out to pose negligible safety issues and therefore are not true hazards.

During the brainstorming session, the moderator encouraged the participants to share their ideas and opinions and interact with each other. The participants are also asked to use the cognitive flow that they are accustomed to in their professional work situations. Once the brainstorm is started, each participant easily recognizes their own professional cognitive flow to have access to the wealth of their operational knowledge and experience.

During the brainstorm session, all inputs generated by the participants were written and presented to the participants. In case of an error or misunderstanding, the contributor of the input can correct it. For each hazard, the name of the contributor was noted; this allows us to contact the contributor in case of follow-up questions during later safety analyses.

3.1.2. Post-Brainstorming Data Processing

After the brainstorming session, the raw data were post-processed by independent safety analysts. First, as the generated raw data were “wide-sense hazards”, they were analyzed as true hazards, i.e., condition, event, or circumstance in aircraft maintenance, which could cause or contribute to aircraft incidents. Second, the terminology and acronyms used in the formulation of the ideas were unified and, when possible, the terminology used for the agent-based model in Section 2 was used. Third, the repetitions of the same idea were analyzed. Ultimately, a list of unique ideas was generated, and repetitions were discarded. As the last step, the obtained hazards were clustered based on whether the hazards are associated only with data-driven PdAM, or with both TBM and PdAM, based on the agent primarily involved with the hazards.

As a result, 41 unique aircraft maintenance hazards were obtained. Out of them, 21 hazards were applicable to generic aircraft maintenance, i.e., these hazards can occur under either TBM or PdAM. The remaining 20 hazards were new hazards associated with the introduction of PdAM, i.e., these hazards can occur only under PdAM.

Table 3. Number of hazards per involved agent.

		Both	Only
	Total	TBM & PdAM	PdAM
Total Number of Hazards	41	21	20
Task generating team (TG)	13	5	8
Data management team (DM)	10	0	10
Mechanics team (ME)	10	8	2
Task planning team (TP)	5	5	0
Flight crews (CR)	3	3	0

shows the number of hazards identified from the brainstorming.

3.2. Analysis of Brainstorming Results

In this section, we analyze the obtained hazards relative to each aircraft maintenance agent, focusing on the 20 new hazards associated with the introduction of data-driven PdAM.

Table 3 shows the number of hazards identified for general aircraft maintenance (both for TBM and PdAM), and for the data-driven predictive aircraft maintenance only (PdAM only). The results show that the greatest number of hazards are identified relative to the task generating team (TG). The main explanation for this result is that TG plays a key role in aircraft maintenance, determining which tasks need to be planned and executed based on the feedback from ME, CR, and DM. The data management team (DM), a new agent supporting PdAM, is associated with 10 new hazards of data-driven PdAM. The mechanics team (ME) is associated with 10 hazards, but only 2 of them are the new hazards of PdAM. This is due to the perception that the role of the mechanics will not change significantly under PdAM because the execution of the tasks is expected to be similar to the execution of tasks under current TBM. Similarly, the task planning team (TP) and the flight crew (CR) do not have new hazards under PdAM because they are expected to work in a similar fashion as under the current TBM.

Below we discuss and analyze in detail the 20 new hazards of data-driven PdAM, identified for the three agents: DM (Table 4), TG (Table 5), and ME (Table 6).

3.2.1. Hazards Associated with the Data Management Team (DM)

The 10 hazards associated with DM are related to (i) the performance of the aircraft condition monitoring systems (ACMS), (ii) the performance of the data-driven algorithms used to generate diagnostics and RUL prognostics for aircraft systems and structures, (iii) communication issues between agents, and (iv) delay in the knowledge and data transfer between agents. Their descriptions and IDs are given in Table 4.

Table 4. Hazards of data-driven PdAM, associated with the Data Management team (DM).

ID	Description
$H_{01}$	DM could not get data because aircraft condition monitoring system is not functioning, or inoperative.
$H_{02}$	DM gets incorrect/inaccurate data because aircraft condition monitoring system is malfunctioning.
$H_{03}$	DM gets incorrect/inaccurate data that is corrupted during data transfer.
$H_{04}$	DM gets data too late because of delays in data transfer from aircraft condition monitoring systems.
$H_{05}$	DM generates wrong prognostics/diagnostics.
$H_{06}$	DM uses unreliable algorithm for prognostics/diagnostics
$H_{07}$	DM does not alert when there is a fault because the threshold is not met.
$H_{08}$	DM alerts when there is no fault because the monitoring parameter is above threshold.
$H_{09}$	DM generates unclear/ambiguous prognostics/diagnostics.
$H_{10}$	DM generates prognostics/diagnostics too late.

Table 4. Hazards of data-driven PdAM, associated with the Data Management team (DM).

ID	Description
$H_{01}$	DM could not get data because aircraft condition monitoring system is not functioning, or inoperative.
$H_{02}$	DM gets incorrect/inaccurate data because aircraft condition monitoring system is malfunctioning.
$H_{03}$	DM gets incorrect/inaccurate data that is corrupted during data transfer.
$H_{04}$	DM gets data too late because of delays in data transfer from aircraft condition monitoring systems.
$H_{05}$	DM generates wrong prognostics/diagnostics.
$H_{06}$	DM uses unreliable algorithm for prognostics/diagnostics
$H_{07}$	DM does not alert when there is a fault because the threshold is not met.
$H_{08}$	DM alerts when there is no fault because the monitoring parameter is above threshold.
$H_{09}$	DM generates unclear/ambiguous prognostics/diagnostics.
$H_{10}$	DM generates prognostics/diagnostics too late.

Four hazards were identified relative to the performance of the ACMS (see hazards $H_{01}$, $H_{02}$, $H_{03}$, and $H_{04}$). First, the ACMS itself can be subject to malfunction or become inoperable (see hazard $H_{01}$). In this case, the streams of condition data are no longer available, and thus DM cannot generate any diagnostics or prognostics. A worse case is when DM does not notice the malfunction of the ACMS. In this case, the malfunction results in the ACMS collecting corrupted data, which is used for diagnostics and prognostics. This is the subject of hazards $H_{02}$ and $H_{03}$. Hazard $H_{02}$ refers to the case when incorrect or inaccurate condition data is used by DM. In turn, the resulting diagnostics and prognostics become unreliable. If these unreliable diagnostics and prognostics are transferred to TG to generate maintenance tasks, then the impact of this hazard is propagated to the entire aircraft maintenance process. Even when the ACMS collects accurate condition data, this data can still become corrupted during data transfer from ACMS to DM (see hazard $H_{03}$). This hazard may trigger additional hazards following the same propagation path as for hazard $H_{02}$. Another important aspect is to obtain the condition monitoring data on time (see hazard $H_{04}$). Hazard $H_{04}$ describes a case when DM obtains the condition monitoring data with delay. Since aircraft are operated under tight and dynamic flight schedules, timely scheduling of maintenance tasks cannot be sustained if the diagnostics and prognostics are generated with delay.

Four hazards were identified related to the accuracy of the diagnostics and prognostics algorithms and their results (see hazards $H_{05}$, $H_{06}$, $H_{07}$, and $H_{08}$). During the brainstorming, erroneous diagnostics and prognostics were identified as the foremost critical hazards (see hazard $H_{05}$). If the diagnostics/prognostics are erroneous, then either no trigger is generated for necessary maintenance tasks in order to prevent failures/malfunctions or triggers are generated for redundant, unnecessary maintenance tasks. The former case may cause incidents/accidents, while the latter case may cause additional, unnecessary work and costs [27].

The possible causes of hazard $H_{05}$ were also identified as hazards, i.e., conditions that make diagnostic/prognostic results erroneous. The errors in the data are already discussed as hazards $H_{02}$ and $H_{03}$. In addition, the used algorithm itself may be unreliable (see hazard $H_{06}$). In this case, regardless of the quality of the data, the diagnostics/prognostics would be unreliable. Furthermore, two different modes of potential error of the prognostics result were discussed. The first case occurs when DM does not provide an alert when there is a fault, i.e., a false negative (see hazard $H_{07}$). Given a false negative, a necessary maintenance task is not triggered. The second case occurs when DM provides an alert when there is actually no fault, i.e., false positive (see hazard $H_{08}$). Although a false positive may not directly affect the safety of the aircraft, it can reduce the efficiency of aircraft maintenance [27]. Moreover, in the case of frequent false positives, the other agents may ignore alerts generated by DM.

Communication issues between agents were also indicated as a hazard during the brainstorming. Assuming that the prognostic results are reliable, an ambiguous or unclear communication between agents about these results was identified as a hazard (see hazard $H_{09}$). Hazard $H_{09}$ outlines various types of miscommunication regarding the diagnostics and RUL prognostics: i) information or alerts generated by DM are not considered by TG because this information is ambiguous or insufficient to determine effective measurement; ii) the digital platform used for communication between DM and TG presents the information in a non-intuitive form (ambiguous graphics, unclear metadata descriptions).

Lastly, the domain experts discussed the delay in obtaining diagnostics and prognostics. If the diagnostic/prognostic results are generated with delay by DM than the other agents, and especially TG, do not have enough time to generate necessary tasks to address the issues raised (see hazard $H_{10}$). In order to cope with tight aircraft flight schedules, it is desirable that diagnostic and prognostic results are delivered to TG, TP, and ME without delay so that necessary tasks can be generated and executed on time. This hazard is related to hazard $H_{04}$ because $H_{04}$ is likely to trigger hazard $H_{10}$. Moreover, these hazards are expected to be propagated to all agents.

3.2.2. Hazards Associated with the Task Generating Team (TG)

There were eight hazards identified for TG under PdAM (see Table 5). Among these eight hazards, three hazards were related to the communication with DM, two hazards were related to TG’s trust in the diagnostics and prognostics generated by DM, and three hazards were related to the process of generating tasks.

Table 5. Hazards of data-driven PdMA associated with the task generating team (TG).

ID	Description
$H_{11}$	TG does not notice the alert from DM.
$H_{12}$	TG misunderstands alerts from DM.
$H_{13}$	TG does not generate a task due to misunderstanding regarding prognostics.
$H_{14}$	TG does not examine/verify the prognostics/diagnostics.
$H_{15}$	TG does not rely on diagnostics/prognostics from DM.
$H_{16}$	TG generates inadequate/ineffective task for a given diagnostics/prognostics.
$H_{17}$	TG generates two identical tasks from two triggers.
$H_{18}$	TG generates a task from prognostics too late.

Table 5. Hazards of data-driven PdMA associated with the task generating team (TG).

ID	Description
$H_{11}$	TG does not notice the alert from DM.
$H_{12}$	TG misunderstands alerts from DM.
$H_{13}$	TG does not generate a task due to misunderstanding regarding prognostics.
$H_{14}$	TG does not examine/verify the prognostics/diagnostics.
$H_{15}$	TG does not rely on diagnostics/prognostics from DM.
$H_{16}$	TG generates inadequate/ineffective task for a given diagnostics/prognostics.
$H_{17}$	TG generates two identical tasks from two triggers.
$H_{18}$	TG generates a task from prognostics too late.

Hazards $H_{11}$, $H_{12}$, and $H_{13}$ address the issues of misunderstanding and miscommunication associated with TG under PdAM. Hazard $H_{11}$ refers to the case when TG does not notice an alert from DM, and thus the necessary maintenance tasks are not generated. Hazard $H_{12}$ refers to the case when TG notices the alert from DM, but misread its meaning. Hazard $H_{12}$ is likely to happen when DM generates unclear/ambiguous diagnostics and prognostics (see hazard $H_{09}$). If either hazards $H_{11}$ or $H_{12}$ occur, TG is likely to not generate a task as required by the alerts (see hazard $H_{13}$).

Hazards $H_{14}$ and $H_{15}$ discuss the level of trust of TG in the data-driven PdAM technologies, such as sensors and data-driven diagnostics and prognostics algorithms. Hazard $H_{15}$ discusses the case when TG does not use the diagnostics and prognostics generated by DM for task generation due to lack of trust. The trust in the new PdAM technologies is constructed not only based on numerical results from experiments, but also based on an accumulated trust over time between the users and the technology [28]. At the other extreme, hazard $H_{14}$ addresses the case when TG fully trusts the new technology and thus TG does not examine or verify the diagnostic and prognostic results. This hazard becomes critical when DM transfers erroneous diagnostics and prognostics (see hazard $H_{05}$). Using erroneous diagnostics and prognostics, TG may not generate necessary tasks (see hazard $H_{13}$) or generate inadequate tasks (see hazard $H_{16}$). Thus, hazard $H_{14}$ links the propagation of hazards from $H_{05}$ to $H_{13}$ and $H_{16}$.

Hazards $H_{16}$, $H_{17}$, and $H_{18}$ are related to the case when the generated tasks are not effective in resolving the issue raised. Hazard $H_{16}$ addresses the case when inadequate/ineffective tasks are generated. In this case, either additional costs are incurred to perform additional tasks, which are actually not necessary, or inadequate tasks are performed on the aircraft’s systems/structures. Hazard $H_{17}$ addresses the case when TG generates multiple identical tasks from different triggers. For example, when an air conditioning system of an aircraft needs maintenance, this task can be generated as a response to a complaint generated by a flight crew, a report filed by the mechanics, and/or following the prognostics results generated by the data management team (see Figure 1). These three independent sources of feedback ensure that an abnormal system performance is indeed reported. However, if the three sources of feedback are not managed properly and as a result, multiple identical tasks are generated (see hazard $H_{17}$), then this leads to confusion in the task generation, task planning, and task execution processes. Lastly, hazard $H_{18}$ discusses the issue of delay in the task generation process. If a task triggered by the diagnostics and prognostics is generated with delay, then the other agents such as TP and ME do not have enough time to plan and execute this task. As such, this hazard is expected to result in missed tasks.

3.2.3. Hazards Associated with the Mechanics Team (ME)

The two hazards associated with ME under the data-driven PdAM are given in Table 6. Here, fewer hazards are identified relative to TG and DM since the role of ME under PdAM is envisioned to be similar to the case of the traditional TBM. However, these two hazards need careful consideration because ME executes the maintenance tasks in the final stage of the aircraft maintenance process, with a direct impact on the aircraft airworthiness.

Table 6. Hazards of data-driven PdAM associated with the mechanics team (ME).

ID	Description
$H_{19}$	Data-driven PdAM would cause more maintenance tasks triggered by diagnostics/prognostics, leading to a higher risk of human error in maintenance by ME.
$H_{20}$	ME performs conventional inspection less carefully due to overconfidence in data-driven PdAM.

Table 6. Hazards of data-driven PdAM associated with the mechanics team (ME).

ID	Description
$H_{19}$	Data-driven PdAM would cause more maintenance tasks triggered by diagnostics/prognostics, leading to a higher risk of human error in maintenance by ME.
$H_{20}$	ME performs conventional inspection less carefully due to overconfidence in data-driven PdAM.

The main concern discussed during the brainstorming relative to ME under PdAM was the quality of the task execution under PdAM. Hazard $H_{19}$ refers to the case when the mechanics are potentially overloaded under PdAM due to additional tasks that are triggered by diagnostics and prognostics algorithms. Furthermore, an overload may occur for the mechanics if DM provides diagnostic and prognostic results with delay (see hazard $H_{10}$), or if TG generates tasks with delay (see hazard $H_{18}$). Under the pressure of executing these data-driven tasks, the risk of human error increases [6,29]. Furthermore, premature tasks can be triggered by the prognostics, which increases the chance of having human errors [30].

Hazard $H_{20}$ describes the case when ME performs the conventional inspection less carefully due to overconfidence in PdAM. This is the result of the ME over-trusting the new PdAM technologies. This hazard is similar to hazard $H_{14}$ for TG.

4. Validation of the Identified Hazards using Reported Aircraft Incidents

In this section, we discuss past aircraft accidents/incidents as a means to validate the hazards identified in the brainstorming session. We first outline the chronology of the events leading to these incidents based on the official investigation reports. Using these reports, we identify similar hazards as those identified in the brainstorming session (see Table 4, Table 5 and Table 6). This analysis shows that the hazards identified in the brainstorming session are also observed in the context of past incidents.

4.1. Nuisance False Positive Alerts Lead to Agents Ignoring a True Positive Alert

An aircraft incident reported in 2017 illustrates how the inadequate handling of alerts from ACMS contributes to the incident [31]. On 29 April 2017 (Day 0), an aircraft was dispatched while the left air conditioning system (ACS) had been disabled, in accordance with the Minimum Equipment List. During the flight, the cabin pressure was lost because the right ACS failed while the left ACS was disabled. The incident investigation established that the component on the right ACS had been changed 11 days before the day of the incident (Day $-11$). After the aircraft returned to service at Day $-9$, the on-board aircraft health monitoring (AHM) system sent an alert message to the operator’s AHM ground-based data system and their engineering department (AHM ground-based data system and their engineering department perform the role of DM and TG in Figure 1). This alert message indicates that a ‘high leakage/low inflow’ of the cabin pressurization system had been detected. The operator assessed the message and the necessary task was planned at Day $+6$. Thereafter, during all the subsequent flights between Day $-9$ and Day 0, maintenance alert messages were sent by AHM, but no further action was taken by the operator.

From the investigation report of this incident, we identify the following hazards that contributed to the incident. The operator generated an inadequate task with too late due date (see hazard $H_{16}$). More importantly, the continuous alert was not taken seriously by the operator because they regarded this as a ‘nuisance’ (see hazard $H_{15}$).

In addition, an indirect, but crucial hazard is identified—the generated diagnostic results had been frequently faulty in the past (see hazards $H_{05}$ and $H_{08}$), and therefore the engineering department classified the true positive alert as faulty (see hazard $H_{13}$). Regarding these hazards, we quote from the investigation report [31]:

The operator later stated that the AHM system provides just over 1200 maintenance alerts. From experience, some maintenance alert messages are inadvertently triggered, which has led to refinements to improve the robustness of the system and reduce the level of ‘nuisance’ alerts. The operator had seen alert message 21-0209-C740 triggered ‘intermittently’ on other aircraft before and this had caused maintenance staff to question the reliability of this particular alert message.

This incident shows that it is critical to ensure the reliability of the diagnostic/prognostic algorithms and the alert systems, in order to make the agents trust the new PdAM technologies.

4.2. Damage Not Identified by Sensors and Inspections

Several incidents are caused by the damage done during hard landings, which was identified neither by the on-board sensors nor by inspections [32,33,34]. Generally, on-board aircraft condition monitoring systems (ACMS) indicate hard landings to the flight crew. In this case, the pilots and the mechanics conduct inspections to identify and evaluate the potential damage, following the manuals.

In 2016, an aircraft damaged by a hard landing was released without addressing the damage [32]. Although the subsequent flight was completed uneventfully, it was found later that the aircraft was in an unsafe condition due to the serious damage made by the previous hard landing.

In the investigation report, it was found that the ACMS did not submit the ‘G-Load’ report to the pilots because the peak load of 3.32 g persisted for less than 1 second only (see Figure 2), while the report is issued when the load persists for at least 2 seconds [32]. Furthermore, the ACMS sent the ‘A15 hard landing report’ to the Maintenance Operation Center (MOC) (MOC performs the role of DM and TG in Figure 1). However, the MOC was not able to interpret the report properly (see hazard $H_{12}$) and on time (see hazards $H_{10}$ and $H_{18}$). Furthermore, the subsequent inspection did not find any damage (see hazard $H_{20}$), and thus, the aircraft was released back to service.

Two similar aircraft incidents occurred in 2013 and 2008 [33,34]. In both cases, the damages to the landing gears were not identified after hard landings. A common contributing factor to these incidents was that the on-board ACMS did not trigger an alert for hard landing since the predefined load threshold had not been exceeded (see hazard $H_{07}$). For the incident in 2008, the engineers reasoned that no inspection was needed because the recorded parameters had not exceeded a predefined threshold, which is in accordance with the aircraft maintenance manual [34]. For the incident in 2013, inspections were performed regardless of the ACMS alert, but the damage was not identified (see hazard $H_{20}$). According to the investigation of this incident in 2013, the other contributing factors were the bad meteorological conditions during the outdoor inspection, and the use of inspection procedures that were not consistent with the aircraft maintenance manual [33].

These incidents show that the parameters and algorithms used for ACMS need to be updated continuously based on the actual operation data in order to properly identify hard landing or other abnormal events (see hazard $H_{07}$). In addition, the inspections carried out by mechanics need to be performed carefully, especially when there is a conflict between reports submitted by flight crews and aircraft condition monitoring systems (see hazard $H_{20}$).

4.3. Unidentified Damage due to Incomprehensible Data Presentation

In 2016, a helicopter lost its yaw control during landing [35]. The helicopter has in place the Health and Usage Monitoring System (HUMS), which monitors the condition parameters such as engine vibration, rotor track balance, engine shaft balance, etc (HUMS performs the role of ACMS for aircraft in Figure 1). One day before the incident (Day $-1$), during flight, HUMS recorded vibration data, including a series of exceedences related to the tail rotor pitch change shaft (TRPCS) bearing. In the routine maintenance following this flight, the HUMS data were downloaded and analyzed. During the analysis, an abnormality for the tail rotor gear box bearing was detected, but the exceedence was not identified. During the first flight of the day of the incident (Day 0), the HUMS recorded further exceedence. However, it was planned to download and analyze the data only after the helicopter returns to the base. During the lift-off of the second flight on Day 0, the helicopter went through an uncommanded yaw. However, this was regarded as the influence of the wind on the helicopter. During landing of the same flight, the helicopter totally lost yaw control and landed expeditiously and heavily. The root cause of the lost yaw control was identified as the damage on the TRPCS caused by the failed bearing. The following two contributing factors were discussed in the investigation report [35]:

Impending failure of the TRPCS bearing was detected by HUMS but was not identified during routine maintenance due to human performance limitations and the design of the HUMS Ground Station Human Machine Interface.

The HUMS Ground Station software in use at the time had a previously-unidentified and undocumented anomaly in the way that data could be viewed by maintenance personnel. The method for viewing data recommended in the manufacturer’s user guide was not always used by maintenance personnel.

For this incident, we identify the hazards related to the unclear communication (see hazards $H_{09}$, $H_{11}$, $H_{12}$, and $H_{13}$), and the delayed data/information sharing (see hazard $H_{18}$). The damage to TRPCS was properly detected by the HUMS before the incident, but this was not identified and resolved by the operator (The operator is performing the role of TG in Figure 1) (see hazards $H_{11}$, $H_{12}$, and $H_{13}$). The first contributing factor was the design of the HUMS Ground Station Human Machine Interface. The information available through this interface needs to be zoomed in to identify the exceedence (see Figure 3), but the two engineers did not address this (see hazard $H_{09}$, $H_{11}$ and $H_{12}$). As a result, a proper inspection was not conducted (see hazard $H_{13}$). In addition, the HUMS data were not shared online, rather the storage card was supposed to be brought back to the base. Thus, the exceedence recorded during the first flight was not reported (see hazard $H_{04}$). Moreover, the global support team who received the HUMS data of the previous day (Day $-1$) identified the exceedence and contacted the operator (The global support team performs as DM and TG in Figure 1). However, the communication was not completed on time (see hazards $H_{04}$ and $H_{18}$) as the incident already occurred by the time the support team transmitted their report.

This case shows the importance of the digital communication platform for data-driven PdAM. The digital platform should visualize the data in an intuitive manner and highlight crucial information to prevent hazards such as hazards $H_{09}$, $H_{11}$, $H_{12}$, and $H_{13}$. In addition, online data sharing is needed to prepare necessary maintenance tasks in advance (see hazard $H_{18}$).

With the analysis above, we validate the hazard list identified during the brainstorming session by revealing similar hazards encountered for actual incidents.

5. Emerging Challenges of Predictive Aircraft Maintenance

In the context of the identified hazards of data-driven PdAM, we discuss the three main challenges. In Figure 4, we group the hazards based on the associated maintenance agents, and mark each hazard based on the associated emerging challenges.

5.1. Reliability of New Technologies

The biggest challenge is to guarantee the reliability of new technologies introduced in data-driven PdAM, e.g., aircraft condition monitoring systems (ACMS), diagnostics and prognostics algorithms, and decision support systems of PdAM. Thus, 9 out of 20 hazards are related to the reliability of new technologies (see hazards $H_{01}$, $H_{02}$, $H_{03}$, $H_{04}$, $H_{05}$, $H_{06}$, $H_{07}$, $H_{08}$, and $H_{10}$). The majority of the maintenance experts perceive the low reliability of diagnostics and prognostics algorithms as a main trigger for most of the hazards associated with data-driven PdAM. Therefore, it is recommended to test the data-driven diagnostic and prognostic algorithms using multiple operational data sets. After all, adequate approval procedures for the design and implementation of data-driven PdAM is needed.

5.2. Communication between the Maintenance Agents

The second challenge is related to communication between the maintenance agents, which is related to 5 out of 20 hazards (see hazards $H_{09}$, $H_{11}$, $H_{12}$, $H_{13}$, and $H_{17}$). In this light, the maintenance experts emphasize the need for an intuitive and effective digital platform to support timely communication at all levels of the data-driven PdAM. Interactive user interfaces and informative visualizations are seen as a means to avoid missed alerts [36]. However, not enough studies discuss user interfaces on aircraft maintenance, although intensive studies are made for other data-driven technologies, such as self-driving cars [37]. Only a few studies discuss the user interface supporting aircraft maintenance tasks (agent ME) [38,39]. In addition, data-driven algorithms also need further improvement in the explainability and interoperability of their prediction results [40]. Overall, further investigation is necessary to improve the effectiveness of communication between all maintenance agents, especially agent TG who is associated with the most number of hazards related to communication (see Table 5 and Figure 4).

5.3. Trust of the Maintenance Agents

A third challenge for data-driven PdAM is to build the trust of the maintenance agents in the new data-driven technologies, which is related to 3 out of 20 hazards (see hazards $H_{14}$, $H_{15}$, and $H_{20}$). The trust in a new technology is based on more than just having systems and algorithms of high accuracy [28]. In fact, trust is equally based on users’ personal cognition on the reputation of these new technologies (cognitive trusting base), their understanding that these new technologies benefit them (calculative trusting base), and their confidence in the human operators behind these new technologies (institutional trusting base) [28]. For the case of aircraft maintenance, the process is even more complex, with multiple agents who use different data-driven technologies locally and who interact with each other at the system level. Therefore, we should build trust both at the level of individual agents, as well as at system-level. At the individual level, trust needs to be built between each agent and the new technologies that they use. At the system level, trust needs to be built in the information transferred from one agent to another.

6. Conclusions

In this paper, we identify hazards associated with the introduction of data-driven predictive aircraft maintenance (PdAM), and discuss the emerging challenges of implementing data-driven PdAM. As a first step, the main agents of data-driven PdAM and their interactions are recognized. Then, a structured brainstorming for hazard identification is conducted with aircraft maintenance experts, each representing one of the maintenance agents. We focus on the emerging hazards associated with the adoption of new technologies, such as aircraft condition monitoring systems (ACMS), data-driven diagnostics and prognostics algorithms, and decision support systems for PdAM. As a result, 20 emerging hazards are uncovered for data-driven PdAM. Two agents, the data management team and task generating team, are associated with the largest number of new hazards of data-driven PdAM. These hazards are validated in the context of past aircraft incidents that occurred between 2008 and 2013.

Following the analysis of the hazards, we discuss three main challenges for safe implementation of data-driven PdAM: (i) guaranteeing the reliability of new data-driven technologies of PdAM, (ii) designing intuitive communication platforms that can facilitate communication between agents under PdAM, and (iii) building the agent’s trust in the new data-driven PdAM process. These challenges guide the future research direction for the successful implementation of data-driven PdAM.