Deep Learning-Driven Automated Fault Detection and Diagnostics Based on a Contextual Environment: A Case Study of HVAC System

Abstract

Indoor thermal comfort affects occupants’ daily activities and health. HVAC systems are necessary to control thermal comfort quality. Tracking and monitoring the effectiveness of HVAC system engines are critical activities because they ensure that the system can produce suitable indoor thermal comfort. However, the operation of such systems depends on practitioners and engineers, which is time-consuming and labor-intensive. Moreover, installing physical sensors into the system engine may keep track of the problem but may also require costs and maintenance. This research addressed this concern by presenting deep learning (DL)-driven automated fault detection and diagnostics (AFDD) for HVAC systems. It employed contextual factors as an indirect measurement to avoid modifying HVAC system engines (e.g., according to standard building appliance warranties) but was still able to effectively detect issues. The design and development of the DL model are proposed to encode complex behaviors of an HVAC system using contextual factors. The experimental results show that the predictive performance of our model achieved an average F-measure of over 97%, which was outstanding compared with the standard ML models. This proposed model will be a natural fit for AFDD for HVAC systems and is ready for future real-world applications as required by building engineering.

1. Introduction

HVAC systems (heating, ventilation, and air conditioning) play essential roles in controlling the thermal comfort quality of indoor environments because 90% of people spend their time in buildings [1]. HVAC systems need proper maintenance to help them work efficiently. Improper maintenance does not always cause HVAC systems to visibly break down, but it can reduce their performance. Uncomfortable environments may affect life quality, such as sleep activity [2,3] and learning activity [4,5], factors that indicate well-being and good health. Thus, problems in HVAC systems can lead to high costs in long-term situations. The regular maintenance for site-specific systems requires engineers and practitioners to understand how HVAC systems work. However, monitoring and checking up on real-world problems is labor-intensive and time-consuming; engineers and practitioners may be insufficient.

Automated fault detection and diagnostics (AFDD) is a software agent technology designed to overcome these limitations. It automatically monitors and investigates HVAC system equipment issues that can reduce performance, known as fault events [6]. AFDD employs multidisciplinary processes—for example, a review of perceiving environmental signals using IoT-based sensors [7], identifying essential features, such as noise detection, filter block, and anomalies in energy consumption [8,9], detecting failures [10], and determining possible causes [11]. It acts based on human-like decisions, similar to engineers and practitioners, to alleviate labor-intensive and time-consuming tasks. AFDD is expected to be a part of future commercial products in building engineering, as it will help to reduce the limitations of experts and meet demands for HVAC system maintenance.

In the past two years, researchers have paid attention to the perception process, based on IoT, that lets systems thoroughly sense environmental signals and detect faults from the signals based on machine learning (ML).

IoT simulates human-like perceptions that allow software agents to recognize how events in environments occur. Valinejadshoubi et al. [12] and Karthick et al. [13] proposed developments of IoT to help software agents solve indoor environmental problems. They noted that well-designed perception components could help software agents detect and diagnose issues. IoT is a fundamental principle that lets software agents manage thermal comfort in a building by sensing environmental signals through human-like perceptions [14]. Liu et al. [15] observed that recent applications employed multi-sensors to perceive environments, as they were the first component in systems. They also added that multi-sensors helped software agents measure fault events with consistent, accurate, and reliable outcomes, which ML models could utilize in the learning process [6].

ML imitates human-like intelligence to learn how to detect and diagnose faults, and it automatically improves its performance given more experience through IoT signals. Yang et al. [16] and Yun et al. [17] proposed an ML-based approach to detect and diagnose faults in HVAC systems. They noted that it helped practitioners understand system failures early. Modern building designs and engineering require advanced HVAC systems to effectively provide comfort to indoor areas. Traditional ML-based AFDD may not detect and diagnose issues properly. Advanced ML is needed to address problems, and deep learning (DL) is a subset of ML that addresses such complex tasks. Xiao et al. [18] and Wei et al. [19] proposed data-driven approaches based on DL for AFDD. They wrote that DL was a powerful tool for addressing multidimensional data and that it could reach a high degree of accuracy in predictive performance. By adding advanced architecture, DL can significantly recognize patterns using multi-sensors that provide covered dimensional data points. However, the accretion of more sensors in commercial sectors increases costs as monitoring HVAC systems requires expensive sensor models [20]. Moreover, life-cycle costs such as maintenance, repair, and replacement are required, raising more issues in commercial sectors. Ma et al. [21] studied the relevant thermal comfort variables and observed that measuring the right factors was a solution to address these limitations.

Contextual fault detecting and diagnosing factors aim to measure indirect preferences of HVAC system factors based on their relevant environments. Notably, the fault detection method we focused on in this study concerned abnormalities based on power consumption and cooling capacity. The detection method observed correlated factors that measured defects inside HVAC systems and was equally likely to detect and diagnose the issues as direct measurement. The goals were twofold; to avoid remodeling an engine based on the warranty terms and conditions of the HVAC system and to reduce the costs of physical sensor installations. However, indirect observations may have lacked some details. Proper factors would be needed to determine such hidden information, and standard ML could not address this concern. Cooperation between contextual factors, well-designed measurement, and DL can be challenging. This study proposed a DL-based approach for AFDD to measure HVAC systems based on contextual factors. The main contributions of this study were as follows:

• To determine how AFDD employs contextual factors to support indirect measurement;
• To design and develop a DL-based AFDD model in the context of HVAC systems;
• To prove that the DL-based AFDD model based on contextual factors is suitable for HVAC systems.

Human-like intelligence and AFDD, contextual observation based on IoT, building engineering, contextual environment, and DL were examined in Section 2. The fault events and contextual factors were proposed in Section 3. In Section 4, we detailed the design and development of the DL model. In Section 5, experiments and analyses of how a DL-based contextual environment could help AFDD were laid out, and conclusions were made in Section 6.

2. Background Knowledge and Related Works

This section considered the relationships between human-like intelligence and AFDD and reviewed the current application of IoT-based context observation. Building engineering, contextual environment, and DL were investigated to discuss the challenges of applying our proposed approach.

2.1. Human-like Intelligence and AFDD

The motivation behind this research was to imitate physician-like prognosis of diseases. Physicians do not directly confirm conditions but intelligently analyze signs and symptoms to consider possible diseases in a process called context measurement. For example, the characteristics and symptoms of coronavirus disease 2019 (COVID-19) can be observed through various symptoms such as fever, cough, loss of taste or smell, and sore throat. Co-occurring disorders that may appear with two or more symptoms may help physicians rapidly diagnose and identify suspected cases. In this way, they may call for advanced testing of clinical specimens of a virus and check for respiratory tract infections that can be treated on time in order to prevent severe disease and death. Physicians utilize contextual information to estimate suspect cases. These do not always require complete data, such as advanced lab testing, which is expensive and time-consuming. AFDD may benefit from this intelligent manner of addressing issues if applied in HVAC systems.

We employed this intelligent understanding to observe environmental features around HVAC systems as symptoms and diagnose their issues as diseases. In other words, a measurement of indoor thermal comfort or power consumption is a sign of HVAC systems. It is an excellent indoor factor. HVAC system issues can thus be considered disease that engineers and practitioners could determine through the observation and analysis of symptoms. Moreover, signs may distinctly appear due to a disease complication, such as a bad environment (e.g., high temperature and humidity) and significantly co-influence the performance of HVAC systems. This understanding motivated us to propose a design for a DL-based model for AFDD based on a contextual environment that employed fewer physical sensors but still provided high predictive performance. The following section reviewed how the IoT perceived relevant signals as a source of symptom detection.

2.2. Context Observation Based on the Internet of Things

Sensor technologies unlock limitations of human perceptions, which roughly measure environments rather than obtain exact values. For instance, occupants could feel uncomfortable but be unable to measure exact temperature and humidity percentages—rendering them unable to confidently confirm the cause of their discomfort. This requires an expert to use an advanced measuring instrument to observe and measure such contextual factors. The exponential progress of IoT has led to relevant sensors that can observe thermal comfort issues, along with intelligent systems to automatically transfer, analyze, and interact with occupants [22].

Many researchers have proposed intelligent system ideas to support HVAC systems. Pawar et al. [23] proposed an IoT-based system for supporting building management, and claimed that advancements in IoT and sensors could help engineers deal with building issues. Guannan et al. [24] used sensor technologies to observe the impact of indoor and outdoor events in different environments. They summarized that sensors help engineers identify HVAC system issues. Tanasiev et al. [25] developed an IoT-based approach for the monitoring and tracking of HVAC systems. They determined that advanced IoT technologies help engineers control HVAC systems with a greater degree of flexibility. Metwally et al. [26] proposed an IoT-based technique for monitoring outdoor environments that could affect indoor thermal comforts. They confirmed that the utilization of IoT helped engineers to more efficiently deal with indoor thermal comfort issues.

Recent research has intensively proposed approaches for monitoring and detecting indoor events. However, these have focused on direct observations from complete sensor installations in HVAC engines and employed them to identify faults. Utilizing sensor settings based on contextual factors could uncover hidden patterns in the data. However, this represents a major challenge, because it would require advanced computing technologies to process and discover such defects. In the following section, we reviewed how computing technologies could be utilized to solve complex problems in HVAC systems.

2.3. Building Engineering, Contextual Factors, and Deep Learning

Context factors play a critical role in AFDD for building engineering, and discovering building information from sensor data requires advanced technology integrations such as IoT and ML [27]. ML models, such as AFDD for HVAC systems, use experience, such as historical data, to solve problems and improve performance, given more background [28]. These models help AFDD identify issues in buildings related to thermal comfort and are a popular research topic.

Li et al. [29] employed ML to extract contextual information using sensor data, claiming that ML helped AFDD improve the performance of the information extraction process. Xiong et al. [30] used ML for thermal building comfort predictions, adaptive to individual participants. They observed that ML helps systems achieve automatic function in indoor thermal comfort. Qavidel Fard et al. [31] reviewed the recent ML applications in the building engineering domain. They summarized that support vector machines, artificial neural networks, and ensemble learning were well-known techniques in building applications. Current research aimed to address problems based on direct observations that focused on a particular aspect of contextual factors in the building. They employed complete data to fit basic ML algorithms, producing specific events (e.g., indoor, outdoor, or HVAC systems). This did not meet our standard assumption, which aimed to employ indirect measurement—and required advanced ML to address problems.

Deep learning (DL) is a sub-concept of ML, an algorithm invented to address complex patterns [32]. Akinosho et al. [33] reviewed the roles of DL in building engineering and application. They observed that DL-based AFDD for HVAC systems would be in demand for research, as future buildings would be complex, with dynamic environments. Zhu et al. [34] and Lee et al. [35] proposed DL-based AFDD approaches for HVAC systems and claimed that DL-based systems could identify HVAC system issues with a high degree of accuracy. However, their research required them to disassemble HVAC engines and install relevant sensors in the systems used. This required engineers who understood the systematic processes of HVAC equipment—which would be difficult for untrained building occupants.

Computing-based contextual factors present a solution for addressing these challenges based on indirect measurement. The method computes relevant ambience data surrounding HVAC systems that explain how they work. Relevant ambient-based factors can be utilized with a new approach to discover and detect abnormal events in HVAC systems. DL is a powerful tool that could be used to encode factors that identify issues in engineer-like intelligence.

In conclusion, this research proposed two main arguments for DL-powered computing in building engineering based on contextual factors. Context was employed to model HVAC systems knowledge based on human-like understanding, and DL was used to address complex factors beyond traditional approaches.

3. Fault Events and Contextual Factors

Faults in HVAC systems cause discomfort in indoor environments. They can be of different types, e.g., airflow problems, refrigerant leaks, and device malfunctions [36]. They affect indoor thermal status and comfort, potentially keeping occupants from resting or desired activities. For example, HVAC system problems can cause indoor environments to be dry, humid, or uncomfortable (e.g., too hot or cold). Moreover, HVAC systems may be defective due to severe outdoor environments, such as temperature or humidity fluctuation [37]. Therefore, the effectiveness of fault detection and diagnostics for HVAC systems depends on contextual factors.

Engineers employ contextual factors to detect faults at early stages and do not directly disassemble HVAC systems to check for problems. They observe relevant environmental factors and infer how HVAC systems could be producing faults. We simplified the relationship between fault events in HVAC systems and their contextual factors, as shown in Figure 1.

Figure 1 illustrates the architecture of the HVAC system, connecting relevant factors. Number one (1) represents the indoor environment, and number two (2) represents the outdoor environment; the two are connected by the HVAC system.

The outdoor environment (2) is dynamic and could lead to discomfort for occupants due to extreme temperatures, affecting daily activity and health. It could transfer outdoor factors, such as temperature and humidity, indoors. The building’s wall could prevent heat transfer, but only temporarily, and certainly not for a long-term season. The HVAC system essentially engages in a heat transfer tradeoff between the indoor and outdoor environments in order to provide comfort to the occupants. It plays an essential role in protecting the indoor environment against the outdoors.

The HVAC system, indoor factors, and outdoor factors all depend on each other. Therefore, measuring indoor and outdoor characteristics could predict the behavior of the HVAC system. For example, outdoor environments during summer can be oppressively hot, while the indoors are kept comfortable. This means that the HVAC system in the building is working effectively. In contrast, if the HVAC system is working, but the indoor environment is as uncomfortable as the outdoors, this implies that the heat transfer process in the HVAC system is not working correctly. The three aspects are correlated, and can provide context for one another to describe the observed phenomena.

AFDD should benefit from contextual factors to infer fault events in HVAC systems. Diagnosis of the fault must measure the relevant factors from physical phenomena and transform them into digital signals. The summarization of factor measurement is shown in

Table 1. The summarization of the HVAC system and its context.

Aspect	Sensor and Factor	Description
Outdoor environment	Outdoor (1) temperature and (2) humidity, assigned by the universe, using AM2315.	They are critical factors ruled by the universe that affect human comforts.
Indoor environment	Indoor (3) temperature and (4) humidity affecting human comfort, using DHT22.	According to outdoor environments, these are directly changed over time, i.e., converging.
HVAC system	(5) Electric current consumed by HVAC system engines, using SCT-013.	This is a primary key indicating the status of the HVAC system engine.
Timestamp	(6) Time of event occurring, produced by ESP32.	Time period is an essential factor indicating HVAC system behaviors.
System behavior	The functioning status of the HVAC system engine (e.g., inactive, normal, defective, or failing).	This requires the above factors in order to determine system behaviors (labeled by technicians and engineers).

.

Table 1 represents the measurable factors, (1) to (5), perceived by physical sensors, as well as measurable factors (6) defined by microcontroller-based software. These are relevant to predicted outcomes (the status of the HVAC system engine) that technicians and engineers employ to investigate whether the HVAC system is inactive, normal, defective, or failing. These statuses reflect the system behavior, in which inactive means the systems are off, normal means the system is functioning properly, and defective means there is something wrong in the system, such as a delay in temperature reduction or a frozen AC. Failing means that the systems cannot control temperature. They can be read as physical signals and modeled into the digital formats of random variable states.

The five measurement aspects are fundamental sources that engineers and technicians employ to predict the function of an HVAC system engine (e.g., outcomes). We observed that that none of the factors were directly installed inside the HVAC system, but employed ambient information to interpret HVAC system statuses.

The following section demonstrates how the system was able to imitate human-like intelligence to operate the factors and predict the results.

4. Contextual-Based Deep Learning Framework

The DL-driven system was designed to address complex problems. It required a well-designed architecture to allow it to consume raw signals and transform them into explicit knowledge. Thus, there were two main proposals herein: (1) an overview of the architecture of our proposed system, and (2) the design and development of the DL model architecture.

4.1. Overview Architecture of Proposed System

An overview system illustrated the AFDD process from beginning to end, sensing signals from the real-world environment until it was able to predict the status of HVAC systems. The conceptual approach of our proposal was founded on machine learning-based analytics [37,38]. It consisted of four components: (1) signal gathering, (2) signal preprocessing, (3) model learning and predicting, and (4) model evaluating and visualizing.

As shown in Figure 2, the system can be divided into two parts: (1) the environment and (2) the system processing. Using sensor technologies allowed the system to encode the environment’s physical factors (see Figure 1) and transform them into digital formats. Our approach took up the available prototype, relevant to three aspects proposed by Sahoh et al. [22], which transferred signals from the environment to the system processing.

The system processing had two parallel subprocesses; above, the dashed-line components represent the learning process of the ML model, and the solid-line components represent the application process. The data gathering acted as human perception organs to sense raw signals such as indoor and outdoor temperature and humidity and observe the HVAC system based on electric currents. The data preprocessing encoded raw data into semantic features that technicians and engineers could understand. For example, temperature, humidity, and electric current could be represented as degrees Celsius, percentage of relative humidity, and ampere, respectively. Our study used the existing data gathering and preprocessing components proposed by Yun et al. [17] and Wei et al. [19]. They produced the main features to input into the model learning and prediction components.

Model learning and prediction were our main contributions. The model imitated human-like thinking to recognize features and decide on actions to express to occupants. In other words, the performance of the systems depended on the effectiveness of the model learning and predictions—which co-considered contextual information to understand the status of HVAC systems. The predicted outcomes from the model could help occupants and engineers detect problems at early stages and prevent them from worsening.

This suggested that the design and development of the model could play an essential role in AFDD for HVAC systems. In the following section, we proposed an architecture of ML for AFDD and the methodology of the model development.

4.2. Deep Learning

Deep learning (DL) aims to discover essential features from unstructured input [38]. It lets systems automatically learn complex issues by connecting and separating two layers (low level) and adding the top, layer-by-layer, to model knowledge (high level). It is able to address AFDD for HVAC systems, a complex problem with multiple relevant factors from various environments. We defined these as context. DL inherited an artificial neural network (ANN) learning process that needed three main elements: (1) architectures (input layer, hidden layer, and output layer), (2) activation functions (a neuron of nonlinearity encoding each layer $X_n^{layer}$∈ ℝ^Dimention), and (3) learning rules (automatic error correction based on loss function). The structural connections of neurons between two layers can be represented as follows:

$$y_j^{ l^n} =\sum (w_{ij}^{ l^n}\times {Neuron}_i^{ l^{n-1}})+b_i^{ l^n},w^{ l^n}\in {ℝ}^{n^{ l^{n-1}}\times n^{ l^n}}$$

(1)

where w_ij is weight as input of neurons between layers (l) (e.g., Neuron_i of l ⁿ⁻¹ and Neuron_j of l ⁿ) to determine whether they should connect or separate, b is a bias to define neuron output behaviors (positive or negative). ${Neuron}_i^{ l^{n-1}}$ represents the value encoded by the activation function (a) as follows:

$$a_i^{ l^{n-1}}={{\displaystyle \int }}_i^{ l^{n-1}}{(y}_i^{ l^{n-1}})$$

(2)

The a lets DL encode nonlinearity between features and target classes and models the context based on the complex curve. The number of hidden layers may vary based on the problems. Thus, DL is suitable for the issues of various factors in AFDD for HVAC systems. Many hidden layers and neurons may help the model recognize HVAC system behaviors in dynamic environments, but computational costs are high.

Backpropagation is an algorithm of learning rules to control errors in the architecture. It aims to minimize the errors of each parameter (weight and bias) in the networks of DL by comparing the desired outcome from engineers and the predicted outcome from DL. It computes the loss and readjusts the parameters based on the chain rule algorithm to update backward from outputs. It helps measure the balance between some neurons, hidden layers, and predictive performances. The loss can be calculated as follows:

$$\theta =ArgMin\frac{1}{n}{{\displaystyle \sum }}_{i=1}^{n}L({predictedoutcome}_i{,desiredoutcome}_i)$$

(3)

where L is a loss function of an overall sample n. Backpropagation and chain rule algorithms may take time to converge to optimal parameters when extensively training data from various contexts, like AFDD for HVAC systems. We employed stochastic gradient descent (SGD) as an optimizer to search for the excellent θ as follows:

$${\theta }_{t+1}{=\theta }_t-\mathsf{\eta }{\nabla }_{{\theta }_t}{L}_{\mathrm{ce}}{(predictedoutcome}_i{,desiredoutcome}_i)$$

(4)

where L_ce is the cross-entropy loss function, and η is a learning rate to define step size automatically. L_ce is computed by

$$L_{\mathrm{ce}}=-{{\displaystyle \sum }}_I{desiredoutcome}_i{\log \left(s\right(predictedoutcome}_i\left)\right)$$

(5)

The s (predicted outcome_i) is based on the softmax function, computed as follows:

$${s(predictedoutcome}_i)=\frac{e^{{outcome}_i}}{{{\displaystyle \sum }}_{n=1}^{outcomes}{e}^{{outcome}_n}}$$

(6)

SGD, based on L_ce, is good at imbalanced categorical training data and redundancy transactions; it speeds up the backpropagation process [39].

In the following section, we covered the design of the DL model, based on the fundamental concepts (architectures, activation functions, and learning rules) of AFDD for HVAC systems, which were in turn based on contextual factors.

4.3. The Deep Learning Model Development

It is a challenge to design and develop DL architecture to achieve the best performance and lowest computation possible, given various contexts. We collected 147,740 events between 16 June and 19 August 2021. Engineers labeled each event relevant to the context. We employed an indoor environment setting (room size) based on 3.2 M × 6 M × 12 M, consisting of six LED light panels, one LED television, and one personal computer. We set the on/off system-based air conditioner with 30,000 BTU (British Thermal Unit), consuming a maximum power of around 11 Amp per hour (according to the product label) to 25 °C. Three subjects (wearing standard uniforms to avoid clothing variations) lived daily in the setting space during office hours (9.00 AM–5.00 PM). No objects were moved in or out of the setting space during data collecting, ensuring no bias during the experiment.

The duration of the study was summertime since it was dynamic and challenging to encode. In this way, the model of HVAC systems during the summertime was able to predict the rest of the desired events (which were less complex). The HVAC systems required training data from various environments to help model more generalizations when applied to real-world problems. We split the collected transactions into training and testing datasets in a ratio of 50:50.

The input layer of our architecture consisted of six factors from contextual environments (see Table 1). The output layer consisted of four aspects of AFDD for HVAC systems (inactive, normal, defective, and failing) concerned with how engineers and practitioners should respond to the events. Our architecture employed ReLU (Rectified Linear Unit) because it was able to address gradient vanishing (compared with hyperbolic tangent and logistic functions) and reduce computation time when the training data were extensive sensor-based signals from HVAC problems.

DL’s most challenging design and development involves determining hidden layers and neurons. Inappropriate numbers of these can cause high computation times and produce useless information. For example, enormous numbers of hidden layers and neurons can cause model overfitting but cannot predict unknown events in a real-world environment. Therefore, this study aimed to determine the number of hidden layers and neurons for HVAC systems in the context of AFDD. We used two regularization techniques to avoid overfilling: dropout and batch normalization, which were able to alleviate the limitation of a particular training dataset.

There is no general rule to decide the number of hidden layers and neurons in DL. We iteratively tuned the configuration by starting from a simple structure and checking for errors in training and validation. Finally, the results show that 100–1000 neurons and two to five hidden layers provided the significant results.

Pytorch is a deep learning model framework [40] that can be built on DL architectures and set regularization, activation functions, and learning rules. Scikit-learn is an efficient framework for predictive analysis [41] that can be used to measure the model performance. They can be implemented in the Python ecosystem to support model design and development.

The SGD optimization was set where the learning rate was 0.001 and momentum was 0.9, as recommended by Cheng et al. [42]. The dropout was set to 0.5 as a generalization, and the learning rate (η) was 0.001, to avoid overfitting. The Pytorch computing employed a GPU-accelerated application based on CUDA (NVIDIA Quadro RTX 4000 model) to learn and readjust DL parameters. We employed the accuracy of model prediction as an evaluation matrix as follows:

$$\mathrm{accuracy}=\frac{(\mathrm{TP}+\mathrm{TN})}{(\mathrm{TP}+\mathrm{FP}+\mathrm{FN}+\mathrm{TN})}$$

(7)

where true positive (TP) means a correct result that was predicted. True negatives (TNs) reflect an accurate result of the undesired outcome that was not predicted. False positive (FP) is an incorrect result that was predicted. False negative (FN) is a factual result that was not predicted.

We designed the architecture of DL, beginning with two hidden layers and up to five hidden layers, and from 100 neurons up to 1000 neurons, respectively. The accuracy of each learning process was recorded based on 10-fold cross-validation, and we repeated it five times to ensure that the accuracy was not random. We considered each DL architecture’s accuracy using minimum, mean, and maximum to choose the best-fit model for AFDD for the HVAC system. The results are shown in Figure 3.

As shown in Figure 3, three points on the diagram were considered to select DL architecture based on the minimum, mean, and maximum accuracy. In the figure, the white point is the average of the lowest results, the blue point is the overall results, and the gray point is the highest. The range between white and gray showed the consistency of model predictions; if the range were balanced and narrow, it could be interpreted that the model effectively fit all folds in the experiments.

Figure 3a represents the two hidden layers of the architecture in different numbers of neurons. The best-fit model of architecture was 600 neurons because it reached a rather high mean accuracy (94.98%) with low variance (minimum and maximum were 93.51% and 96.11%, respectively). The numbers of neurons higher than 600 did not improve mean accuracy significantly, and lower than 600 led to poor predictive performance since the mean was wide and the range between minimum and maximum was high. We considered the architecture to be under-fit with AFDD for HVAC systems.

Figure 3d displays the predictive performances of five hidden layers. The results show that the means of each architecture were lower than 94.98%, compared to two hidden layers, as shown in Figure 3a. This suggested that using five hidden layers was not suitable for AFDD for HVAC systems because, the more hidden layers and neurons were added, the more the model architecture was over-fit to training datasets, while remaining unable to handle unknown situations given by the 10-fold cross-validation.

Figure 3b,c represent three and four hidden layers of architecture, respectively. These reasonably produced predictive performances that reached 95% accuracy over, given 800 neurons. In this way, we chose four hidden layers of architecture. Even though the increased number of hidden layers caused a high computational cost, it encoded abstract features in the AFDD for HVAC systems that could handle unknown events in the future.

Therefore, we simplified our DL model architecture of AFDD for HVAC systems based on the graphical diagram shown in Figure 4.

Figure 4 shows that the input layer consisted of six variables: $X_n^{inputlayer}$ = {X_{indoor, temp}, X_{indoor, humid}, X_{outdoor, temp}, X_{outdoor, humid}, X_amp, X_time}. The architecture ran from the left to the right, which is called forward propagation. The edge flowed through the hidden layers from the input to the output layer. The hidden layers consisted of four sub-layers, each composed of 800 neurons. Finally, the output layer consisted of four events of AFDD for HVAC systems derived from hidden layers. The directed bold edge represents the value (v) of the connection between hidden layer² (${hd}^2$) and hidden layer³ (${hd}^3$) that is computed by $v_j^{h{d}^3}$= ∑($w_{ij}^{h{d}^3}$× ${Neuron}_i^{h{d}^2}$) + $b_i^{h{d}^3}$ where $w^{h{d}^3}$ ∈ ${ℝ}^{n^{h{d}^2}\times n^{h{d}^3}}$. The w indicates the connection and separation between input and output layers. The b is the slope position to achieve the best boundary between input and output layers. The ${Neuron}_i^{h{d}^2}$ represents the value encoded by the activation function (a) where $a_i^{h{d}^2}$ = ${{\displaystyle \int }}_i^{h{d}^2}(v_i^{h{d}^2})$.

The accuracy in Figure 3 shows the overall model performance, which reached over 95%. It summarized that DL-driven AFDD, based on contextual factors, could encode the HVAC system patterns without breaking the HVAC engine. However, accuracy did not reflect significant characteristics of each class (e.g., in the case of rare events). In the following section, we employed this architecture to test and prove how our model performed outstanding predictions compared with traditional machine learning models.

5. Experimental Setup

The experiment evaluated the predictive performance of the proposed model and whether it reached the robustness required in the rarest events to support AFDD for HVAC systems.

5.1. Experimental Objectives

We wished to illustrate how our proposed model could help AFDD for HVAC systems address infrequent events and have expansive effects, e.g., fault and failure. Given the contextual factors, the objective was to measure the predictive performance of the HVAC system’s behaviors (e.g., inactive, normal, defective, or failing). We compared the predictive performance of each behavior with the standard ML algorithms, consisting of three approaches: geometry (k-nearest neighbors: KNN), probability (naïve Bayes: NB), and logical expression (decision tree: DT). We further evaluated the ANN as a fundamental of DL to highlight why our proposal was required to fulfill research contributions. The experimental setup is shown in Figure 5.

Figure 5 shows the procedure of the experiment, which consisted of five processes. The environment setup was set (according to Section 4.3) to observe the contextual factors of HVAC systems. Then, the data gathering retrieved the raw data and sent them to data preprocessing for the extraction of contextual factors. The preprocessed data were utilized as training data for the standard ML models and our proposed DL-based model. While our proposed DL-based model was developed using settings from Section 4.3, the standard KL models were implemented using a well-known Python library—scikit-learn [41]. It is an efficient machine learning library supporting the building of basic ML models with default and calibrated parameters. For example, the KNN model was set with five neighbors and uniform weights. The NB model was set as Gaussian NB with no prior probabilities. The DT model was set with Gini criterion and no maximum depth of the tree. Additionally, the ANN was set with ReLU as the activation function, SGD as the optimization, 0.001 learning rate, and 400 hidden layers. The models were used in this experiment for the predictive performance evaluation.

5.2. Model Testing Metrics

We employed three evaluation metrics to measure the model based on each class’s significant characteristics: precision, recall, and F-measure. Precision (PS) computed a probability of correctly predicted classes against total predicted classes; recall (RC) measured the probability of correctly predicted classes against actual classes; F-measure (F1) figured out an optimal point between PS and RC. Each of the measurement metrics could be considered as follows:

$$\mathrm{PS}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}$$

(8)

$$\mathrm{RC}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}$$

(9)

$$\mathrm{F}1=\frac{2\times (\mathrm{Precision}\times \mathrm{Recall})}{\mathrm{Precision}+\mathrm{Recall}}$$

(10)

The three metrics employed TP, TF, and FN, similar to the accuracy matrices for which we defined semantic meanings in Section 4.3.

5.3. Dataset Description

We derived the testing dataset from Section 4.3 and represented the proportional ratio of each class using a pie chart that was able to determine the distribution, as shown in Figure 6.

Figure 6 illustrates the severely imbalanced dataset of HVAC system behaviors. The failing events were the rarest at 5.62%. Defective events reached 18.21%, which was an imbalance compared with normal and inactive events. This is a common problem in HVAC system behaviors; our proposed model was needed to handle the prediction of HVAC system behaviors.

5.4. Results and Discussions

Measurements focused on the model’s predictive performance dealing with imbalanced events of HVAC system behaviors. The models (DT, KNN, NB, ANN, and our proposed DL) were considered by average PS, RC, and F1. We measured predictive performances focused on four events: (1) inactive, (2) normal, (3) defective, and (4) failing. The results from the models are given in

Table 2. The comparative effectiveness of five ML models.

Model	DT			KNN			NB			ANN			DL
Class\Matrices	PS	RC	F1	PS	RC	F1	PS	RC	F1	PS	RC	F1	PS	RC	F1
Inactive	0.97	0.95	0.96	0.95	0.96	0.96	0.60	0.77	0.68	1.00	1.00	1.00	1.00	1.00	1.00
Failing	0.87	0.85	0.86	0.88	0.82	0.85	0.81	0.54	0.64	0.78	0.78	0.77	0.90	1.00	0.95
Defective	0.78	0.81	0.79	0.81	0.80	0.81	0.41	0.69	0.52	0.81	0.76	0.79	0.93	0.92	0.93
Normal	0.89	0.90	0.90	0.90	0.90	0.90	0.44	0.19	0.27	0.91	0.92	0.91	1.00	0.97	0.98
Average	0.88	0.88	0.88	0.89	0.87	0.88	0.57	0.55	0.53	0.88	0.87	0.87	0.96	0.97	0.97
Accuracy	0.90			0.90			0.53			0.92			0.95

.

Table 2 shows the models’ measured accuracies, representing overall model performance and the PS, RC, and F1 detailed effectiveness of each event. The NB exhibited the lowest performance because the learning process of the model depended on prior knowledge to estimate the probabilities of each occurrence. Unfortunately, the HVAC system behavior could not generate adequate prior knowledge, suggesting that NB was unsuitable for AFDD in our case study.

The overall accuracy of DT, KNN, and ANN was over 90%, and each average PS, RC, and F1 was over 87%. Even though the overall accuracies of ANN and DL were fairly high and acceptable for AFDD applications, the effectiveness of failing and defective circumstances (see the red rows in Table 2) were relatively low. This suggested that traditional models did not adequately fit with AFDD applications for HVAC systems. In contrast, our proposed DL proved itself able to address limitations, especially failing and defective events. The comparative model performances on failing and defective events using F1 are shown in Figure 7.

Figure 7 illustrates an F1 histogram of failing and defective events, with our proposed DL reaching the best values, implying that it could encode HVAC system behaviors even though the classes were imbalanced. This was due to two essential reasons: (1) DL architecture consists of generalizations and optimization functions and connecting multiple layers, and (2) features originating from contextual factors can fit rare events and fulfill unknown relations through the DL architecture.

The results confirm that our proposal met the requirements of AFDD applications for HVAC system behaviors, particularly the ability to predict failures and malfunctions using partial sensor data. These are vital problems in the building management sector.

However, the DL’s PR and RC of the defective event were 93% and 92%, respectively, which were low compared with the rest of the events. The defective events were important because they served as symbols to indicate early stages of system failure and provided essential information that could aid in preventing severe damage such as system failures. Recent research challenges exist in designing and developing architectures, activation functions, and learning rules to enhance the predictive performance on defective events.

6. Conclusions

This research proposed a DL-driven AFDD to help software predict HVAC system behaviors. Our objective was to empower the software to encode the complex problems of HVAC systems and indicate HVAC system behaviors automatically. The best of our contribution was our employment of DL and encoding of contextual factors as indirect measurements to avoid modifying HVAC system engines.

We began our research by reviewing recent works and pointing out existing limitations that did not meet our research contribution goals. We presented contextual factors based on building engineering and showed how they were needed in AFDD for HVAC systems. We featured the DL concept, based on architectures, activation functions, and learning rules that had high potential for modeling AFDD for HVAC systems. Our DL was designed and developed using the optimal point between several neurons and hidden layers that best fit with the prediction of HVAC system behaviors. The results highlighted that the proposed model produced a higher accuracy, precision, recall, and F-measure than traditional models, which naturally fit with the complex problems of HVAC systems, even in the rarest situations. The enhancement could be beneficial for building engineering in the automatic monitoring and detection of abnormal behaviors in HVAC systems. It would enable engineers and practitioners to detect undesirable events early, which would help them to plan and respond to prevent worsening conditions without modifying the physical engine of an HVAC system.

In the future, we plan to improve the predictive performance of defective events and rarest events. The precision and recall values for those were 93% and 92%, respectively, which we considered relatively low. We will also consider the uncertainty between failing, defective, and normal circumstances since the behaviors had unidentical patterns. This may help engineers and practitioners make decisions. Our main concern will be how to encode the transitions between events and explain why they are relatively influenced utilizing cause-and-effect ML.