Enhancing LightGBM for Industrial Fault Warning: An Innovative Hybrid Algorithm

Abstract

The reliable operation of industrial equipment is imperative for ensuring both safety and enhanced production efficiency. Machine learning technology, particularly the Light Gradient Boosting Machine (LightGBM), has emerged as a valuable tool for achieving effective fault warning in industrial settings. Despite its success, the practical application of LightGBM encounters challenges in diverse scenarios, primarily stemming from the multitude of parameters that are intricate and challenging to ascertain, thus constraining computational efficiency and accuracy. In response to these challenges, we propose a novel innovative hybrid algorithm that integrates an Arithmetic Optimization Algorithm (AOA), Simulated Annealing (SA), and new search strategies. This amalgamation is designed to optimize LightGBM hyperparameters more effectively. Subsequently, we seamlessly integrate this hybrid algorithm with LightGBM to formulate a sophisticated fault warning system. Validation through industrial case studies demonstrates that our proposed algorithm consistently outperforms advanced methods in both prediction accuracy and generalization ability. In a real-world water pump application, the algorithm we proposed achieved a fault warning accuracy rate of 90%. Compared to three advanced algorithms, namely, Improved Social Engineering Optimizer-Backpropagation Network (ISEO-BP), Long Short-Term Memory-Convolutional Neural Network (LSTM-CNN), and Grey Wolf Optimizer-Light Gradient Boosting Machine (GWO-LightGBM), its Root Mean Square Error (RMSE) decreased by 7.14%, 17.84%, and 13.16%, respectively. At the same time, its R-Squared value increased by 2.15%, 7.02%, and 3.73%, respectively. Lastly, the method we proposed also holds a leading position in the success rate of a water pump fault warning. This accomplishment provides robust support for the timely detection of issues, thereby mitigating the risk of production interruptions.

1. Introduction

In the current industrial landscape characterized by a fast-paced and highly automated environment, fault warning has become increasingly crucial [1]. As the complexity of production systems rises, the normal operation of industrial equipment becomes essential for maintaining production efficiency and product quality [2]. However, industrial equipment faces various risks such as aging, external environmental changes, and potential technical issues, increasing the likelihood of faults. Therefore, timely and accurate fault warning is a key factor in ensuring production continuity [3]. By detecting potential issues early and taking corresponding measures, businesses can reduce the risk of production interruptions, save on maintenance costs, and enhance production efficiency. The significance of fault warning systems lies in not only avoiding losses caused by unexpected downtime but also optimizing maintenance schedules for more targeted and efficient repairs.

Against this backdrop, the role of machine learning technology in fault warning practices becomes increasingly prominent [4]. It is noteworthy that among numerous machine learning algorithms, LightGBM stands out for its efficient training speed, capacity to handle large-scale data, excellent prediction accuracy, and customizable features, making it perform exceptionally well in fault warning tasks [5]. However, like many advanced technologies, LightGBM faces challenges in practical applications. Its numerous and challenging-to-determine parameters make adjusting and optimizing the model complex and challenging in different application scenarios [6]. To address this issue, innovative methods and strategies are needed to optimize the performance of the LightGBM model, making it better suited for specific industrial fault warning tasks.

With the rapid development of computer technology, metaheuristic algorithms have become powerful tools for tackling this complex challenge [7]. These algorithms are known for their flexibility and robustness in searching the solution space, effectively exploring the optimal solution by simulating natural systems or other heuristic principles. However, in the field of optimization, under the influence of the “no free lunch” theorem [8], continuous development and improvement of appropriate metaheuristic algorithms become particularly important to ensure good algorithm performance in specific problem domains. Therefore, this paper proposes a novel hybrid algorithm, i.e., simulated annealing-based Arithmetic Optimization Algorithm (SAOA), for optimizing the hyperparameters of LightGBM. The algorithm combines chaotic mapping and dynamic reverse learning strategies to generate excellent initial solutions. Additionally, it adopts a nonlinear inertia weight factor and an improved search strategy to enhance the robustness and accuracy of the AOA algorithm. Finally, the SA is applied for global search, further improving the algorithm’s performance.

To validate the effectiveness of the proposed method, we conduct experiments using real industrial system data. The experimental results demonstrate significant performance advantages of this method in fault warning. Compared to other state-of-the-art methods, this method not only exhibits higher accuracy and reliability but also provides new insights for fault warning research in the industrial domain.

In summary, this paper makes the following significant contributions to the field of fault warning:

• Introducing a pioneering hybrid algorithm that combines SA and AOA. This marks the first application of such a hybrid algorithm to fault warning tasks, thereby expanding the applicative scope of AOA;
• Proposing an improved LightGBM model specifically designed for fault warning tasks. In comparison to other advanced algorithms, this tailored model exhibits exceptional performance. Through effective optimization of LightGBM’s hyperparameter selection, the model achieves heightened accuracy and efficiency in predicting faults;
• Conducting rigorous real-world industrial case validations to substantiate the effectiveness of the proposed methods. The validation results affirm the feasibility and efficacy of our approach in actual industrial settings, underscoring its potential and practical value in the realm of fault warning.

The remainder of the paper is structured as follows: Section 2 is an overview and summary of fault warning. Section 3 defines the problem we are trying to solve and the evaluation metrics used in this paper. Section 4 provides a detailed introduction and distributional elaboration of the proposed algorithm, Section 5 applies a real-world case study for the application of the proposed algorithm, Section 6 compares it with other state-of-the-art algorithms, and Section 7 concludes this paper with the conclusions, limitations, and directions that can be explored in the future.

2. Literature Review

In recent years, the field of fault warning has attracted increasing attention, leading to the emergence of numerous research methods and technological applications [9]. Prior to conducting the literature review, we initially select keywords pertinent to fault warning. These keywords encompass ‘fault warning’, ‘predictive maintenance’, ‘machine learning’, and so forth, which aid us in identifying the most relevant research. Subsequently, when we search for literature, we utilize Boolean operators to optimize the search results. For instance, we employ ‘AND’ to ensure all keywords appear in the search results, such as ‘fault warning AND predictive maintenance’. This enables us to locate research that encompasses both topics. We also use ‘OR’ to broaden the search scope, such as ‘fault warning OR machine learning’. This allows us to find research involving either topic. Upon obtaining the search results, we review the title and abstract of each piece of literature to ascertain their relevance to the research topic. We select the most relevant and recent research for reading and analysis. During the reading of the selected literature, we conduct a thorough evaluation. Subsequently, we incorporate this information into the research to form a comprehensive literature review. Finally, the literature we summarized is as follows: Li et al. [10] proposed a diagnostic method for mild-turn-to-turn short circuit faults in synchronous generator excitation windings. They utilized an improved particle swarm optimization algorithm to determine the structure parameters of the gated recurrent unit—convolutional neural network model and used the total bias distance as the fault detection criterion. The effectiveness of this method is verified through experiments. Additionally, Lu et al. [11] introduced a fault warning method for electric vehicle (EV) charging processes based on an adaptive deep belief network. They employed the Nesterov-accelerated adaptive moment estimation for optimization during training and constructed the adaptive deep belief network model for normal charging processes using historical EV charging data. The method achieved real-time fault prediction and demonstrated high accuracy. In the industrial equipment domain, Gao et al. [12] proposed a dynamic modeling approach, combining wavelet packet decomposition and graph theory for early fault detection and identification. Experimental results demonstrated the effectiveness of this method in engineering applications. Furthermore, Lyu et al. [13] presented a battery fault warning and localization method based on acoustic signals. They also proposed a wavelet transform-based false alarm prevention mechanism to ensure the reliability of fault warning and localization. Cai et al. [14] utilized historical observation data of power systems to build a predictive model for power distribution network outages using machine learning, particularly the XGBoost algorithm. The method enabled early warning and maintenance planning for distribution network faults. In the wind turbine field, Tan et al. [15] improved the social engineering optimizer method for optimizing BP networks in hydraulic turbine fault warning systems. Experimental results showed superior performance compared to other methods. Min et al. [16] proposed a fault prediction method for distribution networks using convolutional neural network and LightGBM, enhancing adaptability to imbalanced datasets. The experiments with fault data from a specific region in 2017–2018 demonstrated improved fault prediction performance. Yang et al. [17] focused on monitoring oil and gas pipeline safety using machine learning on distributed fiber optic sensor signals. Their model accurately detected and identified damage events in real time. Jing et al. [18] introduced a LightGBM-based microservices fault identification method, ensuring high availability by analyzing historical runtime information. Tao et al. [19] reduced operation and maintenance costs through an early fault warning system based on real operational data. A novel hybrid algorithm for fault detection and early warning in the manufacturing industry was proposed by Liu et al. [20]. The algorithm combines the stochastic wandering strategy, the balanced optimizer and the adaptive learning strategy of BP neural networks, aiming to overcome the limitations of traditional BP neural networks in practical applications. Zhang et al. [21] proposed a particle swarm optimization algorithm based on nonlinearly decreasing inertia weights and exponentially varying learning factors for optimizing the deep belief network, which is used for solving the early warning problem of wind turbine faults. Zhang et al. [22] established a BP neural network prediction model optimized based on dynamic cuckoo search algorithm for solving modern industrial equipment fault detection and early warning problems. Experiments show that the model has faster convergence speed and higher prediction accuracy. Similarly, Pi et al. [23] also optimized the parameters of the BP neural network. They used an improved sand cat group optimization algorithm and introduced a new search strategy. Wu et al. [24] proposed a three-step framework for multimodal industrial process monitoring. They used a deep local adaptive network, two-stage qualitative trend analysis, and a five-state Bayesian network to gradually implement fault detection, identification, and diagnosis. Huang et al. [25] proposed a gas concentration early warning model involving the spark streaming framework. This model combines the particle swarm optimization algorithm and the gated recurrent unit model in the spark streaming framework. Liu et al. [26] proposed a coupled thermal-hydraulic-mechanical nonlinear model for predict and warn of water inrush in mines. Finally, Kong et al. [27] proposed a fault diagnosis model using a particle swarm optimization algorithm to optimize the structure of a B-spline network, aiming to improve the prediction accuracy of nonlinear time series.

Through the literature analysis, several observations are made:

• In the engineering domain, ensuring the reliable operation of equipment and systems is crucial. While machine learning techniques are widely employed for fault warning tasks, the utilization of efficient methods like LightGBM is relatively limited in practical fault warning applications;
• LightGBM, despite its efficiency, faces challenges such as the complexity of parameter selection, potentially leading to prolonged diagnosis time, increased operational costs, and reduced warning accuracy;
• The combination of metaheuristic algorithms and machine learning technologies proves to be an effective approach for enhancing fault warning performance.

Therefore, this paper proposes a novel fault warning model based on the LightGBM framework. Specifically, the proposed fault warning model, integrating LightGBM and a novel hybrid metaheuristic algorithm, offers a promising solution to the challenges identified in the literature. Through experimentation with real industrial cases, the method showcases enhanced fault warning performance, contributing to the assurance of reliable equipment and system operation while reducing diagnostic time and operational costs.

3. Problem Definition

3.1. Research Purpose

The performance of the LightGBM model is significantly influenced by its parameters. To enhance the model’s performance, it is crucial to identify an appropriate method for adjusting these parameters. Given the superior performance of meta-heuristic algorithms, our objective is to develop a higher-quality meta-heuristic algorithm to determine the parameters of LightGBM and improve the model’s performance. Due to the large number of LightGBM parameters, we optimize the four important hyperparameters to obtain a more accurate fault warning model. The specific information of hyperparameters is shown in

Table 1. LightGBM hyperparameters selection range.

Hyperparameters	Upper and Lower Limits	Role	Meaning
n_estimators	[10,100]	Controls training speed	Estimates
Learning rate	[0.15,0.35]	Impact Accuracy	Model training learning Rate
num_leaves	[5,100]	Impact Accuracy	Number of leaf nodes
max_depth	[3,11]	Avoid overfitting	Maximum depth of tree

, and we adopt the default values for the other parameters. In other words, our goal is to develop an efficient metaheuristic algorithm to find the optimal set of hyperparameters within the given space that yields the best performance for the LightGBM model.

3.2. Evaluation Metrics

Before we delve deeper into our analysis, it is first necessary to clarify the meaning and calculation methods of several evaluation metrics. In this paper, we have chosen three main evaluation metrics: RMSE, R-Squared, and Relative Percentage Deviation (RPD). We use RMSE and R-Squared to evaluate the performance of the model. We use the RPD to calibrate the parameters of SAOA.

RMSE is a metric used to measure the difference between predicted values and actual values. It calculates the square of the prediction errors for each data point, averages these squared errors, and then takes the square root. A smaller RMSE indicates more accurate predictions and less deviation from the actual values. Therefore, RMSE is an important metric for assessing the accuracy of the model [15].

$$\mathrm{RMSE}=\sqrt{\frac{1}{m}\sum _{i=1}^m(y_i-{\widehat{y}}_i{)}^2}$$

(1)

where $m$ is the number of samples, $y_i$ is the actual value, and ${\widehat{y}}_i$ is the predicted value.

R-Squared is a metric that measures the goodness of fit of the model, also known as the coefficient of determination. It represents the proportion of the target variable’s variance explained by the model. R-Squared ranges from 0 to 1, where values closer to 1 indicate a better fit and better explanation of the data variance. When R-Squared is close to 0, it means that the model cannot explain the variability of the target variable. R-Squared is a commonly used statistic for evaluating the model’s fit to the data [15].

$$\mathrm{R}-\mathrm{S}\mathrm{q}\mathrm{u}\mathrm{a}\mathrm{r}\mathrm{e}\mathrm{d}=1-\frac{S{S}_{\mathrm{residual}}}{S{S}_{\mathrm{total}}}$$

(2)

where $S{S}_{\mathrm{residual}}$ is the sum of squares of residuals and $S{S}_{\mathrm{total}}$ is the total sum of squares.

We use RPD to measure the performance of the algorithm under various parameter combinations, that is, the deviation of each experiment from the best experiment’s RPD. RPD is calculated as shown in Equation (3).

$$RPD=\frac{{Alg}_{\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}}-{Min}_{\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}}}{{Min}_{\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}}}$$

(3)

where ${Min}_{\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}}$ represents the minimum RMSE from the algorithm across all experiments, and ${Alg}_{\mathrm{R}\mathrm{M}\mathrm{S}\mathrm{E}}$ represents the value from the algorithm in each individual experiment.

4. Proposed Methods

In this section, we present our hybrid model. We begin by outlining the LightGBM model (Section 4.1) and subsequently introduce the customized hybrid optimization algorithm using SA, AOA and new search strategies tailored for the LightGBM model (Section 4.2).

4.1. LightGBM Model

LightGBM is a fast, distributed, high-performance gradient-boosting framework based on decision tree algorithms, widely used in various machine learning problems such as regression, ranking, and classification [5]. It is a type of Boosting algorithm, which combines multiple weak machine learning models to generate a strong learning model. Boosting algorithms increase the weights of misclassified data and decrease the weights of correctly classified data, so that misclassified classifiers receive more attention in the next round of training. Ultimately, all machine learning models are linearly combined, and the weights of the combined model are adjusted based on the classifier’s error rate [5].

The core concept can be represented by the following Equation (4):

$$f\left(x\right)=\sum _{q=1}^Q{\alpha }_{q}T\left(x,{\theta }_q\right)$$

(4)

where $f\left(x\right)$ is the target value corresponding to the training sample; Q is the number of base learners; ${\alpha }_q$ is the weight coefficient of the qth base learner; x is the training sample; ${\theta }_q$ is the parameter for the learner’s classification; and $T\left(x,{\theta }_q\right)$ is the qth base learner involved in training.

After determining the model’s loss function and training data, the training process of the Boosting algorithm is transformed into an optimization problem to minimize the loss function. The objective function is as follows:

$$\mathrm{a}\mathrm{r}\mathrm{g}min\sum _{h=1}^{H}L(y_h,f(x_h\left)\right)$$

(5)

where H is the number of samples; h is the sample index; $y_h$ is the actual value of the data; $f\left(x_h\right)$ is the target value corresponding to the hth sample; and $L\left(y_h,f\left(x_h\right)\right)$ is the loss function value for the hth sample.

In the GBM, which is a boosting tree model obtained based on the gradient descent algorithm, after each addition of a new sub-model, the selected loss function continuously decreases towards the gradient of the variable with the next highest information content, as show in Equation (6).

$$L\left(F_j\right(x),Y)<L\left(F_{j-1}\right(x),Y)$$

(6)

where $L\left(F_j\right(x),Y)$ and $L\left(F_{j-1}\right(x),Y)$ are the loss function values for the jth and (j − 1)th iterations, respectively, $F_j\left(x\right)$ and $F_{j-1}\left(x\right)$ are the target values corresponding to the jth and (j − 1)th samples, and Y is the true target value of the sample.

As a type of GBM, LightGBM effectively addresses the challenges encountered by GBM when dealing with massive data. This model has two main features [5]:

(a)

Leaf-wise tree growth method: It adopts a leaf-wise growth strategy as opposed to level-wise growth. This method has a smaller computational cost and successfully avoids overfitting issues by controlling the minimum data volume of leaf nodes and the depth of trees;

(b)

Histogram-based decision tree algorithm: LightGBM chooses a histogram-based decision tree algorithm. When selecting features, it only needs to traverse and find the optimal split point based on the discrete values of the histogram, thereby reducing computational and storage costs. This feature enables LightGBM to train and predict more efficiently when handling large-scale datasets.

4.2. Proposed Hybrid Algorithm

While LightGBM has shown remarkable performance across diverse domains, its numerous parameter configurations necessitate optimization for effective adaptation to specific problem contexts [5]. Conventional grid search algorithms for parameter tuning are plagued by extended computation times and high computational demands. Recognizing the complexity of this challenge and the absence of a universal solution, we introduce an innovative SAOA. This algorithm seamlessly integrates AOA, SA, and novel search strategies to streamline parameter adjustments, enhancing the efficiency of the LightGBM model. Our approach aims to tackle the challenges associated with parameter optimization, promoting adaptability in the LightGBM model, and simultaneously reducing computation time, thereby enhancing overall algorithm efficiency.

Given our objective of obtaining the optimal LightGBM parameters through the use of the SAOA, the input to the SAOA in our problem is the LightGBM parameter space that we have defined. This parameter space encompasses all possible combinations of LightGBM parameters that we intend for the SAOA algorithm to explore. We select the RMSE between the actual and predicted values of the LightGBM as the fitness value. Consequently, the output of the SAOA algorithm is the optimal combination of parameters that minimizes the RMSE. This combination of parameters can optimize the performance of the LightGBM model. Subsequently, we utilize the LightGBM model with this set of parameters to carry out training and fault warning tasks. In general, in our problem, the input to the SAOA algorithm is the LightGBM parameter space we have defined, and the output is the optimal combination of parameters. In the following section, we will provide a detailed description of the execution steps of the SAOA. This will include a step-by-step walkthrough of the algorithm, highlighting key operations and decision points, and explaining how these contribute to the overall objective of optimizing the LightGBM parameters.

4.2.1. Population Initialization

Traditional AOA typically begins the optimization process by randomly generating the initial population, which introduces a series of complexities and challenges for subsequent optimization stages [28]. The randomly generated population increases the difficulty for the algorithm to find the global optimum or converge to a stable state. In this case, more effective population initialization strategies and improvements to the optimization algorithm become particularly important. To address this challenge, we introduce chaotic mapping and a dynamic reverse learning strategy to enhance the formation process of the initial population, thereby improving the algorithm’s performance and robustness in optimization problems. Specifically, we use logistic mapping to update the initial population, and the chaotic sequence generation method of the logistic mapping is shown in Equation (7):

$$F_i=x(i+1)=\mu x\left(i\right)(1-x(i\left)\right)$$

(7)

where $F_i$ is the logistic mapping coefficient for the ith individual, and $x\left(i\right)$ is the mapping variable, a random number between 0 and 1. The logistic mapping is in a completely chaotic state when 3.57 < $\mu$ ≤ 4, exhibiting chaotic behavior in this interval. When $\mu$ is set to 4, better results can be obtained; here, we take $\mu$ = 4.

Afterward, we utilize the generated chaotic sequences to form initial individuals that meet the specified conditions, as illustrated in Equation (8).

$$x_{i,j}={LB}_j+F_i({UB}_j-{LB}_j)$$

(8)

where $x_{i,j}$ denotes the value of the jth dimension of the ith individual, ${UB}_j$ is is the lower limit of the value taken in the jth dimension, ${LB}_j$ is the upper limit of the value taken in the jth dimension, and $F_i$ is the chaotic mapping coefficient of the ith individual.

Finally, we apply the dynamic reverse learning strategy to further optimize the initially improved individuals and merge them with individuals improved by logistic mapping. The initial population is then composed of Npop individuals, selected based on their optimal fitness. The dynamic reverse learning strategy is depicted in Equation (9).

$$x_{i,j}=x_{i,j}+c_1·(c_2·({UB}_j+{LB}_j-x_{i,j})-x_{i,j})$$

(9)

where $x_{i,j}$ denotes the value of the jth dimension of the ith individual, ${LB}_j$ is the lower limit of the value taken in the jth dimension, ${UB}_j$ is the upper limit of the value taken in the jth dimension, and $c_1$ and $c_2$ are random numbers in the range (0, 1).

4.2.2. Improved Mathematical Optimizer Acceleration Function

The Arithmetic Optimization Algorithm is primarily divided into two stages: global exploration and local exploitation [28]. In the global exploration stage, operators with high dispersion, such as multiplication or division, are employed for the search. In the local exploitation stage, operators with high density, such as subtraction or addition, are used for the search. The transition between the exploration and exploitation stages is controlled by the function value calculated by the mathematical optimizer acceleration (MOA) function [29]. When $r_1>MOA$, the algorithm performs global exploration, and when $r_1<MOA$, the algorithm executes local exploitation. The mathematical expression for MOA is given by Equation (10) [2]:

$$MOA\left(t\right)=Min+t\ast \left(\frac{Max-Min}{Maxit}\right)$$

(10)

where t is the number of iterations, Min and Max are the minimum and maximum values of the acceleration function, respectively, and Maxit is the maximum number of iterations.

When using MOA to switch between “global exploration” and “local exploitation” in the standard AOA, the algorithm tends to have a lower probability of transitioning to local exploitation in the later stages of the search. This is inconsistent with the common search strategy employed by general intelligent optimization algorithms, which emphasizes global exploration in the early stages and shifts towards local exploitation in the later stages. This discrepancy weakens the algorithm’s ability for local development in the later stages, negatively impacting optimization speed and accuracy. Therefore, this paper proposes a new MOA(t), as shown in Equation (11):

$$MOA\left(t\right)=a\times (l-\mathrm{e}\mathrm{x}\mathrm{p}(-(Maxit-t)/Maxit\left)\right)+b$$

(11)

where t is the current number of iterations, Maxit is the maximum number of iterations, and a and b are control parameters located in the range (0, 1).

4.2.3. AOA Global Exploration Phase

In the global exploration phase of AOA, mathematical calculations involving division and multiplication operators can yield values or decisions that align with the global exploration mechanism [28]. The position update formula during the global exploration phase is represented as Equation (12) [28].

$$x_{i,j}^{t+1}=\left\{\begin{array}{cc}{x}_b^j÷(MOP+\epsilon )\times \left(\right({UB}_j-{LB}_j)\times m\mu +{LB}_j),& \hfill r_2<0.5\\ x_b^j\times MOP\times \left(\right({UB}_j-{LB}_j)\times m\mu +{LB}_j),& \hfill r_2\ge 0.5\end{array}\right.$$

(12)

where $x_{i,j}^{t+1}$ is the updated value of the jth dimension of the ith individual, and $x_b^j$ is the value of the jth dimension of the current optimal individual. ${LB}_j$ is the lower limit of the value of the jth dimension, and ${UB}_j$ is the upper limit of the value of the jth dimension. ε is a minimal value, and $m\mu$ is a control parameter, which is set to be 0. 499 in this paper.

MOP is the mathematical optimizer probability, and the mathematical expression of MOP is shown in Equation (13) [28].

$$MOP\left(t\right)=1-\frac{t^{\frac{1}{a}}}{T^{\frac{1}{a}}}$$

(13)

where t is the current iteration number, Maxit is the maximum iteration number, and α is a sensitivity parameter that defines the probing accuracy and takes the value 5.

4.2.4. AOA Local Exploration Phase

During the local exploitation phase of AOA, mathematical calculations involving subtraction and addition operators can yield high-density values, advantageous for the algorithm’s local exploitation [28]. The position update formula during the local exploitation phase is represented as Equation (14) [28].

$$x_{i,j}^{t+1}=\left\{\begin{array}{c}\begin{array}{cc}{x}_b^j-MOP\times \left(\right({UB}_j-{LB}_j)\times m\mu +{LB}_j),& r_3<0.5\end{array}\\ \begin{array}{cc}{x}_b^j+MOP\times \left(\right({UB}_j-{LB}_j)\times m\mu +{LB}_j),& r_3\ge 0.5\end{array}\end{array}\right.$$

(14)

where $x_{i,j}^{t+1}$ is the updated value of the jth dimension of the ith individual, and $x_b^j$ is the value of the jth dimension of the current optimal individual. ${LB}_j$ is the lower limit of the value taken in the jth dimension, and ${UB}_j$ is the upper limit of the value taken in the jth dimension.

4.2.5. Nonlinear Inertia Weight Factor

To enhance the balance between the exploration and exploitation capabilities of the SAOA algorithm, this paper proposes an improvement to the SAOA formula by introducing a nonlinear inertia weight factor, denoted as w, as shown in Equation (15). Subsequently, the formulas for the global search and local search stages of SAOA are presented in Equations (16) and (17).

$$w\left(t\right)=\left[w_{max}·(1-\mathrm{s}\mathrm{i}\mathrm{n}(\frac{\pi t}{2Maxit}\left)\right)+w_{min}\mathrm{s}\mathrm{i}\mathrm{n}\left(\frac{\pi t}{2Maxit}\right)\right]$$

(15)

where $w_{max}$ and $w_{min}$ are the maximum and minimum values of weight change; we set $w_{min}$= 0.4 and $w_{max}$= 0.9. Maxit is the maximum number of iterations and t is the current number of iterations.

$$x_{i,j}^{t+1}=\left\{\begin{array}{c}\begin{array}{cc}{x}_b^j-MOP\times \left(\right({UB}_j-{LB}_j)\times m\mu +{LB}_j+w{x}_{b,j}^t),& r_3<0.5\end{array}\\ \begin{array}{cc}{x}_b^j+MOP\times \left(\right({UB}_j-{LB}_j)\times m\mu +{LB}_j+w{x}_{b,j}^t),& r_3\ge 0.5\end{array}\end{array}\right.$$

(16)

$$x_{i,j}^{t+1}=\left\{\begin{array}{cc}{x}_b^j÷(MOP+\epsilon )\times (\left({UB}_j-{LB}_j\right)\times m\mu +{LB}_j+\left(1-w\left(t\right)\right)(x_{b,j}^t-x_{i,j}^t\left)\right),& \hfill r_2<0.5\\ x_b^j\times MOP\times \left(\right({UB}_j-{LB}_j)\times m\mu +{LB}_j+\left(1-w\left(t\right)\right)(x_{b,j}^t-x_{i,j}^t\left)\right),& \hfill r_2\ge 0.5\end{array}\right.$$

(17)

where $x_{i,j}^{t+1}$ is the updated value of the jth dimension of the ith individual, and $x_b^j$ is the value of the jth dimension of the current optimal individual. ${LB}_j$ is the lower limit of the jth dimension of the individual, and ${UB}_j$ is the upper limit of the jth dimension of the individual. Ε is a minimal value; $w\left(t\right)$ is the inertia weight of the tth iteration; and $m\mu$ is a control parameter, which is set to 0.499 in this paper.

4.2.6. SA Algorithm

Traditional AOA relies on updating based on the optimal solution obtained in each iteration [28]. However, this strategy may lead the algorithm to converge to a local optimum, hindering global optimization. To address this issue, the SA is introduced.

Key features of the SA include [29]:

• Retention of inferior solutions: In certain probability scenarios, the SA chooses to retain inferior solutions. This practice helps avoid premature convergence to local optima, thereby enhancing the algorithm’s global search capability;
• Increased population diversity: By preserving inferior solutions, the algorithm introduces a level of randomness, augmenting the diversity of the population. This is done to explore a broader range of potential solutions in the search space, rather than being confined to the vicinity of local optima;
• Enhanced escape from local optima: The SA, through the strategy of accepting inferior solutions during the search process, increases the probability of escaping local optima. This makes the algorithm more robust and better adaptable to complex search spaces.

The calculation method is represented by Equation (18).

$$p=\left\{\begin{array}{cc}1,& \mathrm{if}{f}_{\mathrm{new}}<f_{\mathrm{old}}\\ {exp}^{\left(-\frac{f_{\mathrm{new}}-f_{\mathrm{old}}}{T}\right)},& \mathrm{if}{f}_{\mathrm{new}}\ge f_{\mathrm{old}}\end{array}\right.$$

(18)

where p is the likelihood of accepting the new solution, fnew denotes the function value for the new solution, and fold represents the same for the initial solution. Drawing inspiration from the annealing process, T serves as the existing temperature of the SA during this iteration. At the end of this subloop, we update this iteration as shown in Equation (19).

$$T_{u+1}=T_u·C$$

(19)

where $T_u$ is the temperature at the uth search and $C$ is the rate of temperature reduction.

4.2.7. Overall Framework of the Algorithm

Combining the aforementioned components, the specific steps for SAOA’s hyperparameter optimization for LightGBM are as follows:

Step 1: Perform data preprocessing. Data standardization is performed;

Step 2: Set the basic parameters of SAOA algorithm and the range of hyperparameters of LightGBM;

Step 3: Perform SAOA individual initialization;

Step 4: Implement 10-fold cross-validation for LightGBM model training, using the RMRE as the fitness function. Calculate the fitness values of each initial individual.

The 10-fold cross-validation is a technique used in machine learning to assess model performance. The basic idea is to divide the dataset into ten equal parts (folds) and then perform ten rounds of model training and evaluation.

The process is as follows:

Step 1: Divide the dataset into ten parts, usually through random sampling or sequential splitting;

Step 2: Conduct ten rounds of iteration. In each round, use one part as the test set and the other nine parts as the training set. In each iteration, train the model using the training set and evaluate its performance on the test set;

Step 3: Record the model’s performance metric on the test set for each round;

Step 4: After completing ten rounds, average the results of all performance metrics to obtain a more robust performance evaluation;

Using 10-fold cross-validation provides a more robust evaluation and maximizes the use of data. By repeatedly using different data splits, 10-fold cross-validation reduces dependence on a single split, thereby improving the reliability of the evaluation;

Step 5: Select the individual with the optimal fitness value as the current best individual (i.e., the current best hyperparameter combination), and record the best fitness value and individual position in an external file;

Step 6: Use the SAOA operators to update individuals and calculate their fitness values;

Step 7: Find the updated best individual and compare it with the current best individual. According to the simulated annealing strategy, update the best individual’s position and best fitness value. Additionally, record the best fitness value and individual position in an external file;

Step 8: Check if the termination condition is met; if not, return to Step 6, otherwise, output the optimal hyperparameter combination from the external file, and the optimization process concludes.

Finally, Figure 1 shows the flowchart of our proposed algorithm.

5. Case Study

In this case study, we perform a comprehensive fault warning analysis utilizing a water pump within a power plant setting, as depicted in Figure 2. The investigation encompasses eleven critical measurement points linked to the pump’s operational condition: non-drive end bearing temperatures (Non-driving end bearing temperature 1 and Non-driving end bearing temperature 2), non-drive end bearing vibrations in the X and Y directions, stator winding temperatures of the condensate pump (A, B, C phase temperatures 1 and 2), and the pressure drop across the inlet filter. In addition, we use the entrance flow of the pump as an input measurement point because it can reflect many types of faults.

To conduct our analysis, we aggregate these measurement points into a cohesive sample. Our dataset comprises a total of 1500 samples sourced from the supervisory information system. These samples form the basis of our original dataset, which we subsequently partition into training and testing sets at a ratio of 7:3 to facilitate robust model evaluation.

5.1. Data Analysis

As previously mentioned, our data comprises eleven input measurement points and one output measurement point. To facilitate a clearer understanding of the data, we conduct an analysis. The upper and lower limits, standard deviation, median, and average of the data for each measurement point are displayed in

Table 2. Results of data analysis of measurement point input data.

Measurement Point	Unit	Upper Limit	Lower Limit	Standard Deviation	Median	Mean
Condensate pump non-driven end bearing temperature 1	°C	54.60	53.41	0.61	53.87	53.93
Condensate pump non-driving end bearing temperature 2	°C	47.30	45.90	0.77	46.73	46.41
Condensate pump non-driven bearing X-direction vibration	mm/s	0.57	0.48	0.05	0.53	0.54
Condensate pump non-driven end bearing Y-direction vibration	mm/s	0.74	0.64	0.05	0.68	0.70
Condensate pump stator winding phase A temperature 1	°C	65.20	62.61	1.39	64.80	64.20
Condensate pump stator winding phase A temperature 2	°C	64.89	61.82	1.55	63.00	63.24
Condensate pump stator winding phase B temperature 1	°C	65.30	62.87	1.22	64.30	64.16
Condensate pump stator winding phase B temperature 2	°C	66.00	63.20	1.51	63.60	64.27
Condensate pump stator winding phase C temperature 1	°C	64.80	61.70	1.58	62.80	63.07
Condensate pump stator winding phase C temperature 2	°C	67.10	62.50	2.42	64.03	64.37
Inlet screen differential pressure	kPa	5.71	5.28	0.22	5.39	5.48

and

Table 3. Results of data analysis of measurement point output data.

Measurement Point	Unit	Upper Limit	Lower Limit	Standard Deviation	Median	Mean
Entrance flow	ton/h	522.15	325.52	43.45	453.63	440.81

. Additionally, we utilize box plots to illustrate the distribution of the data, which can be found in Appendix A. This comprehensive analysis allows for a more in-depth understanding of the data’s characteristics and underlying patterns, which is crucial for the subsequent modeling process.

5.2. SAOA Algorithm Parameter Calibration

Metaheuristic algorithms play a pivotal role in tackling intricate optimization problems, and their efficacy hinges on the meticulous tuning of parameters [30,31]. Drawing insights from a comprehensive literature analysis [15,20,21,22,29] and initial experiments, we established reference values for each parameter level. SAOA has six parameters, their meanings and predefined values are outlined in

Table 4. SAOA parameter reference levels.

Parameters	Meaning	Level 1	Level 2	Level 3
Npop	Population size	30	40	50
a	MOA control parameters	0.2	0.25	0.3
b	MOA control parameter	0.8	0.85	0.9
Maxit	Iteration number	100	150	200
T1	SA initial temperature	1000	1200	1500
C	Cooling rate	0.92	0.95	0.97

.

However, conducting a full experiment for parameter tuning would require significant computational resources. In light of this, the Taguchi method provides a robust experimental design framework for efficiently exploring the parameter space and determining optimal settings. Following the recommendations of the Taguchi method, we adopted the experimental structure outlined in

Table 5. Results of orthogonal test.

Number of Experiments	Npop	a	b	Maxit	T1	C	RPD
1	1	1	1	1	1	1	0.1572
2	1	1	2	2	3	3	0.1894
3	1	2	1	3	3	2	0.1942
4	1	2	3	1	2	3	0.2137
5	1	3	2	3	2	1	0.2018
6	1	3	3	2	1	2	0.1123
7	2	1	1	3	2	3	0.0189
8	2	1	3	1	3	2	0.2098
9	2	2	2	2	2	2	0.0745
10	2	2	3	3	1	1	0.2236
11	2	3	1	2	3	1	0.1567
12	2	3	2	1	1	3	0.1975
13	3	1	2	3	1	2	0.0304
14	3	1	3	2	2	1	0.1689
15	3	2	1	2	1	3	0.0801
16	3	2	2	1	3	1	0.0823
17	3	3	1	1	2	2	0.1527
18	3	3	3	3	3	3	0.0456

, consisting of 18 experiments.

Next, we calculate the mean value of each parameter across all experiments to determine its optimal level.

We have 18 sets of experiments, each with a performance measure, namely RPD. Our goal is to find a set of parameter settings that optimize the performance of the algorithm. Specifically, we used the following steps to determine the optimal level of each parameter:

Step 1: For the Npop, we first consider its performance at level 1. We calculate the performance of Npop at level 1 (Mean RPD) by adding the RPD values of experiments 1–6 and then dividing by the number of experiments;

Step 2: For parameter a, we also need to calculate its performance at level 1. We add the RPD values of experiments 1, 2, 7, 8, 13, 14, and then divide by the number of experiments to get the performance of a at level 1;

Step 3: We repeat the above process, calculating for each level of each parameter, to get the performance of each parameter at each level;

Step 4: Finally, for each parameter, we choose the level that minimizes the average RPD as the optimal level for that parameter. In this way, we get the optimal setting for each parameter, which is the result of parameter calibration.

This systematic approach allows us to fine-tune the parameters of the algorithm, thereby optimizing its performance. The optimal parameter settings obtained through this process can significantly enhance the accuracy and reliability of the algorithm’s output.

Subsequently, the performance of each parameter at various levels is illustrated in Figure 3. Based on the insights from Figure 3, we establish the following parameter settings: Npop = 50, a = 0.8, b = 0.8, Maxit = 200, T1 = 1500, and C = 0.97.

5.3. Early Warning Strategy

After determining the parameters, we input the SAOA parameters, LightGBM parameter space, and the fault-measuring data points from Section 5 into the algorithm. We then start running the program to obtain the initial SAOA individuals (i.e., possible combinations under the LightGBM parameter space).

Next, we execute the SAOA optimization process with the RMSE of the LightGBM prediction test set serving as the value of the fitness function. This process is aimed at obtaining the optimal LightGBM parameters, which are shown in

Table 6. LightGBM optimal parameters.

Hyperparameters	Optimal Values
n_estimators	15
Learning rate	0.18
num_leaves	20
max_depth	6

.

Finally, we assign the parameters in Table 6 to LightGBM for model training and to output the trained model. The test set outcomes are presented in Figure 4. Subsequently, we compute the residual E, capturing the variance between predicted and actual values, along with its mean ($\mu$) and standard deviation ($\sigma$). For establishing the alarm limit, we set $w_l=\mu -3\sigma \mathrm{a}\mathrm{n}\mathrm{d}{w}_u=\mu +3\sigma$, where $w_l$ denotes the lower warning value, and $w_u$ signifies the upper warning value. To enhance precision, we incorporate a tolerance count (k = 3), indicating that a pump failure warning signal is activated only when the residuals from the predictions of three consecutive operational datasets surpass the alarm limit.

5.4. Early Warning Test

Here, we collected data for twenty sets of each of the four types of faults to verify the accuracy of the proposed method. These four types of faults are high bearing temperature (F1), excessive bearing vibration (F2), high winding temperature (F3), and high inlet filter pressure drop (F4). We input this faulty data into the trained model to assess its fault warning effectiveness. Finally, the results are presented in Figure 5.

The results reveal that the proposed method has effectively delivered precise fault warning outcomes, showcasing notable accuracy in the test data. This validation underscores the efficacy and reliability of the introduced approach. Additionally, we notice that F4 has the highest success rate. This is because our fault determination criterion is the inlet flow rate, and the high inlet filter pressure drop (F4) fault has a greater impact on the inlet flow rate compared to other types of faults, and is more sensitive; that is to say, its fault characteristics are more prominent in the data, the deviation from the data during normal operation is also greater, and it is more adaptable to the fault warning strategy we proposed.

5.5. Algorithm Application Discussion

This paper introduces a novel fault warning method with extensive practical applications, particularly in the realm of industrial equipment fault diagnosis. Our research offers a new optimization method by enhancing LightGBM with a hybrid meta-heuristic algorithm, which can bolster the accuracy and efficiency of fault warnings. This is of significant value for fault warning tasks of industrial equipment that necessitate handling large-scale data and making complex predictions.

Moreover, the SAOA optimization method we proposed can also be applied to other machine learning models that require extensive parameter tuning, thereby increasing its applicability, and broadening the warning effect of industrial equipment. In wider fields such as transportation, energy, and finance, our research results also hold substantial application potential. In the transportation sector, the fault warning system can be employed to monitor the operational status of transportation equipment (such as trains, airplanes, cars, etc.), and detect potential faults in advance, thereby averting accidents and ensuring the safety and efficiency of transportation. In the energy sector, the fault warning system can be utilized to monitor the operational status of energy equipment (such as wind turbines, solar panels, etc.), and detect potential faults in advance, thereby preventing energy waste and enhancing energy utilization efficiency. In the financial sector, the fault warning system can be used to monitor the dynamics of the financial market and detect potential risks in advance, thereby avoiding the occurrence of financial crises and safeguarding the interests of investors.

In conclusion, the novel fault warning method proposed in this paper, along with the SAOA optimization method, not only improves the performance of the LightGBM model but also has wide-ranging applications across various sectors, making it a valuable contribution to the field.

6. Comparison with Other Algorithms

In this section, we conduct a comprehensive comparison of the proposed SAOA algorithm with other methods, including ISEO-BP [15], LSTM-CNN [32], and GWO-LightGBM [33]. The reasons we chose these three algorithms as benchmarks are as follows:

• ISEO-BP: The ISEO-BP is an enhanced version of the traditional BP network. BP networks are commonly used in fault warning systems, but they have some limitations, such as a tendency to fall into local optima. The ISEO method optimizes the BP network to mitigate this issue, thereby providing an efficient fault warning system. ISEO-BP improves the weight update mechanism, enabling the network to avoid local optima during the learning process, which enhances the accuracy and reliability of fault detection;
• LSTM-CNN: The combination of LSTM and CNN is particularly effective for sequence prediction problems with spatial input. In fault warning, this method can help the model understand the time dependence in the data, thereby better predicting potential faults that may occur in the future. LSTM networks can capture long-term dependencies in time series data, while CNNs can extract useful features from spatial data. The combination of these two networks allows the model to handle the complexity of both time and space, thereby improving the accuracy of fault warnings;
• GWO-LightGBM: GWO is a renowned optimization algorithm known for its high-quality solutions and efficiency. GWO simulates the hunting behavior of grey wolves, optimizing problems by searching for the optimal solution. When GWO is combined with LightGBM, it can further enhance the performance and accuracy of the model. By comparing GWO and our developed SAOA, we can demonstrate the superiority of SAOA in handling complex problems.

By comparing with these algorithms, we further validate the effectiveness of SAOA. All algorithm parameters are calibrated using the Taguchi method. To ensure fairness, each algorithm is run ten times, and the average values are taken. To comprehensively evaluate the performance of the algorithms, we use two metrics: RMSE and R-Squared.

Additionally, to ensure the accuracy of the statistical analysis, we perform a 95% confidence interval statistical analysis on the results of each algorithm.

Table 7. Results of algorithm performance comparison (Optimal value is represented in bold).

Algorithms	RMSE	R-Square
ISEO-BP	21.3	95.02
LSTM-CNN	24.1	90.70
GWO-LightGBM	22.8	93.58
SAOA-LightGBM	19.8	97.07

demonstrates the average results of the performance metrics of each algorithm after ten runs, and Figure 6 shows its statistical results.

Based on the results in Table 5, the proposed hybrid algorithm achieves optimal values in both metrics compared to other state-of-the-art algorithms, which fully proves its superiority. In addition, the 95% confidence interval in Figure 6 demonstrates its stability and robustness.

Finally, we apply the models trained by each algorithm to the fault warning test for actual comparison. This step further validates the performance of SAOA in practical applications and provides a clear comparison of the relative performance of the algorithms in fault prediction.

Table 8. Results of the application of each algorithm model (Optimal value is represented in bold).

Algorithms	F1	F2	F3	F4
ISEO-BP	18/20	16/20	17/20	16/20
LSTM-CNN	15/20	15/20	15/20	16/20
GWO-LightGBM	16/20	14/20	16/20	18/20
SAOA-LightGBM	18/20	17/20	18/20	19/20

demonstrates the results of their application testing, and Figure 7 provides a more visual graphical presentation.

The results above underscore the practical significance of SAOA-LightGBM, demonstrating its superior success rates in comparison to other algorithms across four fault warnings.

In summary, the algorithm introduced in this study not only outperforms alternative approaches but also showcases its potential for real-world applications by consistently delivering superior performance.

7. Conclusions and Future Works

In the industrial field, with the advancement of technology and the expansion of the industry, the demand for fault early warning has become increasingly important for the safe operation of equipment and the improvement of production efficiency. As a representative algorithm of machine learning, LightGBM has brought new breakthroughs to fault early warning. However, in practical applications, LightGBM still has a large number of parameters that are difficult to determine; effectively determining these parameters to optimize the performance of the model is a challenging problem.

To solve this problem, this study proposes a hybrid algorithm that combines AOA, SA, and innovative search strategies, namely SAOA. The proposed SAOA cleverly uses the advantages of AOA and SA, and through innovative search strategies, it optimizes the hyperparameters of LightGBM more effectively. Specifically, the combination of AOA and SA can find a good balance between global and local search, and the innovative search strategy can help the algorithm find the optimal solution faster. By combining SAOA with LightGBM, the parameters that make LightGBM perform best can be found to the greatest extent, making it more effective in practical applications.

Through a series of industrial case studies, we found that the proposed method is superior to other advanced methods in terms of fault prediction accuracy and generalization ability. This shows that our solution can detect problems in time, reduce the risk of production interruption, while also providing reliable support for industrial production. Therefore, we believe that this algorithm has broad application prospects in the industrial field and can help enterprises improve safety and production efficiency.

However, we also recognize that although this study has achieved significant results in the field of fault early warning, there are still some limitations, which provide possibilities for future research directions. First, the performance of the algorithm depends on data quality and feature selection, which requires further improvement in key research areas such as data preprocessing and feature engineering [34,35]. Secondly, there is room for improving the computational accuracy of the hybrid algorithm, which requires exploration of more effective optimization strategies and integration of more hybrid methods [36,37]. In addition, as the industrial field develops, future research can delve into combining deep learning and other machine learning methods with LightGBM to further improve the efficiency of fault early warning systems [38].