Research on Variable-Step-Size Adaptive Filter Algorithm with a Momentum Term

: To address the contradiction between the convergence error and convergence rate in the LMS algorithm, this study proposes a variable-step-size adaptive ﬁlter algorithm with a momentum term based on the logistic function. First, the normalization LMS algorithm is obtained by seeking the extremum under the Lagrange function constraint. Second, to reduce the convergence error of the algorithm, the logistic model is used as a function model of step size variation with error, resulting in a variable-step normalization LMS algorithm. Our experimental results demonstrate that this algorithm achieves smaller convergence errors compared to those of the traditional LMS algorithm. Finally, to further improve the convergence rate of the algorithm, a momentum term is introduced into the weight coefﬁcient update process of the LMS algorithm. This leads to the development of a variable-step adaptive ﬁlter algorithm with a momentum term based on the logistic function. The impact of different parameters on the algorithm performance is also investigated. In order to verify the rationality of the proposed algorithm, a dynamic system mathematical model was identiﬁed using the proposed algorithm. The results showed that the proposed algorithm had an identiﬁcation accuracy of over 97% for the mathematical model parameters and a suppression of over 99% for noise. In order to verify the engineering application value of the proposed algorithm, real-time vibration data ﬁtting experiments were conducted in the Aeroelasticity Laboratory of the China Aerodynamics Research and Development Center, and their results were compared with three algorithms: ARMAX, N4SID, and LMS. The results showed that the proposed algorithm had a higher ﬁtting accuracy than the three others. Through simulations and experiments, it is demonstrated that this study has value both theoretically and in engineering applications, promoting engineering applications of adaptive ﬁltering algorithms and providing strong support for the research of adaptive control.


Introduction
Adaptive filtering algorithms play a crucial role in modern information science and control theory.They are widely applied in fields such as system identification, holographic projection, and engineering control [1,2].Among them, the adaptive LMS (Least Mean Square) algorithm, introduced by the American mathematicians Widrow and Hoff in the 1960s and based on the principles of Wiener and Kalman filters, has been extensively studied and utilized [3][4][5][6].One distinguishing feature of this algorithm is that it overcomes the drawback of requiring prior knowledge of the input signal to obtain optimal filter coefficients, as observed in the Wiener and Kalman filter principles.Moreover, it has such advantages as simple algorithm principles and structures as well as ease of implementation.Researchers have proposed numerous improvements to address its limitations [7][8][9][10][11][12].For example, Wang et al. [8] introduced a variable-step-size LMS algorithm with the Tanh function as a model.In the early 20th century, researchers recognized the potential of kernel methods in extending linear algorithms to the nonlinear field, which led to increased focus on their application in the field of nonlinear adaptive filtering.For example, Principe J C et al. [13] combined the LMS algorithm with kernel knowledge to propose the Kernel Least Mean Square (KLMS) algorithm.In the complex domain, Widrow B et al. [7] introduced the Complex Least Mean Square (CLMS) algorithm in 1975.Different from neural network black-box models, adaptive filtering algorithms take full advantages of neural network algorithms and establish white-box data models, providing strong theoretical support for characteristic analyses and the control algorithm design of complex systems.
At present, the PID control algorithm is the most widely used and mature control algorithm in the field of engineering technology.However, modern industrial process control is becoming increasingly complex, and traditional mathematical model identification algorithms and PID control algorithms already face difficulties in meeting these ever-changing needs [14][15][16][17].Moreover, the demand for intelligent control is increasing.With the rapid development of intelligent learning algorithms based on BP neural networks, a large number of intelligent control algorithms have been proposed, which have achieved good results in many studies.However, in practical engineering applications, it is difficult to achieve stable and reliable results.For example, in wind tunnel tests simulating aircraft flight, in order to suppress the vibration of the aircraft in high-speed fluids, PID-algorithm-based control methods were adopted and achieved favorable results.However, for some aircraft with special configurations, their vibration is very severe during wind tunnel testing, and due to the obvious fluid structure coupling effect, the controlled system modal frequency undergoes "drift", resulting in poor PID control [18][19][20][21].The data-driven adaptive control algorithm is an important way to solve the above problems.Although neural networks have performed well in simulation calculations, their stability in engineering is not high, and a large amount of data is required for training in the early stages.These shortcomings make it difficult for neural network algorithms to be applied in wind tunnel tests with high safety requirements.The adaptive filtering algorithm has data fitting capabilities comparable to those of neural networks and does not require a large amount of data for training, saving a great amount of experimental costs.Therefore, the system identification technology based on the adaptive filtering algorithm has been studied in depth, and identifying the controlled object mathematical model more accurately is a key step in achieving adaptive control.The adaptive filtering algorithm shares a core principle with the backpropagation (BP) neural network, in which a cost function is constructed using the error, and the optimal weight coefficient that minimizes the error is obtained through iterative solutions [22].One specific algorithm proposed in this paper is the variable-step-size adaptive filter algorithm, which incorporates the logistic function and a momentum term.This algorithm builds upon the standard LMS algorithm and derives a new weight update formula under the Lagrange function constraint, i.e., the normalization LMS algorithm (NLMS).Due to the inherent contradiction between the convergence rate and convergence error in the LMS algorithm, the logistic function model is introduced into the NLMS algorithm as a function of the step size varying with the error, leading to the variable-step normalization LMS algorithm (VSNLMS).Numerical examples demonstrate that the VSNLMS effectively reduces the convergence error.Furthermore, a momentum term is introduced during the weight iteration process of the VSNLMS algorithm, which consists of a forgetting factor and historical weight.This enhancement enables the algorithm to converge faster, resulting in the momentum-variable step normalization LMS algorithm (M-VSNLMS).Through numerical examples, it is demonstrated that this algorithm achieves a smaller convergence error and shorter steady-state convergence compared to those of the LMS algorithm.To verify the applicability of the proposed algorithm in practical applications, system identification is performed on a dynamic system mathematical model that incorporates noise.The results show that the proposed algorithm exhibits a higher convergence rate and smaller convergence error than those of the traditional LMS algorithm.A system identification was conducted on a dynamic system in the Aeroelasticity Laboratory of the China Aerodynamics Research and Development Center (CARDC).The results showed that the data fitting accuracy of the algorithm was higher than that of the LMS algorithm and traditional N4SID and ARMAX algorithms; thus, it has value in engineering applications and provides strong support for the subsequent implementation of active control.

Principle of LMS Algorithm
The adaptive LMS algorithm is based on the Wiener filter and incorporates the gradient descent method.It minimizes the error between the filter output y(n) and the desired output d(n) by adjusting the weight vector.The cost function used is obtained as follows: By taking the gradient of J(n) with respect to w(n), the weight update formula for the standard LMS algorithm can be obtained as follows: The schematic diagram of system identification based on the LMS algorithm is shown in Figure 1.In summary, the LMS algorithm consists of two main processes: filtering and adaptive adjustment.The specific steps are as follows: (1) Determine the parameters: the global step size parameter β and the number of taps in the filter.(2) Initialize the initial value of the filter.
(3) Algorithm operation process: T represents the input of each tap of the k-th order filter.

Principle of Variable-Step LMS Algorithm
By using the Lagrange multiplier method to find the extreme value under the constraint, the following equations are derived: where Equation ( 3) is the formula for the incremental weight, and Equation ( 4) is the expected value (the constraint).Therefore, the cost function obtained using the Lagrange multiplier algorithm is as follows: where λ is the Lagrange multiplier coefficient.If we take the partial derivative of J(n) about w(n + 1) and obtain 0, then we have the following: Based on the above formula, there is the following expression: The Lagrange coefficient λ can be obtained as follows: The new normalized weight update algorithm obtained from the above formula is as follows: where µ is the iterative step size.To prevent the divergence of the algorithm when x(n) is small, a parameter γ (0 < γ < 1) is introduced.It is evident that the iterative step size of the algorithm remains constant.The logistic function is a widely used nonlinear mapping S-shaped function in deep learning.Its general form is as follows: Therefore, the functional model with the step size varying with the error is as follows: Considering Equations ( 9)-(11) comprehensively, the improved model of the step size varying with the error is obtained as follows:

Principle of Variable-Step LMS Algorithm Based on Incidental Momentum Term
The momentum "accumulates" historical gradients.Therefore, when the weight iteration algorithm includes a momentum term, the weight increment at previous moments will affect the weight change at the current moment.A larger forgetting factor indicates that the weight increment at previous moments has a greater impact on the weight change at the current moment.This discussion only focuses on the impact of the weight increment at the previous moment on the weight increment at the current moment.The new weight algorithm proposed is obtained as follows: where θ is the forgetting factor.Obviously, if the weight increment ∆w(n) at the current moment is in the same direction as the weight increment ∆w(n − 1) at the previous moment, the magnitude of the weight increment will increase.If they are in the opposite direction, the magnitude of the weight increment will decline.Based on the above discussions, the process of the variable-step LMS algorithm with a momentum term can be obtained as follows: (1) Determine the parameter: the number of taps for the filter.
(2) Initialize the initial value of the filter.

Determine the Optimal Algorithm Parameters
The standard wideband signal x(t) is obtained as follows: where f 0 is the starting frequency and k is the rate of frequency change over time.Setting f 0 to 10 Hz, k to 500, and the sampling period to 0.001 s, the standard LMS algorithm and the variable-step normalization LMS (VSNLMS) algorithm are used to filter the noisy signal s(t).The filtered signals s (t) and x(t) are compared for their errors.Figure 3a shows the time domain of the original signal x(t) and the noisy signal s(t) with white noise, and Figure 3b shows the Hilbert time-frequency energy.As shown in Figure 4, the output error diagram of a filter pair based on the LMS algorithm and VSNLMS algorithm is shown.Obviously, the output error of the VSNLMS algorithm is significantly smaller than that of the LMS algorithm, indicating that the VSNLMS outperforms the LMS in filtering.The Pearson correlation coefficient R between s (t) and x(t) is calculated to evaluate the filtering performance of different algorithms.Moreover, to quantify the impact of parameter changes on the filtering performance of the VSNLMS algorithm, the control variable method is used to obtain the R value between s (t) and x(t) when the parameters α, β, and γ range from 0.1 to 0.9.Finally, the optimal filtering parameters for the VSNLMS algorithm are determined.The formula for calculating the Pearson correlation coefficient is obtained as follows: where x i , y i , x, and y represent the sample points and the corresponding means for the two datasets.Table 1 shows the R values obtained based on the LMS and VSNLMS algorithms.According to the above quantitative analysis, the optimal parameters for the VSNLMS algorithm are as follows: α = 0.9, β = 0.9, and γ = 0.1.On this basis, the impact of θ on the M-VSNLMS algorithm is further analyzed.Table 2 reports the R values obtained by the LMS and M-VSNLMS algorithms.By adjusting the value of θ, the variation trend of R with θ is investigated based on the conclusions in Table 1.It can be observed from Tables 1 and 2 that as the parameters α, β, and θ increase, the correlation coefficient R between the filter output sequence and the original sequence also increases, and the algorithm diverges when θ is set to 0.8 or 0.9.Nevertheless, the correlation coefficient R declines as the parameter γ increases.Figure 5 shows the relationship between the Pearson correlation coefficient R and values of the parameters α, β, γ, and θ obtained via the control variable method.From Figure 5, it can be observed that the correlation coefficient between the output sequence of the M-VSNLMS algorithm and the original signal is much higher than the result of the LMS algorithm.Additionally, parameters α and β have the most significant impact on the algorithm, and the momentum term coefficient θ further improves the prediction accuracy of the algorithm.Although the increase in γ results in a decrease in prediction accuracy, this impact is slight.
The above analysis shows that the optimal parameters for the M-VSNLMS algorithm are as follows: α = 0.9, β = 0.9, γ = 0.1, and θ = 0.7.On the basis of the above conclusions, the LMS and M-VSNLMS algorithms are used to filter the noisy signal s(t).Figure 6 illustrates the convergence process of the weight values for both algorithms.For the LMS algorithm, since its weight increment is fixed-step, the weight convergence curve is smooth, but the convergence rate is obviously slow.And at the sampling time of 15 s, the weights are still in a slow iteration process.For the M-VSNLMS algorithm, since its weight increment is variable-step, the weight convergence curve is erratic, but the convergence rate is fast.At t1 = 1.5 s, the weight of each tap quickly converges, and at t1 = 4 s, the weight of each tap roughly converges to a stable state.These demonstrate that the M-VSNLMS algorithm not only has significantly smaller convergence errors compared to those of the LMS algorithm but also has a distinct advantage in terms of the convergence rate.Figure 7 presents the performance of the LMS, VSNLMS, and M-VSNLMS algorithms in filtering the signal s(t).The results demonstrate that the M-VSNLMS algorithm has the best performance on original signal reconstruction and noise removal in the time domain, frequency domain, and energy distribution.Figure 7 shows the Hilbert time-frequency of the s(t) filtering results using different algorithms.

Example Validation
To verify the convergence of the proposed algorithm and the optimization effect of the above conclusions on the algorithm's learning capability, the M-VSNLMS algorithm with α = 0.9, β = 0.9, γ = 0.1, and θ = 0.7 is used to identify the parameters of a dynamic system model.
Among them, when the model parameters are a 1 = −0.5, a 2 = −0.As shown in Table 3, the true values of the estimated parameters are compared with the estimated values.As shown in Table 3, the identification accuracy of system model parameters is above 97%, while the identification accuracy of noise is only 0.55%, indicating a good ability to suppress noise, demonstrating the excellent system identification performance of the M-VSNLMS algorithm.In addition, the algorithm has poor performance in terms of noise model identification, indicating its good noise suppression ability.

Experimental Steps and Results
In order to verify whether the proposed algorithm has good application effects in engineering, instruments and equipment were built in the aerodynamic elasticity laboratory of CARDC, as shown in Figure 9.By applying the driving voltage to the actuator, the aircraft model is vibrated, and the vibration signals are collected in real-time.In Figure 9a   In Table 4, information on the selected measurement and control instruments is shown.The signal processing module uses a series of products from National Instruments (NI) in the United States, the controller is a product from dSPACE in Germany, and the actuator uses piezoelectric stacking materials from Physical Instruments (PI) in Germany.As shown in Figure 10, the real-time data fitting results based on the traditional ARMAX algorithm and N4SID algorithm are as follows: As shown in Figure 11, the real-time data fitting results of the data-driven self-learning algorithms LMS and M-VSNLMS are as follows: Among them, the correlation coefficient δ xy is the ratio of the covariance of data and the expected data to the product of the expected values of the two.X i , Y i , X, and Y respectively represent the sampling and average values of the output data and the expected data, and n represents the sample data capacity.In Table 5, the experimental results of the four algorithms are compared.From Table 5, it can be seen that the data fitting effect of the data-driven self-learning algorithm is superior to that of traditional algorithms, and the data fitting accuracy of the M-VSNLMS algorithm is higher than that of the LMS algorithm.This is consistent with the simulation results and also indicates that the M-VSNLMS algorithm has a stronger real-time identification ability for physical systems than the LMS algorithm.The mathematical model of traditional algorithms has a higher order, which makes the mathematical model more complex, and the correlation between the algorithm output data and the expected data is less than 0.9.The correlation between the output data based on self-learning algorithms and the expected data is above 0.95.The algorithm output data are obtained through the linear convolution operation, and the mathematical model structure of the algorithm is simple.Through the above experiments, it is shown that the algorithm proposed in this article has value for engineering applications.

Figure 1 .
Figure 1.Schematic diagram of system identification based on LMS algorithm.

Figure 1 .Figure 2 .
Figure 1.Schematic diagram of system identification based on LMS algorithm.The detailed model of the adaptive LMS algorithm is depicted in Figure2.

Figure 3 .
Figure 3.Time domain and Hilbert time-frequency energy of original signal and noisy signal.

Figure 3 .
Figure 3.Time domain and Hilbert time-frequency energy of original signal and noisy signal.

Figure 4 .Figure 4 .
Figure 4. Schematic diagram of filtering error based on LMS and VSNLMS algorithms.

Figure 6 .
Figure 6.Weight convergence process of LMS and M-VSNLMS algorithms.

Figure 7 .
Figure 7. Hilbert time-frequency plots of S(t) filtering results using different algorithms.

Figure 7 .
Figure 7. Hilbert time-frequency plots of S(t) filtering results using different algorithms.
2, b 1 = 1.0, b 2 = 1.5, d 1 = −0.8,v(k) is Gaussian white noise with a mean value of 0 and variance of 0.01.The input signal u(t) adopts the white noise sequence.In Figure 8, the predicted results of the model are shown.Appl.Sci.2023, 13, x FOR PEER REVIEW 8 of 11 (a)Time domain (b) Convergence of filter parameters

Figure 8
Figure 8. ( ) t s' and ( ) t e time domain and convergence of various parameters.

Figure 8 .
Figure 8. s (t) and e(t) time domain and convergence of various parameters.
,b, the schematic diagram of the experimental device and the schematic diagram of the measurement and control system are shown respectively.

Figure 11 .Figure 11 .
Figure 11.Real-time data fitting results of LMS and M-VSNLMS algorithms.

Table 1 .
Changes in correlation coefficient R values of LMS and VSNLMS algorithms under different parameter conditions.

Table 2 .
Changes in correlation coefficient R values of LMS and M-VSNLMS algorithms under different parameter conditions.

Table 3 .
Comparison between system identification results and true values.

Table 4 .
Measurement and control instrument information.

Table 5 .
Error analysis between model output value and expected value.