Iterative Parameter Estimation Algorithms for Dual-Frequency Signal Models

: This paper focuses on the iterative parameter estimation algorithms for dual-frequency signal models that are disturbed by stochastic noise. The key of the work is to overcome the difﬁculty that the signal model is a highly nonlinear function with respect to frequencies. A gradient-based iterative (GI) algorithm is presented based on the gradient search. In order to improve the estimation accuracy of the GI algorithm, a Newton iterative algorithm and a moving data window gradient-based iterative algorithm are proposed based on the moving data window technique. Comparative simulation results are provided to illustrate the effectiveness of the proposed approaches for estimating the parameters of signal models.


Introduction
Parameter estimation is used widely in system identification [1][2][3] and signal processing [4,5].The existing parameter estimation methods for signal models can be classified into the following basic categories: the frequency-domain methods and the time-domain methods.The frequency-domain methods based on the fast Fourier transform (FFT) mainly include the Rife method, the phase difference method, etc.The accuracy of the Rife method is high in the case of noiseless or higher signal-to-noise ratio with adaptive sampling points; however, the error given by the Rife method is large if the signal frequency is near the DFT quantization frequency point.Therefore, some improved methods were proposed [6,7].For example, Jacobsen and Kootsookos used three largest spectral lines of the FFT spectrum to calibrate the frequency estimation [8].Deng et al. proposed a modified Rife method for the frequency shift of signals [9].However, there is a risk that the shift direction of the frequency may be error.The time-domain methods mainly include the maximum likelihood algorithm, the subspace method and the self-correlation phase method.The maximum likelihood method is effective estimation to minimize the average risk though imposing significant computational costs.The methods in [10][11][12] used multiple autocorrelation coefficients or multi-steps autocorrelation functions to estimate the frequency so the amount of computations increase.
In practice, actual signals are usually disturbed by various stochastic noise, and the time series signals like vibration signals or biomedical signals are subjected to dynamic excitations, including nonlinear and non-stationary properties.In order to solve the difficulties, time-frequency representations (TFR) provide a powerful tool since a TFR can give information about the frequencies contained in signal over time, such as short-time Fourier transform, wavelet transform, and Hilbert-Huang transform [13].Recently, Amezquita-Sanchez and Adeli presented an adaptive multiple signal classification-empirical wavelet transform methodology for accurate time-frequency representation of noisy non-stationary and nonlinear signals [14].Daubechies et al. used an empirical mode decomposition-like tool to decompose into the blocks functions, with slowly varying amplitudes and frequencies [15].These existing approaches can obtain signal models indirectly, because they are realized based on the transform techniques like the Fourier transform and wavelet transform.In this paper, we propose the direct parameter estimation algorithms for signal modeling.
The iterative methods and/or the recursive methods play an important role not only in finding the solutions of nonlinear matrix equations, but also in deriving parameter estimation algorithms for signal models [16][17][18][19][20][21][22].Furthermore, the iterative algorithms can give more accurate parameter estimates because of making full use of the observed data.Yun utilized the iterative methods to find the roots of nonlinear equations [23].Dehghan and Hajarian presented an iterative algorithm for solving the generalized coupled Sylvester matrix equations over the generalized centro-symmetric matrices [24].Wang et al. proposed an iterative method for a class of complex symmetric linear systems [25].Xu derived a Newton iterative algorithm to the parameter estimation for dynamical systems [26].Pei et al. used a monotone iterative technique to get the existence of positive solutions and to seek the positive minimal and maximal solutions for a Hadamard type fractional integro-differential equation [27].
As an optimization tool, the Newton method is useful for solving roots of nonlinear problems or deriving parameter estimation algorithms from observed data [28][29][30].For a long time, the Newton method has been utilized in much literature, such as transcendental equations, minimization and maximization problems, and numerical verification for solutions of nonlinear equations.Simpson noted that the Newton method can be used to give the generalization to systems of two equations and to solve optimization problems [31].Dennis provided Newton's method or quasi-Newton methods for multidimensional unconstrained optimization and nonlinear equation problems [32].Jürgen studied the accelerated convergence of the Newton method by molding a given function into a new one that the roots remain unchanged, but it looks nearly linear in a neighborhood of the root [33].Djoudi et al. presented a guided recursive Newton method involving inverse iteration to solve the transcendental eigenproblems by reducing it to a generalised linear eigenproblem [34].Benner described the numerical solution of large-scale Lyapunov equations, Riccati equations, and linear-quadratic optimal control problems [35].Seinfeld et al. used a quasi-Newton search technique and a barrier modification to enfore closed-loop stability for the H-infinity control problem [36].Liu et al. presented an iterative identification algorithm for Wiener nonlinear systems using the Newton method [37].The gradient method with the search directions defined by the gradient of the function at the current point has been developed for optimization problems.For instance, Curry used the gradient descent method for minimizing a nonlinear function of n real variables [38].Vranhatis et al. studied the development, convergence theory and numerical testing of a class of gradient unconstrained minimization algorithms with adaptive step-size [39].Hajarian proposed a gradient-based iterative algorithm to find the solutions of the general Sylvester discrete-time periodic matrix equations.
The moving data window with a fixed length is moved as time, which is a first-in-first-out sequence.When a new observation arrives, the data in this moving window are updated by including the new observation and eliminating the oldest one.The length of the moving window remains fixed.The algorithm computes parameter estimates using the observed data in the current window.Recently, Wang et al. presented a moving-window second order blind identification for time-varying transient operational modal parameter identification of linear time-varying structures [41].Al-Matouq and Vincent developed a multiple-window moving horizon estimation strategy that exploits constraint inactivity to reduce the problem size in long horizon estimation problems [42].This paper focuses on the parameter estimation problems of dual-frequency signal models.The main contributions of this paper are twofold.The basic idea is to present a gradient-based iterative (GI) algorithm and to estimate the parameters for signal models.Several estimation errors obtained by the Newton iterative and the moving data window based GI algorithms are compared to the errors given by the GI algorithm.
To close this section, we give the outline of this paper.Section 2 derives a GI parameter estimation algorithm.Sections 3 and 4 propose the Newton and moving data window gradient based iterative parameter estimation algorithms.Section 5 provides an example to verify the effectiveness of the proposed algorithms.Finally, some concluding remarks are given in Section 6.

The Gradient-Based Iterative Parameter Estimation Algorithm
Consider the following dual-frequency cosine signal model: where ω 1 > 0 and ω 2 > 0 are the angular frequencies, a > 0 and b > 0 are the amplitudes, t is a continuous-time variable, y(t) is the observation, and v(t) is a stochastic disturbance with zero mean and variance σ 2 .In actual engineering, we can only get discrete observed data.Suppose that the sampling data are where L is the data length and t i is the sampling time.
As we all know, signals include the sine signal, cosine signal, Gaussian signal, exponential signal, complex exponential signal, etc.Among them, the sine signal and the cosine signal are typical periodic signals whose waveforms are sine and cosine curves in mathematics.Many periodic signals can be decomposed into the sun of multiple sinusoidal signals with different frequencies and different amplitudes by the Fourier series [43].The cosine signal differs from the sine signal by π/2 in the initial phase.This paper takes the double-frequency cosine signal model as an example and derives the parameter estimation algorithms, which are also applicable to the sinusoidal signal models.
Use observed data y(t i ) and the model output to construct the criterion function The criterion function contains the parameters to be estimated, and it represents the error between the observed data and the model output.We hope that this error is as small as possible, which is equivalent to minimizing J 1 (θ) and obtaining the estimates of the parameter vector Letting the partial derivative of J 1 (θ) with respect to θ be zero gives Define the stacked output vector Y(L) and the stacked information matrix Φ(θ, L) as Define the information vector Let f (t i ) := a cos(ω 1 t i ) + b cos(ω 2 t i ) ∈ R, and the model output stacked vector Then, the gradient vector can be expressed as Let k = 1, 2, 3, • • • be an iterative variable and θk−1 be the estimate of θ at iteration k − 1.The gradient at θ = θk−1 is given by grad Using the negative gradient search and minimizing J 1 (θ), introducing an iterative step-size µ k , we can get the gradient-based iterative (GI) algorithm for dual-frequency signal models: The steps of the GI algorithm to compute θk are listed as follows: 1. Let k = 1, give a small number ε and set the initial value θ0 = 1 n /p 0 , p 0 is generally taken to be a large positive number, e.g., p 0 = 10 6 .2. Collect the observed data y(t i ), i = 1, 2, • • • , L, where L is the data length, form Y(L) by Equation (10).3. Form φk by Equation (13) and Φk by Equation (11).4. Form fk by Equation ( 14) and Fk by Equation (12). 5. Choose a larger µ k satisfying Equation ( 16), and update the estimate θk by Equation (15).6. Compare θk with θk−1 , if θk − θk−1 ε, then terminate this procedure and obtain the iteration time k and the estimate θk ; otherwise, increase k by 1 and go to Step 3.

The Newton Iterative Parameter Estimation Algorithm
The gradient method uses the first derivative, and its convergence rate is slow.The Newton method is discussed here, which requires the second derivative, with fast convergence speed.The following derives the Newton iterative algorithm to estimate the parameters of dual-frequency signal models.
According to the criterion function J 1 (θ) in Equation ( 2).Calculating the second partial derivative of the criterion function J 1 (θ) with respect to the parameter θ gives the Hessian matrix Let k = 1, 2, 3, • • • be an iterative variable and θk−1 be the estimate of θ at iteration k − 1.Based on the Newton search, we can derive the Newton iterative (NI) parameter estimation algorithm of dual-frequency signal models: The procedure for computing the parameter estimation vector θk using the NI algorithm in Equations ( 18)-( 35) is listed as follows:

The Moving Data Window Gradient-Based Iterative Algorithm
The gradient method and the Newton method use a batch of data, which is the data from t = 1 to t = L. Here, the moving window data from y(t i−p+1 ) to y(t i ), p used here is the length of the moving data window.Let t i be the current sampling time.The moving window data can be represented as y(t i ), y(t i−1 ), • • • , y(t i−p+1 ).These sampling data change with the sampling time t i .With the increasing of i, the data window moves forward constantly.New data are collected and old data are removed from the window.The following derives the moving data window gradient-based iterative algorithm of dual-frequency signal models.
Define the moving data window criterion function Define the information vector and the stacked information matrix . . .
Then, the gradient vector can be expressed as Let k = 1, 2, 3, • • • be an iterative variable, and θk (t i ) := [ âk (t i ), bk (t i ), ω1,k (t i ), ω2,k (t i )] T ∈ R 4 be the estimate of θ at iteration k and the sampling time t = t i .Minimizing J 2 (θ) and using the negative gradient search, we can obtain the moving data window gradient-based iterative (MDW-GI) algorithm: The steps of the MDW-GI algorithm in Equations ( 41)-( 48) to compute θk are listed as follows.
1. Pre-set the length of p, let i = p, give the parameter estimation accuracy ε and the iterative length k max = 500.2. To initialize, let k = 1, θ0 (t i ) = 1 n /p 0 , p 0 = 10 6 .3. Collect the observed data y(t i ), form Y(p, t i ) by Equation (43) go to Step 2; otherwise, obtain the parameter estimate θk (t i ), terminate this procedure.

Examples
Case 1: The numerical simulations of the three iterative algorithms.Consider the following dual-frequency cosine signal model: The parameter vector to be estimated is In simulation, {v(t)} is taken as a white noise sequence (stationary stochastic noise) with zero mean and variances σ 2 = 0.50 2 , σ 2 = 1.50 2 and σ 2 = 2.50 2 , respectively.Let t = kT, the sampling period T = 0.005, k = 2000, the data length L = 2000.Apply the proposed GI algorithm using the observed data to estimate the parameters of this signal model.The parameter estimates and their estimation errors δ := θk − θ / θ × 100% versus k are as shown in Table 1 and Figure 1.Apply the NI parameter estimation algorithm in Equations ( 18)-( 35) with the finite observed data to estimate the parameters.The data length and the variances are the same as the condition in the GI algorithm.The parameter estimates and their estimation errors are given in Table 2 and the estimation errors δ versus k are shown in Figure 2.
Apply the MDW-GI parameter estimation algorithm in Equations ( 41)-(48) to estimate the parameters of signal models.The length p of the moving data window is 100, 200 and 300, and {v(t)} is taken as a white noise sequence with zero mean and variance σ 2 = 0.50 2 , respectively.The simulation results are shown in Table 3 and Figure 3.In order to validate the obtained models, we use the MDW-GI parameter estimates from next-to-last row in Table 3 with p = 300 to construct the MDW-GI estimated model ŷ(t) = 1.46715 cos(0.07030t)+ 2.23197 cos(0.55621t).
The estimated outputs ŷ(t) and the observed signal model y(t) are plotted in Figure 4, where the solid-line denotes the actual signal y(t) and the dot-line denotes the estimated signal ŷ(t).
From the simulation results, we can draw the following conclusions:

•
The parameter estimation errors obtained by the presented three algorithms gradually decreasing trend as the iterative variable k increases.

•
The parameter estimation errors given by three algorithms become smaller with the noise variance decreasing.

•
Tables 1 and 2 and Figures 1 and 2 show that the proposed NI algorithm has faster convergence speed and more accurate parameter estimates than the GI algorithm under the same simulation conditions.

•
The larger the length p of the moving data window is, the faster the parameter estimation converge speed is (see Figure 3).Moreover, the larger the length p, the higher the accuracy is (see Table 3).

•
In the simulation, these three algorithms are fulfilled in the same conditions (σ 2 = 0.50 2 ), and the estimated models obtained by the MDW-GI and NI algorithms have higher accuracy than the GI algorithm.

•
The outputs of estimated signal ŷ(t) are very close to the actual signal model y(t).In other words, the estimated model can capture the dynamics of the signal.

Conclusions
This paper studies the direct parameter estimation algorithms for signal models only using the discrete observed data.By using the gradient search, we need to select a small step-size in order to ensure the convergence, and this will increase the search time and decrease the convergence rate.The proposed NI and MDW-GI algorithms have higher accuracy than the GI estimation algorithm for estimating the unknown parameters.The MWD-GI is used to obtain the parameter estimates at the current moment based on the estimates of previous data obtained at the moment time.Furthermore, the MDW-GI algorithm based on the moving data window technique can estimate the parameters of signal models in real time.The proposed algorithms can be extended to multi-frequency signal models.In the next work, we will consider the estimation of the initial phase of the signal models, that is to say, the signal model is a highly nonlinear function in regard to the frequencies and phases, and estimate all the parameters of dual-frequency signals including the unknown amplitudes, frequencies and initial phases simultaneously.

Figure 4 .
Figure 4.The estimated outputs ŷ(t) and the actual signal model y(t) versus t.Solid line: the actual output y(t), dot-line: the estimated signal ŷ(t).

Table 1 .
The GI parameter estimates and errors.

Table 2 .
The NI parameter estimates and errors.