Gradient-Based Iterative Identification for Wiener Nonlinear Dynamic Systems with Moving Average Noises

This paper focuses on the parameter identification problem for Wiener nonlinear dynamic systems with moving average noises. In order to improve the convergence rate, the gradient-based iterative algorithm is presented by replacing the unmeasurable variables with their corresponding iterative estimates, and to compute iteratively the noise estimates based on the obtained parameter estimates. The simulation results show that the proposed algorithm can effectively estimate the parameters of Wiener systems with moving average noises.


Introduction
In actual industry processes, block-oriented nonlinear systems are often introduced to model nonlinear systems.Block-oriented nonlinear systems can be commonly divided into Hammerstein systems and Wiener systems [1][2][3].Hammerstein systems consist of a linear block following a static nonlinear block [4][5][6].Wiener systems are composed of a linear block preceding a static nonlinear block [7][8][9].The output of Wiener systems is nonlinear, so the modeling for Wiener systems is more sophisticated than that for Hammerstein systems.Differing from the work in [8,9], this paper focuses on the identification problem for Wiener nonlinear systems with moving average noises which are called Wiener output error moving average (OEMA) systems.In most existing works, the nonlinear part of Wiener systems is assumed a linear combination or a piecewise-linear function [10], or has a invertible and monotone function representation over the operating range [11,12].Wang and Ding derived a least squares-based and a gradient-based iterative identification algorithms for Wiener nonlinear systems by separating one bilinear cost function into two linear cost functions [13].Hagenblad et al. presented a maximum likelihood method to identify Wiener models [14].
The stochastic gradient (SG) algorithm has less computational burden and slower convergence rate than the recursive least squares algorithm [15][16][17].Some new algorithms were presented to improve the convergence rate of the SG algorithm [18][19][20].For example, Ding et al. introduced the convergence index to the SG algorithm and obtained a faster convergence rate than stochastic gradient algorithm [21]; Liu et al. derived the multi-innovation extended stochastic gradient algorithm for controlled autoregressive moving average models by expanding the scalar innovation to an innovation vector and analyzed its performance in detail [22].Recently, the gradient-based iterative (GI) algorithm was also presented to improve the convergence rate of the SG algorithm [23][24][25].By making sufficient use of all the measured information, the GI algorithm can obtain a faster convergence rate than the SG algorithm.Wang et al. presented a gradient-based iterative identification algorithms for Box-Jenkins systems with finite measurement input-output data [26].Li et al. proposed a gradient based iterative algorithm to determine the parameters of a nonlinear system by using the negative gradient search [27].Zhang et al. derived a hierarchical gradient based iterative estimation algorithm for multivariable output error moving average systems using the hierarchical identification principle [28].
To the best of our knowledge, few contributions have addressed the modeling and estimation issues for Wiener nonlinear OEMA systems, which are the focus of this work.For Wiener nonlinear OEMA systems, our objectives are as follows: • To establish the identification model of the Wiener nonlinear OEMA system from input to output.
• To present a gradient-based iterative identification algorithm for the Wiener nonlinear OEMA model.
• To analyze the performances of the proposed algorithm using a numerical simulation, including the convergence rates and the estimation errors of this algorithm.
The rest of this paper is organized as follows.Section 2 establishes the identification model of the Wiener nonlinear OEMA system.Section 3 presents a gradient-based iterative identification algorithm for the Wiener nonlinear OEMA model.Section 4 provides an example to illustrate the effectiveness of the proposed algorithm.The conclusions of the paper are summarized in Section 5.

The Derivation of the Wiener OEMA Model
Let us firstly introduce some notations.The superscript T denotes the matrix transpose; 1 n represents an n-dimensional column vector whose elements are 1; the norm of a matrix X is defined by Figure 1.The Wiener nonlinear OEMA system.
Consider a Wiener nonlinear OEMA system shown in Figure 1: where u(t) is the system input, y(t) is the system output and v(t) is an additive noise with zero mean; the inner variable m(t)(namely, the output of the linear block) is unmeasurable; A(z), B(z) and D(z) are polynomials in the shift operator z The nonlinear part f (•) in the Wiener system is a polynomial of a known order as follows: where n γ is the polynomial orders.Equation ( 1) can be rewritten to In order to get unique parameter estimates, we introduce the key term separation technique presented in [7,29] and let the first coefficient of the nonlinear part be unity, i.e., γ 1 = 1.Then, we have Here m(t) in Equation ( 5) is called as the key term.Substituting Equation (4) into Equation (5) gives Define the information vectors and the parameter vectors Thus, Equation ( 4) can be written in a vector form Combining Equations ( 6) and ( 7), we can obtain the following identification model: The objective of this paper is to present a gradient based iterative identification algorithm to estimate the parameters a i , b i , γ i and d i for the Wiener nonlinear OEMA model using the auxiliary model identification idea in [23].

The Gradient-Based Iterative Algorithm
This section derives the gradient-based iterative identification algorithm for the Wiener nonlinear OEMA model.
Define the stacked output vector Y (N ), the stacked information vector Ψ(N ) and the white noise vector V (N ) as From Equations ( 8) to (11), we have Define a quadratic criterion function Let k = 1, 2, 3, • • • be an iteration variable, and θk be the iterative estimate of ϑ.
Using the negative gradient search for the optimization problem in Equation ( 13), we obtain the iterative algorithm of computing θk as follows: However, Ψ(N ) in Equations ( 14) and ( 15) containing unknown inner variable m(t) and the unmeasurable noise term v(t) lead to a difficulty that the iterative solution θk of ϑ is impossible to be computed.In order to solve this difficulty, the approach here is based on the auxiliary model idea.Let mk (t) and vk (t) be the estimate of m(t) and v(t) at iteration k, respectively, and define where Replacing φ s (t) and ϑ s in Equation (7) with φs,k (t) and θs,k , respectively, the iterative estimate mk (t) can be obtained by the following auxiliary model: Similarly, from Equation ( 8), the estimate v k (t) can be computed by Define Equations ( 14) and ( 15), we have Equations ( 14)-( 21) form the gradient-based iterative (GI) identification algorithm for the Wiener nonlinear OEMA model, which can be summarized as follows: The steps involved in computing the parameter estimate θk in the GI algorithm are listed as follows: 1. Collect the input-output data {u(t), y(t) : t = 1, 2, • • • , N } and form Y (N ) by Equation (25).
6. Compare θk with θk−1 : if θk − θk−1 2 ε, then terminate the procedure and obtain the iterative times k and estimate ϑ k ; otherwise, increment k by 1 and go to step 3.

Example
An example is given to demonstrate the feasibility of the proposed algorithm.Consider the following Wiener nonlinear OEMA system: For this example system, {u(t)} is taken as persistent excitation signal with zero mean and unit variance, and {v(t)} as a white noise process with zero mean and constant variance σ 2 = 0.20 2 and 0.40 2 .Here, we take the whole data lengths N = 1000, and then apply the proposed GI algorithm in Equations ( 22)-(32) to estimate the unknown parameters (a i , b i , γ i , d i ) of this nonlinear system.The parameter estimates and their errors with different noise variances are shown in Tables 1 and 2, and the parameter estimation errors δ versus k are shown in Figure 2, where δ := θk (t) − ϑ / ϑ .
For the sake of performance comparison, we apply the Newton iterative (NI) algorithm in [30] to estimate the unknown parameters of the proposed Wiener nonlinear system.The parameter estimation errors δ versus k are shown in Figure 3.
From Tables 1 and 2, and Figures 2 and 3, we can draw the following conclusions: • It is clear that the estimation errors become smaller (in general) as k increases: see the error curves in Figure 2 and the estimation errors of the last columns of Tables 1 and 2.
• A lower noise level results in a faster rate of convergence of the parameter estimates to the true parameters: see the error curves in Figure 2 and the estimation errors in Tables 1 and 2.
• The NI algorithm has a faster convergence rate than the GI algorithm, but the GI algorithm can generate more accurate parameter estimates than the NI algorithm: see the error curves in Figure 3.

Conclusions
In this paper we have derived the gradient-based iterative identification algorithm for Wiener nonlinear OEMA systems.The proposed algorithm can simultaneously estimate the parameters of the linear and nonlinear parts of Wiener nonlinear OEMA systems.The simulation results showed the parameters of Wiener nonlinear OEMA systems can be estimate effectively by the proposed algorithm.The method in the paper can be applied to study identification problems for other linear or nonlinear systems.