Gradient-Based Iterative Parameter Estimation Algorithms for Dynamical Systems from Observation Data

Abstract

It is well-known that mathematical models are the basis for system analysis and controller design. This paper considers the parameter identification problems of stochastic systems by the controlled autoregressive model. A gradient-based iterative algorithm is derived from observation data by using the gradient search. By using the multi-innovation identification theory, we propose a multi-innovation gradient-based iterative algorithm to improve the performance of the algorithm. Finally, a numerical simulation example is given to demonstrate the effectiveness of the proposed algorithms.

1. Introduction

Parameter estimation deals with the problem of building mathematical models of systems [1,2,3,4,5] based on observation data [6,7,8] and is the basis for system identification [9,10,11]. It has widespread applications in many areas [12,13,14,15,16]. For instance, system identification is used to get the appropriate models for control system designs [17,18], simulations and predictions [19,20] in control and system modelling [21,22]. Recently, Bottegal et al. proposed a two-experiment method to identify Wiener systems by using the data acquired from two separate experiments, in which the first experiment estimates the static nonlinearity and the second experiment identifies the linear block based on the estimated nonlinearity [23]. For nonlinear errors-in-variables systems contaminated with outliers, a robust identification approach is presented by means of the expectation maximization under the framework of the maximum likelihood estimation [24,25].

The recursive identification and the iterative identification are two important types of parameter estimation methods [26,27]. In recursive identification methods, the parameter estimates can be computed recursively in real-time [28,29]. Differing from the recursive methods, the basic idea of the iterative methods is to update the parameter estimates by using batch data [30,31,32]. In recent years, a number of iterative identification methods have been proposed for all kinds of systems. A data filtering-based iterative estimation algorithm is studied for an infinite impulse response filter with colored noise [33]. For  Box-Jenkins models, Liu et al. proposed a least squares-based iterative algorithm according to the auxiliary model identification idea and the iterative search [34]. Other related work includes the recursive algorithms [35,36] and the iterative algorithms [37,38,39].

The multi-innovation theory provides a new idea for system identification [40,41]. The innovation is the useful information that can improve the parameter estimation accuracy. The basic idea of the multi-innovation theory is to expand the dimension of the innovation and to enhance the data utilization. A brief review of the multi-innovation algorithms is listed as follows. An adaptive filtering based multi-innovation stochastic gradient algorithm was derived for bilinear systems with colored noise and can give small parameter estimation errors as the innovation length increases [42]. A  multi-innovation gradient algorithm was developed based on the Kalman filtering to solve the joint state and parameter estimation problem for a nonlinear state space system with time-delay [43].

Identification is to fit a closest model of a practical system from observation data. The commonly used techniques are some optimization methods. These methods include linear programming and convex mixed integer nonlinear programs [44]. Some related optimization algorithms and solvers have been proposed for complete global optimization problems [45], such as the general algebraic modeling system (GAMS) and mixed-integer nonlinear programming (MINLP) solver [46] and the mixed-integer linear programming (MILP) and MINLP approaches for solving nonlinear discrete transportation problem [47]. This paper focuses on the parameter identification problems of controlled autoregressive systems by using the gradient search [48] and the multi-innovation identification theory [49]. The basic idea is to use the iterative technique for computing the parameter estimation through defining and minimizing two quadratic criterion functions. The main contributions of this paper are as follows.

• Based on the gradient search, a gradient-based iterative algorithm is presented for identifying the parameters of controlled autoregressive systems.
• A multi-innovation gradient-based iterative algorithm is derived for improving the performance of the algorithm by using the multi-innovation identification theory.

The outlines of this paper are organized as follows. Section 2 defines some definitions and derives the identification model of controlled autoregressive systems. Section 3 derives a gradient-based iterative algorithm. Section 4 proposes a multi-innovation gradient-based iterative algorithm. Section 5 offers an example to illustrate the effectiveness of the proposed algorithms. Finally, Section 6 gives some concluding remarks.

2. The System Description

Let us introduce some symbols used in this paper. The symbol ${\mathit{I}}_n$ denotes an identity matrix of appropriate size ($n\times n$); ${\mathit{1}}_n$ stands for an n-dimensional column vector whose elements are 1; the superscript T stands for the vector/matrix transpose; the norm of a matrix (or a column vector) $\mathit{X}$ is defined by ${\parallel \mathit{X}\parallel }^2:=\mathrm{tr}[\mathit{X}{\mathit{X}}^{\mathrm{T}}]$.

Consider the dynamic stochastic systems described by the following controlled autoregressive (CAR) model:

$$A(z)y(t)=B(z)u(t)+v(t),$$

(1)

where $\{u(t)\}$ is the input sequence of the system and $\{y(t)\}$ is the output sequence of the system, $\{v(t)\}$ is a white noise sequence with zero mean, $A(z)$ and $B(z)$ are polynomials in the unit backward shift operator [$z^{-1}y(t)=y(t-1)$, $zy(t)=y(t+1)$], and defined as

$$\begin{array}{ccc}\hfill A(z)& :=& 1+a_1{z}^{-1}+a_2{z}^{-2}+\cdots +a_{n_a}{z}^{-n_a},\hfill \\ \hfill B(z)& :=& b_1{z}^{-1}+b_2{z}^{-2}+\cdots +b_{n_b}{z}^{-n_b}.\hfill \end{array}$$

Assume that $n_a$ and $n_b$ are known, and $y(t)=0$, $u(t)=0$ and $v(t)=0$ for $t⩽0$.

Let $n:=n_a+n_b$, define the parameter vectors:

$$\begin{array}{ccc}\hfill \mathit{\vartheta }& :=& \left[\begin{array}{c}\mathit{a}\\ \mathit{b}\end{array}\right]\in {\mathbb{R}}^n,\hfill \\ \hfill \mathit{a}& :=& {[a_1,a_2,\cdots ,a_{n_a}]}^{\mathrm{T}}\in {\mathbb{R}}^{n_a},\hfill \\ \hfill \mathit{b}& :=& {[b_1,b_2,\cdots ,b_{n_b}]}^{\mathrm{T}}\in {\mathbb{R}}^{n_b},\hfill \end{array}$$

and the corresponding information vectors:

$$\begin{array}{ccc}\hfill \mathit{\phi }(t)& :=& \left[\begin{array}{c}{\mathit{\phi }}_a(t)\\ {\mathit{\phi }}_b(t)\end{array}\right]\in {\mathbb{R}}^n,\hfill \\ \hfill {\mathit{\phi }}_a(t)& :=& {[-y(t-1),-y(t-2),\cdots ,-y(t-n_a)]}^{\mathrm{T}}\in {\mathbb{R}}^{n_a},\hfill \\ \hfill {\mathit{\phi }}_b(t)& :=& {[u(t-1),u(t-2),\cdots ,u(t-n_b)]}^{\mathrm{T}}\in {\mathbb{R}}^{n_b}.\hfill \end{array}$$

Through the above definitions, the system in Equation (1) can be rewritten as

$$\begin{array}{ccc}\hfill y(t)& =& [1-A(z)]y(t)+B(z)u(t)+v(t)\hfill \\ & =& (-a_1{z}^{-1}-a_2{z}^{-2}-\cdots -a_{n_a}{z}^{-n_a})y(t)+(b_1{z}^{-1}+b_2{z}^{-2}+\cdots +b_{n_b}{z}^{-n_b})u(t)+v(t)\hfill \\ & =& -a_{1}y(t-1)-a_{2}y(t-2)-\cdots -a_{n_a}y(t-n_a)\hfill \\ & & +b_{1}u(t-1)+b_{2}u(t-2)+\cdots +b_{n_b}u(t-n_b)+v(t)\hfill \\ & =& [-y(t-1),-y(t-2),\cdots ,-y(t-n_a)]\mathit{a}+[u(t-1),u(t-2),\cdots ,u(t-n_b)]\mathit{b}+v(t)\hfill \\ & =& {\mathit{\phi }}_a^{\mathrm{T}}(t)\mathit{a}+{\mathit{\phi }}_b^{\mathrm{T}}(t)\mathit{b}+v(t)\hfill \\ \hfill & =& {\mathit{\phi }}^{\mathrm{T}}(t)\mathit{\vartheta }+v(t).\hfill \end{array}$$

(2)

This identification model for System in (1) involves two different parameter vectors $\mathit{a}$ and $\mathit{b}$, which are merged to a new vector $\mathit{\vartheta }$, and two different information vectors ${\mathit{\phi }}_a(t)$ and ${\mathit{\phi }}_b(t)$, which are merged to a new vector $\mathit{\phi }(t)$ composed of available input-output data $u(t-i)$ and $y(t-i)$. The objective of this paper is by means of the gradient search and based on the multi-innovation identification theory to derive new identification algorithms for estimating the system parameter vectors from available data $\{u(t),y(t)\}$.

3. The Gradient-Based Iterative Algorithm

In this section, we first define a criterion function and present a gradient-based iterative algorithm by using the gradient search. In addition, a brief discussion about the choice of the iterative step-size is given in this section.

Let $k=1,2,3,\cdots$ be an iterative variable and ${\widehat{\mathit{\vartheta }}}_k:=\left[\begin{array}{c}{\widehat{\mathit{a}}}_k\\ {\widehat{\mathit{b}}}_k\end{array}\right]$ be the iterative estimate of $\mathit{\vartheta }=\left[\begin{array}{c}\mathit{a}\\ \mathit{b}\end{array}\right]$ at iteration k. ${\lambda }_{max}[\mathit{X}]$ is the largest eigenvalue of the symmetric matrix $\mathit{X}$.

The observation data are basic for parameter identification algorithms. This paper uses a batch of data with the length L and is based on Model (2), we define the stacked output data vector $\mathit{Y}(L)$ and the stacked data matrix $\mathit{\Phi }(L)$ as

$$\begin{array}{ccc}\hfill \mathit{Y}(L)& :=& \left[\begin{array}{c}y(1)\\ y(2)\\ ⋮\\ y(L)\end{array}\right]\in {\mathbb{R}}^L,\mathit{\Phi }(L):=\left[\begin{array}{c}{\mathit{\phi }}^{\mathrm{T}}(1)\\ {\mathit{\phi }}^{\mathrm{T}}(2)\\ ⋮\\ {\mathit{\phi }}^{\mathrm{T}}(L)\end{array}\right]\in {\mathbb{R}}^{L\times n}.\hfill \end{array}$$

Define a static criterion function

$$J_1(\mathit{\vartheta }):=\frac{1}{2}\sum _{t=1}^L{[y(t)-{\mathit{\phi }}^{\mathrm{T}}(t)\mathit{\vartheta }]}^2,$$

which can be equivalently expressed as

$$J_1(\mathit{\vartheta }):=\frac{1}{2}{\parallel \mathit{Y}(L)-\mathit{\Phi }(L)\mathit{\vartheta }\parallel }^2.$$

By means of the negative gradient search, computing the partial derivative of $J_1(\mathit{\vartheta })$ with respect to $\mathit{\vartheta }$, we can obtain the iterative relation:

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\vartheta }}}_k& =& {\widehat{\mathit{\vartheta }}}_{k-1}-\mu \mathrm{grad}[J_2({\widehat{\mathit{\vartheta }}}_{k-1})]\hfill \\ & =& {\widehat{\mathit{\vartheta }}}_{k-1}+\mu {\mathit{\Phi }}^{\mathrm{T}}(L)[\mathit{Y}(L)-\mathit{\Phi }(L){\widehat{\mathit{\vartheta }}}_{k-1}]\hfill \\ & =& [{\mathit{I}}_n-\mu {\mathit{\Phi }}^{\mathrm{T}}(L)\mathit{\Phi }(L)]{\widehat{\mathit{\vartheta }}}_{k-1}+\mu {\mathit{\Phi }}^{\mathrm{T}}(L)\mathit{Y}(L),\hfill \end{array}$$

where $\mu >0$ is an iterative step-size or a convergence factor. The above equation can be seen as a discrete-time system. To ensure the convergence of ${\widehat{\mathit{\vartheta }}}_k$, all the eigenvalues of matrix $[\mathit{I}-\mu {\widehat{\mathit{\Phi }}}^{\mathrm{T}}(L)\widehat{\mathit{\Phi }}(L)]$ must be in the unit circle, that is to say $\mu$ should satisfy $-{\mathit{I}}_n⩽{\mathit{I}}_n-\mu {\mathit{\Phi }}^{\mathrm{T}}(L)\mathit{\Phi }(L)⩽{\mathit{I}}_n$, or $\mathit{0}⩽\mu {\mathit{\Phi }}^{\mathrm{T}}(L)\mathit{\Phi }(L)⩽2{\mathit{I}}_n$, so a conservative choice of $\mu$ is

$$\mu ⩽\frac{2}{{\lambda }_{max}[{\mathit{\Phi }}^{\mathrm{T}}(L)\mathit{\Phi }(L)]}=2{\lambda }_{max}^{-1}[{\mathit{\Phi }}^{\mathrm{T}}(L)\mathit{\Phi }(L)].$$

In order to avoid computing the complicated eigenvalues of a square matrix and to reduce computational cost, we use the trace of the matrix and take another way for selecting the step-size

$$\mu ⩽\frac{2}{{\parallel \mathit{\Phi }(L)\parallel }^2}=2{\parallel \mathit{\Phi }(L)\parallel }^{-2}.$$

Then we can obtain the gradient-based iterative (GI) algorithm for estimating the parameter vector $\mathit{\vartheta }$ of the CAR system in (1) [4]:

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\vartheta }}}_k& =& {\widehat{\mathit{\vartheta }}}_{k-1}+\mu {\mathit{\Phi }}^{\mathrm{T}}(L)[\mathit{Y}(L)-\mathit{\Phi }(L){\widehat{\mathit{\vartheta }}}_{k-1}],k=1,2,3,\cdots \hfill \end{array}$$

(3)

$$\begin{array}{ccc}\hfill \mu & =& 2{\lambda }_{max}^{-1}[{\mathit{\Phi }}^{\mathrm{T}}(L)\mathit{\Phi }(L)],\mathrm{or}\mu =2{\parallel \mathit{\Phi }(L)\parallel }^{-2},\hfill \end{array}$$

(4)

$$\begin{array}{ccc}\hfill \mathit{Y}(L)& =& {[y(1),y(2),\cdots ,y(L)]}^{\mathrm{T}},\hfill \end{array}$$

(5)

$$\begin{array}{ccc}\hfill \mathit{\Phi }(L)& =& {[\mathit{\phi }(1),\mathit{\phi }(2),\cdots ,\mathit{\phi }(L)]}^{\mathrm{T}},\hfill \end{array}$$

(6)

$$\begin{array}{ccc}\hfill \mathit{\phi }(t)& =& \left[\begin{array}{c}{\mathit{\phi }}_a(t)\\ {\mathit{\phi }}_b(t)\end{array}\right],t=1,2,\cdots ,L,\hfill \end{array}$$

(7)

$$\begin{array}{ccc}\hfill {\mathit{\phi }}_a(t)& =& {[-y(t-1),-y(t-2),\cdots ,-y(t-n_a)]}^{\mathrm{T}},\hfill \end{array}$$

(8)

$$\begin{array}{ccc}\hfill {\mathit{\phi }}_b(t)& =& {[u(t-1),u(t-2),\cdots ,u(t-n_b)]}^{\mathrm{T}}.\hfill \end{array}$$

(9)

The steps of computing ${\widehat{\mathit{\vartheta }}}_k$ involved in the GI algorithm in Equations (3)–(9) are summarized in the following. The pseudo-code of implementing the gradient-based iterative algorithm is shown in Algorithm 1.

• For $t⩽0$, all the variables are set to zero. Let $k=1$, give the data length L ($L\gg n$) and set the initial values: ${\widehat{\mathit{\vartheta }}}_0={\mathit{1}}_n/p_0$, $p_0={10}^6$, and the parameter estimation accuracy $\epsilon$.
• Collect the input and output data $u(t)$ and $y(t)$, $t=1$, 2, ⋯, L.
• Form the information vectors ${\mathit{\phi }}_a(t)$, ${\mathit{\phi }}_b(t)$ and $\mathit{\phi }(t)$ using Equations (8)–(9) and (7).
• Construct the stacked output vector $\mathit{Y}(L)$ by Equation (5) and the stacked information matrix $\mathit{\Phi }(L)$ by Equation (6), select a large $\mu$ according to Equation (4).
• Update the parameter vector estimate ${\widehat{\mathit{\vartheta }}}_k$ using Equation (3).
• Compare ${\widehat{\mathit{\vartheta }}}_k$ with ${\widehat{\mathit{\vartheta }}}_{k-1}$: If $\parallel {\widehat{\mathit{\vartheta }}}_k-{\widehat{\mathit{\vartheta }}}_{k-1}\parallel >\epsilon$, increase k by 1 and go to step 5; otherwise, obtain the iteration k and the parameter estimation vector ${\widehat{\mathit{\vartheta }}}_k$.

Algorithm 1 The pseudo-code of implementing the gradient-based iterative algorithm.

Data:  $\{u(t),y(t),t=1,2,\cdots ,L\}$, $n_a$, $n_b$, $\epsilon$, N   Result:  ${\widehat{\mathit{\vartheta }}}_k$  1: Initialization:  ${\widehat{\mathit{\vartheta }}}_0={[1,1,\cdots ,1]}^{\mathrm{T}}/p_0\in {\mathbb{R}}^{n_a+n_b}$,  $p_0={10}^6$. 2: for$t=1:L$do  3:     Form the information vectors ${\mathit{\phi }}_a(t)$, ${\mathit{\phi }}_b(t)$ and $\mathit{\phi }(t)$ using (8)–(9) and (7).  4: end for  5: Construct the stacked output vector $\mathit{Y}(L)$ by (5).  6: Construct the stacked information matrix $\mathit{\Phi }(L)$ by (6).  7: for$k=1$do  8:     Compute the step-size $\mu$ using (4):  9:     $\mu =2{\lambda }_{max}^{-1}[{\mathit{\Phi }}^{\mathrm{T}}(L)\mathit{\Phi }(L)]$ or $\mu ={2\parallel \mathit{\Phi }(L)\parallel }^{-2}$;  10:    Update the parameter vector estimate ${\widehat{\mathit{\vartheta }}}_k$ using (3):  11:    ${\widehat{\mathit{\vartheta }}}_{k+1}={\widehat{\mathit{\vartheta }}}_k+\mu {\mathit{\Phi }}^{\mathrm{T}}(L)[\mathit{Y}(L)-\mathit{\Phi }(L){\widehat{\mathit{\vartheta }}}_k]$;  12:    If $\parallel {\widehat{\mathit{\vartheta }}}_k-{\widehat{\mathit{\vartheta }}}_{k-1}\parallel >\epsilon$  13:    $k=k+1$;  14:    else  15:    Obtain the iteration k and the parameter estimation vector ${\widehat{\mathit{\vartheta }}}_k$, break;  16:    end  17: end for

The flowchart of computing ${\widehat{\mathit{\vartheta }}}_k$ in the GI algorithm is shown in Figure 1.

Remark 1.

The criterion function is quadratic and and a convex optimization problem. For a static optimization problem, a constant step-size is often used for simplification in parameter estimation algorithms.

Remark 2.

The computation is an important property of an algorithm, and is represented by the floating-point operation of multiplications/divisions and additions/subtractions (the flop for short).

Table 1. The computational efficiency of the gradient-based iterative (GI) algorithm.

Variables	Expressions	Multiplications	Additions
${\widehat{\mathit{\vartheta }}}_k$	${\widehat{\mathit{\vartheta }}}_k={\widehat{\mathit{\vartheta }}}_{k-1}+\mu {\mathit{\Phi }}^{\mathrm{T}}(L)[\mathit{Y}(L)-\mathit{\Phi }(L){\widehat{\mathit{\vartheta }}}_{k-1}]\in {\mathbb{R}}^n$	$(2nL+n)k$	$2nLk$
$\mu$	$\mu ⩽{2\parallel \mathit{\Phi }(L)\parallel }^{-2}\in \mathbb{R}$	$nL+1$	$nL-1$
Sum		$(2nL+n)k+nL+1$	$2nLk+nL-1$
Total flops		$N_1:=(4nL+n)k+2nL$

gives the computational efficiency of the GI algorithm.

Remark 3.

The gradient algorithm can be used to find the optimal solution for quadratic optimization problems and nonlinear optimization problems. It can handle not only the linear regression systems with the known information vector, but also the linear and nonlinear systems with the unknown information vector. However, Under the same data length L, the estimation accuracy of the GI algorithm becomes high as the iterative index k increases, and when the data length L increases, the estimation accuracy also becomes high. Thus, we choose a sufficient large data length in practice

Remark 4.

If we choose the step-size as $\mu ={2\parallel \mathit{\Phi }(L)\parallel }^{-2}$ and eliminate the intermediate variables $\mathit{Y}(L)$ and $\mathit{\Phi }(L)$, the GI algorithm can be equivalently transformed into

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\vartheta }}}_k& =& {\widehat{\mathit{\vartheta }}}_{k-1}+2{\left[\sum _{j=1}^L{\parallel \mathit{\phi }(j)\parallel }^2\right]}^{-1}\left[\sum _{j=1}^L\mathit{\phi }(j)[y(j)-{\mathit{\phi }}^{\mathrm{T}}(j){\widehat{\mathit{\vartheta }}}_{k-1}]\right].\hfill \end{array}$$

(10)

In order to improve the performance of the GI algorithm, we introduce a forgetting factor $0⩽\lambda ⩽1$ and obtain the forgetting factor GI algorithm:

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\vartheta }}}_k& =& {\widehat{\mathit{\vartheta }}}_{k-1}+2{\left[\sum _{t=1}^L{\lambda }^{L-t}{\parallel \mathit{\phi }(t)\parallel }^2\right]}^{-1}\left[\sum _{t=1}^L{\lambda }^{L-t}\mathit{\phi }(t)[y(t)-{\mathit{\phi }}^{\mathrm{T}}(t){\widehat{\mathit{\vartheta }}}_{k-1}]\right].\hfill \end{array}$$

(11)

4. The Multi-Innovation Gradient-Based Iterative Algorithm

The multi-innovation identification algorithm uses the data in a moving data window to update the parameter estimates. The moving data window, which is also called the dynamical data window, moves along with t. The length of the moving data window can be variable or invariable. In this section, we derive a multi-innovation gradient-based iterative algorithm with the constant window length.

Consider the newest p data from $j=t-p+1$ to $j=t$ (p represents the data length, i.e., the length of the moving data window), and define the stacked output vector $\mathit{Y}(p,t)$ and the stacked information matrix $\mathit{\Phi }(p,t)$ as

$$\begin{array}{ccc}\hfill \mathit{Y}(p,t)& :=& \left[\begin{array}{c}y(t)\\ y(t-1)\\ ⋮\\ y(t-p+1)\end{array}\right]\in {\mathbb{R}}^p,\mathit{\Phi }(p,t):=\left[\begin{array}{c}{\mathit{\phi }}^{\mathrm{T}}(t)\\ {\mathit{\phi }}^{\mathrm{T}}(t-1)\\ ⋮\\ {\mathit{\phi }}^{\mathrm{T}}(t-p+1)\end{array}\right]\in {\mathbb{R}}^{p\times n}.\hfill \end{array}$$

(12)

According to the identification model in (2), define a dynamic data window criterion function:

$$J_2(\mathit{\vartheta }):=\frac{1}{2}\sum _{j=t-p+1}^t{[y(j)-{\mathit{\phi }}^{\mathrm{T}}(j)\mathit{\vartheta }]}^2=\frac{1}{2}{\parallel \mathit{Y}(p,t)-\mathit{\Phi }(p,t)\mathit{\vartheta }\parallel }^2.$$

Using the negative gradient search to minimize $J_2(\mathit{\vartheta })$, we can obtain the following iterative relation:

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\vartheta }}}_k(t)& =& {\widehat{\mathit{\vartheta }}}_{k-1}(t)-\mu (t)\mathrm{grad}[J_6({\widehat{\mathit{\vartheta }}}_{k-1}(t))]\hfill \\ & =& {\widehat{\mathit{\vartheta }}}_{k-1}(t)+\mu (t){\mathit{\Phi }}^{\mathrm{T}}(p,t)[\mathit{Y}(p,t)-\mathit{\Phi }(p,t){\widehat{\mathit{\vartheta }}}_{k-1}(t)]\hfill \\ & =& [{\mathit{I}}_n-\mu (t){\mathit{\Phi }}^{\mathrm{T}}(p,t)\mathit{\Phi }(p,t)]{\widehat{\mathit{\vartheta }}}_{k-1}(t)+\mu (t){\mathit{\Phi }}^{\mathrm{T}}(p,t)\mathit{Y}(p,t),\hfill \end{array}$$

(13)

where $\mu (t)⩾0$ is the step-size of the time t at iteration k. Similarly, in order to guarantee the convergence of the parameter estimation vector ${\widehat{\mathit{\vartheta }}}_k(t)$, $\mu (t)$ can be conservatively chosen as

$$\mu (t)⩽\frac{2}{{\lambda }_{max}[{\mathit{\Phi }}^{\mathrm{T}}(p,t)\mathit{\Phi }(p,t)]}=2{\lambda }_{max}^{-1}[{\mathit{\Phi }}^{\mathrm{T}}(p,t)\mathit{\Phi }(p,t)].$$

(14)

In consideration of the computational cost of the eigenvalue calculation, by using the norm $\mu (t)$ is to take as

$$\mu (t)⩽\frac{2}{{\parallel \mathit{\Phi }(p,t)\parallel }^2}=2{\parallel \mathit{\Phi }(p,t)\parallel }^{-2}.$$

(15)

Equations (12)–(15) and (7)–(9) form the multi-innovation gradient-based iterative (MIGI) algorithm for estimating the parameter vector $\mathit{\vartheta }$ [4]:

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\vartheta }}}_k(t)& =& {\widehat{\mathit{\vartheta }}}_{k-1}(t)+\mu (t){\mathit{\Phi }}^{\mathrm{T}}(p,t)[\mathit{Y}(p,t)-\mathit{\Phi }(p,t){\widehat{\mathit{\vartheta }}}_{k-1}(t)],k=1,2,3,\cdots \hfill \end{array}$$

(16)

$$\begin{array}{ccc}\hfill \mu (t)& ⩽& 2{\lambda }_{max}^{-1}[{\mathit{\Phi }}^{\mathrm{T}}(p,t)\mathit{\Phi }(p,t)],\mathrm{or}\mu (t)⩽2{\parallel \mathit{\Phi }(p,t)\parallel }^{-2},\hfill \end{array}$$

(17)

$$\begin{array}{ccc}\hfill \mathit{Y}(p,t)& =& {[y(t),y(t-1),\cdots ,y(t-p+1)]}^{\mathrm{T}},\hfill \end{array}$$

(18)

$$\begin{array}{ccc}\hfill \mathit{\Phi }(p,t)& =& {[\mathit{\phi }(t),\mathit{\phi }(t-1),\cdots ,\mathit{\phi }(t-p+1)]}^{\mathrm{T}},\hfill \end{array}$$

(19)

$$\begin{array}{ccc}\hfill \mathit{\phi }(t)& =& \left[\begin{array}{c}{\mathit{\phi }}_a(t)\\ {\mathit{\phi }}_b(t)\end{array}\right],\hfill \end{array}$$

(20)

$$\begin{array}{ccc}\hfill {\mathit{\phi }}_a(t)& =& {[-y(t-1),-y(t-2),\cdots ,-y(t-n_a)]}^{\mathrm{T}},\hfill \end{array}$$

(21)

$$\begin{array}{ccc}\hfill {\mathit{\phi }}_b(t)& =& {[u(t-1),u(t-2),\cdots ,u(t-n_b)]}^{\mathrm{T}}.\hfill \end{array}$$

(22)

The identification steps of the MIGI algorithm in Equations (16)–(22) for computing ${\widehat{\mathit{\vartheta }}}_k(t)$ are listed as follows.

• For $t⩽0$, all the variables are set to zero. Let $t=1$, give the data length p ($p\gg n$) and set the initial values: ${\widehat{\mathit{\vartheta }}}_0(t)={\mathit{1}}_n/p_0$, $p_0={10}^6$, the maximum iteration $k_{max}$ and the accuracy $\epsilon$.
• Let $k=1$, collect the input and output data $u(t)$ and $y(t)$.
• Form the information vectors ${\mathit{\phi }}_a(t)$, ${\mathit{\phi }}_b(t)$ and $\mathit{\phi }(t)$ using Equations (21)–(22) and (20).
• Construct the stacked output vector $\mathit{Y}(p,t)$ by Equation (18) and the stacked information matrix $\mathit{\Phi }(p,t)$ by Equation (19), select a large $\mu (t)$ according to Equation (17).
• Update the parameter vector estimate ${\widehat{\mathit{\vartheta }}}_k(t)$ using Equation (16).
• If $k<k_{max}$, increase k by 1 and go to Step 5; otherwise, proceed with the next step.
• Compare ${\widehat{\mathit{\vartheta }}}_k(t)$ with ${\widehat{\mathit{\vartheta }}}_{k-1}(t)$: If $\parallel {\widehat{\mathit{\vartheta }}}_k(t)-{\widehat{\mathit{\vartheta }}}_{k-1}(t)\parallel >\epsilon$, set ${\widehat{\mathit{\vartheta }}}_0(t+1):={\widehat{\mathit{\vartheta }}}_k(t)$ and increase k by 1 and go to step 2; otherwise, obtain the parameter estimation vector ${\widehat{\mathit{\vartheta }}}_k(t)$.

The flowchart of computing computing the parameter estimation vector ${\widehat{\mathit{\vartheta }}}_k(t)$ is shown in Figure 2.

Remark 5.

The multi-innovation gradient identification method can improve the parameter estimation accuracy by expanding the dimension of the innovation and making full use of the system information. The MIGI algorithm in Equations (16)–(22) can be seen as an application of the multi-innovation theory in the iterative identification. In particular, we use the newest p data from $j=t-p+1$ to $j=t$ to do the iterative estimation. When $k=k_{max}$, move the data window forward to the next moment, introduce new observation data and eliminate the oldest data to keep p data in the data window. The MISG algorithm enhances the utilization of data and improves the parameter estimation accuracy.

Remark 6.

When take $p=t=L$, the MIGI algorithm reduces to the GI algorithm. That is the MIGI algorithm is the extension of the GI algorithm, or the GI algorithm is the special case of the MIGI algorithm.

Furthermore, the proposed approaches in the paper can combine other mathematical tools [50,51,52] and statistical strategies [53,54,55,56,57,58,59,60] to study the parameter estimation problems of linear and nonlinear systems with different disturbance noises [61,62,63,64], and can be applied to other literature [65,66,67,68] such as signal processing [69,70,71,72,73,74] and neural networks [75,76,77].

5. Example

In industrial processes, some control task can be simplified into a water tank control plant, as is shown in Figure 3 [78].

In this system, the manipulated variable is the position of the inlet water valve, denoted $u(t)$; the measured variable is the water level in the tank, denoted $y(t)$. For a two-level water tank plant, it can be described by a second-order controlled autoregressive system, we assume that its mathematical model is given by

$$\begin{array}{ccc}\hfill A(z)y(t)& =& B(z)u(t)+v(t),\hfill \\ \hfill A(z)& =& 1+a_1{z}^{-1}+a_2{z}^{-2}=1+1.35z^{-1}+0.75z^{-2},\hfill \\ \hfill B(z)& =& b_1{z}^{-1}+b_2{z}^{-2}=1.68z^{-1}+2.32z^{-2}.\hfill \end{array}$$

The parameter vector to be identified is given by

$$\mathit{\vartheta }={[a_1,a_2,b_1,b_2]}^{\mathrm{T}}={[1.35,0.75,1.68,2.32]}^{\mathrm{T}}.$$

In simulation, the input $\{u(t)\}$ is taken as an uncorrelated uniform distribution random signal sequence with zero mean and unit variance, $\{v(t)\}$ is taken as a normal distribution white noise sequence with zero mean and variance ${\sigma }^2$. Using the simulation model parameters and the input signal generates the output signal $\{y(t)\}$.

Taking the noise variances ${\sigma }^2={0.20}^2$, ${\sigma }^2={1.00}^2$ and ${\sigma }^2={2.00}^2$, respectively, the corresponding noise-to-signal ratios were ${\delta }_{\mathrm{ns}}=13.51\%$, ${\delta }_{\mathrm{ns}}=67.55\%$, and ${\delta }_{\mathrm{ns}}=135.11\%$. Setting the data length $L=3000$, the convergence factor $\mu ={\lambda }_{max}^{-1}[{\mathit{\Phi }}^{\mathrm{T}}(L)\mathit{\Phi }(L)]$, applying the GI algorithm and the input-output data $\{u(t),y(t)\}$ to estimate the parameters of this example system, the GI parameter estimates and errors versus with different noise variances are shown in

Table 2. The GI estimates and their errors with ${\sigma }^2={0.20}^2$.

k	${\mathit{a}}_1$	${\mathit{a}}_2$	${\mathit{b}}_1$	${\mathit{b}}_2$	$\mathit{\delta }(\%)$
1	0.36787	−0.03138	0.08512	0.00447	94.61743
2	0.49364	0.09219	0.16729	0.03988	90.39550
5	0.72013	0.30680	0.39043	0.18999	80.09229
10	0.88253	0.44424	0.69306	0.47742	66.48636
20	1.02561	0.54320	1.10288	0.97583	46.48051
50	1.22656	0.67165	1.56551	1.80449	16.83784
100	1.32469	0.73450	1.67354	2.21545	3.34572
150	1.34439	0.74715	1.68095	2.29850	0.68915
200	1.34836	0.74970	1.68146	2.31527	0.16042
True values	1.35000	0.75000	1.68000	2.32000	-

,

Table 3. The GI estimates and their errors with ${\sigma }^2={1.00}^2$.

k	${\mathit{a}}_1$	${\mathit{a}}_2$	${\mathit{b}}_1$	${\mathit{b}}_2$	$\mathit{\delta }(\%)$
1	0.39074	−0.07246	0.05593	0.00225	95.24273
2	0.51599	0.05176	0.11093	0.02612	91.71095
5	0.75720	0.28762	0.26578	0.12989	83.37742
10	0.93536	0.45376	0.49189	0.34387	72.56593
20	1.05985	0.55458	0.84302	0.75447	55.60085
50	1.20990	0.65849	1.39092	1.55649	25.60909
100	1.30606	0.72444	1.63629	2.08855	7.40476
150	1.33488	0.74426	1.67889	2.24897	2.23868
200	1.34356	0.75024	1.68613	2.29741	0.74605
True values	1.35000	0.75000	1.68000	2.32000	-

and

Table 4. The GI estimates and their errors with ${\sigma }^2={2.00}^2$.

k	${\mathit{a}}_1$	${\mathit{a}}_2$	${\mathit{b}}_1$	${\mathit{b}}_2$	$\mathit{\delta }(\%)$
1	0.41321	−0.11502	0.02712	0.00069	95.88923
2	0.53665	0.00817	0.05426	0.01247	93.10387
5	0.79437	0.26457	0.13329	0.06494	87.03542
10	1.00553	0.47258	0.25709	0.18223	80.07592
20	1.13655	0.59653	0.47660	0.43831	69.11109
50	1.21869	0.66123	0.95452	1.06581	44.78973
100	1.28208	0.70753	1.37278	1.68366	21.85326
150	1.31416	0.73098	1.55522	1.99714	10.70952
200	1.33042	0.74288	1.63461	2.15631	5.25922
True values	1.35000	0.75000	1.68000	2.32000	-

, the GI estimation errors $\delta :=\parallel \widehat{\mathit{\vartheta }}(t)-\mathit{\vartheta }\parallel /\parallel \mathit{\vartheta }\parallel$ versus k are shown in Figure 4 with ${\sigma }^2={0.20}^2$, ${\sigma }^2={1.00}^2$ and ${\sigma }^2={2.00}^2$, and the GI estimates of the parameters $a_1$, $a_2$, $b_1$ and $b_2$ versus k are shown in Figure 5 for ${\sigma }^2={0.20}^2$.

From Table 2, Table 3 and Table 4 and Figure 4 and Figure 5, we can draw the following conclusions.

• As the noise levels decrease, the GI algorithm can give more accurate parameter estimates—see the parameter estimation errors in the last columns in Table 2, Table 3 and Table 4 and the parameter estimation error curves in Figure 4 under different noise variances.
• The GI parameter estimates approach to their true values for sufficiently large data length as the iteration k increases—see Figure 5.

6. Conclusions

Modeling a dynamical system is the first step for system analysis and design in control engineering. Through defining and minimizing a quadratic criterion function by using observation data, this paper studies and derives a gradient-based iterative algorithm for stochastic systems described by the controlled autoregressive model based on the gradient search. Furthermore, in order to track time-varying parameters, a multi-innovation gradient-based iterative algorithm has been proposed by means of the multi-innovation identification theory. The proposed algorithms have the following advantages.

• For a lower noise level, the gradient-based algorithm can give more accurate parameter estimates. The parameter estimation errors become smaller as the iterative index increases.
• The gradient-based iterative parameter estimates approach their true values for sufficiently large data length and iterative index.
• The multi-innovation gradient-based iterative algorithm can track time-varying parameters of dynamical systems, improving the performance of the algorithms.
• The simulation results indicate that the proposed algorithms are effective for estimating the parameters of stochastic systems.
• The proposed methods in this paper can be extended to model industrial processes and network systems [79,80,81,82,83,84] by means of some other mathematical tools and approaches [85,86,87,88,89,90].