Decomposition Least-Squares-Based Iterative Identification Algorithms for Multivariable Equation-Error Autoregressive Moving Average Systems

Abstract

This paper is concerned with the identification problem for multivariable equation-error systems whose disturbance is an autoregressive moving average process. By means of the hierarchical identification principle and the iterative search, a hierarchical least-squares-based iterative (HLSI) identification algorithm is derived and a least-squares-based iterative (LSI) identification algorithm is given for comparison. Furthermore, a hierarchical multi-innovation least-squares-based iterative (HMILSI) identification algorithm is proposed using the multi-innovation theory. Compared with the LSI algorithm, the HLSI algorithm has smaller computational burden and can give more accurate parameter estimates and the HMILSI algorithm can track time-varying parameters. Finally, a simulation example is provided to verify the effectiveness of the proposed algorithms.

1. Introduction

With the development of modern industry, multivariable systems have provided rich possibilities to system modeling and process control [1,2]. Compared with scalar systems, multivariable systems have more complex structures, complicated relationships between variables, high dimensions and stochastic disturbances. For decades, multivariable system identification has attracted increasing attention and many identification approaches have been reported [3]. Among them, how to improve the identification accuracy and to increase the identification efficiency of multivariable systems has become the core problems for researchers. Liu et al. proposed a partially-coupled gradient identification algorithm to obtain the more accurate estimates by filtering the input and output data and by transform the system into two subsystems [4].

Many methods have been used to deal with system identification problems [5,6,7,8]. Minimizing different criterion leads to different identification methods, such as neural network methods, fuzzy logic system identification methods, wavelet network system identification methods and so on. According to certain criterion function adopting different search strategies leads to different identification approaches, such as Newton identification methods, least-squares identification methods [9] and gradient identification methods [10,11]. The recursive and iterative identification techniques can handle the parameter estimates of the system models. The recursive identification algorithms can avoid the matrix inversion and can be operated on-line [12,13] and the iterative algorithms can make use of all the given input–output data sufficiently and can improve the parameter estimation accuracy [14,15,16,17]. A filtering based multi-innovation gradient estimation algorithm has been proposed for nonlinear dynamical systems [18].

On the other hand, some identification principles are widely applied to parameter estimation for linear and nonlinear systems and many methods have been obtained for identifying multi-input multi-output systems. The multi-innovation identification theory is beneficial for deriving more accurate estimation algorithms by expanding the innovation from a scalar to a vector and from the vector to the matrix. The hierarchical identification principle is a decomposition-based identification technique, whose key is to decompose a system into several subsystems which can be identified easier by using the gradient-based or the least-squares-based identification algorithms. Hierarchical identification methods are suitable for large-scale and complex systems since they can reduce the dimension of the systems to be identified and reduce the computation load. The filtering technique is potential for identifying systems with colored noises [19,20]. An adaptive filtering based multi-innovation stochastic gradient algorithm was derived for bilinear systems with colored noise and can give small parameter estimation errors as the innovation length increases [21]. A multi-innovation gradient algorithm was developed based on the Kalman filtering to solve the joint state and parameter estimation problem for a nonlinear state space system with time-delay [22]. Many identification methods can be found in [23,24,25,26,27,28,29,30,31,32] and they can be applied to many areas [33,34,35,36,37].

Recently, using the hierarchical identification principle and the multi-innovation identification theory, some algorithms have been proposed to track the parameters of nonlinear systems and multivariable systems [38,39]. Considering that the least-squares algorithms have high estimation accuracy and rapid convergence [40,41,42], this paper focuses on the parameter identification problem of multivariable equation-error systems with autoregressive moving average noise process and presents iterative identification algorithms using the hierarchical identification principle and the multi-innovation method. The key is to decompose a system into two subsystems and to iteratively estimate the matrix of unknown parameters of each subsystem separately. The main contribution of this work are as follows.

• A decomposition least-squares-based iterative identification algorithm is derived for multivariable equation-error autoregressive moving average systems by using the hierarchical identification principle.
• Compared with the least-squares-based iterative algorithm, the proposed algorithm can improve the estimation accuracy and decrease the computation burden.
• A hierarchical multi-innovation least-squares-based iterative identification algorithm is proposed by using the multi-innovation theory, which can track time-varying parameters.

The remainder of the paper is organized as follows. Section 2 derives the identification models of multivariable equation-error systems with different forms of colored noise processes. A Least-squares-based iterative identification algorithm is introduced in Section 3. In Section 4, a hierarchical least-squares-based iterative algorithm is derived. Section 5 presents a hierarchial multi-innovation least-squares-based iterative identification algorithm. The numerical example results are presented to illustrate the effectiveness of the proposed algorithms in Section 6. Finally, Section 7 concludes the work.

2. System Description and Identification Model

Symbols Meaning
$\mathbf{0}$:	The zero matrix of appropriate sizes.
${\mathbf{1}}_{m\times n}$:	An $m\times n$ matrix whose entries are all 1.
${\mathbf{1}}_n$:	An n dimensional column vector whose entries are all 1.
$\mathit{I}$ or ${\mathit{I}}_n$:	The identity matrix of appropriate sizes or $n\times n$.
$\mathrm{tr}\left[\mathit{X}\right]$:	The trace of the square matrix $\mathit{X}$.
${\mathit{X}}^{\mathrm{T}}$:	The transpose of the vector or matrix $\mathit{X}$.
$\parallel \mathit{X}\parallel$:	${\parallel \mathit{X}\parallel }^2:=\mathrm{tr}\left[\mathit{X}{\mathit{X}}^{\mathrm{T}}\right]$.
$A=:X$:	X is defined by A.
$X:=A$:	X is defined by A.
s:	The time variable.
$\widehat{\mathit{\theta }}$:	The estimate of the parameter matrix $\mathit{\theta }$.
$\widehat{\mathit{\theta }}\left(s\right)$:	The estimate of $\mathit{\theta }$ at time s.
$p_0$:	A large positive constant, e.g., $p_0={10}^6$.

Consider the following multivariable equation-error system with the colored noise,

$$\mathit{A}\left(r\right)\mathit{y}\left(s\right)=\mathit{B}\left(r\right)\mathit{u}\left(s\right)+\mathit{w}\left(s\right),$$

(1)

where $\mathit{u}\left(s\right):={[u_1\left(s\right),u_2\left(s\right),\cdots ,u_r\left(s\right)]}^{\mathrm{T}}\in {\mathbb{R}}^r$ is the input vector, $\mathit{y}\left(s\right):={[y_1\left(s\right),y_2\left(s\right),\cdots ,y_m\left(s\right)]}^{\mathrm{T}}\in {\mathbb{R}}^m$ is the output vector, $\mathit{A}\left(r\right)$ and $\mathit{B}\left(r\right)$ are the matrix-coefficient polynomials in the unit backward shift operator $r^{-1}$ (i.e., $r^{-1}\mathit{x}\left(s\right)=\mathit{x}(s-1)$) with degrees $n_a$ and $n_b$, and they are defined as

$$\begin{array}{ccc}\hfill \mathit{A}\left(r\right)& :=& {\mathit{I}}_m+{\mathit{A}}_1{r}^{-1}+{\mathit{A}}_2{r}^{-2}+\cdots +{\mathit{A}}_{n_a}{r}^{-n_a},{\mathit{A}}_i\in {\mathbb{R}}^{m\times m},\hfill \\ \hfill \mathit{B}\left(r\right)& :=& {\mathit{B}}_1{r}^{-1}+{\mathit{B}}_2{r}^{-2}+\cdots +{\mathit{B}}_{n_b}{r}^{-n_b},{\mathit{B}}_i\in {\mathbb{R}}^{m\times r},\hfill \end{array}$$

where $\mathit{w}\left(s\right)\in {\mathbb{R}}^m$ is a stochastic noise vector with zero mean, which may be a moving average (MA) process, an autoregressive (AR) process or an autoregressive moving average process (ARMA).

Considering the ARMA process, there are four forms for the description of the noise term as follows.

• The first form is $\mathit{w}\left(s\right)=\frac{d\left(r\right)}{c\left(r\right)}\mathit{v}\left(s\right)\in {\mathbb{R}}^m$, where $\mathit{v}\left(s\right)\in {\mathbb{R}}^m$ is a white noise vector, and $c\left(r\right)$ and $d\left(r\right)$ are scalar polynomials in $r^{-1}$ with degrees $n_c$ and $n_d$, which are expressed as

  $$\begin{array}{ccc}\hfill c\left(r\right)& :=& 1+c_1{r}^{-1}+c_2{r}^{-2}+\cdots +c_{n_c}{r}^{-n_c},c_i\in \mathbb{R},\hfill \\ \hfill d\left(r\right)& :=& 1+d_1{r}^{-1}+d_2{r}^{-2}+\cdots +d_{n_d}{r}^{-n_d},d_i\in \mathbb{R}.\hfill \end{array}$$
• The second form is $\mathit{w}\left(s\right)=d\left(r\right){\mathit{C}}^{-1}\left(r\right)\mathit{v}\left(s\right)\in {\mathbb{R}}^m$, where $d\left(r\right)$ is a scalar polynomial and $\mathit{C}\left(r\right)$ is a matrix polynomial in $r^{-1}$, which is expressed as

  $$\mathit{C}\left(r\right):={\mathit{I}}_m+{\mathit{C}}_1{r}^{-1}+{\mathit{C}}_2{r}^{-2}+\cdots +{\mathit{C}}_{n_c}{r}^{-n_c},{\mathit{C}}_i\in {\mathbb{R}}^{m\times m}.$$
• The third form is $\mathit{w}\left(s\right)=\frac{\mathit{D}\left(r\right)}{c\left(r\right)}\mathit{v}\left(s\right)\in {\mathbb{R}}^m$, where $c\left(r\right)$ is a scalar polynomial and $\mathit{D}\left(r\right)$ is a matrix polynomial in $r^{-1}$, which is expressed as

  $$\mathit{D}\left(r\right):={\mathit{I}}_m+{\mathit{D}}_1{r}^{-1}+{\mathit{D}}_2{r}^{-2}+\cdots +{\mathit{D}}_{n_d}{r}^{-n_d},{\mathit{D}}_i\in {\mathbb{R}}^{m\times m}.$$
• The last form is $\mathit{w}\left(s\right)={\mathit{C}}^{-1}\left(r\right)\mathit{D}\left(r\right)\mathit{v}\left(s\right)\in {\mathbb{R}}^m$, where $\mathit{C}\left(r\right)$ and $\mathit{D}\left(r\right)$ are matrix polynomials in $r^{-1}$ with degrees $n_c$ and $n_d$.

Without loss of generality, we consider the multivariable systems with ARMA noise process, i.e.,

$$\mathit{A}\left(r\right)\mathit{y}\left(s\right)=\mathit{B}\left(r\right)\mathit{u}\left(s\right)+{\mathit{C}}^{-1}\left(r\right)\mathit{D}\left(r\right)\mathit{v}\left(s\right),$$

(2)

where ${\mathit{A}}_i$, ${\mathit{B}}_i$, ${\mathit{C}}_i$ and ${\mathit{D}}_i$ are the coefficient matrices to be estimated. Assume that the orders $n_a$, $n_b$, $n_c$ and $n_d$ are known and $n:=m{n}_a+r{n}_b+m{n}_c+m{n}_d$, $n_1:=m{n}_a+r{n}_b$, $n_2:=m{n}_c+m{n}_d$, $\mathit{u}\left(s\right)=\mathbf{0}$, $\mathit{y}\left(s\right)=\mathbf{0}$ and $\mathit{v}\left(s\right)=\mathbf{0}$ for $s⩽0$.

Define the parameter matrices $\mathit{\theta }$, $\mathit{\alpha }$ and $\mathit{\beta }$ as

$$\begin{array}{ccc}\hfill {\mathit{\theta }}^{\mathrm{T}}& :=& [{\mathit{\alpha }}^{\mathrm{T}},{\mathit{\beta }}^{\mathrm{T}}]\in {\mathbb{R}}^{m\times n},\hfill \\ \hfill {\mathit{\alpha }}^{\mathrm{T}}& :=& [{\mathit{A}}_1,{\mathit{A}}_2,\cdots ,{\mathit{A}}_{n_a},{\mathit{B}}_1,{\mathit{B}}_2,\cdots ,{\mathit{B}}_{n_b}]\in {\mathbb{R}}^{m\times n_1},\hfill \\ \hfill {\mathit{\beta }}^{\mathrm{T}}& :=& [{\mathit{C}}_1,{\mathit{C}}_2,\cdots ,{\mathit{C}}_{n_c},{\mathit{D}}_1,{\mathit{D}}_2,\cdots ,{\mathit{D}}_{n_d}]\in {\mathbb{R}}^{m\times n_2},\hfill \end{array}$$

and the information vectors $\mathit{\phi }\left(s\right)$, ${\mathit{\varphi }}_{\alpha }\left(s\right)$ and $\mathit{\psi }\left(s\right)$ as

$$\begin{array}{ccc}\hfill \mathit{\phi }\left(s\right)& :=& {[{\mathit{\varphi }}_{\alpha }^{\mathrm{T}}\left(s\right),{\mathit{\psi }}^{\mathrm{T}}\left(s\right)]}^{\mathrm{T}}\in {\mathbb{R}}^n,\hfill \end{array}$$

$$\begin{array}{ccc}\hfill {\mathit{\varphi }}_{\alpha }\left(s\right)& :=& {[-{\mathit{y}}^{\mathrm{T}}(s-1),-{\mathit{y}}^{\mathrm{T}}(s-2),\cdots ,-{\mathit{y}}^{\mathrm{T}}(s-n_a),{\mathit{u}}^{\mathrm{T}}(s-1),{\mathit{u}}^{\mathrm{T}}(s-2),\cdots ,{\mathit{u}}^{\mathrm{T}}(s-n_b)]}^{\mathrm{T}},\hfill \end{array}$$

(3)

$$\begin{array}{ccc}\hfill \mathit{\psi }\left(s\right)& :=& {[-{\mathit{w}}^{\mathrm{T}}(s-1),-{\mathit{w}}^{\mathrm{T}}(s-2),\cdots ,-{\mathit{w}}^{\mathrm{T}}(s-n_c),{\mathit{v}}^{\mathrm{T}}(s-1),{\mathit{v}}^{\mathrm{T}}(s-2),\cdots ,{\mathit{v}}^{\mathrm{T}}(s-n_d)]}^{\mathrm{T}}.\hfill \end{array}$$

(4)

It follows that,

$$\begin{array}{ccc}\hfill \mathit{w}\left(s\right)& =& [{\mathit{I}}_m-\mathit{C}\left(r\right)]\mathit{w}\left(s\right)+\mathit{D}\left(r\right)\mathit{v}\left(s\right)\hfill \\ & =& [{\mathit{I}}_m-\mathit{C}\left(r\right)]\mathit{w}\left(s\right)+[\mathit{D}\left(r\right)-{\mathit{I}}_m]\mathit{v}\left(s\right)+\mathit{v}\left(s\right)\hfill \\ & =& {\mathit{\beta }}^{\mathrm{T}}\mathit{\psi }\left(s\right)+\mathit{v}\left(s\right).\hfill \end{array}$$

(5)

Substituting Equation (5) into Equation (1) gives

$$\begin{array}{ccc}\hfill \mathit{y}\left(s\right)& =& [{\mathit{I}}_m-\mathit{A}\left(r\right)]\mathit{y}\left(s\right)+\mathit{B}\left(r\right)\mathit{u}\left(s\right)+\mathit{w}\left(s\right)\hfill \end{array}$$

$$\begin{array}{cc}=& {\mathit{\alpha }}^{\mathrm{T}}{\mathit{\varphi }}_{\alpha }\left(s\right)+{\mathit{\beta }}^{\mathrm{T}}\mathit{\psi }\left(s\right)+\mathit{v}\left(s\right)\hspace{1em}\hspace{1em} \hfill \end{array}$$

(6)

$$\begin{array}{cc}=& {\mathit{\theta }}^{\mathrm{T}}\mathit{\phi }\left(s\right)+\mathit{v}\left(s\right).\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em} \hfill \end{array}$$

(7)

In this identification model, $\mathit{\theta }$ is the parameter matrix to be identified, which consists of two parameter matrices: the parameter matrix $\mathit{\alpha }$ of the system model and the parameter matrix $\mathit{\beta }$ of the noise model.

Note that the unknown parameter matrices ${\mathit{A}}_i$, ${\mathit{B}}_i$, ${\mathit{C}}_i$ and ${\mathit{D}}_i$ are included in $\mathit{\theta }$, the identification problem has many parameters to be estimated and it is significant to enhance the computational efficiency of the multivariable system identification algorithms. Aiming at this goal, this work studies the decomposition least-squares-based iterative estimation algorithms for the multivariable CARARMA systems by using the hierarchical identification principle from the measured input–output data {$\mathit{u}\left(s\right),\mathit{y}\left(s\right)$: $s=1,2,\cdots ,L$} (L denotes the data length).

3. The Least-Squares-Based Iterative Algorithm

Let $k=1,2,3,\cdots$ be an iterative variable and ${\widehat{\mathit{\theta }}}_k:=\left[\begin{array}{c}{\widehat{\mathit{\alpha }}}_k\\ {\widehat{\mathit{\beta }}}_k\end{array}\right]$ be the estimate of $\mathit{\theta }=\left[\begin{array}{c}\mathit{\alpha }\\ \mathit{\beta }\end{array}\right]$ at iteration k. Assume that L is the data length ($L\gg n$). According to the data {$\mathit{u}\left(s\right),\mathit{y}\left(s\right)$: $s=1,2,\cdots ,L$} and based on Equation (7), define the stacked output matrix $\mathit{Y}\left(L\right)$ as

$$\mathit{Y}\left(L\right)=[\mathit{y}\left(1\right),\mathit{y}\left(2\right),\cdots ,\mathit{y}\left(L\right)]\in {\mathbb{R}}^{m\times L},$$

and the stacked information matrix $\mathit{\Omega }\left(L\right)$ is defined as

$$\mathit{\Omega }\left(L\right)=[\mathit{\phi }\left(1\right),\mathit{\phi }\left(2\right),\cdots ,\mathit{\phi }\left(L\right)]\in {\mathbb{R}}^{n\times L}.$$

Define a quadratic criterion function,

$$\begin{array}{ccc}\hfill J\left(\mathit{\theta }\right)& :=& \parallel \mathit{Y}\left(L\right)-{\mathit{\theta }}^{\mathrm{T}}{\mathit{\Omega }\left(L\right)\parallel }^2\hfill \\ & =& \sum _{s=1}^L{[\mathit{y}\left(s\right)-{\mathit{\theta }}^{\mathrm{T}}\mathit{\phi }\left(s\right)]}^{\mathrm{T}}[\mathit{y}\left(s\right)-{\mathit{\theta }}^{\mathrm{T}}\mathit{\phi }\left(s\right)].\hfill \end{array}$$

(8)

To minimize $J\left(\mathit{\theta }\right)$, letting its partial derivatives with $\mathit{\theta }$ at $\mathit{\theta }=\widehat{\mathit{\theta }}$ be zero gives

$$\frac{\partial J\left(\mathit{\theta }\right)}{\partial \mathit{\theta }}{|}_{\mathit{\theta }=\widehat{\mathit{\theta }}}=-2\mathit{\Omega }\left(L\right){[\mathit{Y}\left(L\right)-{\widehat{\mathit{\theta }}}^{\mathrm{T}}\mathit{\Omega }\left(L\right)]}^{\mathrm{T}}=\mathbf{0}$$

or

$$\frac{\partial J\left(\mathit{\theta }\right)}{\partial \mathit{\theta }}{|}_{\mathit{\theta }=\widehat{\mathit{\theta }}}=-2\sum _{s=1}^L\mathit{\phi }\left(s\right){[\mathit{y}\left(s\right)-{\widehat{\mathit{\theta }}}^{\mathrm{T}}\mathit{\phi }\left(s\right)]}^{\mathrm{T}}=\mathbf{0}.$$

Then, the estimates of the parameter matrix $\mathit{\theta }$ is given by

$$\begin{array}{ccc}\hfill \widehat{\mathit{\theta }}& =& {\left[\mathit{\Omega }\left(L\right){\mathit{\Omega }}^{\mathrm{T}}\left(L\right)\right]}^{-1}\mathit{\Omega }\left(L\right){\mathit{Y}}^{\mathrm{T}}\left(L\right)={\left[\sum _{s=1}^L\mathit{\phi }\left(s\right){\mathit{\phi }}^{\mathrm{T}}\left(s\right)\right]}^{-1}\sum _{s=1}^L\mathit{\phi }\left(s\right){\mathit{y}}^{\mathrm{T}}\left(s\right).\hfill \end{array}$$

(9)

It is observed that the information vector $\mathit{\phi }\left(s\right)$ in $\mathit{\Omega }\left(L\right)$ involves the unknown noise terms $\mathit{w}(s-i)$ ($i=1,2,\cdots ,n_c$) and $\mathit{v}(s-j)$ ($j=1,2,\cdots ,n_d$), therefore the estimate $\widehat{\mathit{\theta }}$ in Equation (9) cannot been computed. To resolve this problem, we replace the unknown noise terms $\mathit{w}(s-i)$ and $\mathit{v}(s-j)$ in $\mathit{\phi }\left(s\right)$ with their corresponding estimates ${\widehat{\mathit{w}}}_{k-1}(s-i)$ and ${\widehat{\mathit{v}}}_{k-1}(s-j)$ at iteration $k-1$. Then, we can obtain the least-squares-based iterative (LSI) algorithm as follows:

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\theta }}}_k& =& {\left[{\widehat{\mathit{\Omega }}}_k\left(L\right){\widehat{\mathit{\Omega }}}_k^{\mathrm{T}}\left(L\right)\right]}^{-1}{\widehat{\mathit{\Omega }}}_k\left(L\right){\mathit{Y}}^{\mathrm{T}}\left(L\right)\hfill \\ & =& {\left[\sum _{s=1}^L{\widehat{\mathit{\phi }}}_k\left(s\right){\widehat{\mathit{\phi }}}_k^{\mathrm{T}}\left(s\right)\right]}^{-1}\sum _{s=1}^L{\widehat{\mathit{\phi }}}_k\left(s\right){\mathit{y}}^{\mathrm{T}}\left(s\right),k=1,2,3,\cdots \hfill \end{array}$$

(10)

$$\begin{array}{ccc}\hfill \mathit{Y}\left(L\right)& =& \left[\mathit{y}\right(1),\mathit{y}(2),\cdots ,\mathit{y}(L\left)\right],\hfill \end{array}$$

(11)

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\Omega }}}_k\left(L\right)& =& [{\widehat{\mathit{\phi }}}_k\left(1\right),{\widehat{\mathit{\phi }}}_k\left(2\right),\cdots ,{\widehat{\mathit{\phi }}}_k\left(L\right)],\hfill \end{array}$$

(12)

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\phi }}}_k\left(s\right)& =& [{\mathit{\varphi }}_{\alpha }^{\mathrm{T}}\left(s\right),-{\widehat{\mathit{w}}}_{k-1}^{\mathrm{T}}(s-1),-{\widehat{\mathit{w}}}_{k-1}^{\mathrm{T}}(s-2),\cdots ,-{\widehat{\mathit{w}}}_{k-1}^{\mathrm{T}}(s-n_c),\hfill \\ & & {\widehat{\mathit{v}}}_{k-1}^{\mathrm{T}}(s-1),{\widehat{\mathit{v}}}_{k-1}^{\mathrm{T}}(s-2),\cdots ,{\widehat{\mathit{v}}}_{k-1}^{\mathrm{T}}(s-n_d){]}^{\mathrm{T}},\hfill \end{array}$$

(13)

$$\begin{array}{ccc}\hfill {\mathit{\varphi }}_{\alpha }\left(s\right)& =& [-{\mathit{y}}^{\mathrm{T}}(s-1),-{\mathit{y}}^{\mathrm{T}}(s-2),\cdots ,-{\mathit{y}}^{\mathrm{T}}(s-n_a),\hfill \\ & & {\mathit{u}}^{\mathrm{T}}(s-1),{\mathit{u}}^{\mathrm{T}}(s-2),\cdots ,{\mathit{u}}^{\mathrm{T}}(s-n_b){]}^{\mathrm{T}},\hfill \end{array}$$

(14)

$$\begin{array}{ccc}\hfill {\widehat{\mathit{w}}}_k\left(s\right)& =& \mathit{y}\left(s\right)-{\widehat{\mathit{\alpha }}}_k^{\mathrm{T}}{\mathit{\varphi }}_{\alpha }\left(s\right),s=1,2,\cdots ,L,\hfill \end{array}$$

(15)

$$\begin{array}{ccc}\hfill {\widehat{\mathit{v}}}_k\left(s\right)& =& \mathit{y}\left(s\right)-{\widehat{\mathit{\theta }}}_k^{\mathrm{T}}\widehat{\mathit{\phi }}\left(s\right),\hfill \end{array}$$

(16)

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\theta }}}_k^{\mathrm{T}}& =& [{\widehat{\mathit{\alpha }}}_k^{\mathrm{T}},{\widehat{\mathit{\beta }}}_k^{\mathrm{T}}],\hfill \end{array}$$

(17)

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\alpha }}}_k^{\mathrm{T}}& =& [{\widehat{\mathit{A}}}_{1,k},{\widehat{\mathit{A}}}_{2,k},\cdots ,{\widehat{\mathit{A}}}_{n_a,k},{\widehat{\mathit{B}}}_{1,k},{\widehat{\mathit{B}}}_{2,k},\cdots ,{\widehat{\mathit{B}}}_{n_b,k}],\hfill \end{array}$$

(18)

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\beta }}}_k^{\mathrm{T}}& =& [{\widehat{\mathit{C}}}_{1,k},{\widehat{\mathit{C}}}_{2,k},\cdots ,{\widehat{\mathit{C}}}_{n_c,k},{\widehat{\mathit{D}}}_{1,k},{\widehat{\mathit{D}}}_{2,k},\cdots ,{\widehat{\mathit{D}}}_{n_d,k}].\hfill \end{array}$$

(19)

The computational efficiency of the LSI algorithm for the multivariable CARARMA systems is given in

Table 1. The computational efficiency of the least-squares-based iterative identification algorithm.

Expressions	Number of Multiplications	Number of Additions
${\widehat{\mathit{\theta }}}_k={\mathit{S}}_k^{\prime }{\mathit{\beta }}_k\in {\mathbb{R}}^{n\times m}$	$m{n}^{2}k$	$mn(n-1)k$
${\mathit{S}}_k^{\prime }:={\mathit{S}}_k^{-1}\in {\mathbb{R}}^{n\times n}$	$(n^3+n^2-n+1)k$	$(n^3+n^2-2n)k$
${\mathit{S}}_k:={\widehat{\mathit{\Omega }}}_k\left(L\right){\widehat{\mathit{\Omega }}}_k^{\mathrm{T}}\left(L\right)\in {\mathbb{R}}^{n\times n}$	$n^{2}Lk$	$n^2(L-1)k$
${\mathit{\beta }}_k:={\widehat{\mathit{\Omega }}}_k\left(L\right){\mathit{Y}}^{\mathrm{T}}\left(L\right)\in {\mathbb{R}}^{n\times m}$	$mnLk$	$mn(L-1)k$
${\widehat{\mathit{w}}}_k\left(s\right)=\mathit{y}\left(s\right)-{\widehat{\mathit{\alpha }}}_k^{\mathrm{T}}{\mathit{\varphi }}_{\alpha }\left(s\right)\in {\mathbb{R}}^m$	$m{n}_{1}Lk$	$m{n}_{1}Lk$
${\widehat{\mathit{v}}}_k\left(s\right)={\widehat{\mathit{w}}}_k-{\widehat{\mathit{\beta }}}_k^{\mathrm{T}}{\widehat{\mathit{\psi }}}_k\left(s\right)\in {\mathbb{R}}^m$	$m{n}_{2}Lk$	$m{n}_{2}Lk$
Sum (Number of multiplications)	$\left[n^3+(m+1+L)n^2+(2mL-1)n+1\right]k$
Sum (Number of additions)	$\left[n^3+(m+L)n^2+2(mL-m-1)n\right]k$
Total flops	$N_1:=\left[2n^3+(2m+2L+1)n^2+(4mL-2m-3)n+1\right]k$

.

4. The Hierarchical Least-Squares-Based Iterative Algorithm

The multivariable system in Equation (6) is decomposed into two subsystems, one containing $\mathit{\alpha }$ and the other containing $\mathit{\beta }$, and each subsystem is identified separately by using the hierarchical identification principle and the iterative technique. Define two intermediate variables,

$$\begin{array}{ccc}\hfill \mathit{h}\left(s\right)& :=& \mathit{y}\left(s\right)-{\mathit{\beta }}^{\mathrm{T}}\mathit{\psi }\left(s\right),\hfill \end{array}$$

(20)

$$\begin{array}{ccc}\hfill \mathit{w}\left(s\right)& :=& \mathit{y}\left(s\right)-{\mathit{\alpha }}^{\mathrm{T}}{\mathit{\varphi }}_{\alpha }\left(s\right).\hfill \end{array}$$

(21)

The system in Equation (1) can be decomposed into the following two fictitious subsystems:

$$\begin{array}{ccc}\hfill \mathit{h}\left(s\right)& =& {\mathit{\alpha }}^{\mathrm{T}}{\mathit{\varphi }}_{\alpha }\left(s\right)+\mathit{v}\left(s\right),\hfill \end{array}$$

(22)

$$\begin{array}{ccc}\hfill \mathit{w}\left(s\right)& =& {\mathit{\beta }}^{\mathrm{T}}\mathit{\psi }\left(s\right)+\mathit{v}\left(s\right).\hfill \end{array}$$

(23)

Let L be the data length ($L\gg n$) and define the stacked output matrices $\mathit{Y}\left(L\right)$, $\mathit{H}\left(L\right)$, $\mathit{W}\left(L\right)$ and the stacked information matrices $\mathit{\Phi }\left(L\right)$ and $\mathit{\Psi }\left(L\right)$ as

$$\begin{array}{ccc}\hfill \mathit{Y}\left(L\right)& :=& [\mathit{y}\left(1\right),\mathit{y}\left(2\right),\cdots ,\mathit{y}\left(L\right)]\in {\mathbb{R}}^{m\times L},\hfill \end{array}$$

(24)

$$\begin{array}{ccc}\hfill \mathit{H}\left(L\right)& :=& [\mathit{h}\left(1\right),\mathit{h}\left(2\right),\cdots ,\mathit{h}\left(L\right)]\in {\mathbb{R}}^{m\times L},\hfill \end{array}$$

$$\begin{array}{ccc}\hfill \mathit{W}\left(L\right)& :=& [\mathit{w}\left(1\right),\mathit{w}\left(2\right),\cdots ,\mathit{w}\left(L\right)]\in {\mathbb{R}}^{m\times L},\hfill \end{array}$$

$$\begin{array}{ccc}\hfill \mathit{\Phi }\left(L\right)& :=& [{\mathit{\varphi }}_{\alpha }\left(1\right),{\mathit{\varphi }}_{\alpha }\left(2\right),\cdots ,{\mathit{\varphi }}_{\alpha }\left(L\right)]\in {\mathbb{R}}^{n_1\times L},\hfill \end{array}$$

(25)

$$\begin{array}{ccc}\hfill \mathit{\Psi }\left(L\right)& :=& [\mathit{\psi }\left(1\right),\mathit{\psi }\left(2\right),\cdots ,\mathit{\psi }\left(L\right)]\in {\mathbb{R}}^{n_2\times L}.\hfill \end{array}$$

From Equations (20) and (21), we have

$$\begin{array}{ccc}\hfill \mathit{H}\left(L\right)& =& \mathit{Y}\left(L\right)-{\mathit{\beta }}^{\mathrm{T}}\mathit{\Psi }\left(L\right),\hfill \end{array}$$

(26)

$$\begin{array}{ccc}\hfill \mathit{W}\left(L\right)& =& \mathit{Y}\left(L\right)-{\mathit{\alpha }}^{\mathrm{T}}\mathit{\Phi }\left(L\right).\hfill \end{array}$$

(27)

Define quadratic criterion functions:

$$\begin{array}{ccc}\hfill J_1\left(\mathit{\alpha }\right)& :=& \parallel \mathit{H}\left(L\right)-{\mathit{\alpha }}^{\mathrm{T}}{\mathit{\Phi }\left(L\right)\parallel }^2=\sum _{s=1}^L{[\mathit{h}\left(s\right)-{\mathit{\alpha }}^{\mathrm{T}}{\mathit{\varphi }}_{\alpha }\left(s\right)]}^{\mathrm{T}}[\mathit{h}\left(s\right)-{\mathit{\alpha }}^{\mathrm{T}}{\mathit{\varphi }}_{\alpha }\left(s\right)],\hfill \\ \hfill J_2\left(\mathit{\beta }\right)& :=& \parallel \mathit{W}\left(L\right)-{\mathit{\beta }}^{\mathrm{T}}{\mathit{\Psi }\left(L\right)\parallel }^2=\sum _{s=1}^L{[\mathit{w}\left(s\right)-{\mathit{\beta }}^{\mathrm{T}}\mathit{\psi }\left(s\right)]}^{\mathrm{T}}[\mathit{w}\left(s\right)-{\mathit{\beta }}^{\mathrm{T}}\mathit{\psi }\left(s\right)].\hfill \end{array}$$

Let $k=1,2,3,\cdots$ be an iterative variable, and ${\widehat{\mathit{\alpha }}}_k$ and ${\widehat{\mathit{\beta }}}_k$ be the estimates of parameter matrices $\mathit{\alpha }$ and $\mathit{\beta }$ at iteration k, respectively. Minimizing $J_1\left(\mathit{\alpha }\right)$ and $J_2\left(\mathit{\beta }\right)$ and letting their partial derivatives with respect to $\mathit{\alpha }$ and $\mathit{\beta }$ be zero, we can obtain the least-squares-based iterative relations, respectively:

$$\frac{\partial J_1\left(\mathit{\alpha }\right)}{\partial \mathit{\alpha }}{|}_{\mathit{\alpha }=\widehat{\mathit{\alpha }}}=-2\mathit{\Phi }\left(L\right){[\mathit{H}\left(L\right)-{\widehat{\mathit{\alpha }}}^{\mathrm{T}}\mathit{\Phi }\left(L\right)]}^{\mathrm{T}}=\mathbf{0}$$

or

$$\frac{\partial J_1\left(\mathit{\alpha }\right)}{\partial \mathit{\alpha }}{|}_{\mathit{\alpha }=\widehat{\mathit{\alpha }}}=-2\sum _{s=1}^L{\mathit{\varphi }}_{\alpha }\left(s\right){[\mathit{h}\left(s\right)-{\widehat{\mathit{\alpha }}}^{\mathrm{T}}{\mathit{\varphi }}_{\alpha }\left(s\right)]}^{\mathrm{T}}=\mathbf{0},$$

$$\frac{\partial J_2\left(\mathit{\beta }\right)}{\partial \mathit{\beta }}{|}_{\mathit{\beta }=\widehat{\mathit{\beta }}}=-2\mathit{\Psi }\left(L\right){[\mathit{W}\left(L\right)-{\widehat{\mathit{\beta }}}^{\mathrm{T}}\mathit{\Psi }\left(L\right)]}^{\mathrm{T}}=\mathbf{0}$$

or

$$\frac{\partial J_2\left(\mathit{\beta }\right)}{\partial \mathit{\beta }}{|}_{\mathit{\beta }=\widehat{\mathit{\beta }}}=-2\sum _{s=1}^L\mathit{\psi }\left(s\right){[\mathit{w}\left(s\right)-{\widehat{\mathit{\beta }}}^{\mathrm{T}}\mathit{\psi }\left(s\right)]}^{\mathrm{T}}=\mathbf{0}.$$

Then, the estimates of the parameter matrices $\mathit{\alpha }$ and $\mathit{\beta }$ are given by

$$\begin{array}{ccc}\hfill \widehat{\mathit{\alpha }}& =& {\left[\mathit{\Phi }\left(L\right){\mathit{\Phi }}^{\mathrm{T}}\left(L\right)\right]}^{-1}\mathit{\Phi }\left(L\right){[\mathit{Y}\left(L\right)-{\widehat{\mathit{\beta }}}^{\mathrm{T}}\mathit{\Psi }\left(L\right)]}^{\mathrm{T}},\hfill \end{array}$$

(28)

$$\begin{array}{ccc}\hfill \widehat{\mathit{\beta }}& =& {\left[\mathit{\Psi }\left(L\right){\mathit{\Psi }}^{\mathrm{T}}\left(L\right)\right]}^{-1}\mathit{\Psi }\left(L\right){[\mathit{Y}\left(L\right)-{\widehat{\mathit{\alpha }}}^{\mathrm{T}}\mathit{\Phi }\left(L\right)]}^{\mathrm{T}}.\hfill \end{array}$$

(29)

Notice that the information vector $\mathit{\psi }\left(s\right)$ in $\mathit{\Psi }\left(L\right)$ contains the unknown noise items $\mathit{w}(s-i)$ and $\mathit{v}(s-j)$, thus the algorithm in Equations (28) and (29) cannot be implemented. The solution is to use the estimates ${\widehat{\mathit{w}}}_{k-1}(s-i)$ and ${\widehat{\mathit{v}}}_{k-1}(s-j)$ of the unknown noise vectors $\mathit{w}(s-i)$ and $\mathit{v}(s-j)$ at iteration $k-1$ to define the estimate ${\widehat{\mathit{\psi }}}_k\left(s\right)$ of $\mathit{\psi }\left(s\right)$ as

$${\widehat{\mathit{\psi }}}_k\left(s\right):={[-{\widehat{\mathit{w}}}_{k-1}^{\mathrm{T}}(s-1),-{\widehat{\mathit{w}}}_{k-1}^{\mathrm{T}}(s-2),\cdots ,-{\widehat{\mathit{w}}}_{k-1}^{\mathrm{T}}(s-n_c),{\widehat{\mathit{v}}}_{k-1}^{\mathrm{T}}(s-1),{\widehat{\mathit{v}}}_{k-1}^{\mathrm{T}}(s-2),\cdots ,{\widehat{\mathit{v}}}_{k-1}^{\mathrm{T}}(s-n_d)]}^{\mathrm{T}}.$$

(30)

Replacing the unknown parameter matrix $\mathit{\alpha }$ in Equation (21) with its estimate ${\widehat{\mathit{\alpha }}}_k$ , we have the estimate of $\mathit{w}\left(s\right)$ at iteration k as

$${\widehat{\mathit{w}}}_k\left(s\right):=\mathit{y}\left(s\right)-{\widehat{\mathit{\alpha }}}_k^{\mathrm{T}}{\mathit{\varphi }}_{\alpha }\left(s\right).$$

(31)

According to Equations (5) and (23), replacing $\mathit{w}\left(s\right)$ and $\mathit{\alpha }$ with their estimates ${\widehat{\mathit{w}}}_k\left(s\right)$ and ${\widehat{\mathit{\alpha }}}_k$, we have the estimate of $\mathit{v}\left(s\right)$ at iteration k as

$$\begin{array}{ccc}\hfill {\widehat{\mathit{v}}}_k\left(s\right)& :=& {\widehat{\mathit{w}}}_k\left(s\right)-{\widehat{\mathit{\beta }}}_k^{\mathrm{T}}{\widehat{\mathit{\psi }}}_k\left(s\right)\hfill \\ & =& \mathit{y}\left(s\right)-{\widehat{\mathit{\alpha }}}_k^{\mathrm{T}}{\mathit{\varphi }}_{\alpha }\left(s\right)-{\widehat{\mathit{\beta }}}_k^{\mathrm{T}}{\widehat{\mathit{\psi }}}_k\left(s\right).\hfill \end{array}$$

(32)

Use the estimate ${\widehat{\mathit{\psi }}}_k\left(s\right)$ of $\mathit{\psi }\left(s\right)$ to define the estimate ${\widehat{\mathit{\Psi }}}_k\left(L\right)$ of $\mathit{\Psi }\left(L\right)$ as

$${\widehat{\mathit{\Psi }}}_k\left(L\right):=[{\widehat{\mathit{\psi }}}_k\left(1\right),{\widehat{\mathit{\psi }}}_k\left(2\right),\cdots ,{\widehat{\mathit{\psi }}}_k\left(L\right)]\in {\mathbb{R}}^{n_2\times L}.$$

(33)

Replacing $\mathit{\beta }$, $\mathit{\alpha }$ and $\mathit{\Psi }\left(L\right)$ in Equations (28) and (29) with their estimates ${\widehat{\mathit{\beta }}}_{k-1}$, ${\widehat{\mathit{\alpha }}}_{k-1}$ and ${\widehat{\mathit{\Psi }}}_k\left(L\right)$, and combining Equations (24), (25), (33), (3), and (30)–(32), we have the hierarchical least-squares-based iterative (HLSI) identification algorithm as follows:

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\alpha }}}_k& =& {\left[\mathit{\Phi }\left(L\right){\mathit{\Phi }}^{\mathrm{T}}\left(L\right)\right]}^{-1}\mathit{\Phi }\left(L\right){[\mathit{Y}\left(L\right)-{\widehat{\mathit{\beta }}}_{k-1}^{\mathrm{T}}{\widehat{\mathit{\Psi }}}_k\left(L\right)]}^{\mathrm{T}}\hfill \\ & =& {\left[\sum _{s=1}^L{\mathit{\varphi }}_{\alpha }\left(s\right){\mathit{\varphi }}_{\alpha }^{\mathrm{T}}\left(s\right)\right]}^{-1}\sum _{s=1}^L{\mathit{\varphi }}_{\alpha }\left(s\right){[\mathit{y}\left(s\right)-{\widehat{\mathit{\beta }}}_{k-1}^{\mathrm{T}}{\widehat{\mathit{\psi }}}_k\left(s\right)]}^{\mathrm{T}},k=1,2,3,\cdots \hfill \end{array}$$

(34)

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\beta }}}_k& =& {\left[{\widehat{\mathit{\Psi }}}_k\left(L\right){\widehat{\mathit{\Psi }}}_k^{\mathrm{T}}\left(L\right)\right]}^{-1}{\widehat{\mathit{\Psi }}}_k\left(L\right){[\mathit{Y}\left(L\right)-{\widehat{\mathit{\alpha }}}_{k-1}^{\mathrm{T}}\mathit{\Phi }\left(L\right)]}^{\mathrm{T}}\hfill \\ & =& {\left[\sum _{s=1}^L{\widehat{\mathit{\psi }}}_k\left(s\right){\widehat{\mathit{\psi }}}_k^{\mathrm{T}}\left(s\right)\right]}^{-1}\sum _{s=1}^L{\widehat{\mathit{\psi }}}_k\left(s\right){[\mathit{y}\left(s\right)-{\widehat{\mathit{\alpha }}}_{k-1}^{\mathrm{T}}{\mathit{\varphi }}_{\alpha }\left(s\right)]}^{\mathrm{T}},\hfill \end{array}$$

(35)

$$\begin{array}{ccc}\hfill \mathit{Y}\left(L\right)& =& \left[\mathit{y}\right(1),\mathit{y}(2),\cdots ,\mathit{y}(L\left)\right],\hfill \end{array}$$

(36)

$$\begin{array}{ccc}\hfill \mathit{\Phi }\left(L\right)& =& [{\mathit{\varphi }}_{\alpha }\left(1\right),{\mathit{\varphi }}_{\alpha }\left(2\right),\cdots ,{\mathit{\varphi }}_{\alpha }\left(L\right)],\hfill \end{array}$$

(37)

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\Psi }}}_k\left(L\right)& =& [{\widehat{\mathit{\psi }}}_k\left(1\right),{\widehat{\mathit{\psi }}}_k\left(2\right),\cdots ,{\widehat{\mathit{\psi }}}_k\left(L\right)],\hfill \end{array}$$

(38)

$$\begin{array}{ccc}\hfill {\mathit{\varphi }}_{\alpha }\left(s\right)& =& {[-{\mathit{y}}^{\mathrm{T}}(s-1),-{\mathit{y}}^{\mathrm{T}}(s-2),\cdots ,-{\mathit{y}}^{\mathrm{T}}(s-n_a),{\mathit{u}}^{\mathrm{T}}(s-1),{\mathit{u}}^{\mathrm{T}}(s-2),\cdots ,{\mathit{u}}^{\mathrm{T}}(s-n_b)]}^{\mathrm{T}},\hfill \end{array}$$

(39)

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\psi }}}_k\left(s\right)& =& [-{\widehat{\mathit{w}}}_{k-1}^{\mathrm{T}}(s-1),-{\widehat{\mathit{w}}}_{k-1}^{\mathrm{T}}(s-2),\cdots ,-{\widehat{\mathit{w}}}_{k-1}^{\mathrm{T}}(s-n_c),\hfill \\ & & {\widehat{\mathit{v}}}_{k-1}^{\mathrm{T}}(s-1),{\widehat{\mathit{v}}}_{k-1}^{\mathrm{T}}(s-2),\cdots ,{\widehat{\mathit{v}}}_{k-1}^{\mathrm{T}}(s-n_d){]}^{\mathrm{T}},\hfill \end{array}$$

(40)

$$\begin{array}{ccc}\hfill {\widehat{\mathit{w}}}_k\left(s\right)& =& \mathit{y}\left(s\right)-{\widehat{\mathit{\alpha }}}_k^{\mathrm{T}}{\mathit{\varphi }}_{\alpha }\left(s\right),s=1,2,3,\cdots ,L,\hfill \end{array}$$

(41)

$$\begin{array}{ccc}\hfill {\widehat{\mathit{v}}}_k\left(s\right)& =& \mathit{y}\left(s\right)-{\widehat{\mathit{\alpha }}}_k^{\mathrm{T}}{\mathit{\varphi }}_{\alpha }\left(s\right)-{\widehat{\mathit{\beta }}}_k^{\mathrm{T}}{\widehat{\mathit{\psi }}}_k\left(s\right),\hfill \end{array}$$

(42)

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\theta }}}_k^{\mathrm{T}}& =& [{\widehat{\mathit{\alpha }}}_k^{\mathrm{T}},{\widehat{\mathit{\beta }}}_k^{\mathrm{T}}],\hfill \end{array}$$

(43)

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\alpha }}}_k^{\mathrm{T}}& =& [{\widehat{\mathit{A}}}_{1,k},{\widehat{\mathit{A}}}_{2,k},\cdots ,{\widehat{\mathit{A}}}_{n_a,k},{\widehat{\mathit{B}}}_{1,k},{\widehat{\mathit{B}}}_{2,k},\cdots ,{\widehat{\mathit{B}}}_{n_b,k}],\hfill \end{array}$$

(44)

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\beta }}}_k^{\mathrm{T}}& =& [{\widehat{\mathit{C}}}_{1,k},{\widehat{\mathit{C}}}_{2,k},\cdots ,{\widehat{\mathit{C}}}_{n_c,k},{\widehat{\mathit{D}}}_{1,k},{\widehat{\mathit{D}}}_{2,k},\cdots ,{\widehat{\mathit{D}}}_{n_d,k}].\hfill \end{array}$$

(45)

The steps of the HLSI algorithm to compute the parameter estimates ${\widehat{\mathit{\alpha }}}_k$ and ${\widehat{\mathit{\beta }}}_k$ are listed as follows.

• To initialize, let $k=1$, give the data length L ($L\gg n$) and some small positive $\epsilon$, and set the initial values: ${\widehat{\mathit{\alpha }}}_0={\mathbf{1}}_{n_1\times m}/p_0$, ${\widehat{\mathit{\beta }}}_0={\mathbf{1}}_{n_2\times m}/p_0$, $p_0={10}^6$.
• Collect the input–output data {$\mathit{u}\left(s\right),\mathit{y}\left(s\right)$: $s=1,2,\cdots ,L$}. Let ${\widehat{\mathit{w}}}_0\left(s\right)$ and ${\widehat{\mathit{v}}}_0\left(s\right)$ be random variables, and form the information vector ${\mathit{\varphi }}_{\alpha }\left(s\right)$ by Equation (39), the stacked output matrix $\mathit{Y}\left(L\right)$ by Equation (36) and the stacked information matrix $\mathit{\Phi }\left(L\right)$ by Equation (37).
• Form the information vector ${\widehat{\mathit{\psi }}}_k\left(s\right)$ by Equation (40), $s=1,2,\cdots ,L$, and form the stacked information matrix ${\widehat{\mathit{\Psi }}}_k\left(L\right)$ by Equation (38).
• Update the parameter estimates ${\widehat{\mathit{\alpha }}}_k$ by Equation (34) and ${\widehat{\mathit{\beta }}}_k$ by Equation (35).
• Compute ${\widehat{\mathit{w}}}_k\left(s\right)$ and ${\widehat{\mathit{v}}}_k\left(s\right)$ by Equations (41) and (42).
• If $\parallel {\widehat{\mathit{\alpha }}}_k-{\widehat{\mathit{\alpha }}}_{k-1}\parallel +\parallel {\widehat{\mathit{\beta }}}_k-{\widehat{\mathit{\beta }}}_{k-1}\parallel >\epsilon$, increase k by 1 and go to Step 3; otherwise, obtain the iterative time k and the estimates ${\widehat{\mathit{\alpha }}}_k$ and ${\widehat{\mathit{\beta }}}_k$, and terminate this procedure.

Remark 1.

In the above algorithms, the unknown variables are replaced with their corresponding previous iteration estimates. Under some conditions and ensuring that the information matrices are persistently exciting, the identification algorithms based on the replacement of the unknown variables with their estimates can give satisfactory parameter estimates.

Remark 2.

The computation is very important to evaluate the performance of the algorithm and it can be measured by the numbers of multiplication and addition operations. In particular, one division operation can be counted as one multiplication operation and one subtraction operation can be counted as one addition operation. One multiplication or one addition operation is called one floating point operation (flop for short). Here, we give the computational efficiency by measuring the flop numbers involved in the identification algorithms.

The multiplication and addition numbers of the HLSI algorithm are shown in

Table 2. The computational efficiency of the hierarchical least-squares-based iterative identification algorithm.

Expressions	Number of Multiplications	Number of Additions
${\widehat{\mathit{\alpha }}}_k={\mathit{S}}_1^{\prime }{\mathit{\beta }}_{1,k}\in {\mathbb{R}}^{n_1\times m}$	$m{n}_1^{2}k$	$m{n}_1(n_1-1)k$
${\mathit{S}}_1^{\prime }:={\mathit{S}}_1^{-1}\in {\mathbb{R}}^{n_1\times n_1}$	$n_1^3+n_1^2-n_1+1$	$n_1^3+n_1^2-2n_1$
${\mathit{S}}_1:={\mathit{\varphi }}_{\alpha }\left(L\right){\mathit{\varphi }}_{\alpha }^{\mathrm{T}}\left(L\right)\in {\mathbb{R}}^{n_1\times n_1}$	$n_1^{2}L$	$n_1^2(L-1)$
${\mathit{\beta }}_{1,k}:=\mathit{\Phi }\left(L\right){\mathit{E}}_k^{\mathrm{T}}\left(L\right)\in {\mathbb{R}}^{n_1\times m}$	$m{n}_{1}Lk$	$m{n}_1(L-1)k$
${\mathit{E}}_k\left(L\right):=\mathit{Y}\left(L\right)-{\widehat{\mathit{\alpha }}}_{k-1}^{\mathrm{T}}\mathit{\Phi }\left(L\right)-{\widehat{\mathit{\beta }}}_{k-1}^{\mathrm{T}}{\widehat{\mathit{\Psi }}}_k\left(L\right)$	$mnLk$	$mnLk$
${\widehat{\mathit{\beta }}}_k={\mathit{S}}_{2,k}^{\prime }{\mathit{\beta }}_{2,k}\in {\mathbb{R}}^{n_2\times m}$	$m{n}_2^{2}k$	$m{n}_2(n_2-1)k$
${\mathit{S}}_{2,k}^{\prime }:={\mathit{S}}_{2,k}^{-1}\in {\mathbb{R}}^{n_2\times n_2}$	$[n_2^3+n_2^2-n_2+1]k$	$[n_2^3+n_2^2-2n_2]k$
${\mathit{S}}_{2,k}:={\widehat{\mathit{\Psi }}}_k\left(L\right){\widehat{\mathit{\Psi }}}_k^{\mathrm{T}}\left(L\right)\in {\mathbb{R}}^{n_2\times n_2}$	$n_2^{2}Lk$	$n_2^2(L-1)k$
${\mathit{\beta }}_{2,k}:={\widehat{\mathit{\Psi }}}_k\left(L\right){\mathit{E}}_k^{\mathrm{T}}\left(L\right)\in {\mathbb{R}}^{n_2\times m}$	$m{n}_{2}Lk$	$m{n}_2(L-1)k$
${\widehat{\mathit{w}}}_k\left(s\right)=\mathit{y}\left(s\right)-{\widehat{\mathit{\alpha }}}_k^{\mathrm{T}}{\mathit{\varphi }}_{\alpha }\left(s\right)\in {\mathbb{R}}^m$	$m{n}_{1}Lk$	$m{n}_{1}Lk$
${\widehat{\mathit{v}}}_k\left(s\right)={\widehat{\mathit{w}}}_k-{\widehat{\mathit{\beta }}}_k^{\mathrm{T}}{\widehat{\mathit{\psi }}}_k\left(s\right)\in {\mathbb{R}}^m$	$m{n}_{2}Lk$	$m{n}_{2}Lk$
Sum (Number of multiplications)	$[m(n_1^2+n_2^2)+3mnL+n_2^3+n_2^2(L+1)-n_2+1]k$
	$+n_1^3+n_1^2(L+1)-n_1+1$
Sum (Number of additions)	$[m(n_1^2+n_2^2)+mn(3L-2)+n_2^3+n_2^{2}L-2n_2]k$
	$+n_1^3+n_1^{2}L-2n_1$
Total flops	$N_2:=[2m(n_1^2+n_2^2)+2mn(3L-2)+2n_2^3+n_2^2(2L+1)-3n_2+1]k$
	$+2n_1^3+n_1^2(2L+1)-3n_1+1$

.

The amount of calculations of the LSI and HLSI identification algorithms are compared as follows

$$\begin{array}{ccc}\hfill N_1-N_2& =& [2n^3+(2m+2L+1)n^2+(4mL-2m-3)n+1]k\hfill \\ & & -[2m(n_1^2+n_2^2)+2mn(3L-2)+2n_2^3+n_2^2(2L+1)-3n_2+1]\hfill \\ & & -[2n_1^3+n_1^2(2L+1)-3n_1+1]\hfill \\ & =& [(4m^2+2mL-4m-3)n+3n_1^3+{(2L+3n_2+1)}_1^2+3n_1{n}_2^2+3n_2-1]k\hfill \\ & & -[2n_1^3+n_1^2(2L+1)-3n_1+1]>0.\hfill \end{array}$$

It is obvious that the HLSI identification algorithm requires less computation than the LSI identification algorithm.

Compared with the LSI algorithm, the HLSI algorithm can obtain more accurate parameter estimates and higher computational efficiency for multivariable systems with the colored noise. Furthermore, the proposed approaches in the paper can combine other mathematical optimization approaches [43,44,45,46,47] and statistical strategies [48,49,50,51,52,53,54] to study the parameter estimation problems of linear and nonlinear systems with different disturbances [55,56,57,58,59,60], and can be applied to other fields [61,62,63,64] such as signal processing [65,66,67,68,69,70].

5. The Hierarchical Multi-Innovation Least-Squares-Based Iterative Identification Algorithm

To possess higher identification accuracy of the HLSI algorithm, we propose a hierarchical multi-innovation least-squares-based iterative algorithm by using the multi-innovation identification theory in this section.

Let p be the innovation length. Consider the data from time $l=s-p+1$ to $l=s$ and define the stacked information matrices:

$$\begin{array}{ccc}\hfill \mathit{Y}(p,s)& :=& [\mathit{y}\left(s\right),\mathit{y}(s-1),\cdots ,\mathit{y}(s-p+1)]\in {\mathbb{R}}^{m\times p},\hfill \end{array}$$

(46)

$$\begin{array}{ccc}\hfill \mathit{\Phi }(p,s)& :=& [{\mathit{\varphi }}_{\alpha }\left(s\right),{\mathit{\varphi }}_{\alpha }(s-1),\cdots ,{\mathit{\varphi }}_{\alpha }(s-p+1)]\in {\mathbb{R}}^{n_1\times p},\hfill \end{array}$$

(47)

$$\begin{array}{ccc}\hfill \mathit{\Psi }(p,s)& :=& [\mathit{\psi }\left(s\right),\mathit{\psi }(s-1),\cdots ,\mathit{\psi }(s-p+1)]\in {\mathbb{R}}^{n_2\times p},\hfill \end{array}$$

(48)

$$\begin{array}{ccc}\hfill \mathit{H}(p,s)& :=& [\mathit{h}\left(s\right),\mathit{h}(s-1),\cdots ,\mathit{h}(s-p+1)]=\mathit{Y}(p,s)-{\mathit{\beta }}^{\mathrm{T}}\mathit{\Psi }(p,s)\in {\mathbb{R}}^{m\times p},\hfill \end{array}$$

$$\begin{array}{ccc}\hfill \mathit{W}(p,s)& :=& [\mathit{w}\left(s\right),\mathit{w}(s-1),\cdots ,\mathit{w}(s-p+1)]=\mathit{Y}(p,s)-{\mathit{\alpha }}^{\mathrm{T}}\mathit{\Phi }(p,s)\in {\mathbb{R}}^{m\times p}.\hfill \end{array}$$

From the identification model in Equation (6), we define two cost functions:

$$\begin{array}{ccc}\hfill J_3\left(\mathit{\alpha }\right)& :=& \parallel \mathit{H}(p,s)-{\mathit{\alpha }}^{\mathrm{T}}{\mathit{\Phi }(p,s)\parallel }^2\hfill \\ & =& \sum _{l=s-p+1}^s{[\mathit{h}\left(l\right)-{\mathit{\alpha }}^{\mathrm{T}}{\mathit{\varphi }}_{\alpha }\left(l\right)]}^{\mathrm{T}}[\mathit{h}\left(l\right)-{\mathit{\alpha }}^{\mathrm{T}}{\mathit{\varphi }}_{\alpha }\left(l\right)],\hfill \\ \hfill J_4\left(\mathit{\beta }\right)& :=& \parallel \mathit{W}(p,s)-{\mathit{\beta }}^{\mathrm{T}}{\mathit{\Psi }(p,s)\parallel }^2\hfill \\ & =& \sum _{l=s-p+1}^s{[\mathit{w}\left(l\right)-{\mathit{\beta }}^{\mathrm{T}}\mathit{\psi }\left(l\right)]}^{\mathrm{T}}[\mathit{w}\left(l\right)-{\mathit{\beta }}^{\mathrm{T}}\mathit{\psi }\left(l\right)].\hfill \end{array}$$

Let ${\widehat{\mathit{\alpha }}}_k\left(s\right)$ and ${\widehat{\mathit{\beta }}}_k\left(s\right)$ be the kth iterative estimates of $\mathit{\alpha }$ and $\mathit{\beta }$ at the current time s. Minimizing $J_3\left(\mathit{\alpha }\right)$ and $J_4\left(\mathit{\beta }\right)$, we have the iterative relations as follows:

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\alpha }}}_k\left(s\right)& =& {\left[\mathit{\Phi }(p,s){\mathit{\Phi }}^{\mathrm{T}}(p,s)\right]}^{-1}\mathit{\Phi }(p,s){[\mathit{Y}(p,s)-{\widehat{\mathit{\beta }}}_{k-1}^{\mathrm{T}}\left(s\right)\mathit{\Psi }(p,s)]}^{\mathrm{T}},\hfill \end{array}$$

(49)

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\beta }}}_k\left(s\right)& =& {\left[\mathit{\Psi }(p,s){\mathit{\Psi }}^{\mathrm{T}}(p,s)\right]}^{-1}\mathit{\Psi }(p,s){[\mathit{Y}(p,s)-{\widehat{\mathit{\alpha }}}_{k-1}^{\mathrm{T}}\left(s\right)\mathit{\Phi }(p,s)]}^{\mathrm{T}}.\hfill \end{array}$$

(50)

Notice that the information vector $\mathit{\psi }\left(s\right)$ in $\mathit{\Psi }(p,s)$ contains the unmeasured variables $\mathit{w}(s-i)$ and $\mathit{v}(s-j)$, thus the algorithm in Equations (49) and (50) cannot be implemented. The solution is to use the estimates ${\widehat{\mathit{w}}}_{k-1}(s-i)$ and ${\widehat{\mathit{v}}}_{k-1}(s-j)$ of the unknown noise vectors $\mathit{w}(s-i)$ and $\mathit{v}(s-j)$ to define the estimate ${\widehat{\mathit{\psi }}}_k\left(s\right)$ of $\mathit{\Psi }(p,s)$ as

$${\widehat{\mathit{\psi }}}_k\left(s\right):={[-{\widehat{\mathit{w}}}_{k-1}^{\mathrm{T}}(s-1),-{\widehat{\mathit{w}}}_{k-1}^{\mathrm{T}}(s-2),\cdots ,-{\widehat{\mathit{w}}}_{k-1}^{\mathrm{T}}(s-n_c),{\widehat{\mathit{v}}}_{k-1}^{\mathrm{T}}(s-1),{\widehat{\mathit{v}}}_{k-1}^{\mathrm{T}}(s-2),\cdots ,{\widehat{\mathit{v}}}_{k-1}^{\mathrm{T}}(s-n_d)]}^{\mathrm{T}}.$$

(51)

Replacing the unknown parameter matrix $\mathit{\alpha }$ in Equation (22) with its estimate ${\widehat{\mathit{\alpha }}}_k\left(s\right)$, we have the estimate of $\mathit{w}\left(s\right)$ at iteration k as

$${\widehat{\mathit{w}}}_k\left(s\right):=\mathit{y}\left(s\right)-{\widehat{\mathit{\alpha }}}_k^{\mathrm{T}}\left(s\right){\mathit{\varphi }}_{\alpha }\left(s\right).$$

(52)

Equation (23) gives the estimate of $\mathit{v}\left(s\right)$ at iteration k as

$${\widehat{\mathit{v}}}_k\left(s\right):=\mathit{y}\left(s\right)-{\widehat{\mathit{\alpha }}}_k^{\mathrm{T}}\left(s\right){\mathit{\varphi }}_{\alpha }\left(s\right)-{\widehat{\mathit{\beta }}}_k^{\mathrm{T}}\left(s\right){\widehat{\mathit{\psi }}}_k\left(s\right).$$

(53)

Define the estimate ${\widehat{\mathit{\Psi }}}_k(p,s)$ of $\mathit{\Psi }(p,s)$ with the estimate ${\widehat{\mathit{\psi }}}_k\left(s\right)$ of $\mathit{\psi }\left(s\right)$ as

$${\widehat{\mathit{\Psi }}}_k(p,s):=[{\widehat{\mathit{\psi }}}_k\left(s\right),{\widehat{\mathit{\psi }}}_k(s-1),\cdots ,{\widehat{\mathit{\psi }}}_k(s-p+1)]\in {\mathbb{R}}^{n_2\times p}.$$

(54)

Replacing $\mathit{\beta }$, $\mathit{\alpha }$ and $\mathit{\Psi }(p,t)$ in Equations (49) and (50) with their estimates ${\widehat{\mathit{\beta }}}_{k-1}\left(s\right)$, ${\widehat{\mathit{\alpha }}}_{k-1}\left(s\right)$ and ${\widehat{\mathit{\Psi }}}_k(p,t)$, and combining Equations (46), (47), (54), (3), and (51)–(53), we have the hierarchical multi-innovation least-squares-based iterative identification algorithm for the multivariable CARARMA systems as follows:

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\alpha }}}_k\left(s\right)& =& {\left[\mathit{\Phi }(p,s){\mathit{\Phi }}^{\mathrm{T}}(p,s)\right]}^{-1}\mathit{\Phi }(p,s){[\mathit{Y}(p,s)-{\widehat{\mathit{\beta }}}_{k-1}^{\mathrm{T}}\left(s\right){\widehat{\mathit{\Psi }}}_k(p,s)]}^{\mathrm{T}}\hfill \\ & =& {\left[\sum _{l=s-p+1}^s{\mathit{\varphi }}_{\alpha }\left(l\right){\mathit{\varphi }}_{\alpha }^{\mathrm{T}}\left(l\right)\right]}^{-1}\sum _{l=s-p+1}^s{\mathit{\varphi }}_{\alpha }\left(l\right){[\mathit{y}\left(l\right)-{\widehat{\mathit{\beta }}}_{k-1}^{\mathrm{T}}\left(s\right){\widehat{\mathit{\psi }}}_k\left(l\right)]}^{\mathrm{T}},\hfill \end{array}$$

(55)

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\beta }}}_k\left(s\right)& =& {\left[{\widehat{\mathit{\Psi }}}_k(p,s){\widehat{\mathit{\Psi }}}_k(p,s)\right]}^{-1}{\widehat{\mathit{\Psi }}}_k(p,s){[\mathit{Y}(p,s)-{\widehat{\mathit{\alpha }}}_{k-1}^{\mathrm{T}}\left(s\right){\mathit{\varphi }}_{\alpha }(p,s)]}^{\mathrm{T}}\hfill \\ & =& {\left[\sum _{l=s-p+1}^s{\widehat{\mathit{\psi }}}_k\left(l\right){\widehat{\mathit{\psi }}}_k^{\mathrm{T}}\left(l\right)\right]}^{-1}\sum _{l=s-p+1}^s{\widehat{\mathit{\psi }}}_k\left(l\right){[\mathit{y}\left(l\right)-{\widehat{\mathit{\alpha }}}_{k-1}^{\mathrm{T}}\left(s\right){\mathit{\varphi }}_{\alpha }\left(l\right)]}^{\mathrm{T}},\hfill \end{array}$$

(56)

$$\begin{array}{ccc}\hfill \mathit{Y}(p,s)& =& \left[\mathit{y}\right(s),\mathit{y}(s-1),\cdots ,\mathit{y}(s-p+1\left)\right],\hfill \end{array}$$

(57)

$$\begin{array}{ccc}\hfill \mathit{\Phi }(p,s)& =& [{\mathit{\varphi }}_{\alpha }\left(s\right),{\mathit{\varphi }}_{\alpha }(s-1),\cdots ,{\mathit{\varphi }}_{\alpha }(s-p+1)],\hfill \end{array}$$

(58)

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\Psi }}}_k(p,s)& =& [{\widehat{\mathit{\psi }}}_k\left(s\right),{\widehat{\mathit{\psi }}}_k(s-1),\cdots ,{\widehat{\mathit{\psi }}}_k(s-p+1)],\hfill \end{array}$$

(59)

$$\begin{array}{ccc}\hfill {\mathit{\varphi }}_{\alpha }\left(s\right)& =& [-{\mathit{y}}^{\mathrm{T}}(s-1),-{\mathit{y}}^{\mathrm{T}}(s-2),\cdots ,-{\mathit{y}}^{\mathrm{T}}(s-n_a),\hfill \\ & & {\mathit{u}}^{\mathrm{T}}(s-1),{\mathit{u}}^{\mathrm{T}}(s-2),\cdots ,{\mathit{u}}^{\mathrm{T}}(s-n_b){]}^{\mathrm{T}},\hfill \end{array}$$

(60)

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\psi }}}_k\left(s\right)& =& [-{\widehat{\mathit{w}}}_{k-1}^{\mathrm{T}}(s-1),-{\widehat{\mathit{w}}}_{k-1}^{\mathrm{T}}(s-2),\cdots ,-{\widehat{\mathit{w}}}_{k-1}^{\mathrm{T}}(s-n_c),\hfill \\ & & {\widehat{\mathit{v}}}_{k-1}^{\mathrm{T}}(s-1),{\widehat{\mathit{v}}}_{k-1}^{\mathrm{T}}(s-2),\cdots ,{\widehat{\mathit{v}}}_{k-1}^{\mathrm{T}}(s-n_d){]}^{\mathrm{T}},\hfill \end{array}$$

(61)

$$\begin{array}{ccc}\hfill {\widehat{\mathit{w}}}_k\left(s\right)& =& \mathit{y}\left(s\right)-{\widehat{\mathit{\alpha }}}_k^{\mathrm{T}}\left(s\right){\mathit{\varphi }}_{\alpha }\left(s\right),s=1,2,\cdots ,s,\hfill \end{array}$$

(62)

$$\begin{array}{ccc}\hfill {\widehat{\mathit{v}}}_k\left(s\right)& =& \mathit{y}\left(s\right)-{\widehat{\mathit{\alpha }}}_k^{\mathrm{T}}\left(s\right){\mathit{\varphi }}_{\alpha }\left(s\right)-{\widehat{\mathit{\beta }}}_k^{\mathrm{T}}\left(s\right){\widehat{\mathit{\psi }}}_k\left(s\right),\hfill \end{array}$$

(63)

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\theta }}}_k^{\mathrm{T}}\left(s\right)& =& [{\widehat{\mathit{\alpha }}}_k^{\mathrm{T}}\left(s\right),{\widehat{\mathit{\beta }}}_k^{\mathrm{T}}\left(s\right)],\hfill \end{array}$$

(64)

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\alpha }}}_k^{\mathrm{T}}\left(s\right)& =& [{\widehat{\mathit{A}}}_{1,k}\left(s\right),{\widehat{\mathit{A}}}_{2,k}\left(s\right),\cdots ,{\widehat{\mathit{A}}}_{n_a,k}\left(s\right),{\widehat{\mathit{B}}}_{1,k}\left(s\right),{\widehat{\mathit{B}}}_{2,k}\left(s\right),\cdots ,{\widehat{\mathit{B}}}_{n_b,k}\left(s\right)],\hfill \end{array}$$

(65)

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\beta }}}_k^{\mathrm{T}}\left(s\right)& =& [{\widehat{\mathit{C}}}_{1,k}\left(s\right),{\widehat{\mathit{C}}}_{2,k}\left(s\right),\cdots ,{\widehat{\mathit{C}}}_{n_c,k}\left(s\right),{\widehat{\mathit{D}}}_{1,k}\left(s\right),{\widehat{\mathit{D}}}_{2,k}\left(s\right),\cdots ,{\widehat{\mathit{D}}}_{n_d,k}\left(s\right)].\hfill \end{array}$$

(66)

The steps of the HMIGI algorithm for computing ${\widehat{\mathit{\alpha }}}_k\left(s\right)$ and ${\widehat{\mathit{\beta }}}_k\left(s\right)$ are listed as follows.

• To initialize, choose an innovation length p, let $s=p$, give some small positive $\epsilon$ and set the maximum iterative number $k_{max}$. Set the initial values: ${\widehat{\mathit{\theta }}}_0=\left[\begin{array}{c}{\widehat{\mathit{\alpha }}}_0\left(s\right)\\ {\widehat{\mathit{\beta }}}_0\left(s\right)\end{array}\right]={\mathbf{1}}_{n\times m}/p_0$, ${\widehat{\mathit{w}}}_0\left(s\right)$ and ${\widehat{\mathit{v}}}_0\left(s\right)$ to be random vectors.
• Let $k=1$ and collect the input–output data $\left\{\mathit{u}\right(s),\mathit{y}(s\left)\right\}$. Form the stacked output matrix $\mathit{Y}(p,s)$ by Equation (57), form the information vector ${\mathit{\varphi }}_{\alpha }\left(s\right)$ by Equation (60) and the stacked information matrix $\mathit{\Phi }(p,s)$ by Equation (58).
• Form the information vector ${\widehat{\mathit{\psi }}}_k\left(s\right)$ by Equation (61) and the stacked information matrix ${\widehat{\mathit{\Psi }}}_k(p,s)$ by Equation (59).
• Update the parameter estimates ${\widehat{\mathit{\alpha }}}_k\left(s\right)$ by Equation (55) and ${\widehat{\mathit{\beta }}}_k\left(s\right)$ by Equation (56).
• Compute ${\widehat{\mathit{w}}}_k\left(s\right)$ and ${\widehat{\mathit{v}}}_k\left(s\right)$ by Equations (62) and (63).
• If $k<k_{max}$, increase k by 1 and go to Step 3; otherwise, proceed to the next step.
• Compare ${\widehat{\mathit{\theta }}}_k\left(s\right):=\left[\begin{array}{c}{\widehat{\mathit{\alpha }}}_k\left(s\right)\\ {\widehat{\mathit{\beta }}}_k\left(s\right)\end{array}\right]$ and ${\widehat{\mathit{\theta }}}_{k-1}\left(s\right)$, for the given small positive $\epsilon$, if $\parallel {\widehat{\mathit{\alpha }}}_k\left(s\right)-{\widehat{\mathit{\alpha }}}_{k-1}\left(s\right)\parallel +\parallel {\widehat{\mathit{\beta }}}_k\left(s\right)-{\widehat{\mathit{\beta }}}_{k-1}\left(s\right)\parallel >\epsilon$, let ${\widehat{\mathit{\theta }}}_0(s+1):={\widehat{\mathit{\theta }}}_k\left(s\right)$ and increase s by 1 and go to Step 2; otherwise, obtain the iterative estimate ${\widehat{\mathit{\theta }}}_k\left(s\right)$ and terminate this procedure.

6. Example

Consider the following simulation system

$$\begin{array}{ccc}\hfill \mathit{A}\left(r\right)\mathit{y}\left(s\right)& =& \mathit{B}\left(r\right)\mathit{u}\left(s\right)+{\mathit{C}}^{-1}\left(r\right)\mathit{D}\left(r\right)\mathit{v}\left(s\right),\hfill \\ \hfill \mathit{A}\left(r\right)& =& {\mathit{I}}_2+{\mathit{A}}_1{r}^{-1}=\left[\begin{array}{cc}\hfill 1+0.50r^{-1}& \hfill 0.40r^{-1}\\ \hfill -0.70r^{-1}& \hfill 1+0.80r^{-1}\end{array}\right],\hfill \\ \hfill \mathit{B}\left(r\right)& =& {\mathit{B}}_1{r}^{-1}=\left[\begin{array}{cc}\hfill 1.00r^{-1}& \hfill -0.50r^{-1}\\ \hfill -0.40r^{-1}& \hfill 1.20r^{-1}\end{array}\right],\hfill \\ \hfill \mathit{C}\left(r\right)& =& {\mathit{I}}_2+{\mathit{C}}_1{r}^{-1}=\left[\begin{array}{cc}\hfill 1-0.55r^{-1}& \hfill 0.20r^{-1}\\ \hfill -1.90r^{-1}& \hfill 1+0.03r^{-1}\end{array}\right],\hfill \\ \hfill \mathit{D}\left(r\right)& =& {\mathit{I}}_2+{\mathit{D}}_1{r}^{-1}=\left[\begin{array}{cc}\hfill 1+0.05r^{-1}& \hfill -0.05r^{-1}\\ \hfill -0.05r^{-1}& \hfill 1+0.05r^{-1}\end{array}\right],\hfill \\ \hfill {\mathit{\theta }}^{\mathrm{T}}& =& [{\mathit{A}}_1,{\mathit{B}}_1,{\mathit{C}}_1,{\mathit{D}}_1].\hfill \end{array}$$

In simulation, $\left\{u_1\left(s\right)\right\}$ and $\left\{u_2\left(s\right)\right\}$ are taken as two independent persistent excitation signal sequences with zero mean and unit variances, $\left\{v_1\left(s\right)\right\}$ and $\left\{v_2\left(s\right)\right\}$ as two independent white noise sequences with zero mean and variances ${\sigma }_1^2$ for $v_1\left(s\right)$ and ${\sigma }_2^2$ for $v_2\left(s\right)$. Taking the data length $L=3000$, ${\sigma }_1^2={\sigma }_2^2={0.05}^2$, ${\sigma }_1^2={\sigma }_2^2={1.00}^2$ and ${\sigma }_1^2={\sigma }_2^2={2.00}^2$, respectively. Applying the LSI and HLSI algorithms to estimate the parameters of this example system, the parameter estimates and the estimation errors $\delta :=\parallel {\widehat{\mathit{\theta }}}_k-\mathit{\theta }\parallel /\parallel \mathit{\theta }\parallel$ of the LSI algorithm are shown in

Table 3. The LSI estimates and errors (${\sigma }_1^2={\sigma }_2^2={0.50}^2$).

k	${\mathit{a}}_{11}$	${\mathit{a}}_{12}$	${\mathit{b}}_{11}$	${\mathit{b}}_{12}$	${\mathit{c}}_{11}$	${\mathit{c}}_{12}$	${\mathit{d}}_{11}$	${\mathit{d}}_{12}$
1	0.45255	0.40643	0.98869	−0.50900	−0.00688	0.00264	−0.00796	−0.00359
2	0.48634	0.38699	0.99227	−0.50971	−0.54477	0.22674	−0.00537	−0.00023
3	0.50958	0.39596	0.99203	−0.50976	−0.56916	0.21497	−0.00523	0.00100
4	0.50615	0.40461	0.99211	−0.50968	−0.56544	0.20941	−0.00531	0.00003
5	0.49758	0.40336	0.99219	−0.50966	−0.55529	0.21183	−0.00537	−0.00049
10	0.49925	0.40139	0.99217	−0.50968	−0.55793	0.21329	−0.00534	−0.00021
True values	0.50000	0.40000	1.00000	−0.50000	−0.55000	0.20000	0.05000	−0.05000
$\mathit{k}$	${\mathit{a}}_{\mathbf{21}}$	${\mathit{a}}_{\mathbf{22}}$	${\mathit{b}}_{\mathbf{21}}$	${\mathit{b}}_{\mathbf{22}}$	${\mathit{c}}_{\mathbf{21}}$	${\mathit{c}}_{\mathbf{22}}$	${\mathit{d}}_{\mathbf{21}}$	${\mathit{d}}_{\mathbf{22}}$	$\mathit{\delta }(\%)$
1	−0.79945	0.73056	−0.38776	1.18668	−0.02309	−0.02288	−0.02250	0.00397	68.20082
2	−0.65715	0.75762	−0.39320	1.19655	−1.91411	0.05582	−0.01435	0.01394	4.08239
3	−0.66034	0.80889	−0.39353	1.19729	−1.91203	−0.00101	−0.01470	0.00996	3.81620
4	−0.70285	0.80968	−0.39302	1.19743	−1.86645	0.00885	−0.01503	0.00676	3.61680
5	−0.71357	0.79623	−0.39296	1.19737	−1.85580	0.02099	−0.01502	0.00710	3.66328
10	−0.70093	0.79640	−0.39309	1.19732	−1.87117	0.01880	−0.01492	0.00802	3.45265
True values	−0.70000	0.80000	−0.40000	1.20000	−1.90000	0.03000	−0.05000	0.05000

,

Table 4. The LSI estimates and errors (${\sigma }_1^2={\sigma }_2^2={1.00}^2$).

k	${\mathit{a}}_{11}$	${\mathit{a}}_{12}$	${\mathit{b}}_{11}$	${\mathit{b}}_{12}$	${\mathit{c}}_{11}$	${\mathit{c}}_{12}$	${\mathit{d}}_{11}$	${\mathit{d}}_{12}$
1	0.37911	0.41821	0.97582	−0.51920	−0.02077	0.00610	−0.01571	−0.00216
2	0.45862	0.36770	0.98462	−0.51944	−0.51850	0.24583	−0.01069	−0.00056
3	0.52330	0.37841	0.98443	−0.51962	−0.58539	0.22828	−0.01071	0.00344
4	0.52177	0.41221	0.98429	−0.51938	−0.58362	0.20060	−0.01068	0.00061
5	0.49205	0.40950	0.98436	−0.51932	−0.54986	0.20628	−0.01068	−0.00126
10	0.49662	0.40205	0.98442	−0.51939	−0.55669	0.21270	−0.01067	−0.00036
True values	0.50000	0.40000	1.00000	−0.50000	−0.55000	0.20000	0.05000	−0.05000
$\mathit{k}$	${\mathit{a}}_{\mathbf{21}}$	${\mathit{a}}_{\mathbf{22}}$	${\mathit{b}}_{\mathbf{21}}$	${\mathit{b}}_{\mathbf{22}}$	${\mathit{c}}_{\mathbf{21}}$	${\mathit{c}}_{\mathbf{22}}$	${\mathit{d}}_{\mathbf{21}}$	${\mathit{d}}_{\mathbf{22}}$	$\mathit{\delta }(\%)$
1	−0.93419	0.64539	−0.39021	1.17886	−0.05758	−0.04446	−0.04030	0.01929	67.62883
2	−0.59263	0.66521	−0.38479	1.19285	−1.97468	0.14646	−0.02931	0.03218	8.64671
3	−0.57894	0.81485	−0.38713	1.19469	−1.99036	−0.01829	−0.02970	0.02306	6.64949
4	−0.69557	0.83966	−0.38683	1.19518	−1.86685	−0.02063	−0.02958	0.01271	4.35994
5	−0.74887	0.79252	−0.38647	1.19502	−1.81223	0.02568	−0.02959	0.01277	4.77498
10	−0.70220	0.78945	−0.38650	1.19486	−1.86665	0.02395	−0.02957	0.01625	3.46273
True values	−0.70000	0.80000	−0.40000	1.20000	−1.90000	0.03000	−0.05000	0.05000

and

Table 5. The LSI estimates and errors (${\sigma }_1^2={\sigma }_2^2={2.00}^2$).

k	${\mathit{a}}_{11}$	${\mathit{a}}_{12}$	${\mathit{b}}_{11}$	${\mathit{b}}_{12}$	${\mathit{c}}_{11}$	${\mathit{c}}_{12}$	${\mathit{d}}_{11}$	${\mathit{d}}_{12}$
1	0.30598	0.43273	0.95321	−0.54078	−0.05132	0.00896	−0.03407	0.00703
2	0.42316	0.35154	0.96908	−0.53879	−0.48423	0.26160	−0.02134	−0.00104
3	0.54052	0.33373	0.97007	−0.53902	−0.60385	0.27146	−0.02198	0.00733
4	0.55436	0.42567	0.96900	−0.53866	−0.61803	0.18606	−0.02160	0.00172
5	0.48269	0.42471	0.96851	−0.53864	−0.54117	0.19036	−0.02131	−0.00235
10	0.49093	0.40387	0.96891	−0.53872	−0.55204	0.21020	−0.02138	−0.00053
True values	0.50000	0.40000	1.00000	−0.50000	−0.55000	0.20000	0.05000	−0.05000
$\mathit{k}$	${\mathit{a}}_{\mathbf{21}}$	${\mathit{a}}_{\mathbf{22}}$	${\mathit{b}}_{\mathbf{21}}$	${\mathit{b}}_{\mathbf{22}}$	${\mathit{c}}_{\mathbf{21}}$	${\mathit{c}}_{\mathbf{22}}$	${\mathit{d}}_{\mathbf{21}}$	${\mathit{d}}_{\mathbf{22}}$	$\mathit{\delta }(\%)$
1	−1.05293	0.57820	−0.40261	1.16284	−0.12697	−0.08390	−0.08149	0.05918	66.29107
2	−0.52615	0.53991	−0.36897	1.18798	−2.03624	0.27031	−0.05907	0.05664	15.54031
3	−0.44989	0.78904	−0.37305	1.18991	−2.11502	0.00375	−0.06016	0.04746	12.73434
4	−0.67116	0.92651	−0.37609	1.19057	−1.88402	−0.10881	−0.05836	0.02468	8.21885
5	−0.82963	0.79822	−0.37539	1.19022	−1.72334	0.01847	−0.05820	0.02430	8.51037
10	−0.71231	0.78084	−0.37425	1.19008	−1.85093	0.03082	−0.05865	0.03211	4.13713
True values	−0.70000	0.80000	−0.40000	1.20000	−1.90000	0.03000	−0.05000	0.05000

, and the parameter estimates and the estimation errors $\delta :=\sqrt{\frac{\parallel {\widehat{\mathit{\alpha }}}_k{-\mathit{\alpha }\parallel }^2+{\parallel {\widehat{\mathit{\beta }}}_k-\mathit{\beta }\parallel }^2}{{\parallel \mathit{\alpha }\parallel }^2+{\parallel \mathit{\beta }\parallel }^2}}$ of the HLSI algorithm are shown in

Table 6. The HLSI estimates and errors (${\sigma }_1^2={\sigma }_2^2={0.50}^2$).

k	${\mathit{a}}_{11}$	${\mathit{a}}_{12}$	${\mathit{b}}_{11}$	${\mathit{b}}_{12}$	${\mathit{c}}_{11}$	${\mathit{c}}_{12}$	${\mathit{d}}_{11}$	${\mathit{d}}_{12}$
1	0.45263	0.40637	0.98875	−0.50909	−0.04541	0.02727	0.02581	0.01375
2	0.45491	0.40356	0.98880	−0.51009	−0.51110	0.20729	−0.00526	0.00002
3	0.48853	0.39232	0.99200	−0.50967	−0.51309	0.20921	−0.00489	0.00030
4	0.50095	0.39372	0.99204	−0.50988	−0.54442	0.21420	−0.00477	0.00107
5	0.50362	0.40108	0.99221	−0.50987	−0.55317	0.21647	−0.00489	0.00036
10	0.49955	0.40119	0.99217	−0.50969	−0.55804	0.21307	−0.00534	−0.00017
True values	0.50000	0.40000	1.00000	−0.50000	−0.55000	0.20000	0.05000	−0.05000
$\mathit{k}$	${\mathit{a}}_{\mathbf{21}}$	${\mathit{a}}_{\mathbf{22}}$	${\mathit{b}}_{\mathbf{21}}$	${\mathit{b}}_{\mathbf{22}}$	${\mathit{c}}_{\mathbf{21}}$	${\mathit{c}}_{\mathbf{22}}$	${\mathit{d}}_{\mathbf{21}}$	${\mathit{d}}_{\mathbf{22}}$	$\mathit{\delta }(\%)$
1	−0.79936	0.73063	−0.38834	1.18748	0.01185	−0.13866	0.00423	0.01831	69.14794
2	−0.79589	0.74816	−0.39168	1.18917	−1.77185	0.08294	−0.00952	0.01716	7.20851
3	−0.66539	0.75709	−0.39207	1.19513	−1.76251	0.05227	−0.01002	0.01671	6.24422
4	−0.67843	0.79006	−0.39212	1.19557	−1.88355	0.05344	−0.01313	0.01080	3.52902
5	−0.68501	0.80187	−0.39217	1.19649	−1.88049	0.02427	−0.01446	0.00823	3.41217
10	−0.70042	0.79617	−0.39310	1.19731	−1.87272	0.01863	−0.01494	0.00806	3.43715
True values	−0.70000	0.80000	−0.40000	1.20000	−1.90000	0.03000	−0.05000	0.05000

,

Table 7. The HLSI estimates and errors (${\sigma }_1^2={\sigma }_2^2={1.00}^2$).

k	${\mathit{a}}_{11}$	${\mathit{a}}_{12}$	${\mathit{b}}_{11}$	${\mathit{b}}_{12}$	${\mathit{c}}_{11}$	${\mathit{c}}_{12}$	${\mathit{d}}_{11}$	${\mathit{d}}_{12}$
1	0.37942	0.41810	0.97612	−0.51907	−0.06555	0.02995	0.02371	0.03103
2	0.38912	0.41194	0.97660	−0.52021	−0.43935	0.19532	−0.01176	0.00257
3	0.45524	0.38505	0.98314	−0.51937	−0.45002	0.19959	−0.01084	0.00294
4	0.48057	0.37803	0.98320	−0.51970	−0.51466	0.21120	−0.01033	0.00417
5	0.50132	0.39142	0.98407	−0.51981	−0.52689	0.22128	−0.00933	0.00318
10	0.49770	0.39981	0.98448	−0.51946	−0.56257	0.21197	−0.01094	−0.00043
True values	0.50000	0.40000	1.00000	−0.50000	−0.55000	0.20000	0.05000	−0.05000
$\mathit{k}$	${\mathit{a}}_{\mathbf{21}}$	${\mathit{a}}_{\mathbf{22}}$	${\mathit{b}}_{\mathbf{21}}$	${\mathit{b}}_{\mathbf{22}}$	${\mathit{c}}_{\mathbf{21}}$	${\mathit{c}}_{\mathbf{22}}$	${\mathit{d}}_{\mathbf{21}}$	${\mathit{d}}_{\mathbf{22}}$	$\mathit{\delta }(\%)$
1	−0.93342	0.64534	−0.39098	1.18112	0.02877	−0.15072	−0.00904	−0.00210	70.25212
2	−0.93091	0.68495	−0.39579	1.18382	−1.63403	0.16609	−0.02072	0.04766	15.04949
3	−0.64662	0.66320	−0.38416	1.18926	−1.61660	0.09883	−0.02194	0.04712	12.33823
4	−0.65462	0.72677	−0.38511	1.18991	−1.87187	0.13390	−0.02323	0.02942	5.98792
5	−0.61429	0.75987	−0.38266	1.19109	−1.85595	0.06066	−0.02495	0.02355	5.08516
10	−0.70984	0.79018	−0.38689	1.19508	−1.88460	0.02909	−0.03028	0.01504	3.34293
True values	−0.70000	0.80000	−0.40000	1.20000	−1.90000	0.03000	−0.05000	0.05000

and

Table 8. The HLSI estimates and errors (${\sigma }_1^2={\sigma }_2^2={2.00}^2$).

k	${\mathit{a}}_{11}$	${\mathit{a}}_{12}$	${\mathit{b}}_{11}$	${\mathit{b}}_{12}$	${\mathit{c}}_{11}$	${\mathit{c}}_{12}$	${\mathit{d}}_{11}$	${\mathit{d}}_{12}$
1	0.30659	0.43256	0.95390	−0.54000	−0.10583	0.03531	0.01951	0.06559
2	0.33248	0.42280	0.95566	−0.54103	−0.36770	0.18045	−0.02732	0.01179
3	0.40565	0.38654	0.96508	−0.53914	−0.39917	0.18838	−0.02489	0.01057
4	0.44228	0.36624	0.96510	−0.53908	−0.46950	0.19745	−0.02448	0.01207
5	0.47839	0.38036	0.96662	−0.53919	−0.48983	0.22303	−0.02097	0.01083
10	0.50459	0.38505	0.96907	−0.53937	−0.58299	0.21447	−0.02257	−0.00096
True values	0.50000	0.40000	1.00000	−0.50000	−0.55000	0.20000	0.05000	−0.05000
$\mathit{k}$	${\mathit{a}}_{\mathbf{21}}$	${\mathit{a}}_{\mathbf{22}}$	${\mathit{b}}_{\mathbf{21}}$	${\mathit{b}}_{\mathbf{22}}$	${\mathit{c}}_{\mathbf{21}}$	${\mathit{c}}_{\mathbf{22}}$	${\mathit{d}}_{\mathbf{21}}$	${\mathit{d}}_{\mathbf{22}}$	$\mathit{\delta }(\%)$
1	−1.05128	0.57789	−0.40369	1.16799	0.06260	−0.17483	−0.03557	−0.04291	72.22302
2	−1.06002	0.63997	−0.40964	1.17231	−1.51135	0.23182	−0.05330	0.10898	22.60909
3	−0.65570	0.56386	−0.37654	1.18036	−1.48940	0.13195	−0.05586	0.11046	18.48183
4	−0.64791	0.63257	−0.37934	1.18113	−1.85360	0.21998	−0.04887	0.06935	10.56573
5	−0.51920	0.65844	−0.37044	1.18255	−1.81902	0.13118	−0.04833	0.06187	10.21901
10	−0.72624	0.78837	−0.37432	1.19006	−1.96662	0.08720	−0.05894	0.01722	5.15136
True values	−0.70000	0.80000	−0.40000	1.20000	−1.90000	0.03000	−0.05000	0.05000

.

From the simulation results in Table 3, Table 4, Table 5, Table 6, Table 7 and Table 8, we can draw the following conclusions.

• The estimation errors given by the LSI and HLSI algorithms become smaller as the iterative variable k increases (see the estimation errors $\delta$ in Table 3, Table 4, Table 5, Table 6, Table 7 and Table 8).
• The HLSI algorithm leads to smaller parameter estimation errors than the LSI algorithm for the same data length L and noise level (see Table 4 and Table 7).
• With iterations increase, the parameter estimations given by the HLSI algorithm are close to the true parameters, which verify the effectiveness of the HLSI algorithm proposed in this paper (see Table 6, Table 7 and Table 8).

To validate the obtained model, we use the LSI estimates and the HLSI estimates to construct the estimated model, respectively. The predicted outputs are

$$\begin{array}{c}\hfill \widehat{\mathit{y}}\left(s\right)=\mathit{y}\left(s\right)-{\mathit{D}}^{-1}\left(r\right)\mathit{C}\left(r\right)[\mathit{A}\left(r\right)\mathit{y}\left(s\right)-\mathit{B}\left(r\right)\mathit{u}\left(s\right)].\end{array}$$

(67)

Using the LSI parameter estimates in Table 4 with the noise variance ${\sigma }_1^2={\sigma }_2^2={0.50}^2$ and $k=10$ to construct the LSI estimated model:

$$\begin{array}{ccc}\hfill {\widehat{\mathit{y}}}_{LS}\left(s\right)& =& \mathit{y}\left(s\right)-{\widehat{\mathit{D}}}_{LS}^{-1}\left(r\right){\widehat{\mathit{C}}}_{LS}\left(r\right)[{\widehat{\mathit{A}}}_{LS}\left(r\right)\mathit{y}\left(s\right)-{\widehat{\mathit{B}}}_{LS}\left(r\right)\mathit{u}\left(s\right)],\hfill \\ \hfill {\widehat{\mathit{A}}}_{LS}\left(r\right)& =& {\mathit{I}}_2+{\widehat{\mathit{A}}}_{LS1}{r}^{-1}=\left[\begin{array}{cc}\hfill 1+0.49925r^{-1}& \hfill 0.40139r^{-1}\\ \hfill -0.70093r^{-1}& \hfill 1+0.7640r^{-1}\end{array}\right],\hfill \\ \hfill {\widehat{\mathit{B}}}_{LS}\left(r\right)& =& {\widehat{\mathit{B}}}_{LS1}{r}^{-1}=\left[\begin{array}{cc}\hfill 0.992172r^{-1}& \hfill -0.50968r^{-1}\\ \hfill -0.39309r^{-1}& \hfill 1.19732r^{-1}\end{array}\right],\hfill \\ \hfill {\widehat{\mathit{C}}}_{LS}\left(r\right)& =& {\mathit{I}}_2+{\widehat{\mathit{C}}}_{LS1}{r}^{-1}=\left[\begin{array}{cc}\hfill 1-0.55793r^{-1}& \hfill 0.21329r^{-1}\\ \hfill -1.87117r^{-1}& \hfill 1+0.01880r^{-1}\end{array}\right],\hfill \\ \hfill {\widehat{\mathit{D}}}_{LS}\left(r\right)& =& {\mathit{I}}_2+{\widehat{\mathit{D}}}_{LS1}{r}^{-1}=\left[\begin{array}{cc}\hfill 1-0.00534r^{-1}& \hfill -0.00021r^{-1}\\ \hfill -0.01492r^{-1}& \hfill 1+0.00802r^{-1}\end{array}\right].\hfill \end{array}$$

Using the HLSI parameter estimates in Table 7 with the noise variance ${\sigma }_1^2={\sigma }_2^2={0.50}^2$ and $k=10$ to construct the HLSI estimated model as

$$\begin{array}{ccc}\hfill {\widehat{\mathit{y}}}_H\left(s\right)& =& \mathit{y}\left(s\right)-{\widehat{\mathit{D}}}_H^{-1}\left(r\right){\widehat{\mathit{C}}}_H\left(r\right)[{\widehat{\mathit{A}}}_H\left(r\right)\mathit{y}\left(s\right)-{\widehat{\mathit{B}}}_H\left(r\right)\mathit{u}\left(s\right)],\hfill \\ \hfill {\widehat{\mathit{A}}}_H\left(r\right)& =& {\mathit{I}}_2+{\widehat{\mathit{A}}}_{H1}{r}^{-1}=\left[\begin{array}{cc}\hfill 1+0.49955r^{-1}& \hfill 0.40119r^{-1}\\ \hfill -0.70042r^{-1}& \hfill 1+0.79617r^{-1}\end{array}\right],\hfill \\ \hfill {\widehat{\mathit{B}}}_H\left(r\right)& =& {\widehat{\mathit{B}}}_{H1}{r}^{-1}=\left[\begin{array}{cc}\hfill 0.99217r^{-1}& \hfill -0.50969r^{-1}\\ \hfill -0.39310r^{-1}& \hfill 1.19731r^{-1}\end{array}\right],\hfill \\ \hfill {\widehat{\mathit{C}}}_H\left(r\right)& =& {\mathit{I}}_2+{\widehat{\mathit{C}}}_{H1}{r}^{-1}=\left[\begin{array}{cc}\hfill 1-0.55804r^{-1}& \hfill 0.21307r^{-1}\\ \hfill -1.87272r^{-1}& \hfill 1+0.01863r^{-1}\end{array}\right],\hfill \\ \hfill {\widehat{\mathit{D}}}_H\left(r\right)& =& {\mathit{I}}_2+{\widehat{\mathit{D}}}_{H1}{r}^{-1}=\left[\begin{array}{cc}\hfill 1-0.00534r^{-1}& \hfill -0.00017r^{-1}\\ \hfill -0.01494r^{-1}& \hfill 1+0.00806r^{-1}\end{array}\right].\hfill \end{array}$$

For model validation, we use a different dataset with a length of $L_r=100$, whose data are from $s=L+1$ to $s=L+L_r$, to compute the predicted outputs $\widehat{\mathit{y}}\left(s\right)={[{\widehat{y}}_1\left(s\right),{\widehat{y}}_2\left(s\right)]}^{\mathrm{T}}\in {\mathbb{R}}^2$ of the system. The actual outputs ${\mathit{y}}_i\left(s\right)$ ($i=1,2$), the predicted output ${\widehat{\mathit{y}}}_{LSI}\left(s\right)$ based on the LSI parameter estimates, and the predicted output ${\widehat{\mathit{y}}}_{HLSI}\left(s\right)$ based on the HLSI parameter estimates are plotted in Figure 1 and Figure 2. Define the root-mean-square errors (RMSEs) as

$$Error\left(\widehat{\mathit{y}}\right):=\sqrt{\frac{1}{L_r}\sum _{s=L}^{L+L_r}{[\widehat{\mathit{y}}\left(s\right)-\mathit{y}\left(s\right)]}^2}.$$

(68)

Using the estimated outputs to compute the root-mean-square errors for the LSI and HLSI estimated models as

$$\begin{array}{ccc}\hfill Error\left({\widehat{\mathit{y}}}_{LS}\right)& :=& \sqrt{\frac{1}{L_r}\sum _{s=L}^{L+L_r}{[{\widehat{\mathit{y}}}_{LS}\left(s\right)-\mathit{y}\left(s\right)]}^2}={[0.89203,1.24336]}^{\mathrm{T}}\in {\mathbb{R}}^2,\hfill \\ \hfill Error\left({\widehat{\mathit{y}}}_H\right)& :=& \sqrt{\frac{1}{L_r}\sum _{s=L}^{L+L_r}{[{\widehat{\mathit{y}}}_H\left(s\right)-\mathit{y}\left(s\right)]}^2}={[0.65907,1.22223]}^{\mathrm{T}}\in {\mathbb{R}}^2.\hfill \end{array}$$

Remark 3.

Figure 1 and Figure 2 demonstrate that the predicted outputs are close to the measurement outputs, which means that the LSI and HLSI estimation algorithms are effective and they can give accurate estimates of the multivariable CARARMA systems.

Remark 4.

In Figure 1 and Figure 2, we can see that the predicted outputs of the LSI and HLSI algorithms are close to true values for $y_1\left(s\right)$. However, for $y_2\left(s\right)$, the predicted outputs of the LSI algorithm has a significant discrepancy while the HLSI estimates remain close to true values, which shows that the LSI method has a limitation for application to this example. On the other hand, the proposed HLSI algorithm achieves good estimation performance for identifying the multivariable CARARMA systems. Generally speaking, the combination of the decomposition techniques and the iterative technique results in more accurate parameter estimates and better computational efficiency for multivariable systems.

7. Conclusions

This work proposes a hierarchical least-squares-based iterative identification algorithm and a hierarchical multi-innovation least-squares-based iterative identification algorithm for multivariable CARARMA systems based on the hierarchical identification principle and the iterative technique. Compared with the least-squares-based iterative identification algorithm, the proposed algorithms achieve highly accurate parameter estimates and improve the performance the algorithms. The proposed decomposition least-squares-based iterative identification algorithms for multivariable equation-error autoregressive moving average systems can combine other estimation methods [71,72,73,74] and the mathematical techniques [75,76,77] to explore the parameter identification methods of other scalar, multivariable linear, nonlinear systems with colored noises [78,79,80], and can be extended to other scientific fields [81,82,83,84,85,86,87,88] such as signal modeling and communication networked systems [89,90,91,92].