Iterative Identification for Multivariable Systems with Time-Delays Based on Basis Pursuit De-Noising and Auxiliary Model

Abstract

This paper focuses on the joint estimation of parameters and time-delays of the multiple-input single-output output-error systems. Since the time-delays are unknown, an effective identification model with a high dimensional and sparse parameter vector is established based on overparameterization. Then, the identification problem is converted to a sparse optimization problem. Based on the basis pursuit de-noising criterion and the auxiliary model identification idea, an auxiliary model based basis pursuit de-noising iterative algorithm is presented. The parameters are estimated by solving a quadratic program, and the unavailable terms in the information vector are updated by the auxiliary model outputs iteratively. The time-delays are estimated according to the sparse structure of the parameter vector. The proposed method can obtain effective estimates of the parameters and time-delays from few sampled data. The simulation results illustrate the effectiveness of the proposed algorithm.

1. Introduction

This paper focuses on the identification of the multiple-input single-output output-error systems with unknown time-delays. The investigation is introduced from its background, the formulation of the problem, the literature survey, and the scope and the contributions in this section.

1.1. Background

System identification is a process of developing mathematical models of dynamical systems based on observed data [1]. The dynamical system to be modeled can be a mechanical system, an industrial system or other types of system. Many identification methods have been proposed for the specific systems. For example, some identification techniques for mechanical systems can be found in [2,3,4]. In this paper, we study the identification of multivariable systems because most of the industrial processes can be modeled as multivariable systems.

Multivariable systems are a class of systems which often present significant interactions between the multiple-input and multiple-output channels. A wealth of literature is available on effective identification algorithms for such systems. For example, an expectation-maximization technique was employed for the estimation of the linear time-invariant (LTI) multiple-input multiple-output (MIMO) systems [5]. A matchable-observable linear identification with zero-order oracle filter tuning method was developed for the LTI MIMO systems by comprising the problems of parameter estimation, filter tuning and model structure selection [6]. The methods proposed in [5,6] are applicable for the MIMO systems without time-delays. In many industry processes, time-delays are usually unavoidable. Therefore, it is of great interest to study the identification of multivariable systems with unknown time-delays.

For the estimation of time-delays in multivariable systems, several methods have been proposed. In [7], a parametric model is first estimated by using generalized orthonormal basis filters. Then, the time-delay estimates are generated by analyzing the simulated step response of the resulting model. However, it is required that the model structure is available a priori. To address this problem, a non-parametric method was proposed in [8]. The central idea is to break up a general $m\times n$ MIMO system into $mn$ decoupled single-input single-output systems using partial coherence functions, and then to apply the frequency-domain method to estimate the time-delay of each subsystem. Since the time-delay estimation procedure was repeated for $mn$ times, this method requires a large amount of computation and is limited for high order systems. In this paper, we attempt to find a simple and efficient method to address the joint estimation of time-delays and parameters of multivariable systems.

1.2. Formulation of the Problem of Interest for this Investigation

For simplicity, we consider the identification problem of multiple-input single-output output-error (MISO-OE) systems with unknown input time-delays. The main body of the MISO-OE model is the impulse transfer functions between the inputs and output. The identification challenge lies in that the system output is nonlinear in the polynomial parameters.

For the identification of the multivariable systems with time-delays, it is needed to form an appropriate identification model first. Taking into account the unknown time-delays, a high dimensional identification model is formed with a sparse parameter vector, which contains many zeros [9,10]. For the identification of high dimensional models, traditional identification methods, such as the least squares method and the stochastic gradient method, require many observations as well as heavy computational burden. However, there are many situations in reality where only limited observations are available such as setpoint-operated processes, online estimation and linear time-variant system identification [11]. It has proven that traditional identification is unreliable in these cases and the estimates often tend to be biased or have large variances [12]. Therefore, it is necessary to develop new identification methods and reduce the identification cost.

The compressive sensing theory originally arose in the signal processing community, which enables the reconstruction of sparse or compressible signals under the undersampling condition [13]. In view of the sparse characteristic of the identification model, the estimation of the sparse parameter vector can be addressed as a reconstruction problem of the sparse signal.

1.3. Literature Survey

Greedy algorithms and convex optimization approaches are most widely used reconstruction methods. Greedy algorithms are commonly applied for solving minimum $l_0$ norm problem with advantages of fast speed and easy implement [14]. Convex optimization approaches are a class of global optimization methods, which address minimum $l_1$ norm problems with high stability and strong applicability [15]. Basis pursuit, basis pursuit de-noising (BPDN) [16], the least absolute shrinkage and selection operator (LASSO) [17], and the least angle regression [18] are typical convex optimization approaches. These approaches can be converted to linear programming or quadratic programming form, and then the optimal solutions can be obtained [19]. Recently, the reconstruction methods have been applied for system identification. For example, the orthogonal matching pursuit algorithm was applied to obtain the joint estimation of parameters and time-delays of the MISO finite impulse response systems and MISO controlled autoregressive systems [20,21]. The compressive sampling matching pursuit algorithm was modified by employing the instrumental variable method to identify a class of closed-loop systems [22]. The LASSO approach is investigated for computing efficient model structure of the overparameterized nonlinear autoregressive, moving average exogenous systems [23]. In this paper, we aim to identify the MISO-OE systems with time-delays based on a convex optimization approach.

Compared with the systems studied in [20,21], the structure of the MISO-OE systems is more general and complex. Considering that the system output is nonlinear in the polynomial parameters, the MISO-OE system model can be transformed into a linear regression form by introducing intermediate variables. However, the intermediate variables are unmeasurable and make the identification difficult. Fortunately, the auxiliary model identification idea can solve this problem by replacing unmeasured variables with outputs of the auxiliary model which is constructed of measurable variables [24]. Combining the auxiliary model idea with conventional methods, many new identification methods have been developed such as the auxiliary model based stochastic gradient algorithm [25], and the auxiliary model based least squares (AM-RLS) algorithm [26]. It is indicated that the AM-RLS algorithm can obtain unbiased and consistent estimation of parameters under a persistent excitation condition. Inspired by the algorithms in [25,26], we attempt to combine the auxiliary model idea with the convex optimization approach to deal with the joint estimation of parameters and time-delays of MISO-OE systems.

1.4. Scope and Contribution of this Study

This investigation deals with the identification of the MISO-OE systems with unknown time-delays based on convex optimization and auxiliary model. The structure of the MISO-OE systems is different from the systems studied in [20,21], which means that different methods need to be applied to form the identification model. In this paper, the auxiliary model technique is employed to address the nonlinearity of parameters, and an effective identification model with a high dimensional and sparse parameter vector is derived due to the unknown time-delays. In addition, a different identification method is applied here. The methods proposed in [20,21] are modifications of a greedy algorithm. In this paper, a convex optimization approach, the BPDN approach, is modified for identification due to its robustness. By converting the BPDN approach to a quadratic programming form, the parameters are calculated from the optimal solutions, and the unmeasurable terms in the information vector are updated iteratively. The unknown time-delays can be read from the estimated parameter vector. The effectiveness of the proposed algorithm is tested by simulation examples.

1.5. Organization of the Paper

The rest of this paper is organized as follows. Section 2 introduces the MISO-OE systems with time-delays and describes the identification problem. Section 3 presents an auxiliary model-basis pursuit de-noising iterative (AM-BPDNI) algorithm. Section 4 provides a simulation example to show the effectiveness of the AM-BPDNI algorithm. Finally, some summaries are given in Section 5.

2. Problem Description

Consider an MISO-OE system

$$y\left(t\right)=\sum _{i=1}^r{z}^{-d_i}\frac{B_i\left(z\right)}{A_i\left(z\right)}{u}_i\left(t\right)+v\left(t\right),$$

(1)

where $u_i\left(t\right)$ and $d_i$ are the input and time-delay of the ith input channel, $y\left(t\right)$ is the output, $v\left(t\right)$ is a white noise vector with zero mean and variance ${\sigma }^2$, and $A_i\left(z\right)$ and $B_i\left(z\right)$ are the time-invariant polynomials with constant coefficients in the unit backward shift operator $z^{-1}$$[\mathrm{i}.\mathrm{e}.,u\left(t\right)z^{-1}=u(t-1)]$,

$$\begin{array}{ccc}\hfill A_i\left(z\right)& :=& 1+a_{i1}{z}^{-1}+a_{i2}{z}^{-2}+\cdots +a_{i{n}_{ai}}{z}^{-n_{ai}},\hfill \\ \hfill B_i\left(z\right)& :=& b_{i1}{z}^{-1}+b_{i2}{z}^{-2}+\cdots +b_{i{n}_{bi}}{z}^{-n_{bi}}.\hfill \end{array}$$

Assume that the orders $n_{ai}$ and $n_{bi}$ are known, $y\left(t\right)=0$, $u_i\left(t\right)=0$ and $v\left(t\right)=0$ for $t<0$.

To form the identification model, an intermediate variable is introduced [24],

$$\begin{array}{c}\hfill x_i\left(t\right):=z^{-d_i}\frac{B_i\left(z\right)}{A_i\left(z\right)}{u}_i\left(t\right)=[1-A_i\left(z\right)]x_i\left(t\right)+z^{-d_i}{B}_i\left(z\right)u_i\left(t\right).\end{array}$$

Since the time-delay $d_i$ of each input channel is unknown, an overparameterization method is applied by setting a maximum input regression length l which satisfies $l\ge \max (d_i+n_{bi})$ [20]. Then, $x_i\left(t\right)$ can be written in an impact form

$$\begin{array}{c}\hfill x_i\left(t\right)={\mathit{\phi }}_i^T\left(t\right){\mathit{\theta }}_i,\end{array}$$

where

$$\begin{array}{ccc}\hfill {\mathit{\phi }}_i\left(t\right)& :=& [-x_i(t-1),-x_i(t-2),\cdots ,-x_i(t-n_{ai}),u_i(t-1),\cdots ,u_i(t-d_i),\hfill \\ \hfill & & u_i(t-d_i-1),\cdots ,u_i(t-d_i-n_{bi}),\cdots ,u_i{(t-l)]}^T\in {\mathbb{R}}^{n_{ai}+l},\hfill \end{array}$$

(2)

$$\begin{array}{ccc}\hfill {\mathit{\theta }}_i& :=& {[a_{i1},a_{i2},\cdots ,a_{i{n}_{ai}},\underset{d_i}{\underbrace{0,\cdots ,0}},b_{i1},b_{i2},\cdots ,b_{i{n}_{bi}},\underset{l-d_i-n_{bi}}{\underbrace{0,\cdots ,0}}]}^T\in {\mathbb{R}}^{n_{ai}+l}.\hfill \end{array}$$

(3)

The identification model of the system in Equation (1) can be formed as

$$\begin{array}{ccc}\hfill y\left(t\right)& =& \sum _{i=1}^r{x}_i\left(t\right)+v\left(t\right)={\mathit{\phi }}^T\left(t\right)\mathit{\theta }+v\left(t\right),\hfill \end{array}$$

(4)

where

$$\begin{array}{ccc}\hfill \mathit{\phi }\left(t\right)& :=& {[{\mathit{\phi }}_1^T\left(t\right),{\mathit{\phi }}_2^T\left(t\right),\cdots ,{\mathit{\phi }}_r^T\left(t\right)]}^T\in {\mathbb{R}}^n,\hfill \end{array}$$

(5)

$$\begin{array}{ccc}\hfill \mathit{\theta }& :=& {[{\mathit{\theta }}_1^T,{\mathit{\theta }}_2^T,\cdots ,{\mathit{\theta }}_r^T]}^T\in {\mathbb{R}}^n,\hfill \end{array}$$

(6)

$$\begin{array}{ccc}\hfill n& :=& \sum _{i=1}^r(n_{ai}+l).\hfill \end{array}$$

(7)

It can be seen from Equations (3) and (6) that the parameter vector $\mathit{\theta }$ contains many zeros, therefore $\mathit{\theta }$ is a sparse vector and the system in Equation (4) is a sparse system. The sparsity level can be measured by $K:={\displaystyle \sum _{i=1}^r}(n_{ai}+n_{bi})$, which denotes the number of non-zero elements in $\mathit{\theta }$. The identification objective is to estimate the unknown parameters $a_{ij}$, $b_{ij}$ as well as the time-delays $d_i$ from observations.

3. Identification Algorithm

If we have m observations from $t=1$ to $t=m$, Equation (4) can be written in a stacked form

$$\begin{array}{c}\hfill \mathit{Y}=\Phi \mathit{\theta }+\mathit{V},\end{array}$$

(8)

where

$$\begin{array}{ccc}\hfill \mathit{Y}& :=& {[y\left(1\right),y\left(2\right),\cdots ,y\left(m\right)]}^T\in {\mathbb{R}}^m,\hfill \end{array}$$

(9)

$$\begin{array}{ccc}\hfill \Phi & :=& {[\mathit{\phi }\left(1\right),\mathit{\phi }\left(2\right),\cdots ,\mathit{\phi }\left(m\right)]}^T\in {\mathbb{R}}^{m\times n},\hfill \\ \hfill \mathit{V}& :=& {[v\left(1\right),v\left(2\right),\cdots ,v\left(m\right)]}^T\in {\mathbb{R}}^m.\hfill \end{array}$$

(10)

From Equations (2), (5) and (10), we can see that the information matrix $\Phi$ contains many unknown intermediate terms. Therefore, it is difficult to perform the identification directly. According to the auxiliary model identification idea [24,27], the information matrix $\Phi$ can be replaced with its estimate

$$\begin{array}{c}\hfill {\widehat{\mathbf{\Phi }}}_k:={[{\widehat{\mathit{\phi }}}_k\left(1\right),{\widehat{\mathit{\phi }}}_k\left(2\right),\cdots ,{\widehat{\mathit{\phi }}}_k\left(m\right)]}^T,\end{array}$$

(11)

where

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\phi }}}_k\left(t\right)& :=& {[{\widehat{\mathit{\phi }}}_{1,k}^T\left(t\right),{\widehat{\mathit{\phi }}}_{2,k}^T\left(t\right),\cdots ,{\widehat{\mathit{\phi }}}_{r,k}^T\left(t\right)]}^T,\hfill \end{array}$$

(12)

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\phi }}}_{i,k}\left(t\right)& :=& {[-{\widehat{x}}_{i,k-1}(t-1),\cdots ,-{\widehat{x}}_{i,k-1}(t-n_{ai}),u_i(t-1),\cdots ,u_i(t-l)]}^T.\hfill \end{array}$$

(13)

Note that the unmeasurable terms $x_i(t-j)$ are replaced with their auxiliary model output estimates ${\widehat{x}}_i(t-j)$. Then, the parameter vector $\mathit{\theta }$ can be estimated by the auxiliary model based least squares iterative (AM-LSI) algorithm [28],

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\theta }}}_k& =& {\left[{\widehat{\mathbf{\Phi }}}_k^T{\widehat{\mathbf{\Phi }}}_k\right]}^{-1}{\widehat{\mathbf{\Phi }}}_k^T\mathit{Y},k=1,2,3,\cdots \hfill \end{array}$$

(14)

$$\begin{array}{ccc}\hfill {\widehat{x}}_{1,k}\left(t\right)& =& {\widehat{\mathit{\phi }}}_{1,k}^T\left(t\right){\widehat{\mathit{\theta }}}_k(1:n_{a1}+l),\hfill \end{array}$$

(15)

$$\begin{array}{ccc}\hfill {\widehat{x}}_{i,k}\left(t\right)& =& {\widehat{\mathit{\phi }}}_{i,k}^T\left(t\right){\widehat{\mathit{\theta }}}_k\left(1+\sum _{j=1}^{i-1}(n_{aj}+l):\sum _{j=1}^i(n_{aj}+l)\right),i=2,3,\cdots ,r,\hfill \end{array}$$

(16)

where ${\widehat{\mathit{\theta }}}_k$ denotes the parameter vector estimate at the kth iteration. According to the least squares (LS) theory, the AM-LSI algorithm is efficient if it is satisfied that $m\gg n$. However, from Equation (7), we can see that the dimension of the system in Equation (8) is high. Therefore, it would take a lot of time and efforts to obtain enough observations to meet the identification requirement. Moreover, Equation (14) shows that the AM-LSI algorithm requires computing the inverse matrix ${\left[{\widehat{\mathbf{\Phi }}}_k^T{\widehat{\mathbf{\Phi }}}_k\right]}^{-1}$ at each iteration, which leads to a heavy computational burden. Furthermore, the sparse solution cannot be obtained [23], and the time-delays cannot be effectively estimated. Thus, the AM-LSI algorithm is infeasible for the high dimensional and sparse system identification.

Inspired by the compressive sensing theory, the identification of the sparse system in Equation (8) can be further expressed as an optimization problem

$$\begin{array}{c}\hfill \widehat{\mathit{\theta }}={\arg min\parallel \mathit{\theta }\parallel }_0,\mathrm{s}.\mathrm{t}.{\parallel \mathit{Y}-\Phi \mathit{\theta }\parallel }_2^2\le \mathbf{\epsilon },\end{array}$$

where ${\parallel ·\parallel }_0$ represents the $l_0$ norm, ${\parallel ·\parallel }_2$ the $l_2$ norm, and $\epsilon$ the error tolerance, which is a priori chosen. However, it is a non-convex problem and is difficult to solve in practice. A commonly used alternative is to replace the $l_0$ norm with a relaxed convex $l_1$ norm [9],

$$\begin{array}{c}\hfill \widehat{\mathit{\theta }}={\arg min\parallel \mathit{\theta }\parallel }_1,\mathrm{s}.\mathrm{t}.{\parallel \mathit{Y}-\Phi \mathit{\theta }\parallel }_2^2\le \mathbf{\epsilon },\end{array}$$

(17)

where ${\parallel ·\parallel }_1$ is the $l_1$ norm. It is proved that the minimization of $l_1$ norm is equivalent to the minimization of $l_0$ norm when the Restricted Isometry Property (RIP) is satisfied [13]. Then, the convex optimization problem in Equation (17) is solvable with the BPDN criterion [16]. The BPDN approach can effectively reduce the interference of noise and has good robustness. Inspired by the auxiliary model and the iterative identification idea in the AM-LSI algorithm, we apply the BPDN criterion to modify the AM-LSI algorithm by replacing the LS step with the BPDN criterion. The parameters are estimated by the BPDN criterion and the unmeasurable terms are replaced by the outputs of the auxiliary model in each iteration.

Let $k=1,2,\cdots$ be the iterative number. To get accurate reconstruction from Equation (17), the information matrix should satisfy some conditions, such as the RIP [29] or the exact recovery condition (ERC) [9]. The ERC guarantee and the consistency properties for the identification of the controlled autoregressive models have been investigated in [12]. To meet the ERC, we normalize the information matrix ${\widehat{\mathbf{\Phi }}}_k$ defined in Equations (11)–(13) by dividing the elements in each column by the $l_2$ norm of that column [30,31]. Denote the element of the ith row and jth column in ${\widehat{\mathbf{\Phi }}}_k$ by ${\widehat{\Phi }}_{k,ij}$, and the normalized information matrix ${\widehat{\mathbf{\Phi }}}_{k,n}$ is constructed by

$$\begin{array}{c}\hfill {\widehat{\mathbf{\Phi }}}_k:={\widehat{\mathbf{\Phi }}}_{k,n}{\widehat{\mathbf{\Phi }}}_{l_2},{\widehat{\mathbf{\Phi }}}_{l_2}:=\left[\begin{array}{cccc}{\widehat{\Phi }}_k\left(1\right)& 0& \cdots & 0\\ 0& {\widehat{\Phi }}_k\left(2\right)& ⋮\\ ⋮& \ddots & 0\\ 0& \cdots & 0& {\widehat{\Phi }}_k\left(n\right)\end{array}\right]\in {\mathbb{R}}^{n\times n},\end{array}$$

(18)

where ${\widehat{\Phi }}_k\left(j\right):=\sqrt{{\sum }_{i=1}^m{\left({\widehat{\Phi }}_{k,ij}\right)}^2}$. Similarly, the normalized parameter vector ${\mathit{\theta }}_{k,n}$ can be defined as

$$\begin{array}{c}\hfill {\mathit{\theta }}_n:={\widehat{\mathbf{\Phi }}}_{l_2}\mathit{\theta }.\end{array}$$

(19)

Note that the location of non-zeros in ${\mathit{\theta }}_n$ is identical to that in $\mathit{\theta }$. Accordingly, the constrained optimization problem in Equation (17) equals

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\theta }}}_n& =& \arg min\parallel {\mathit{\theta }}_n{\parallel }_1,\mathrm{s}.\mathrm{t}.{\parallel \mathit{Y}-{\widehat{\mathbf{\Phi }}}_{k,n}{\mathit{\theta }}_n\parallel }_2^2\le \mathbf{\epsilon }.\hfill \end{array}$$

(20)

The problem in Equation (20) is closely related to the following unconstrained convex optimization problem

$$\begin{array}{c}\hfill {\widehat{\mathit{\theta }}}_n=\underset{{\mathit{\theta }}_n}{min}\frac{1}{2}\parallel \mathit{Y}-{\widehat{\mathbf{\Phi }}}_{k,n}{\mathit{\theta }}_n{\parallel }_2^2+\lambda {\parallel {\mathit{\theta }}_n\parallel }_1,\end{array}$$

(21)

where $\lambda$ is a nonnegative parameter. Since the information matrix ${\widehat{\mathbf{\Phi }}}_k$ is normalized, we can set $\lambda$ to the value $\lambda =\sigma \sqrt{2log\left(n\right)}$ [16].

The key step of the BPDN approach is to express Equation (21) as a quadratic program (QP) problem [32]. To begin with, two nonnegative vectors ${\mathit{u}}_n$ and ${\mathit{v}}_n$ are introduced to express ${\mathit{\theta }}_n$. Let ${\theta }_{nj}$, $u_{nj}$ and $v_{nj}$ be the jth element of the vectors ${\mathit{\theta }}_n$, ${\mathit{u}}_n$ and ${\mathit{v}}_n$, respectively, where $u_{nj}\ge 0$ and $v_{nj}\ge 0$ for all $j=1,2,\cdots ,n$. Define

$$\begin{array}{c}\hfill u_{nj}:={\left({\theta }_{nj}\right)}_{+},v_{nj}:={(-{\theta }_{nj})}_{+},{(\ast )}_{+}:=max\{0,\ast \}.\end{array}$$

Then, ${\mathit{\theta }}_n$ can be rewritten as

$$\begin{array}{c}\hfill {\mathit{\theta }}_n={\mathit{u}}_n-{\mathit{v}}_n.\end{array}$$

Accordingly, the $l_1$ regularization term $\parallel {\mathit{\theta }}_n{\parallel }_1$ can be expressed as

$$\begin{array}{ccc}\hfill \parallel {\mathit{\theta }}_n{\parallel }_1& =& {\mathbf{1}}_n^T{\mathit{u}}_n+{\mathbf{1}}_n^T{\mathit{v}}_n={\mathbf{1}}_{2n}^T{\mathbf{z}}_n,\hfill \end{array}$$

(22)

where ${\mathbf{1}}_n^T:=[1,\cdots ,1]\in {\mathbb{R}}^n$, ${\mathbf{1}}_{2n}^T:=[1,\cdots ,1]\in {\mathbb{R}}^{2n}$, and

$$\begin{array}{c}\hfill {\mathbf{z}}_n:={[{\mathit{u}}_n^T,{\mathit{v}}_n^T]}^T\in {\mathbb{R}}^{2n}.\end{array}$$

(23)

Note that all elements in ${\mathbf{z}}_n$ are nonnegative. Similarly, the quadratic error term can be written as

$$\begin{array}{cc}\hfill & \parallel \mathit{Y}-{\widehat{\mathbf{\Phi }}}_{k,n}{\mathit{\theta }}_n{\parallel }_2^2\hfill \\ =& \parallel \mathit{Y}-[{\widehat{\mathbf{\Phi }}}_{k,n},-{\widehat{\mathbf{\Phi }}}_{k,n}]{\mathbf{z}}_n{\parallel }_2^2\hfill \\ =& {\mathit{Y}}^T\mathit{Y}-{\mathit{Y}}^T[{\widehat{\mathbf{\Phi }}}_{k,n},-{\widehat{\mathbf{\Phi }}}_{k,n}]{\mathbf{z}}_n-{\mathbf{z}}_n^T{[{\widehat{\mathbf{\Phi }}}_{k,n},-{\widehat{\mathbf{\Phi }}}_{k,n}]}^T\mathit{Y}+{\mathbf{z}}_n^T{[{\widehat{\mathbf{\Phi }}}_{k,n},-{\widehat{\mathbf{\Phi }}}_{k,n}]}^T[{\widehat{\mathbf{\Phi }}}_{k,n},-{\widehat{\mathbf{\Phi }}}_{k,n}]{\mathbf{z}}_n.\hfill \end{array}$$

(24)

Since ${\mathit{Y}}^T[{\widehat{\mathbf{\Phi }}}_{k,n},-{\widehat{\mathbf{\Phi }}}_{k,n}]{\mathbf{z}}_n$ is a scalar, it follows that

$$\begin{array}{c}\hfill {\mathit{Y}}^T[{\widehat{\mathbf{\Phi }}}_{k,n},-{\widehat{\mathbf{\Phi }}}_{k,n}]{\mathbf{z}}_n={\left({\mathit{Y}}^T[{\widehat{\mathbf{\Phi }}}_{k,n},-{\widehat{\mathbf{\Phi }}}_{k,n}]{\mathbf{z}}_n\right)}^T={\mathbf{z}}_n^T{[{\widehat{\mathbf{\Phi }}}_{k,n},-{\widehat{\mathbf{\Phi }}}_{k,n}]}^T\mathit{Y}=[{\mathit{Y}}^T{\widehat{\mathbf{\Phi }}}_{k,n},-{\mathit{Y}}^T{\widehat{\mathbf{\Phi }}}_{k,n}]{\mathbf{z}}_n.\end{array}$$

Let

$$\begin{array}{c}\hfill b:={\widehat{\mathbf{\Phi }}}_{k,n}^T\mathit{Y}\in {\mathbb{R}}^n.\end{array}$$

(25)

Equation (24) can be further written as

$$\begin{array}{ccc}\hfill \parallel \mathit{Y}-{\widehat{\mathbf{\Phi }}}_{k,n}{\mathit{\theta }}_n{\parallel }_2^2& =& {\mathit{Y}}^T\mathit{Y}-2[b^T,-b^T]{\mathbf{z}}_n+{\mathbf{z}}_n^T{[{\widehat{\mathbf{\Phi }}}_{k,n},-{\widehat{\mathbf{\Phi }}}_{k,n}]}^T[{\widehat{\mathbf{\Phi }}}_{k,n},-{\widehat{\mathbf{\Phi }}}_{k,n}]{\mathbf{z}}_n\hfill \\ & =& {\mathit{Y}}^T\mathit{Y}-2[b^T,-b^T]{\mathbf{z}}_n+{\mathbf{z}}_n^T\left[\begin{array}{cc}{\widehat{\mathbf{\Phi }}}_{k,n}^T{\widehat{\mathbf{\Phi }}}_{k,n}& -{\widehat{\mathbf{\Phi }}}_{k,n}^T{\widehat{\mathbf{\Phi }}}_{k,n}\\ -{\widehat{\mathbf{\Phi }}}_{k,n}^T{\widehat{\mathbf{\Phi }}}_{k,n}& {\widehat{\mathbf{\Phi }}}_{k,n}^T{\widehat{\mathbf{\Phi }}}_{k,n}\end{array}\right]{\mathbf{z}}_n\hfill \\ \hfill & =& {\mathit{Y}}^T\mathit{Y}-2[b^T,-b^T]{\mathbf{z}}_n+{\mathbf{z}}_n^{T}b{\mathbf{z}}_n,\hfill \end{array}$$

(26)

where

$$\begin{array}{c}\hfill b:=\left[\begin{array}{cc}{\widehat{\mathbf{\Phi }}}_{k,n}^T{\widehat{\mathbf{\Phi }}}_{k,n}& -{\widehat{\mathbf{\Phi }}}_{k,n}^T{\widehat{\mathbf{\Phi }}}_{k,n}\\ -{\widehat{\mathbf{\Phi }}}_{k,n}^T{\widehat{\mathbf{\Phi }}}_{k,n}& {\widehat{\mathbf{\Phi }}}_{k,n}^T{\widehat{\mathbf{\Phi }}}_{k,n}\end{array}\right]\in {\mathbb{R}}^{2n\times 2n}.\end{array}$$

(27)

From Equations (22) and (26), we have

$$\begin{array}{cc}\hfill & \underset{{\mathit{\theta }}_n}{min}\frac{1}{2}\parallel \mathit{Y}-{\widehat{\mathbf{\Phi }}}_{k,n}{\mathit{\theta }}_n{\parallel }_2^2+\lambda {\parallel {\mathit{\theta }}_n\parallel }_1\hfill \\ =& \underset{{\mathbf{z}}_{\mathbf{n}}}{min}\frac{1}{2}{\mathit{Y}}^T\mathit{Y}-[b^T,-b^T]{\mathbf{z}}_n+\frac{1}{2}{\mathbf{z}}_n^{T}b{\mathbf{z}}_n+\lambda {\mathbf{1}}_{2n}^T{\mathbf{z}}_n\hfill \\ =& \underset{{\mathbf{z}}_{\mathbf{n}}}{min}\frac{1}{2}{\mathit{Y}}^T\mathit{Y}+{\mathit{C}}^T{\mathbf{z}}_n+\frac{1}{2}{\mathbf{z}}_n^{T}b{\mathbf{z}}_n,\hfill \end{array}$$

(28)

where

$$\begin{array}{c}\hfill \mathit{C}:=\lambda {\mathbf{1}}_{2n}+{[-b^T,b^T]}^T=\mathbf{\sigma }\sqrt{2log\left(n\right)}{\mathbf{1}}_{2n}+{[-b^T,b^T]}^T\in {\mathbb{R}}^{2n}.\end{array}$$

(29)

Since $\frac{1}{2}{\mathit{Y}}^T\mathit{Y}$ is a constant, Equation (28) can be constructed in a standard QP framework,

$$\begin{array}{c}\hfill \underset{{\mathbf{z}}_{\mathbf{n}}}{min}{\mathit{C}}^T{\mathbf{z}}_n+\frac{1}{2}{\mathbf{z}}_n^{T}b{\mathbf{z}}_n,\mathrm{s}.\mathrm{t}.z_{nj}\ge 0,j=1,2,\cdots ,n.\end{array}$$

(30)

For the inequality constrained QP problem in Equation (30), the common solution is the active set method [33]. While for the sake of simplicity, the QP problem can be directly solved by calling the relevant function from the standard scientific software toolbox. For example, the MATLAB toolbox provides a function “quadprog”. Then, the parameter vector ${\widehat{\mathit{\theta }}}_n$ can be obtained from the optimum solution ${\widehat{\mathbf{z}}}_n$,

$$\begin{array}{c}\hfill {\widehat{\mathit{\theta }}}_n={\widehat{\mathbf{z}}}_n(1:n)-{\widehat{\mathbf{z}}}_n(n+1:2n),\end{array}$$

(31)

where ${\widehat{\mathbf{z}}}_n(1:n)$ represents a vector formed by the first n elements of ${\widehat{\mathbf{z}}}_n$, and ${\widehat{\mathbf{z}}}_n(n+1:2n)$ a vector formed by the last n elements of ${\widehat{\mathbf{z}}}_n$. Considering that the system in Equation (8) is contaminated with noise, the parameter estimation error can be large. To further reduce the estimation error, a small threshold TH $=ϵ$ can be set to filter the elements close to zero in ${\widehat{\mathit{\theta }}}_n$. Let ${\widehat{\theta }}_{nj}=0$ if ${\widehat{\theta }}_{nj}<ϵ$ and denote the filtered parameter vector as ${\widehat{\mathit{\theta }}}_{n,ϵ}$. Then, the parameter vector estimate ${\widehat{\mathit{\theta }}}_k$ can be recovered according to Equation (19),

$$\begin{array}{c}\hfill {\widehat{\mathit{\theta }}}_k={\widehat{\mathbf{\Phi }}}_{l_2}^{-1}{\widehat{\mathit{\theta }}}_{n,ϵ}.\end{array}$$

(32)

The estimates of the intermediate variables $x_{i,k}\left(t\right)$ can be refreshed by ${\widehat{\mathit{\theta }}}_k$ as shown in Equations (15) and (16).

Equations (9), (11)–(13), (15), (16), (18), (23), (25), (27), and (29)–(32) form the auxiliary model-basis pursuit de-noising iterative (AM-BPDNI) algorithm for the MISO-OE system. The implementation procedures are listed as follows:

• Collect the input–output data {$u_i\left(t\right)$, $y\left(t\right)$: $i=1,2,\cdots ,r$; $t=1,2,\cdots ,m$} and set the parameter estimation accuracy ${\epsilon }_0$.
• Construct the stacked output vector Y by Equation (9).
• Initialize the iteration: let $k=1$ and ${\widehat{x}}_{i,0}\left(t\right)$ be random sequences.
• Construct the information matrix ${\widehat{\mathbf{\Phi }}}_k$ by Equations (11)–(13) and normalize ${\widehat{\mathbf{\Phi }}}_k$ by (18).
• Form the vectors and matrix ${\mathbf{z}}_n$, b, b and C by Equations (23), (25), (27), and (29) and formulate the QP by Equation (30).
• Call the function to obtain the optimum solution ${\widehat{\mathbf{z}}}_n$ and compute ${\widehat{\mathit{\theta }}}_n$ by Equation (31).
• Set a threshold to obtain ${\widehat{\mathit{\theta }}}_{n,ϵ}$ and recover the parameter vector estimate ${\widehat{\mathit{\theta }}}_k$ by (32).
• Compare ${\widehat{\mathit{\theta }}}_k$ with ${\widehat{\mathit{\theta }}}_{k-1}$: if $\parallel {\widehat{\mathit{\theta }}}_k-{\widehat{\mathit{\theta }}}_{k-1}\parallel >{\epsilon }_0$, update the auxiliary model outputs $x_{i,k}\left(t\right)$ by Equations (15) and (16) and go to Step 4. Otherwise, stop the iteration and obtain the parameter vector estimate $\widehat{\mathit{\theta }}$.

The unknown time-delay of each input channel can be estimated according to the location of zero-blocks and the number of zeros in $\widehat{\mathit{\theta }}$. It can be seen from Equations (3) and (6) that there are $2r$ zero-blocks in $\widehat{\mathit{\theta }}$. Denote the number of zeros in each zero-block by $z_i$ $(i=1,2,\cdots ,2r)$. Then, time-delays can be estimated by

$$\begin{array}{c}\hfill {\widehat{d}}_i=z_{2i-1},i=1,2,\cdots ,2r.\end{array}$$

(33)

4. Simulation Example

Example 1.

Consider the following MISO-OE system with time-delays,

$$\begin{array}{ccc}\hfill y\left(t\right)& =& \sum _{i=1}^3{z}^{-d_i}\frac{B_i\left(z\right)}{A_i\left(z\right)}{u}_i\left(t\right)+v\left(t\right),\hfill \\ \hfill A_1\left(z\right)& =& 1-0.1z^{-1}+0.7z^{-2},B_1\left(z\right)=1.5z^{-1}+0.9z^{-2},\hfill \\ \hfill A_2\left(z\right)& =& 1+0.3z^{-1}+0.5z^{-2},B_2\left(z\right)=0.2z^{-1}+1.8z^{-2},\hfill \\ \hfill A_3\left(z\right)& =& 1-0.2z^{-1}-0.4z^{-2},B_3\left(z\right)=-0.1z^{-1}+2z^{-2},\hfill \\ \hfill d_1& =& 20,d_2=10,d_3=30.\hfill \end{array}$$

(34)

The system in Equation (34) is a second order system with three inputs and one output. The inputs $\left\{u_i\left(t\right)\right\}$, $i=1,2,3$ are taken as random uncorrelated signal sequences with zero mean and unit variances, and $\left\{v\right(t\left)\right\}$ as a white noise sequence with zero mean and variances ${\sigma }^2$. Let the maximum input regression length be $l=50$. Then, the parameter vector to be identified is

$$\begin{array}{ccc}\hfill {\mathit{\theta }}_1& =& {[-0.1,0.7,{\mathbf{0}}_{20},1.5,0.9,{\mathbf{0}}_{28},0.3,0.5,{\mathbf{0}}_{10},0.2,1.8,{\mathbf{0}}_{38},-0.2,-0.4,{\mathbf{0}}_{30},-0.1,2,{\mathbf{0}}_{18}]}^T\in {\mathbb{R}}^{156},\hfill \end{array}$$

where ${\mathbf{0}}_i$ denotes the zero-block with i zeros. Note that the number of non-zero elements is $K={\parallel \mathit{\theta }\parallel }_0={\displaystyle \sum _{i=1}^3}(n_{ai}+n_{bi})=12$.

Taking $m=130$ and $\mathrm{TH}=0.001$, apply the AM-LSI algorithm and the AM-BPDNI algorithm to perform the identification, respectively. The parameter estimation errors $\delta :=\parallel \mathit{\theta }-\widehat{\mathit{\theta }}\parallel /\parallel \mathit{\theta }\parallel$ versus different noise levels are shown in Table 1. When ${\sigma }^2={0.10}^2$, the estimation errors $\delta$ versus the iterative number k are shown in Figure 1. It can be seen that the AM-BPDNI algorithm performs better than the AM-LSI algorithm and is insensitive to noise.

Table 1. The parameter estimation errors $\delta$ (%) versus the noise variances ${\sigma }^2$ ($m=130$).

$\mathit{\sigma }$	0.10	0.15	0.20	0.25	0.30	0.40
AM-LSI	53.4284	53.7175	54.2753	55.3337	55.8435	57.2966
AM-BPDNI	1.9974	2.9825	4.7985	8.6665	8.9512	9.8808

Figure 1. The parameter estimation error $\delta$ versus k ($m=130$, ${\sigma }^2={0.10}^2$).

Let the variance of $\left\{v\right(t\left)\right\}$ be ${\sigma }^2={0.10}^2$. Apply the AM-BPDNI algorithm to obtain the estimated model of the system in Equation (34) with the first $m=130$ data. Then, validate the estimated model by using $m_e=200$ samples from $t=131$ to 330. The predicted output of the estimated model, the true output of the system and their errors are plotted in Figure 2. It is shown that the predicted outputs $\widehat{y}\left(t\right)$ are close to the true outputs $y\left(t\right)$. Moreover, the average predicted output error

$$\begin{array}{c}\hfill {\delta }_v:=\sqrt{\frac{1}{m_e}\sum _{t=131}^{330}{[\widehat{y}\left(t\right)-y\left(t\right)]}^2}=0.1422\end{array}$$

is small and close to the standard deviation of the noise $\sigma =0.10$. It follows that the estimated model can well capture the system dynamics.

Figure 2. Predicted output $\widehat{y}\left(t\right)$, true output $y\left(t\right)$ and errors (from t =131 to 330).

Let $m=130$ and ${\sigma }^2={0.10}^2$. Using the AM-BPDNI algorithm to estimate the sparse parameter vector $\mathit{\theta }$, the non-zero parameter estimates versus k are shown in Table 2 and Figure 3.

Table 2. The non-zero parameter estimates and estimation error $\delta$ versus k ($m=130$, ${\sigma }^2={0.10}^2$).

k	${\mathit{a}}_{11}$	${\mathit{a}}_{12}$	${\mathit{b}}_{11}$	${\mathit{b}}_{12}$	${\mathit{a}}_{21}$	${\mathit{a}}_{22}$	${\mathit{b}}_{21}$	${\mathit{b}}_{22}$	${\mathit{a}}_{31}$	${\mathit{a}}_{32}$	${\mathit{b}}_{31}$	${\mathit{b}}_{32}$	$\mathit{\delta }$(%)
1	0.000	0.000	1.483	1.058	0.000	0.000	0.148	1.742	0.000	0.000	–0.076	1.963	69.2788
2	–0.077	0.685	1.491	0.908	0.296	0.477	0.170	1.771	–0.190	–0.375	–0.104	1.983	3.5707
5	–0.095	0.684	1.470	0.893	0.289	0.484	0.168	1.778	–0.190	–0.396	–0.078	1.960	2.1595
10	–0.100	0.694	1.474	0.888	0.290	0.482	0.170	1.778	–0.192	–0.396	–0.080	1.963	1.9974
True value	–0.100	0.700	1.500	0.900	0.300	0.500	0.200	1.800	–0.200	–0.400	–0.100	2.000

Figure 3. (a) The parameter estimates ${\widehat{a}}_{\mathit{ij}}$ versus k. (b) The parameter estimates ${\widehat{b}}_{\mathit{ij}}$ versus k. The non-zero parameter estimates versus k with m = 130, $\sigma$² = 0.10².

With 10 iterations, the estimated parameter vector is

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\theta }}}_1& =& [-0.1001,0.6938,{\mathbf{0}}_{20},1.4737,0.8882,{\mathbf{0}}_{28},0.2902,0.4818,{\mathbf{0}}_{10},0.1699,\hfill \\ & & 1.7783,{\mathbf{0}}_{38},-0.1917,-0.3961,{\mathbf{0}}_{30},-0.08002,1.9628,{\mathbf{0}}_{18}{]}^T.\hfill \end{array}$$

(35)

It can be seen from Equation (35) that there are six zero-blocks in $\widehat{\mathit{\theta }}$ and the number of zeros in each zero-block are $z_1=20$, $z_2=28$, $z_3=10$, $z_4=38$, $z_5=30$ and $z_6=18$. Then, the time-delay of each input channel can be estimated according to Equation (33),

$$\begin{array}{c}\hfill {\widehat{d}}_1=z_1=20,{\widehat{d}}_2=z_3=10,{\widehat{d}}_3=z_5=30.\end{array}$$

Obviously, the time-delay estimates are agreement with the true time-delays.

Example 2.

Consider the following MISO-OE system with time-delays,

$$\begin{array}{ccc}\hfill y\left(t\right)& =& \sum _{i=1}^4{z}^{-d_i}\frac{B_i\left(z\right)}{A_i\left(z\right)}{u}_i\left(t\right)+v\left(t\right),\hfill \\ \hfill A_1\left(z\right)& =& 1-0.1z^{-1}+0.7z^{-2},B_1\left(z\right)=1.5z^{-1}+0.9z^{-2},\hfill \\ \hfill A_2\left(z\right)& =& 1+0.3z^{-1}+0.5z^{-2},B_2\left(z\right)=0.2z^{-1}+1.8z^{-2},\hfill \\ \hfill A_3\left(z\right)& =& 1-0.2z^{-1}-0.4z^{-2},B_3\left(z\right)=-0.1z^{-1}+2z^{-2},\hfill \\ \hfill A_4\left(z\right)& =& 1-0.4z^{-1}-0.1z^{-2},B_4\left(z\right)=1.1z^{-1}+0.8z^{-2},\hfill \\ \hfill d_1& =& 20,d_2=10,d_3=30,d_4=40.\hfill \end{array}$$

(36)

Compared with the system in Equation (34), the system in Equation (36) has one more input. Thus, the number of parameters is increased. Let $l=50$ and the true parameter vector is

$$\begin{array}{ccc}\hfill {\mathit{\theta }}_2& =& [-0.1,0.7,{\mathbf{0}}_{20},1.5,0.9,{\mathbf{0}}_{28},0.3,0.5,{\mathbf{0}}_{10},0.2,1.8,{\mathbf{0}}_{38},-0.2,-0.4,{\mathbf{0}}_{30},-0.1,2,{\mathbf{0}}_{18},-0.4,\hfill \\ & \hfill & -0.1,{\mathbf{0}}_{40},1.1,0.8,{\mathbf{0}}_8{]}^T\in {\mathbb{R}}^{208}.\hfill \end{array}$$

Taking $m=130$, ${\sigma }^2=0.{10}^2$, and $\mathrm{TH}=0.001$, employ the AM-BPDNI algorithm to identify the system in Equation (36). The estimated parameter vector is

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\theta }}}_2& =& [-0.0874,0.6849,{\mathbf{0}}_{20},1.4682,0.9218,{\mathbf{0}}_{28},0.2880,0.4711,{\mathbf{0}}_{10},0.1658,1.7708,{\mathbf{0}}_{38},-0.1842,\hfill \\ & \hfill & -0.3968,{\mathbf{0}}_{30},-0.0759,1.9503,{\mathbf{0}}_{18},-0.4563,-0.0556,{\mathbf{0}}_{40},1.0429,0.7457,{\mathbf{0}}_8{]}^T.\hfill \end{array}$$

The time-delay estimates are

$$\begin{array}{c}\hfill {\widehat{d}}_1=z_1=20,{\widehat{d}}_2=z_3=10,{\widehat{d}}_3=z_5=30,{\widehat{d}}_4=z_7=40,\end{array}$$

which are identical to the true time-delays.

The parameter estimation errors of the systems in Equations (34) and (36) versus k are shown in Figure 4.

Figure 4. The parameter estimation errors $\delta$ of different systems versus k.

The running time of the proposed method for the systems is $t_1=1.374637$ s and $t_2=2.398602$ s. It can be concluded that the computational burden increases as the dimension of the parameter vector increases.

The simulation results show that for the MISO-OE model, the proposed AM-BPDNI algorithm can obtain efficient estimation of parameters from few observations ($m<n$) with good robustness. Moreover, the AM-BPDNI algorithm can effectively estimate the time-delays according to the sparse characteristic of the estimated parameter vector. However, as the number of the input channels increases, the computational burden of the proposed algorithm increases.

5. Conclusions

This paper proposes an AM-BPDNI algorithm for the identification of MISO-OE systems with unknown time-delays. Based on the BPDN criterion and the auxiliary model identification idea, the sparse parameters and multiple-input time-delays can be effectively and simultaneously estimated. The proposed algorithm requires few sampled data and is robust to noise.