Data Filtering Based Recursive and Iterative Least Squares Algorithms for Parameter Estimation of Multi-Input Output Systems

: This paper discusses the parameter estimation problems of multi-input output-error autoregressive (OEAR) systems. By combining the auxiliary model identiﬁcation idea and the data ﬁltering technique, a data ﬁltering based recursive generalized least squares (F-RGLS) identiﬁcation algorithm and a data ﬁltering based iterative least squares (F-LSI) identiﬁcation algorithm are derived. Compared with the F-RGLS algorithm, the proposed F-LSI algorithm is more effective and can generate more accurate parameter estimates. The simulation results conﬁrm this conclusion.


Introduction
System modeling and identification of single variable processes have been well studied.However, most industrial processes are multivariable systems [1][2][3], including multiple-input multiple-output (MIMO) systems and multiple-input single-output (MISO) systems.For example, in chemical and process industries, the heat exchangers are MIMO systems, in which the state of a heat exchanger often is represented by four field input variables: the cold inlet temperature, the hot inlet temperature, the cold mass flow and the hot mass flow; the outputs are respectively the cold outlet temperature and hot outlet temperature [4].In wireless communication systems, the MIMO technology can increase wireless channel capacity and bandwidth by using the multiple antennas without the need of additional power [5].In computing system technology, the power consumption model for host servers can be identified using a MISO model, in which the system inputs have different forms, such as the rate of the change in the CPU frequency and the rate of the change in the CPU time share, and the system outputs are the changes in power consumption [6].With the development of the industrial process, the identification of multivariable processes is in great demand.Researchers have studied the problem of identification for multichannel systems from different fields [7][8][9], and many methods have been proposed for multivariable cases [10,11].
Recursive algorithms and iterative algorithms have wide applications in system modeling and system identification [12][13][14].For example, Wang et al. derived the hierarchical least squares based iterative algorithm for the Box-Jenkins system [15]; and Dehghan and Hajarian presented the iterative method for solving systems of linear matrix equations over reflexive and anti-reflexive matrices [16].Compared with the recursive identification algorithm, the iterative identification algorithm uses all the measured data to refresh parameter estimation, so the parameter estimation accuracy can be greatly improved, and the iterative identification methods have been successfully applied to many different models [17][18][19].
In the field of system identification, the filtering technique is efficient to improve the computational efficiency [20][21][22], and it has been widely used in parameter estimation of different models [23,24].Particularly, Basin et al. discussed the parameter estimation for linear stochastic time-delay systems based state filtering [25]; Scarpiniti et al. discussed the identification of Wiener-type nonlinear systems using the adaptive filter [26]; and Wang et al. presented a gradient based iterative algorithm for identification of a class nonlinear systems by filtering the input-output data [27].
This paper combines the filtering technique with the auxiliary model identification idea to estimate parameters of multi-input output error autoregressive (OEAR) systems.By using a linear filter to filter the input-output data, a multi-input OEAR system is transformed into two identification models, and the dimensions of the covariance matrices of the decomposed two models become smaller than that of the original OEAR model.The contributions of this paper are as follows: • By using the data filtering technique and the auxiliary model identification idea, a data filtering based recursive generalized least squares (F-RGLS) identification algorithm is derived for the multi-input OEAR system.• A data filtering based iterative least squares (F-LSI) identification algorithm is developed for the multi-input OEAR system.• The proposed F-LSI identification algorithm updates the parameter estimation by using all of the available data, and can produce highly accurate parameter estimates compared to the F-RGLS identification algorithm.
The rest of this paper is organized as follows: Section 2 gives a description for multi-input OEAR systems.Section 3 gives an F-RGLS algorithm for the multi-input OEAR system by using the data filtering technique.Section 4 derives an F-LSI algorithm by using the data filtering technique and the iterative identification method.Two examples to illustrate the effectiveness of the proposed algorithms are given in Section 5. Finally, Section 6 gives some concluding remarks.

The System Description
Consider the following multi-input OEAR system: where ∈ R is random white noise with zero mean, and w(t) ∈ R is random colored noise.Assume that the orders n j and n c are known, y(t) = 0, u j (t) = 0 and v(t) = 0 for t ≤ 0. The parameters a ji , b ji and c i are to be identified from input-output data {u j (t), y(t), j = 1, 2, • • • , r}.Define the parameter vectors: and the information vectors as The information vector ϕ(t) is unknown due to the unmeasured variables x j (t − i) and w(t − i).By means of the above definitions, Equations ( 2)-( 4) can be expressed as Equation ( 7) is the identification model of the multi-input OEAR system, and the parameter vector θ contains all the parameters to be estimated.

The Data Filtering Based Recursive Least Squares Algorithm
Define a unit backward shift operator q −1 as q −1 u(t) := u(t − 1) and a rational function In this section, we use the linear filter C(q) to filter the input-output data and derive an F-RGLS algorithm.
For the multi-input OEAR system in Equations ( 1)-(4), we define the filtered input-output data u jf (t) and y f (t) as Multiplying both sides of Equations ( 1) and (4) by C(q), we can obtain the following filtered output-error model: Define the filtered information vectors φ f (t) and φ jf (t) as Equations ( 10) and ( 11) can be rewritten as Taking advantage of the idea in [28] for the filtered identification model in Equation ( 13), we can get Since the information vector φ f (t) contains the unknown variables u jf (t) and x jf (t), the algorithm in Equation ( 14) cannot be applied to estimate θ(t) directly.According to the idea in [29], the unknown variables x jf (t) are replaced with the outputs of relevant auxiliary model, and the unmeasurable terms u jf (t) and y f (t) are replaced with their estimates ûjf (t) and ŷf (t), respectively.The derivation process is as follows.
Let θ(t), ĉ(t) and θj (t) be the estimates of ϑ, c and ϑ j at time t, respectively.Use the estimates xj (t − i), xjf (t − i) and ûjf (t − i) to define the estimates of φ(t) and φ f (t) as where the estimates xj (t) and xjf (t) can be computed by Define the covariance matrix and the gain vector Equation ( 14) can be written as Applying the matrix inversion formula (A + BC) 15), we can obtain the following recursive least squares algorithm for estimating θ(t): Applying the least squares principle to the noise model in Equation ( 6), we can obtain the following algorithm to estimate the parameter vector c: The noise information vector ψ(t) involves the unknown term w(t − i).From Equations ( 6) and (7), once the estimate θ(t) is obtained, the estimate ŵ(t) can be computed by Replace the unmeasurable noise terms w(t − i) in ψ(t) with estimates ŵ(t − i) and define the estimate of ψ(t) as to form the estimate of C(q) as follows: The estimates of the filtered input u jf (t) and the filtered output y f (t) can be computed through Replacing φ f (t) and y f (t) in Equations ( 16)- (18) with their estimates φf (t) and ŷf (t), and replacing ψ(t) and w(t) in Equations ( 19)- (21) with their estimates ψ(t) and ŵ(t), we can summarize the data filtering based recursive generalized least squares (F-RGLS) algorithm for the multi-input OEAR systems: The F-RGLS estimation algorithm involves two steps: the parameter identification of the system model-see Equations ( 22)-( 29)-and the parameter identification of the noise model-see Equations ( 30)-(37).The F-RGLS algorithm can generate the parameter estimation of the multi-input OEAR system; however, the algorithm uses only the measured data {u j (i), y(i) : i = 0, 1, 2, • • • , t} up to time t, not including the data {u(i), y(i) Next, we will make full use of all the measured data to improve the parameter estimation accuracy by adopting the iterative identification approach.

The Data Filtering Based Iterative Least Squares Algorithm
Suppose that the data length L n 0 + n c .Based on the identification models in Equations ( 6) and ( 13), we define two quadratic criterion functions Minimizing the above two quadratic criterion functions, we can obtain the estimation algorithm of computing θ(t) and ĉ(t): Because the vectors φ f (t) and ψ(t) are unknown, the parameter estimates θ(t) and ĉ(t) cannot be computed directly.Here, we adopt the iterative estimation theory.Let k = 1, 2, 3, • • • be an iterative variable, θk and ĉk denote the iterative estimates of ϑ and c at iteration k.Let xj,k (t) and ŵk (t) be the estimates of x j (t) and w(t) at iteration k.Replacing φ j (t) and ϑ j in Equation ( 5) with their estimates φj,k (t) and θj,k at iteration k, φ(t) and ϑ in Equation ( 6) with their estimates φk (t) at iteration k and the θk−1 at iteration k − 1, the estimate xj,k (t) and ŵk (t) can be calculated by , the estimates φk (t), φj,k (t) and ψk (t) can be obtained by Using the parameter estimate ĉk = [ ĉ1,k , ĉ2,k , • • • , ĉn c ,k ] T to form the estimate of C(q) at iteration k: Filtering the input-output data u j (t) and y(t) by Ĉk (q), we can obtain the estimates of u jf (t) and y f (t) Replacing φ f (t) and y f (t) in Equation ( 41) with their estimates φf,k (t) and ŷf,k (t), and replacing ψ(t) and w(t) in Equation ( 42) with their estimates ψk (t) and ŵk (t), we can obtain the data filtering based iterative least squares (F-LSI) algorithm of estimating the parameter vectors ϑ and c: From Equations ( 43)-(54), we can summarize the F-LSI algorithm as follows: We list the steps for computing the estimates θk and ĉk as iteration k increases: 1. To initialize, let k = 1, xjf,0 (t) = 1/p 0 , ûjf,0 (t) = 1/p 0 , ŷf,0 (t) = 1/p 0 , xj,0 (t) = 1/p 0 , ŵ0 (t) = 1/p 0 , p 0 = 10 6 .2. Collect the input-output data {u 1 (t), u 2 (t),

Remark:
The computational complexity implies the computational amount of multiplications and adds in the algorithm, depending on the sizes and lengths.

Examples
Example 1: Consider the following multi-input OEAR system: The parameter vector to be estimated is The inputs {u 1 (t), u 2 (t)} are taken as two persistent excitation signal sequences with zero mean and unit variance, and {v(t)} as a white noise sequence with zero mean and variance σ 2 = 0.20 2 and σ 2 = 0.60 2 .
Applying the F-RGLS algorithm to estimate the parameters of this example system, the parameter estimates and their estimation errors δ := θ(t) − θ / θ are shown in Table 1.Applying the F-LSI algorithm to estimate the parameters of this example system, when the data length L = 2000 the parameter estimates and their estimation errors δ := θk − θ / θ are shown in Table 2 with different noise variances.When the data length L = 4000, the parameter estimates and their errors δ are shown in Table 3 with different noise variances.Under different noise variances and different data lengths, the parameter estimates and their errors δ are shown in Table 4 when the iteration k = 10.Under different noise variances, the parameter estimation errors δ versus k are shown in Figure 1.From Tables 1-4 and Figure 1, we can draw the following conclusions: • Increasing the data length L can improve the parameter estimation accuracy of the F-RGLS algorithm and the F-LSI algorithm, and as the data length L increases, the parameter estimates are getting more stationary.• Under the same data length, the estimation accuracy of the F-RGLS algorithm and the F-LSI algorithm increases as the noise variance decreases.• Under the same data length and noise variance, the estimation errors of the F-LSI algorithm are smaller than the F-RGLS algorithm.• The F-LSI algorithm has fast convergence speed, and the parameter estimates only need several iterations close to their true values.
Example 2: Consider the industrial process with colored noise, which has two inputs and one output as shown in Figure 2 and described as The simulation conditions are the same as those of Example 1, and the noise variance σ 2 = 0.30 2 .Applying the F-RGLS and the F-LSI algorithms to estimate the parameters of the system, the parameter estimates and their errors are presented in Tables 5 and 6.
From Tables 5 and 6, we can see that the estimation errors become smaller with the increase of t and the F-LSI algorithm can get accurate parameter estimates by only several iterations, which shows the effectiveness of the proposed algorithms.
The power consumption in host servers can be concerned by the model of Example 2. The two inputs are the changes in CPU frequency of the host server and the changes in the guest server's time share to use the physical CPU of the host server, and the power consumption is the system output.The configuration and the allocation of memory, storage and network bandwidth for the guest server are the random disturbances of the system.True values 0.35000 0.30000 0.50000 0.84000 −0.25000 0.30000 0.40000 0.75000 −0.25000

Conclusions
This paper discusses the parameter estimation problem for multi-input OEAR systems.Based on the data filtering technique, an F-RGLS algorithm and an F-LSI algorithm are developed.The proposed methods are effective for estimating the parameters of multi-input OEAR systems.The simulation results indicate that the proposed F-LSI algorithm achieves higher estimation accuracies than the F-RGLS algorithm, and the convergence rate of the proposed methods can be improved by increasing the data length.The methods used in this paper can be extended to study the identification of other linear systems, nonlinear systems, state space systems and time delay systems.

Figure 2 .
Figure 2. The diagram of a multi-input OEAR system.

Table 1 .
The F-RGLS estimates and their errors for Example 1.

Table 2 .
The F-LSI parameter estimates and errors for Example 1 (L = 2000).

Table 3 .
The F-LSI parameter estimates and errors for Example 1 (L = 4000).

Table 4 .
The F-LSI parameter estimates and errors for Example 1 (k = 10).

Table 6 .
The F-LSI parameter estimates and errors for Example 2.