Symmetrical Augmented System of Equations for the Parameter Identification of Discrete Fractional Systems by Generalized Total Least Squares

Abstract

This paper is devoted to the identification of the parameters of discrete fractional systems with errors in variables. Estimates of the parameters of such systems can be obtained using generalized total least squares (GTLS). A GTLS problem can be reduced to a total least squares (TLS) problem. A total least squares problem is often ill-conditioned. To solve a TLS problem, a classical algorithm based on finding the right singular vector or an algorithm based on an augmented system of equations with complex coefficients can be applied. In this paper, a new augmented system of equations with real coefficients is proposed to solve TLS problems. A symmetrical augmented system of equations was applied to the parameter identification of discrete fractional systems. The simulation results showed that the use of the proposed symmetrical augmented system of equations can shorten the time for solving such problems. It was also shown that the proposed system can have a smaller condition number.

1. Introduction

Equations with fractional-order derivatives and differences are widely used to describe various processes and phenomena. Models based on equations with fractional-order derivatives and differences find wide application in hydrology, economics, and forecasts of network data usage [1,2,3].

Most of the research in this field deals with the parameter identification of fractional-order differential equations using the error in an equation or an output error. References [4,5] described time domain identification methods based on the least squares technique. References [6,7,8] presented an overview of different methods for identifying systems with fractional-order derivatives. The use of total least squares for estimating the parameters of fractional-order differential equations was proposed in [9,10].

Autoregressions with fractional differences are widely used in the analysis of time series with long memory [11,12]. There are a large number of different models with generalizations of fractional differences, such as Gegenbauer autoregressive moving average (GARMA) [13,14], fractional ARUMA [15], seasonal autoregressive fractionally integrated moving average (SARFIMA) [16,17], and autoregressive tempered fractionally integrated moving average (ARTFIMA) [18,19]. Various aspects of using fractional differences for time series analysis have been considered [20,21].

In [22], a discrete Kalman filter with fractional differences was proposed. Equations with fractional-order differences were also used to approximate differential equations with Grunwald–Letnikov derivatives [23]. Meanwhile, the modeling of various physical processes based on equations with fractional-order differences was considered in [24,25,26].

All articles cited assumed that the input signal is observed without noise. The problem of identifying integer or fractional-order systems with errors in variables is much more difficult. Reference [27] provided an overview of the contemporary state of this problem.

A small number of articles are devoted to the parameter identification of fractional-order systems with errors-in-variables. The known methods are based on cumulant least squares [28,29], cumulant instrumental variables [30], bias-compensated least squares [31], generalized total least squares [32,33,34], and generalized instrumental variables [35].

The problem of estimating the parameters of fractional systems with error in variables can be solved based on generalized total least squares [32]. A least squares solution is possible based on the SVD approach or based on the secular equation approach. The secular equation approach is more preferable than the SVD approach if [36]:

• The problem is sensitive: The minimum singularities have a small gap;
• The problem is large: The SVD of a large matrix is very expensive. Instead, we can approximate the secular equation in the large-scale problem by Gaussian quadrature rules.

When applying the secular equation approach, it is necessary to solve a biased normal system. A biased normal system has more conditionality than a solution based on the SVD approach.

The problem of estimating the parameters of fractional systems is often ill-conditioned. To reduce the condition number, augmented systems of equations can be used [37,38].

For TLS, [39] proposed an augmented system of equations with complex coefficients equivalent to a biased normal system.

In article [34], an algorithm was proposed for the parameter identification of discrete fractional systems based on an augmented system of equations.

This article proposes a symmetrical augmented system of equations with real coefficients for the parameter identification of discrete fractional systems by generalized total least squares. The proposed symmetrical augmented system of equations has a smaller dimension than the augmented system [39], as well as real coefficients.

2. Materials and Methods

2.1. Problem Statement

The dynamical system is described by linear stochastic equations with fractional differences:

$$z_i={\displaystyle \sum _{m=1}^r{b}_{}^{(m)}{\Delta }^{{\alpha }_m}}{z}_{i-1}+{\displaystyle \sum _{m=1}^{r_1}{a}_{}^{(m)}}{\Delta }^{{\beta }_m^{}}{x}_i^{},y_i=z_i+{\xi }_i,w_i^{}=x_i^{}+{\zeta }_i^{},$$

(1)

where $b_{}^{(m)}$ and $a_{}^{(m)}$ are constant coefficients; $0<{\alpha }_1\dots <{\alpha }_r$, $0<{\beta }_1^{}\dots <{\beta }_{r_1}^{}$, $\Gamma (\alpha )={\displaystyle \underset{0}{\overset{\infty }{\int }}{e}^{-t}{t}^{\alpha -1}dt}$, ${\Delta }^{{\alpha }_m}{z}_i={\displaystyle \sum _{j=0}^i{(-1)}^j\left(\begin{array}{c}{\alpha }_m\\ j\end{array}\right)z_{i-j}},$ and ${\Delta }^{{\beta }_m^{}}{x}_i^{}={\displaystyle \sum _{j=0}^i{(-1)}^j\left(\begin{array}{c}{\beta }_m^{}\\ j\end{array}\right)x_{i-j}^{}}$ are fractional differences [1];

$\left(\begin{array}{c}{\alpha }_m\\ j\end{array}\right)=\frac{\Gamma ({\alpha }_m+1)}{\Gamma (j+1)\Gamma ({\alpha }_m-j+1)},$ and $\left(\begin{array}{c}{\beta }_m^{}\\ j\end{array}\right)=\frac{\Gamma ({\beta }_m^{}+1)}{\Gamma (j+1)\Gamma ({\beta }_m^{}-j+1)}$ are generalized binomial coefficients.

The following assumptions are introduced:

A1. The dynamic system (1) is asymptotically stable. Results on the asymptotic stability of discrete time fractional difference systems can be found in [40,41].

A2. The noise sequences $\left\{{\xi }_i\right\}$, $\left\{{\zeta }_i\right\}$ are independent sequences with ${\rm E}\left({\xi }_i\right)=0$, ${\rm E}\left({\zeta }_i\right)=0$, ${\rm E}\left({\xi }_i^2\right)={\sigma }_{\xi }^2$, and ${\rm E}\left({\zeta }_i^2\right)={\sigma }_{\zeta }^2$, where ${\rm E}$ is the expectation operator.

A3. The input and output sequences $\{x_i\},\{z_i\}$ are independent of noise sequences $\{{\xi }_i\},\{{\zeta }_i\}$. The noise sequences $\{{\xi }_i\},\{{\zeta }_i\}$ are mutually independent.

A4. The input signal $\left\{x_i^{}\right\}$ persistently excites the order $r_1$ [42].

A5. The noise variance ratio $\gamma ={\sigma }_{\zeta }^2/{\sigma }_{\xi }^2$ is known.

It is required to estimate the unknown coefficients of the dynamic system described by Equation (1) from the observed sequences $\left\{y_i\right\},\left\{w_i\right\}$ with noise for the known orders $r,$ $r_1,$ ${\alpha }_m,$ and ${\beta }_m^{}$.

If $r,$ $r_1,$ ${\alpha }_m,$ and ${\beta }_m^{}$ are unknown, it is necessary to apply algorithms based on global optimization, such as genetic algorithms [43].

2.2. The Objective Function for Parameter Estimation

In [13], the following objective function was proposed for estimating the parameters:

$$\underset{\theta }{\min }{\displaystyle \sum _{i=1}^N\frac{{\left(y_i^{}-{\phi }_i^T\theta \right)}^2}{1+b_{}^T{H}_{\xi }b+\gamma a_{}^T{H}_{\zeta }^{}a}},$$

(2)

where:

$$H_{\xi }^{(mk)}=\underset{N\to \infty }{\lim }\frac{1}{N}\left({\displaystyle \sum _{j=0}^{N-1}\left(\begin{array}{c}{\alpha }_m\\ j\end{array}\right)}\left(\begin{array}{c}{\alpha }_k\\ j\end{array}\right)\cdot \frac{N-j}{N}\right),m=\overline{1,r_{}},k=\overline{1,r},$$

$$H_{\zeta }^{(mk)}=\underset{N\to \infty }{\lim }\frac{1}{N}\left({\displaystyle \sum _{j=0}^{N-1}\left(\begin{array}{c}{\beta }_m\\ j\end{array}\right)}\left(\begin{array}{c}{\beta }_k\\ j\end{array}\right)\cdot \frac{N-j}{N}\right),m=\overline{1,r_1},k=\overline{1,r_1},$$

$$\theta ={\left(\begin{array}{cc}{b}_{}^{(1)},\dots ,b_{}^{(r)}& a_{}^{(1)},\dots ,a_{}^{(r_1)}\end{array}\right)}^T\in {ℝ}^{r+r_1},{\phi }_i^{}={\left(\begin{array}{cc}{\Delta }^{{\alpha }_1}{y}_{i-j-1},\dots ,{\Delta }^{{\alpha }_r}{y}_{i-j-1}& {\Delta }^{{\beta }_1}{w}_{i-j},\dots ,{\Delta }^{{\beta }_{r_1}}{w}_{i-j}\end{array}\right)}^T\in {ℝ}^{r+r_1}.$$

Theorem 1.

Let the dynamic system described by Equation (1) with initial zero conditions and Assumptions A1–A5 be introduced. Then, the estimate of the coefficients determined by expression (2) exists, is unique, and converges to the true value of the coefficients with probability 1, i.e.,

$$\widehat{\theta }\underset{a.s.}{\to }\theta .$$

The proof of the theorem is similar to the proof given in [33].

The objective function (2) is a generalized total least squares objective function. In [32], an iterative procedure based on the minimization of the generalized Rayleigh ratio was proposed. The simulation results in [34] showed that for ill-conditioned systems, the algorithm has a low accuracy of parameter estimation.

We transformed the objective function (2) into the following form:

$$\underset{\theta }{\min }\frac{{\Vert d-C\theta \Vert }_2^2}{1+{\theta }_{}^T{H^{\prime }}_{\xi \zeta }\theta },$$

(3)

$$d={\left(y_1\dots y_N\right)}^T\in {ℝ}^N,H_{\xi \zeta }=\left(\begin{array}{cc}{H}_{\xi }& 0\\ 0& \gamma H_{\zeta }\end{array}\right)\in {ℝ}^{(r+r_1)\times (r+r_1)},C={\left(\begin{array}{ccc}{\phi }_1^T& \dots & {\phi }_N^T\end{array}\right)}^T\in {ℝ}^{(r+r_1)\times N}.$$

Under Assumptions A1–A5, the matrix $H_{\xi \zeta }$ is nonsingular. The generalized total least squares problem (3) can be reduced to the total least squares problem [44].

$$\vartheta =U_{\xi \zeta }\theta ,H_{\xi \zeta }=U_{\xi \zeta }^T{U}_{\xi \zeta }.$$

(4)

Then objective function (3) can be defined as:

$$\underset{\vartheta }{\min }\frac{{\Vert d-C{U}_{\xi \zeta }^{-1}\vartheta \Vert }_2^2}{1+{\vartheta }^T\vartheta }$$

(5)

The generalized total least squares algorithm for a nonsingular matrix $H_{\xi \zeta }$ has this form (Algorithm 1).

Algorithm 1. Ref. [34].

Step 1. Change variables ${H^{\prime }}_{\xi \zeta }=U_{\xi \zeta }^T{U}_{\xi \zeta }$ and $\vartheta =U_{\xi \zeta }\theta$

Step 2. Calculate the estimate of the vector of parameter $\widehat{\vartheta }$ by the objective function (5).

Step 3. Perform inverse change of variable $\widehat{\theta }=U_{\xi \zeta }^{-1}\widehat{\vartheta }.$

2.3. Numerical Method for the Total Least Squares Problem

There are many algorithms for solving total least squares problems. The classical algorithm for solving the total least squares problem is based on SVD (singular value decomposition) [45]. The solution to a total least squares problem based on augmented systems of the equation was considered in [40]. To solve a large dimension or with a sparse matrix, iterative methods for solving total least squares problems are used, such as Newton’s method [46,47], Rayleigh iterations [48], and Lanschoz iterations [49].

The problem of finding the singular vector is a nonlinear vector problem. Solving this problem numerically involves significant difficulties [50] related to convergence issues, high computational complexity, and the stability of search algorithms. Another approach is based on solving a biased normal system. Reference [51] showed that given the satisfaction of the condition:

$$\sigma ={\sigma }_{\min }\left(\begin{array}{cc}C{U}_{\xi \zeta }^{-1}& d\end{array}\right)<{\sigma }_{\min }\left(C{U}_{\xi \zeta }^{-1}\right),$$

where ${\sigma }_{\min }\left(\begin{array}{cc}C{U}_{\xi \zeta }^{-1}& d\end{array}\right)$ and ${\sigma }_{\min }\left(C{U}_{\xi \zeta }^{-1}\right)$ are minimal singular values of matrices $\left(\begin{array}{cc}C{U}_{\xi \zeta }^{-1}& d\end{array}\right)$ and $\left(C{U}_{\xi \zeta }^{-1}\right)$.

The minimum of function (5) can be found as a solution to the biased normal system of equations

$$\left({\left(C{U}_{\xi \zeta }^{-1}\right)}^{T}C{U}_{\xi \zeta }^{-1}-{\sigma }_{}^{2}I\right)\upsilon ={\left(C{U}_{\xi \zeta }^{-1}\right)}^{T}d.$$

(6)

When system (6) is solved, only the scalar problem of finding the minimal singular value $\sigma$ remains nonlinear. This problem is always well-conditioned [48]. The vector problem is linear, but the system (6) is often ill-conditioned. The spectral condition number of the biased normal matrix is obtainable from the expression:

$$\begin{array}{ll}{\kappa }_2\left({\left(C{U}_{\xi \zeta }^{-1}\right)}^{T}C{U}_{\xi \zeta }^{-1}-{\sigma }^{2}I\right)\triangleq & {\Vert {\left(C{U}_{\xi \zeta }^{-1}\right)}^{T}C{U}_{\xi \zeta }^{-1}-{\sigma }^{2}I\Vert }_2{\Vert {\left({\left(C{U}_{\xi \zeta }^{-1}\right)}^{T}C{U}_{\xi \zeta }^{-1}-{\sigma }^{2}I\right)}^{-1}\Vert }_2,\\ & {\kappa }_2\left({\left(C{U}_{\xi \zeta }^{-1}\right)}^{T}C{U}_{\xi \zeta }^{-1}-{\sigma }^{2}I\right)=\frac{{\sigma }_{\max }^2\left(C{U}_{\xi \zeta }^{-1}\right)-{\sigma }^{2}I}{{\sigma }_{\min }^2\left(C{U}_{\xi \zeta }^{-1}\right)-{\sigma }^{2}I}.\end{array}$$

The use of Cholesky decomposition can make the solution more stable. As this method applies to systems of linear equations with symmetrical, positively definite matrices, it can also be used for the biased normal system (6). However, Cholesky decomposition for ill-conditioned matrices leads to large errors.

2.3.1. Augmented System of Equations for Total Least Squares

In [39], an approach was proposed for solving normal systems of equations based on equivalent augmented systems of equations. Normal systems of equations corresponding to ordinary least squares:

$${\left(C{U}_{\xi \zeta }^{-1}\right)}^{T}C{U}_{\xi \zeta }^{-1}\upsilon ={\left(C{U}_{\xi \zeta }^{-1}\right)}^{T}d,$$

Can be represented as an equivalent augmented system of equations:

$$\left(\begin{array}{cc}{k}_0{I}_{}& C{U}_{\xi \zeta }^{-1}\\ {\left(C{U}_{\xi \zeta }^{-1}\right)}^T& 0\end{array}\right)\left(\begin{array}{c}{k}_0^{-1}\epsilon \\ \upsilon \end{array}\right)=\left(\begin{array}{c}d\\ 0\end{array}\right),$$

(7)

where $k_0=\frac{{\sigma }_{\min }\left(C{U}_{\xi \zeta }^{-1}\right)}{\sqrt{2}},$ $\epsilon =\left(d-C{U}_{\xi \zeta }^{-1}\upsilon \right).$

The number of conditions (7) is defined as:

$${\kappa }_2\left(A_{OLS}\right)=0.5+\sqrt{0.25+2{\kappa }_2\left({\left(C{U}_{\xi \zeta }^{-1}\right)}^{T}C{U}_{\xi \zeta }^{-1}\right)},$$

where $A_{OLS}=\left(\begin{array}{cc}{k}_0{I}_N& C{U}_{\xi \zeta }^{-1}\\ {\left(C{U}_{\xi \zeta }^{-1}\right)}^T& 0_{r+r_1}\end{array}\right)\in {ℝ}^{(N+r+r_1)\times (N+r+r_1)}.$

In [38], a generalization of the augmented system (7) to the case of the biased normal system (6) is proposed. It was proposed to use a complex matrix $G$ and vector $g$ defined as:

$$G=\left(\begin{array}{c}C{U}_{\xi \zeta }^{-1}\\ j\sigma I\end{array}\right)\in {ℂ}^{(N+r+r_1)\times (N+r+r_1)},g=\left(\begin{array}{c}d\\ 0\end{array}\right)\in {ℝ}^{N+r+r_1},$$

(8)

Using notation (8), the biased normal system (6) can be defined as:

$$\left(\begin{array}{cc}k{I}_{}& G\\ G^T& 0\end{array}\right)\left(\begin{array}{c}{k}_{}^{-1}\overline{\epsilon }\\ \upsilon \end{array}\right)=\left(\begin{array}{c}g\\ 0\end{array}\right),$$

(9)

Or:

$$A\left(k\right)\overline{\upsilon }=\overline{d},$$

where $k$ is an arbitrary positive variable factor:

$$\begin{array}{l}A\left(k\right)=\left(\begin{array}{ccc}k{I}_N& 0_{N\times (r+r_1)}& C{U}_{\xi \zeta }^{-1}\\ 0_{(r+r_1)\times N}& k{I}_{r+r_1}& j\sigma I_{{}_{r+r_1}}\\ {\left(C{U}_{\xi \zeta }^{-1}\right)}^T& j\sigma I_{{}_{r+r_1}}& 0_{r+r_1}\end{array}\right)\in {ℂ}^{(N+2(r+r_1))\times (N+2(r+r_1))},\\ \overline{\upsilon }=\left(\begin{array}{c}{k}^{-1}\epsilon \\ k^{-1}{\epsilon }_{\xi \zeta }\\ \widehat{\upsilon }\end{array}\right)\in {ℂ}^{N+2(r+r_1)},\overline{d}=\left(\begin{array}{c}d\\ 0\\ 0\end{array}\right)\in {ℝ}^{N+2(r+r_1)}\end{array}$$

The number of conditions (9) is defined as [17]:

$${\kappa }_2\left(A\left(k\right)\right)=\frac{\sqrt{1+k^2{\mu }_{\max }+k^2{\sigma }_{}^2}}{\sqrt{1+k^2{\mu }_{\min }+k^2{\sigma }_{}^2}}\left|\frac{\cos \left(\frac{1}{3}\arccos \left(\frac{3\sqrt{3}}{2}\frac{k^2\left({\mu }_{\max }-{\sigma }_{}^2\right)}{{\left(1+k^2{\mu }_{\max }+k^2{\sigma }_{}^2\right)}^{3/2}}\right)\right)}{\cos \left(\frac{\pi }{3}+\frac{1}{3}\arccos \left(\frac{3\sqrt{3}}{2}\frac{k^2\left({\mu }_{\min }-{\sigma }_{}^2\right)}{{\left(1+k^2{\mu }_{\min }+k^2{\sigma }_{}^2\right)}^{3/2}}\right)\right)}\right|,$$

(10)

where ${\mu }_{\max }$ and ${\mu }_{\min }$ are the maximal and minimal eigenvalues of a matrix $\left(C{U}_{\xi \zeta }^{-1}\right){\left(C{U}_{\xi \zeta }^{-1}\right)}^T$, respectively.

The problem of finding the minimum value of the condition number can be considered as a problem of choosing the optimal factor $k$:

$$\underset{k>0}{\inf }{\kappa }_2\left(A\left(k\right)\right).$$

(11)

Problem (9) has no analytical solution but can be solved by numerical methods. In [38], an estimate was proposed for factor ${\widehat{k}}_{opt}$:

$${\widehat{k}}_{opt}=\frac{{\sigma }_{\max }\left(C{U}_{\xi \zeta }^{-1}\right)+\sigma }{{\sigma }_{\min }\left(C{U}_{\xi \zeta }^{-1}\right)+\sigma }\sqrt{\frac{2}{{\sigma }_{\max }^2\left(C{U}_{\xi \zeta }^{-1}\right)+{\sigma }_{}^2}}.$$

(12)

Equation (7) can be solved by standard methods for solving systems of equations, such as using LU decomposition [52].

Generalized total least squares with the augmented system of Equation (9) in the second step algorithm were proposed for estimating the parameters of the system (1) [15].

The simulation results in [15,39] showed that the use of the augmented system of equations (7) is superior in accuracy to methods based on minimization of the generalized Rayleigh ratio or Cholesky decomposition for the biased normal system of equations.

The limitations of the augmented system (9) include the following:

• The augmented system of equations has complex coefficients, even if the original system has real coefficients.
• The matrix $A\left(k\right)$ is non-Hermitian. It is impossible to apply LDL^T decomposition for the solution of the system of Equation (9) [52].
• The matrix $A\left(k\right)$ has the dimensions $2\left(r+r_1\right)+N$.
• It is necessary to determine the optimal factor $k_{opt}$.

2.3.2. Symmetry Augmented System of Equations for Total Least Squares

Let us obtain an alternative augmented system of equations free of these limitations.

We transformed the system of Equation (6):

$$\begin{array}{c}{\left(C{U}_{\xi \zeta }^{-1}\right)}^{T}C{U}_{\xi \zeta }^{-1}\upsilon -{\sigma }_{}^2\upsilon ={\left(C{U}_{\xi \zeta }^{-1}\right)}^{T}d\Rightarrow {\left(C{U}_{\xi \zeta }^{-1}\right)}^T\left(d-C{U}_{\xi \zeta }^{-1}\upsilon \right)+{\sigma }_{}^2\upsilon =0\Rightarrow \\ \Rightarrow {\left(C{U}_{\xi \zeta }^{-1}\right)}^T\left(d-C{U}_{\xi \zeta }^{-1}\upsilon \right)+{\sigma }_{}^2\upsilon =0\Rightarrow {\left(C{U}_{\xi \zeta }^{-1}\right)}^T\epsilon +{\sigma }_{}^2\upsilon =0\Rightarrow {\sigma }_{}^{-1}{\left(C{U}_{\xi \zeta }^{-1}\right)}^T\epsilon +\sigma \upsilon =0.\end{array}$$

(13)

We combined Equation (13) with the equation $\epsilon =\left(d-C{U}_{\xi \zeta }^{-1}\upsilon \right)$:

$$\{\begin{array}{c}\epsilon +C{U}_{\xi \zeta }^{-1}\upsilon =d,\\ {\sigma }_{}^{-1}{\left(C{U}_{\xi \zeta }^{-1}\right)}^T\epsilon +\sigma \upsilon =0.\end{array}$$

(14)

Let us represent this in matrix form:

$$\tilde{A}\tilde{\upsilon }=\tilde{d},$$

(15)

Or:

$$\left(\begin{array}{cc}\sigma I& C{U}_{\xi \zeta }^{-1}\\ {\left(C{U}_{\xi \zeta }^{-1}\right)}^T& \sigma I\end{array}\right)\left(\begin{array}{c}{\sigma }^{-1}\epsilon \\ \upsilon \end{array}\right)=\left(\begin{array}{c}d\\ 0\end{array}\right).$$

(16)

The resulting augmented system has a real symmetric matrix. The dimension of the matrix $\tilde{A}$ is $r+r_1+N$.

Theorem 2.

The spectral condition number of the system of Equation (15) is defined as:

$${\kappa }_2\left(\tilde{A}\right)=\frac{{\sigma }_{\max }^{}+\sigma }{{\sigma }_{\min }^{}-\sigma }.$$

(17)

Proof of Theorem 2.

We introduced a new variable, $\rho ={\sigma }^{-1}\epsilon$, and considered the eigenvalue problem: □

$$\left(\begin{array}{cc}\sigma I& C{U}_{\xi \zeta }^{-1}\\ {\left(C{U}_{\xi \zeta }^{-1}\right)}^T& \sigma I\end{array}\right)\left(\begin{array}{c}\rho \\ \upsilon \end{array}\right)=\lambda \left(\begin{array}{c}\rho \\ \upsilon \end{array}\right)\iff \{\begin{array}{c}\sigma \rho +C{U}_{\xi \zeta }^{-1}\upsilon =\lambda \rho ,\\ {\left(C{U}_{\xi \zeta }^{-1}\right)}^T\rho +\sigma \upsilon =\lambda \upsilon .\end{array}$$

(18)

We transformed the system of Equation (17):

$$\begin{array}{c}\{\begin{array}{c}\rho ={\left(\lambda -\sigma \right)}^{-1}C{U}_{\xi \zeta }^{-1}\upsilon ,\\ {\left(\lambda -\sigma \right)}^{-1}{\left(C{U}_{\xi \zeta }^{-1}\right)}^{T}C{U}_{\xi \zeta }^{-1}\upsilon +\sigma \upsilon =\lambda \upsilon .\end{array}\\ {\left(C{U}_{\xi \zeta }^{-1}\right)}^{T}C{U}_{\xi \zeta }^{-1}\upsilon ={\left(\lambda -\sigma \right)}^2\upsilon \Rightarrow {\left(C{U}_{\xi \zeta }^{-1}\right)}^{T}C{U}_{\xi \zeta }^{-1}\upsilon ={\left({\lambda }^2-2\lambda \sigma +{\sigma }^2\right)}^2\upsilon \Rightarrow \\ \Rightarrow {\left(C{U}_{\xi \zeta }^{-1}\right)}^{T}C{U}_{\xi \zeta }^{-1}\upsilon ={\left({\lambda }^2-2\lambda \sigma +{\sigma }^2\right)}^2\upsilon \Rightarrow \left({\left(C{U}_{\xi \zeta }^{-1}\right)}^{T}C{U}_{\xi \zeta }^{-1}-{\sigma }^{2}I\right)\upsilon =\left({\lambda }^2-2\lambda \sigma \right)\upsilon .\end{array}$$

(19)

From Equation (18), it follows that if $\upsilon \ne 0$, then the eigenvector $\upsilon$ and $\left({\lambda }^2-2\lambda \sigma \right)$ are the corresponding eigenvalue of the matrix $\left({\left(C{U}_{\xi \zeta }^{-1}\right)}^{T}C{U}_{\xi \zeta }^{-1}-{\sigma }^{2}I\right),$ i.e.,

$$\left({\lambda }_i{}^2-2{\lambda }_i\sigma \right)={\sigma }_i^2-{\sigma }^2,$$

where the eigenvalues of the matrix by solving the quadratic equation are:

$${\lambda }_i{}^2-2{\lambda }_i\sigma -{\sigma }_i^2+{\sigma }^2=0$$

(20)

Solving Equation (19), we can obtain ${\lambda }_i=\sigma \pm {\sigma }_i^{}$.

Then, the spectral condition number is:

$${\kappa }_2\left(\tilde{A}\right)=\frac{{\sigma }_{\max }^{}+\sigma }{{\sigma }_{\min }^{}-\sigma }.$$

3. Results

Test cases were compared to the normalized root mean square error (NRMSE) of parameter estimation, defined as:

$$\delta \theta =\sqrt{{\Vert \widehat{\theta }-{\theta }_0\Vert }^2/{\Vert {\theta }_0\Vert }^2\cdot }100\raisebox{1ex}{$0$}\!\left/ \!\raisebox{-1ex}{$0$}\right.$$

The results were based on 50 independent Monte-Carlo simulations. The number of data points N in each simulation was 200. The input signal was a step function. The dynamic system is described by the equation:

$$z_i=0.5{\Delta }^{0.2}{z}_{i-1}+0.3{\Delta }^{0.1999}{z}_{i-1}-0.3{\Delta }^{0.3}{z}_{i-1}+{\Delta }^{0.2}{x}_i^{}+0.9{\Delta }^{0.15}{x}_i^{},y_i=z_i+{\xi }_i,w_i^{}=x_i^{}+{\zeta }_i^{}.$$

(21)

Figure 1 shows the step response of the system (20).

Example 1.

The noise-to-signal ratios were${\sigma }_{\xi }^{}/{\sigma }_z^{}={10}^{-12}$and${\sigma }_{\zeta }^{}/{\sigma }_x^{}={10}^{-12}.$The gap between the minimal singular values of the matrix$\left(\begin{array}{cc}C{U}_{\xi \zeta }^{-1}& d\end{array}\right)$is$g=4.5199\pm 0.0048$.

Table 1. The mean values of the condition numbers of the matrices and their standard deviations.

${\kappa }_2\left({\left(C{U}_{\xi \zeta }^{-1}\right)}^{T}C{U}_{\xi \zeta }^{-1}-{\sigma }_{}^{2}I\right)$	${\kappa }_2\left(\tilde{A}\right)$	${\kappa }_2\left(A\left(k\right)\right)$
8246.5 ± 886.6	90.684 ± 4.839	215.410 ± 13.147

shows the mean values of the condition numbers of the matrices and their standard deviations.

Figure 2 shows the histogram of the decision time.

Table 2. The mean values of the decision time and their standard deviations.

System (7) Based on LU Decomposition, s	System (11) Based on LU Decomposition, s	System (11) Based on LDLT Decomposition, s
0.0084 ± 0.0018	0.0050 ± 0.0038	0.0046 ± 0.0048

shows the mean values of the decision time and their standard deviations.

Figure 3 shows the histogram of the normalized root mean square error of the parameters.

Table 3. The mean values of the normalized root mean square error of the parameter estimation and their standard deviations.

Classical Algorithm [20], %	System (7) Based on LU Decomposition, %	System (11) Based on LU Decomposition, %	System (11) Based on LDLT Decomposition, %
(2.542 ± 2.005)10−7	(2.547 ± 2.032) 10−7	(2.561 ± 2.166) 10−7	(2.533 ± 2.058) 10−7

shows the mean values of the normalized root mean square error of the parameter estimation and their standard deviations.

Example 2.

The noise-to-signal ratios were${\sigma }_{\xi }^{}/{\sigma }_z^{}={10}^{-12}$and${\sigma }_{\zeta }^{}/{\sigma }_x^{}={10}^{-1}.$

The gap between the minimal singular values of the matrix is $g=(7.628\pm 0.4828){10}^{-12}$.

Table 4. The mean values of the condition numbers of the matrices and their standard deviations.

${\kappa }_2\left({\left(C{U}_{\xi \zeta }^{-1}\right)}^{T}C{U}_{\xi \zeta }^{-1}-{\sigma }_{}^{2}I\right)$	${\kappa }_2\left(\tilde{A}\right)$	${\kappa }_2\left(A\left(k\right)\right)$
(1.6530 ± 0.1216)1038	(4.0269 ± 0.1828)1018	(6.4410 ± 0.1510)109

shows the mean values of the condition numbers of the matrices and their standard deviations.

Figure 4 shows the histogram of the decision time.

Table 5. The mean values of the decision time and their standard deviations.

System (7) Based on LU Decomposition, s	System (11) Based on LU Decomposition, s	System (11) Based on LDLT Decomposition, s
0.0087 ± 0.0068	0.0037 ± 0.0037	0.0036 ± 0.0023

shows the mean values of the decision time and their standard deviations.

Figure 5 shows the histogram of the normalized root mean square error of the parameters.

Table 6. The mean values of the normalized root mean square error of parameter estimation and their standard deviations.

Classical Algorithm [20], %	System (7) Based on LU Decomposition, %	System (11) Based on LU Decomposition, %	System (11) Based on LDLT Decomposition, %
1167.4 ± 1041.51	9.9926 ± 0.4832	9.9932 ± 0.4800	9.9936 ± 0.4824

shows the mean values of the normalized root mean square error of parameter estimation and their standard deviations.

In Table 3, the standard deviations and mean values of the normalized root mean square error of the parameters are nearly equal. This is due to the fact that the condition numbers were large enough and the resulting estimates were sensitive to the initial data.

In Table 6, the standard deviations for the classical algorithm significantly exceed the mean values of the normalized root mean square error of the parameters. This indicates that in some experiments the results are very inaccurate, suggesting that the algorithm is numerically unstable and cannot be applied without regularization methods.

As can be seen from Examples 1,2, conditionality is highly dependent on the signal-to-noise ratios upon input and output. Let us investigate how the condition number changes with a change in the signal-to-noise ratio upon input and output. Upon output, there is a signal–noise ratio of ${\sigma }_{\xi }^{}/{\sigma }_z^{}={10}^{-12}$, while there is a signal-to-noise ratio upon input of ${\sigma }_{\zeta }^{{}^{\left(m\right)}}/{\sigma }_x^{}={10}^{-\left(12-0.2m\right)}$. Figure 2 shows $\lg \left({\kappa }_2\left(\tilde{A}\right)\right)$ and $\lg \left({\kappa }_2\left(A(k)\right)\right)$. As can be seen from Figure 6, the conditionality of the proposed system is lower than the conditionality of the system (8).

4. Discussion

This paper proposed an estimation method of the parameters of discrete fractional systems by generalized total least squares. The simulation results showed that the parameter estimates obtained with the augmented systems of the equation can exceed the accuracy of the SVD-based estimates. This is due to the fact that methods based on augmented systems are less sensitive to the size of the gap between the smallest singular numbers.

In Table 6, showing the standard deviations for the classical algorithm, the mean values of normalized root mean square error of parameters is significantly exceeded. This shows that the method based on the SVD approach in example 2 is numerically unstable.

The solution time for the proposed method is less than that used in [34]. It is also not required to determine the optimal parameter $k_{opt}$. The LDL^T decomposition-based implementation turned out to be faster than the LU decomposition-based implementation.

The application of the proposed augmented system is especially justified in cases when it is necessary to repeatedly estimate the parameters of discrete fractional systems, such as in algorithms that estimate both linear and nonlinear parameters [43]. In the future, it is planned to investigate the use of regularized versions of these systems.