Model Reduction for Discrete-Time Systems via Optimization over Grassmann Manifold

Abstract

In this paper, we investigate $h_2$-optimal model reduction methods for discrete-time linear time-invariant systems. Similar to the continuous-time case, we will formulate this problem as an optimization problem over a Grassmann manifold. We consider constructing reduced systems by both one-sided and two-sided projections. For one-sided projection, by utilizing the principle of the Grassmann manifold, we propose a gradient flow method and a sequentially quadratic approximation approach to solve the optimization problem. For two-sided projection, we apply the strategies of alternating direction iteration and sequentially quadratic approximation to the minimization problem and develop a numerically efficient method. One main advantage of these methods, based on the formulation of optimization over a Grassmann manifold, is that stability can be preserved in the reduced system. Several numerical examples are provided to illustrate the effectiveness of the methods proposed in this paper.

1. Introduction

Model reduction, which is often employed in approximating various very large-scale, complex dynamical systems, has received considerable attention in the past several decades. These systems often arise in various applications, such as fluid dynamics [1], structural dynamics [2], electrical circuits [3], aerodynamics [4], micro-electro-mechanical systems [5], weather prediction [6], and so on. Roughly speaking, the goal of model reduction is to replace a given mathematical model, described via differential equations or difference equations, with a model of the same form but with much smaller state space dimension than the original one, such that the reduced model still describes at least appropriately certain aspects of the original system.

In this paper we consider the model reduction of a discrete-time linear time-invariant (LTI) system described by the following system of difference equations

$$\Sigma :\left\{\begin{array}{ccc}\hfill x_{k+1}& =& A{x}_k+B{u}_k,\hfill \\ \hfill y_k& =& C{x}_k,\hfill \end{array}\right.$$

(1)

where the matrices $A\in {\mathbb{R}}^{n\times n}$, $B\in {\mathbb{R}}^{n\times m},$ and $C\in {\mathbb{R}}^{p\times n}$ represent the state matrix, the input matrix, and the output matrix, respectively. The vectors $x_k\in {\mathbb{R}}^n$, $u_k\in {\mathbb{R}}^m$, and $y_k\in {\mathbb{R}}^p$ are the state vector, the input vector, and the output vector of the discrete-time LTI system at time $t_k$, respectively. For simplicity, we assume that we have a zero initial condition $x_0=0$ in (1). Discrete-time systems arise naturally when continuous-time systems are discretized by numerical approximation to differentiation. Moreover, they directly appear in some important applications such as economics [7] and transportation network [8]. For a comprehensive treatment of discrete systems, including system characterization, structural properties, stability, optimal control, and applications, we refer to the book [9].

In this paper we aim at constructing a reduced-order model of $\Sigma$ via projection. Specifically, we hope to find two matrices $W\in {\mathbb{R}}^{n\times r}$ and $V\in {\mathbb{R}}^{n\times r}$ with $r\ll n$, such that a reduced system $\widehat{\Sigma }$ is given by

$$\widehat{\Sigma }:\left\{\begin{array}{ccc}\hfill {\widehat{x}}_{k+1}& =& \widehat{A}{\widehat{x}}_k+\widehat{B}{u}_k,\hfill \\ \hfill {\widehat{y}}_k& =& \widehat{C}{\widehat{x}}_k\hfill \end{array}\right.$$

(2)

with $\widehat{A}=W^{T}AV$, $\widehat{B}=W^{T}B$, and $\widehat{C}=CV$.

Much research has addressed the construction of the reduced-order model. For continuous-time LTI systems, there exist various model reduction methods. For example, classical model reduction methods, which are based on system Gramians, have been well established, including the balanced truncation method [10], the optimal Hankel norm approximation method [11], and the singular perturbation approximation method [12]. The other class of projection methods are the Krylov subspace [13] and rational Krylov subspace [14] methods, which become increasingly popular for large-scale linear dynamical systems such as those arising from structure dynamics, circuit simulations, and micro-electro-mechanical systems (see [15,16,17,18,19]). Rational Krylov subspace methods have been further improved to develop an iterative rational Krylov subspace algorithm [20], in which the interpolation points are iteratively updated so that the established reduced-order model satisfies the interpolation-based first-order necessary conditions for $H_2$-optimal model reduction of single-input single-output continuous-time systems. In [21,22], an iterative rational Krylov subspace method (IRKA) is extended to derive a tangential rational interpolation framework for model reduction of multi-input multi-output (MIMO) systems. $H_2$-optimal model reduction can also be formulated as an optimization problem over a Grassmann manifold. Based on this fact and by making use of one-sided projection, Xu and Zeng [23] proposed a fast gradient flow algorithm and a sequentially quadratic approximation for solving the optimization problem to construct stable reduced systems. The model reduction method based on Grassmann manifold optimization is further improved in [24] by employing two-sided projection. Model order reduction on Grassmann manifolds for other classes of systems are also considered. In [25], model order reduction on Grassmann manifolds has been extended to a special class of linear parameter-varying systems. Xu et al. [26] proposed a preserving–periodic Riemannian descent model reduction iterative method for linear discrete-time periodic systems. In [27], a parametric interpolation parallel MOR method for discrete-time parametric systems is proposed by making use of Grassmann manifolds and discrete Laguerre polynomials. In [28], $H_2$ order reduction based on Grassmann manifolds is extended to bilinear systems. Otto et al. [29] studied model order reduction over the product of two Grassmann manifolds for nonlinear systems. Padovan et al. [30] considered data-driven model reduction for nonlinear systems by solving an optimization problem over the product of two manifolds. In [31], a novel differential geometric framework for model reduction on smooth manifolds is proposed. This general framework can capture and generalize several existing MOR techniques, such as preserving the structures for Lagrangian or Hamiltonian dynamics. Zimmermann [32] reviewed matrix manifolds and outlined the principal approaches to data interpolation and Taylor-like extrapolation on matrix manifolds. For the reduced-order modeling of high-dimensional dynamical systems, Sashittal and Bodony [33] proposed low-rank dynamic mode decomposition by solving a matrix manifold optimization problem with a rank constraint on the solution.

Some researchers have considered $h_2$-optimal model reduction for discrete-time LTI systems. For example, Van Dooren, Gallivan, and Absil [22] derived two first-order necessary conditions, the discrete-time counterparts of Wilson’s conditions [34] and tangential rational interpolation conditions for $h_2$-optimal model reduction. In [35], the authors gave the discrete-time counterparts of Hyland–Bernstein Gramian-based first-order necessary $h_2$-optimality conditions. They showed that Wilson’s conditions, the Hyland–Bernstein conditions, and the tangential interpolation-based conditions are equivalent. Moreover, based on the tangential interpolation-based conditions, a MIMO iterative rational interpolation algorithm is developed for model reduction of discrete-time MIMO LTI systems.

In this paper, we consider constructing reduced systems by both one-sided and two-sided projection. Similar to the continuous-time case, we formulate the $h_2$-optimal model reduction problems for discrete-time LTI systems as optimization problems over Grassmann manifolds. Based on this formulation, in the one-side projection case, we apply the gradient flow method and the sequentially quadratic approximation method to solve the related optimization problem. For two-sided projection, we combine the techniques of alternating direction iteration and sequentially quadratic approximation to develop an iterative method for the Grassmann manifold optimization problem. Numerical experiments demonstrate the effectiveness of model reduction methods based on Grassmann manifold optimization for discrete-time LTI systems.

Throughout this paper the following notation is used: The sets of all real and complex $n\times m$ matrices are denoted by ${\mathbb{R}}^{n\times m}$ and ${\mathbb{C}}^{n\times m}$, respectively. The identity matrix of dimension n is denoted by $I_n$ and the zero matrix by 0. If the dimension of $I_m$ is apparent from the context, we drop the index and simply use I. The actual dimension of 0 will always be apparent from the context. The superscripts T and H denote the transpose and the complex conjugate transpose of a vector or a matrix. The notation $\mathrm{span}\left\{V\right\}$ denotes the space spanned by the column vectors of the matrix V, and $\mathrm{span}\{V_1,V_2,\dots ,V_j\}$ denotes the space spanned by the matrix sequence $V_1,V_2,\dots ,V_j$. For two matrices $U,V\in {\mathbb{R}}^{n\times m}$, the inner product of U and V is defined as $⟨U,V⟩=\mathrm{trace}\left(V^{T}U\right)$, where $\mathrm{trace}(·)$ is the trace of a matrix. The notation $\parallel U\parallel$ denotes the Frobenius matrix norm, which is defined by $\parallel U\parallel =\sqrt{⟨U,U⟩}$.

The main contributions of this paper include the following:

• The $h_2$-optimal model reduction methods based on Grassmann manifold optimization are extended to discrete-time LTI systems.
• For one-sided projection, a gradient flow method and a sequentially quadratic approximation approach are proposed to solve the optimization problem. For two-sided projection, the optimization problem is solved by applying the strategies of alternating direction iteration and sequentially quadratic approximation.
• We present the details of implementation, such as how to efficiently solve sparse–dense discrete-time Sylvester equations.
• The effectiveness of the proposed methods in this paper is demonstrated with two numerical examples.

The remainder of this paper is organized as follows: Section 2 is devoted to a short review of some important results on the $h_2$-norm of discrete-time LTI systems and Stiefel and Grassmann manifolds. Moreover, we also outline the framework of the gradient flow method for solving optimization problems over Grassmann manifolds. In Section 3, we consider the one-sided projection case. Two iterative methods, one relying on the gradient flow and the other on sequentially quadratic approximation, are proposed. We consider two-sided projection in Section 4. An iterative method based on the combination of alternating direction iteration and sequentially quadratic approximation is given. In Section 5, some issues related to the implementation of these algorithms are investigated, including the initial projection matrix selection and the termination criterion. We also discuss how to solve Stein equations, which appear in the methods proposed in this paper. In Section 6, several numerical examples are presented to illustrate the effectiveness of these model reduction methods. Finally, some conclusions are drawn in Section 7.

2. Preliminaries

In this section we review some important results on the $h_2$-norm of discrete-time LTI systems and the Grassmann manifold. Moreover, we also outline the framework of the gradient flow method for solving optimization problems over Grassmann manifolds. Much of the material in this section is standard and can be found in [35,36,37,38].

2.1. $h_2$-Norm of Discrete-Time LTI Systems

For a vector sequence $x_k$, $k=0,1,2,\dots$, its z-transform is defined by

$$x\left(z\right)=\sum _{k=0}^{\infty }{x}_k{z}^{-1}.$$

By applying the z-transform to the system $\Sigma$, we obtain, in the frequency domain, the following input–output relation:

$$y\left(z\right)=H\left(z\right)u\left(z\right),$$

(3)

where $y\left(z\right)$ and $u\left(z\right)$ are the z-transformations of the input and output, respectively, and 

$$H\left(z\right)=C{(z{I}_n-A)}^{-1}B$$

(4)

is called the transfer function of the system $\Sigma$.

In this paper, we assume that the discrete-time system $\Sigma$ is asymptotically stable, i.e., all the eigenvalues of A lie inside the unit circle. In this case, the controllability and observability Gramians of the discrete-time system $\Sigma$ are well defined and formulated as

$$\begin{array}{ccc}\hfill P& =& \sum _{k=0}^{\infty }{A}^{k}B{B}^T{\left(A^T\right)}^k,\hfill \end{array}$$

(5)

$$\begin{array}{ccc}\hfill Q& =& \sum _{k=0}^{\infty }{\left(A^T\right)}^k{C}^{T}C{A}^k.\hfill \end{array}$$

(6)

These Gramians also satisfy the Stein equations, also called the discrete Lyapunov equations:

$$\begin{array}{ccc}& & AP{A}^T-P+B{B}^T=0,\hfill \end{array}$$

(7)

$$\begin{array}{ccc}& & A^{T}QA-Q+C^{T}C=0.\hfill \end{array}$$

(8)

For an asymptotically stable discrete-time LTI system $\Sigma$, its squared $h_2$-norm is defined by

$${\parallel \Sigma \parallel }_{h_2}^2={\parallel H\parallel }_{h_2}^2=\mathrm{trace}\left({\displaystyle \frac{1}{2\pi }}{\int }_0^{2\pi }{H}^{*}\left(e^{i\theta }\right)H\left(e^{i\theta }\right)d\theta \right).$$

(9)

Using these systems’ Gramians P and Q, the $h_2$-norm can be calculated via the following equation:

$${\parallel \Sigma \parallel }_{h_2}^2={\parallel H\parallel }_{h_2}^2=\mathrm{trace}\left(CP{C}^T\right)=\mathrm{trace}\left(B^{T}QB\right).$$

(10)

2.2. Stiefel and Grassmann Manifolds

In this subsection, we briefly review the definitions of Stiefel manifold and Grassmann manifold and some basic results on these two manifolds.

Definition 1. 

The real Stiefel manifold $\mathbf{St}(r,n)$ with $r\le n$ is defined as the set of all $n\times r$ orthonormal matrices, i.e.,

$$\mathbf{St}(r,n)=\{U\in {\mathbb{R}}^{n\times r}|U^{T}U=I_r\}.$$

Definition 2. 

Two matrices $U_1,U_2$ in the real Stiefel manifold $\mathbf{St}(r,n)$ are defined to be equivalent if their columns span the same r-dimensional subspaces.

From Definition 2, $U_1,U_2\in \mathbf{St}(r,n)$ are equivalent if and only if there exists an orthogonal matrix $Q\in {\mathbb{R}}^{r\times r}$ such that $U_1=U_{2}Q$. So, we can define the equivalent class $\left[U\right]$ of a point $U\in \mathbf{St}(r,n)$ to be

$$\left[U\right]=\left\{UQ\right|Q\in {\mathbf{O}}_r\},$$

where ${\mathbf{O}}_r$ denotes the set of all orthogonal matrices in ${\mathbb{R}}^{r\times r}$.

Definition 3. 

The real Grassmann manifold $\mathbf{Gr}(r,n)$ with $r\le n$ is defined as the set of all r-dimensional real linear subspaces of ${\mathbb{R}}^n$.

From the above definition, we know that a point in the real Grassmann manifold $\mathbf{Gr}(r,n)$ is an r-dimensional real linear subspace, which can be spanned by the columns of a matrix in $\mathbf{St}(r,n)$. So, there is a one-to-one correspondence between a point in the real Grassmann manifold $\mathbf{Gr}(r,n)$ and an equivalent class of $\mathbf{St}(r,n)$. Thus, the Grassmann manifold $\mathbf{Gr}(r,n)$ can be seen as a quotient manifold, i.e.,

$$\mathbf{Gr}(r,n)=\mathbf{St}(r,n)/{\mathbf{O}}_r.$$

Definition 4. 

For a Stiefel manifold $\mathbf{St}(r,n)$, the tangent space at $U\in \mathbf{St}(r,n)$ is defined as

$${\mathbb{T}}_U\mathbf{St}(r,n)=\{Z\in {\mathbb{R}}^{n\times r}|U^{T}Z+Z^{T}U=0\}.$$

Correspondingly, we have a definition of the tangent space of the Grassmann manifold.

Definition 5. 

The tangent space at point $\left[U\right]$ in the Grassmann manifold $\mathbf{Gr}(r,n)$ is described as

$${\mathbb{T}}_{\left[U\right]}\mathbf{Gr}(r,n)=\{Z\in {\mathbb{R}}^{n\times r}|U^{T}Z=0\}.$$

Let $J\left(U\right)$ be a function of $U\in {\mathbb{R}}^{n\times r}$. Then we can define the matrix $J_U$ of all partial derivatives of J with respect to U:

$${\left(J_U\right)}_{ij}={\displaystyle \frac{\partial J}{\partial U_{ij}}}.$$

Definition 6. 

Let $J\left(U\right)$ be a function defined in a Stiefel manifold $\mathbf{St}(r,n)$. Then, its gradient at U in the Stiefel manifold is defined to be the tangent vector $\nabla J$ such that

$$\mathrm{trace}\left(J_U^{T}V\right)=\mathrm{trace}\left({(\nabla J)}^T\left(I-{\displaystyle \frac{1}{2}}U{U}^T\right)V\right).$$

holds for all tangent vectors $V\in {\mathbb{T}}_U\mathbf{St}(r,n)$.

Definition 7. 

Let $J\left(U\right)$ be a function defined in a Grassmann manifold $\mathbf{Gr}(r,n)$. Then, its gradient at point $\left[U\right]\in \mathbf{Gr}(r,n)$ is defined to be the tangent vector $\nabla J$ such that

$$\mathrm{trace}\left(J_U^{T}V\right)=\mathrm{trace}\left({(\nabla J)}^{T}V\right).$$

holds for all tangent vectors $V\in {\mathbb{T}}_{\left[U\right]}\mathbf{Gr}(r,n)$.

It has been shown in [36,37] that the gradient of J defined in a Stiefel manifold $\mathbf{St}(r,n)$ is expressed as

$$\nabla J=J_U-U{J}_U^{T}U,$$

(11)

while the gradient of J defined in Grassmann manifold $\mathbf{Gr}(r,n)$ is formulated as

$$\nabla J=(I-U{U}^T)J_U.$$

(12)

2.3. The Gradient Flow Method

We briefly overview the framework of this method in the rest of this section.

Given an optimization problem based on Grassmann manifold $\mathbf{Gr}(r,n)$,

$$\underset{\left[U\right]\in \mathbf{Gr}(r,n)}{min}J\left(U\right),$$

(13)

where J is the objective function.

It is well known [36] that the gradient flow method is one of the most efficient iterative methods for solving optimization problems based on Grassmann manifolds.

Let $J_U$ denote the matrix of all partial derivatives of J with respect to $\left[U\right]$. Then the gradient of J at point U of Grassmann manifold $\mathbf{Gr}(r,n)$ is $\nabla J=(I-U{U}^T)J_U$. Clearly, the solution U of minimization problem (13) must be a point of $\mathbf{Gr}(r,n)$ such that the gradient $\nabla J=0$, i.e., U, satisfies

$$\nabla J=(I-U{U}^T)J_U=0,U^{T}U=I_r.$$

To find a zero point of the above equation, it is proposed to solve the following gradient flow problem on the Grassmann manifold $\mathbf{Gr}(r,n)$:

$${\displaystyle \frac{dU\left(t\right)}{dt}}=-\nabla J=\left(U\left(t\right)U^T\left(t\right)-I_n\right)J_{U\left(t\right)}.$$

(14)

Given an arbitrary square matrix A, the matrix functions $sin\left(A\right)$ and $cos\left(A\right)$ are defined, respectively, as [39]

$$\begin{array}{ccc}\hfill sin\left(A\right)& =& \sum _{j=0}^{\infty }{\displaystyle \frac{{(-1)}^j}{(2j+1)!}}{A}^{2j+1}=A-{\displaystyle \frac{A^3}{3!}}+{\displaystyle \frac{A^5}{5!}}+{\displaystyle \frac{A^7}{7!}}\dots ,\hfill \\ \hfill cos\left(A\right)& =& \sum _{j=0}^{\infty }{\displaystyle \frac{{(-1)}^j}{\left(2j\right)!}}{A}^{2j}=I-{\displaystyle \frac{A^2}{2!}}+{\displaystyle \frac{A^4}{4!}}-{\displaystyle \frac{A^6}{6!}}\dots .\hfill \end{array}$$

Note that for any diagonal matrix $\Gamma =\mathrm{diag}\{{\sigma }_1,{\sigma }_2,\dots ,{\sigma }_n\}$, it holds that

$$\begin{array}{ccc}\hfill sin(\Gamma )& =& \mathrm{diag}\{sin{\sigma }_1,sin{\sigma }_2,\dots ,sin{\sigma }_n\},\hfill \\ \hfill cos(\Gamma )& =& \mathrm{diag}\{cos{\sigma }_1,cos{\sigma }_2,\dots ,cos{\sigma }_n\}.\hfill \end{array}$$

Given a matrix $F\in {\mathbb{R}}^{n\times r}$, let $F=\Phi \Gamma {\Psi }^T$ be the singular value decomposition of F, where $\Phi \in {\mathbb{R}}^{n\times r},\Gamma \in {\mathbb{R}}^{r\times r},$ and $\Psi \in {\mathbb{R}}^{r\times r}$. The geodesic on the Grassmann manifold $\mathbf{Gr}(r,n)$ at point $\left[U\right]$ with the direction F is defined by

$$U\Psi cos(t\Gamma ){\Psi }^T+\Phi sin(t\Gamma ){\Psi }^T,t\in \mathbb{R}.$$

With the help of the geodesic and by choosing an appropriate time step $t_j$, the gradient flow iteration scheme, which iteratively constructs the approximate solution of (14) along the geodesic at the negative gradient direction $-\nabla J$, is formulated as

$$U_{j+1}=U_j{\Psi }_{j}cos\left(t_j{\Gamma }_j\right){\Psi }_j^T+{\Phi }_{j}sin\left(t_j{\Gamma }_j\right){\Psi }_j^T,$$

(15)

where ${\Phi }_j{\Gamma }_j{\Psi }_j^T$ is the singular value decomposition of the negative gradient of J at $U_j$.

3. One-Sided Projection via Optimization on Grassmann Manifold

This section is concerned with model order reduction of the discrete-time linear LTI system in (1). One objective of model order reduction is to reduce the error between the original system (1) and the reduced system (2). In this paper we will be measuring the error in the $h_2$-norm. For the sake of the derivation of necessary conditions for $h_2$-optimal model reduction over a Grassmann manifold, let us define the transfer function of the error system ${\Sigma }_e$:

$$H_e\left(z\right)=H\left(z\right)-\widehat{H}\left(z\right)=C{(z{I}_n-A)}^{-1}B-\widehat{C}{(z{I}_r-\widehat{A})}^{-1}\widehat{B}.$$

(16)

It is clear that the error system ${\Sigma }_e$ has a realization $(A_e,B_e,C_e)$ with

$$A_e=\left(\begin{array}{cc}A& 0\\ 0& \widehat{A}\end{array}\right),B_e=\left(\begin{array}{c}B\\ \widehat{B}\end{array}\right),C_e=\left(\begin{array}{cc}C& -\widehat{C}\end{array}\right),$$

(17)

i.e., the transfer function of the error system ${\Sigma }_e$ can be also formulated as

$$H_e\left(z\right)=C_e{(z{I}_{n+r}-A_e)}^{-1}{B}_e.$$

(18)

Following the knowledge in Section 2, the squared $h_2$-norm of the error system ${\Sigma }_e$ can be written in terms of its controllability Gramian $P_e$ and observability $Q_e$ as

$$\parallel {\Sigma }_e{\parallel }_{h_2}^2={\parallel H_e\parallel }_{h_2}^2=\mathrm{trace}\left(C_e{P}_e{C}_e^T\right)=\mathrm{trace}\left(B_e^T{Q}_e{B}_e\right).$$

(19)

Note that $P_e$ and $Q_e$ are also the solutions of the following two Stein equations associated with the error system:

$$\begin{array}{ccc}& & A_e{P}_e{A}_e^T-P_e+B_e{B}_e^T=0,\hfill \end{array}$$

(20)

$$\begin{array}{ccc}& & A_e^T{Q}_e{A}_e-Q_e+C_e^T{C}_e=0.\hfill \end{array}$$

(21)

Let us partition the matrices $P_e$, $Q_e$ into

$$P_e=\left(\begin{array}{cc}P& X\\ X^T& \widehat{P}\end{array}\right),Q_e=\left(\begin{array}{cc}Q& Y\\ Y^T& \widehat{Q}\end{array}\right)$$

(22)

with $P,Q\in {\mathbb{R}}^{n\times n}$, $\widehat{P},\widehat{Q}\in {\mathbb{R}}^{r\times r}$, and $X,Y\in {\mathbb{R}}^{n\times r}$. Then, the Stein Equation in (20) is equivalent to the following equations:

$$\begin{array}{ccc}& & AP{A}^T-P+B{B}^T=0,\hfill \end{array}$$

(23)

$$\begin{array}{ccc}& & \widehat{A}\widehat{P}{\widehat{A}}^T-\widehat{P}+\widehat{B}{\widehat{B}}^T=0,\hfill \end{array}$$

(24)

$$\begin{array}{ccc}& & AX{\widehat{A}}^T-X+B{\widehat{B}}^T=0,\hfill \end{array}$$

(25)

On the other hand, the Stein Equation in (21) is equivalent to

$$\begin{array}{ccc}& & A^{T}QA-Q+C^{T}C=0,\hfill \\ & & {\widehat{A}}^T\widehat{Q}\widehat{A}-\widehat{Q}+{\widehat{C}}^T\widehat{C}=0,\hfill \\ & & A^{T}Y\widehat{A}-Y-C^T\widehat{C}=0.\hfill \end{array}$$

Based on the expression of the squared $h_2$-norm of the error system in (19) and the partition of $P_e$ and $Q_e$ in (22), the squared $h_2$-norm of the error system can now be rewritten as

$$\begin{array}{ccc}\hfill \parallel H_e{\parallel }_{h_2}^2& =& \mathrm{trace}(CP{C}^T+\widehat{C}\widehat{P}{\widehat{C}}^T-2CX{\widehat{C}}^T)\hfill \end{array}$$

(26)

$$\begin{array}{ccc}& =& \mathrm{trace}(B^{T}QB+{\widehat{B}}^T\widehat{Q}\widehat{B}+2B^{T}Y\widehat{B}).\hfill \end{array}$$

(27)

$H_2$-norm model reduction on a Grassmann manifold for continuous-time systems was considered in [23,40]. In this section, we aim to extend the one-sided projection technique based on Grassmann manifold optimization to the discrete-time case. The one-sided projection-based reduction approach involves finding one projection matrix $U\in {\mathbb{R}}^{n\times r}$, $U^{T}U=I_r$. With U, the reduced system $\widehat{\Sigma }$ is constructed by setting

$$\widehat{A}=U^{T}AU,\widehat{B}=U^{T}B,\widehat{C}=CU.$$

(28)

In the one-sided projection scheme, the objective function $J\left(U\right)$ is defined by

$$\begin{array}{ccc}\hfill J\left(U\right)=\parallel H_e{\parallel }_{h_2}^2& =& \mathrm{trace}(CP{C}^T+\widehat{C}\widehat{P}{\widehat{C}}^T-2CX{\widehat{C}}^T)\hfill \\ & =& \mathrm{trace}\left(C^{T}C(P+U\widehat{P}{U}^T-2X{U}^T)\right).\hfill \end{array}$$

(29)

Now, from (23)–(25) and (28), it follows that P, $\widehat{P}$, and X satisfy

$$\begin{array}{ccc}& & AP{A}^T-P+B{B}^T=0,\hfill \end{array}$$

(30)

$$\begin{array}{ccc}& & U^{T}AU\widehat{P}{U}^T{A}^{T}U-\widehat{P}+U^{T}B{B}^{T}U=0,\hfill \end{array}$$

(31)

$$\begin{array}{ccc}& & AX{U}^T{A}^{T}U-X+B{B}^{T}U=0.\hfill \end{array}$$

(32)

The objective function $J\left(U\right)$ is also reformulated as

$$\begin{array}{ccc}\hfill J\left(U\right)=\parallel H_e{\parallel }_{h_2}^2& =& \mathrm{trace}(B^{T}QB+{\widehat{B}}^T\widehat{Q}\widehat{B}+2B^{T}Y\widehat{B})\hfill \\ & =& \mathrm{trace}\left(B{B}^T(Q+U\widehat{Q}{U}^T+2Y{U}^T)\right),\hfill \end{array}$$

where Q, $\widehat{Q}$, and Y are the solutions to the equations

$$\begin{array}{ccc}& & A^{T}QA-Q+C^{T}C=0,\hfill \end{array}$$

(33)

$$\begin{array}{ccc}& & U^T{A}^{T}U\widehat{Q}{U}^{T}AU-\widehat{Q}+U^T{C}^{T}CU=0,\hfill \end{array}$$

(34)

$$\begin{array}{ccc}& & A^{T}Y{U}^{T}AU-Y-C^{T}CU=0.\hfill \end{array}$$

(35)

respectively.

We will be deriving an expression of the matrix $J_U$ of all partial derivatives of J with respect to U. The following lemma is useful in the derivation of our result.

Lemma 1. 

Let $M,N\in {\mathbb{R}}^{n\times m}$, $A\in {\mathbb{R}}^{n\times n}$, $B\in {\mathbb{R}}^{m\times m}$, and $X,Y\in {\mathbb{R}}^{n\times m}$. If $M,N$ satisfy

$$AMB-M+X=0,A^{T}N{B}^T-N+Y=0,$$

then the following holds:

$$\mathrm{trace}\left(Y^{T}M\right)=\mathrm{trace}\left(X^{T}N\right).$$

Proof. 

Obviously, it holds that

$$\begin{array}{ccc}\hfill \mathrm{trace}\left(Y^{T}M\right)& =& \mathrm{trace}(N^{T}M-B{N}^{T}AM)\hfill \\ & =& \mathrm{trace}\left(N^T(M-AMB)\right)\hfill \\ & =& \mathrm{trace}\left({(M-AMB)}^{T}N\right)\hfill \\ & =& \mathrm{trace}\left(X^{T}N\right).\hfill \end{array}$$

This completes the proof.    □

Theorem 1. 

Let $\widehat{P}$, X, $\widehat{Q}$, and Y be the solutions of (31), (32), (34), and (35), respectively. Then, the matrix $J_U$ of all partial derivatives of J with respect to U can be expressed as

$$J_U=2R,$$

(36)

where R is defined by

$$\begin{array}{ccc}\hfill R& =& (AU\widehat{P}{U}^T{A}^{T}U\widehat{Q}+A^{T}U\widehat{Q}{U}^{T}AU\widehat{P}+B{B}^{T}U\widehat{Q})\hfill \\ & & +(A^{T}U{Y}^{T}AX+AU{X}^T{A}^{T}Y+B{B}^{T}Y)+\left(C^{T}CU\widehat{P}-C^{T}CX\right).\hfill \end{array}$$

(37)

Proof. 

Let $E_{ij}$ be the single-entry matrix being one in entry $(i,j)$ and zero otherwise. By differentiating J with respect to $U_{ij}$, we obtain

$$\begin{array}{ccc}\hfill {\left(J_U\right)}_{ij}& =& {\displaystyle \frac{\partial }{\partial U_{ij}}}\mathrm{trace}\left(C^{T}C(P+U\widehat{P}{U}^T-2X{U}^T)\right)\hfill \\ & =& \mathrm{trace}\left(C^{T}C\left(E_{ij}\widehat{P}{U}^T+U{\displaystyle \frac{\partial \widehat{P}}{\partial U_{ij}}}{U}^T+U\widehat{P}{E}_{ij}^T-2{\displaystyle \frac{\partial X}{\partial U_{ij}}}{U}^T-2X{E}_{ij}^T\right)\right)\hfill \\ & =& \mathrm{trace}\left({\displaystyle \frac{\partial \widehat{P}}{\partial U_{ij}}}{U}^T{C}^{T}CU-2U^T{C}^{T}C{\displaystyle \frac{\partial X}{\partial U_{ij}}}+2\left(C^{T}CU\widehat{P}-C^{T}CX\right)E_{ij}^T\right).\hfill \end{array}$$

(38)

Differentiating Equations (31) and (32) with respect to $U_{ij}$ leads to

$$\begin{array}{ccc}& & U^{T}AU{\displaystyle \frac{\partial \widehat{P}}{\partial U_{ij}}}{U}^T{A}^{T}U-{\displaystyle \frac{\partial \widehat{P}}{\partial U_{ij}}}+Z+Z^T=0,\hfill \end{array}$$

(39)

$$\begin{array}{ccc}& & A{\displaystyle \frac{\partial X}{\partial U_{ij}}}{U}^T{A}^{T}U-{\displaystyle \frac{\partial X}{\partial U_{ij}}}+S=0,\hfill \end{array}$$

(40)

where Z and S are defined by

$$\begin{array}{ccc}\hfill Z& =& E_{ij}^{T}AU\widehat{P}{U}^T{A}^{T}U+U^{T}A{E}_{ij}\widehat{P}{U}^T{A}^{T}U+E_{ij}^{T}B{B}^{T}U,\hfill \\ \hfill S& =& AX{E}_{ij}^T{A}^{T}U+AX{U}^T{A}^T{E}_{ij}+B{B}^T{E}_{ij}.\hfill \end{array}$$

Applying Lemma 1 to Equations (34) and (39) leads to

$$\begin{array}{ccc}& & \mathrm{trace}\left(U^T{C}^{T}CU{\displaystyle \frac{\partial \widehat{P}}{\partial U_{ij}}}\right)\hfill \\ & =& \mathrm{trace}\left({(Z+Z^T)}^T\widehat{Q}\right)\hfill \\ & =& 2\mathrm{trace}\left(Z\widehat{Q}\right)\hfill \\ & =& 2\mathrm{trace}\left(E_{ij}^{T}AU\widehat{P}{U}^T{A}^{T}U\widehat{Q}+U^{T}A{E}_{ij}\widehat{P}{U}^T{A}^{T}U\widehat{Q}+E_{ij}^{T}B{B}^{T}U\widehat{Q}\right)\hfill \\ & =& 2\mathrm{trace}\left((AU\widehat{P}{U}^T{A}^{T}U\widehat{Q}+A^{T}U\widehat{Q}{U}^{T}AU\widehat{P}+B{B}^{T}U\widehat{Q})E_{ij}^T\right).\hfill \end{array}$$

Similarly, from (35), (40), and Lemma 1, it follows that

$$\begin{array}{ccc}& & \mathrm{trace}\left(-U^T{C}^{T}C{\displaystyle \frac{\partial X}{\partial U_{ij}}}\right)\hfill \\ & =& \mathrm{trace}\left(S^{T}Y\right)\hfill \\ & =& \mathrm{trace}\left(U^{T}A{E}_{ij}{X}^T{A}^{T}Y+E_{ij}^{T}AU{X}^T{A}^{T}Y+E_{ij}^{T}B{B}^{T}Y\right)\hfill \\ & =& \mathrm{trace}\left((A^{T}U{Y}^{T}AX+AU{X}^T{A}^{T}Y+B{B}^{T}Y)E_{ij}^T\right).\hfill \end{array}$$

Then, we obtain

$$\begin{array}{ccc}\hfill {\left(J_U\right)}_{ij}& =& 2\mathrm{trace}\left(R{E}_{ij}^T\right),\hfill \end{array}$$

where R is defined by (37). Therefore, we can obtain $J_U=2R$.    □

Given a stable discrete-time system (1), one-sided projection $h_2$-optimal model order reduction aims to minimize the squared $h_2$-norm of the error between the stable full system and the stable reduced-order system; i.e., it seeks to solve the minimization problem

$$\underset{U^{T}U=I_r}{min}J\left(U\right)=\underset{U^{T}U=I_r}{min}\mathrm{trace}\left(C^{T}C(P+U\widehat{P}{U}^T-2X{U}^T)\right),$$

(41)

where $\widehat{P}$ and X are the solutions of (31) and (32), respectively. Since U is a point of the Stiefel manifold $\mathbf{St}(r,n)$, the minimization problem can be also expressed as an optimization problem based on the Stiefel manifold $\mathbf{St}(r,n)$.

It is known [23] that in the continuous-time case, the minimization problem ${min}_{U^{T}U=I_r}J\left(U\right)$ can be equivalently rewritten as a minimization problem based on the Grassmann manifold $\mathbf{Gr}(r,n)$ if the cost function $J\left(U\right)$ depends only on the space spanned by the columns of the matrix U, i.e., if $J\left(U\right)=J\left(UQ\right)$ for any $Q\in {\mathbb{R}}^{r\times r}$. For the discrete-time system considered in this paper, due to the same reason, minimization problem (41) can be reformulated as an equivalent optimization problem based on a Grassmann manifold:

$$\underset{\left[U\right]\in \mathbf{Gr}(r,n)}{min}J\left(U\right)=\underset{\left[U\right]\in \mathbf{Gr}(r,n)}{min}\mathrm{trace}\left(C^{T}C(P+U\widehat{P}{U}^T-2X{U}^T)\right).$$

(42)

Clearly, the solution U of minimization problem (42) must be a point of $\mathbf{Gr}(r,n)$ such that the gradient $\nabla J_U=0$, i.e., U, satisfies the necessary conditions

$$\nabla J_U=(I-U{U}^T)J_U=0,U^{T}U=I_r,$$

where $J_U$ is given by (36).

3.1. Solving the Optimization Problem via the Gradient Flow Approach

In this subsection, we consider the application of the gradient flow method reviewed in Section 2.3 to the optimization problem over a Grassmann manifold (42).

Suppose that for $j=0,1,2,\dots$, $U_j\in {\mathbb{R}}^{n\times r}$, with $U_j^T{U}_j=I_r$, is the j-th iterate. From Theorem (1), it follows that the matrix $J_{U_j}$ of all partial derivatives of J with respect to $U_j$ can be expressed as

$$J_{U_j}=2R_j,$$

(43)

where $R_j$ is given by

$$\begin{array}{ccc}\hfill R_j& =& A{U}_j\left({\widehat{P}}_j{U}_j^T{A}^T{U}_j{\widehat{Q}}_j+X_j^T{A}^T{Y}_j\right)+B{B}^T\left(Y_j+U_j{\widehat{Q}}_j\right)\hfill \\ & & +A^T{U}_j\left(Y_j^{T}A{X}_j+{\widehat{Q}}_j{U}_j^{T}A{U}_j{\widehat{P}}_j\right)+C^{T}C\left(U_j{\widehat{P}}_j-X_j\right),\hfill \end{array}$$

(44)

where ${\widehat{P}}_j$, $X_j$, ${\widehat{Q}}_j$, and $Y_j$ are the solutions of Equations (31), (32), (34), and (35) with $U=U_j$, respectively. With $J_{U_j}$, we see from (12) that the gradient of J at point $\left[U_j\right]$ in Grassmann manifold $\mathbf{Gr}(r,n)$ can be formulated as

$$\nabla J_{U_j}=(I-U_j{U}_j^T)R_j.$$

(45)

Then, the gradient flow method computes the $(j+1)$-th approximate solution to minimization problem (41) as

$$U_{j+1}=U_j{\Psi }_{j}cos\left(t_j{\Gamma }_j\right){\Psi }_j^T+{\Phi }_{j}sin\left(t_j{\Gamma }_j\right){\Psi }_j^T,$$

(46)

where $-\nabla J_{U_j}=(U_j{U}_j^T-I_n)R_j={\Phi }_j{\Gamma }_j{\Psi }_j^T$ is the singular value decomposition of the negative gradient of J at $U_j$.

In iteration (46), the step size $t_j$ can be chosen according to the inexact line search with the Armijo rule as follows: Define

$${\mathbf{U}}_j\left(t\right)=U_j{\Psi }_{j}cos\left(t{\Gamma }_j\right){\Psi }_j^T+{\Phi }_{j}sin\left(t{\Gamma }_j\right){\Psi }_j^T,$$

where t denotes the step size parameter. For some given $c,\delta \in (0,1),\gamma >0$, the Armijo step size is $t_j={\delta }^l\gamma$, where l is the smallest nonnegative integer such that the adequate reduction condition holds:

$$J\left(U_j\right)-J\left({\mathbf{U}}_j\left({\delta }^l\gamma \right)\right)\ge c{\delta }^l\gamma ⟨\nabla J_{U_j},\nabla J_{U_j}⟩.$$

(47)

Let ${\widehat{P}}_j$ and $X_j$ be the solutions of (31) and (32) with $U=U_j$, and ${\widehat{\mathbf{P}}}_j$ and ${\mathbf{X}}_j$ be the solutions of (31) and (32) with $U={\mathbf{U}}_j\left({\delta }^l\gamma \right)$. Then, by (29), we have

$$\begin{array}{ccc}\hfill J\left(U_j\right)-J\left({\mathbf{U}}_j\left({\delta }^l\gamma \right)\right)& =& \mathrm{trace}\left(C\left(U_j{\widehat{P}}_j{U}_j^T-2X_j{U}_j^T\right)C^T\right)\hfill \\ & & -\mathrm{trace}\left(C\left({\mathbf{U}}_j\left({\delta }^l\gamma \right){\widehat{\mathbf{P}}}_j{\mathbf{U}}_j^T\left({\delta }^l\gamma \right)-2{\mathbf{X}}_j{\mathbf{U}}_j^T\left({\delta }^l\gamma \right)\right)C^T\right).\hfill \end{array}$$

This shows that we do not need to solve (30) for P to apply the inexact linear search with the Armijo condition.

We now outline the gradient flow method for $h_2$ model reduction of discrete-time system (1) in Algorithm 1.

Algorithm 1 Gradient flow method

Input:  the original system $\Sigma =(A,B,C)$ with $A\in {\mathbb{R}}^{n\times n},B\in {\mathbb{R}}^{n\times m},C\in {\mathbb{R}}^{p\times n}$, $c,\delta \in (0,1),\gamma >0$, and the dimension r of the reduced system. Output: : the reduced system $\widehat{\Sigma }=(\widehat{A},\widehat{B},\widehat{C})$ with $\widehat{A}\in {\mathbb{R}}^{r\times r},\widehat{B}\in {\mathbb{R}}^{r\times m},\widehat{C}\in {\mathbb{R}}^{p\times r}$. Choose an initial matrix $U_0\in {\mathbb{R}}^{n\times r}$ such that $U_0^T{U}_0=I_r$. Set ${\widehat{A}}_0=U_0^{T}A{U}_0$, ${\widehat{B}}_0=U_0^{T}B$, $j=0$. While not converge do Compute ${\widehat{P}}_j$, $X_j$, ${\widehat{Q}}_j$, $Y_j$ by solving the Equations (31), (32), (34) and (35) with $U=U_j$, $\widehat{P}={\widehat{P}}_j$, $X=X_j$, $\widehat{Q}={\widehat{Q}}_j$, and $Y=Y_j$. Compute $R_j$ according to (44). Compute the gradient direction $\nabla J_{U_j}=(I_n-U_j{U}_j^T)R_j$. Compute the SVD: $-\nabla J_{U_j}={\Phi }_j{\Gamma }_j{\Psi }_j^T$. Choose the step size $t_j={\delta }^l\gamma$ so that l is the smallest nonnegative integer satisfying $$J\left(U_j\right)-J\left({\mathbf{U}}_j\left({\delta }^l\gamma \right)\right)\ge c{\delta }^l\gamma ⟨\nabla J_{U_j},\nabla J_{U_j}⟩.$$ Compute $U_{j+1}=U_j{\Psi }_{j}cos\left(t_j{\Gamma }_j\right){\Psi }_j^T+{\Phi }_{j}sin\left(t_j{\Gamma }_j\right){\Psi }_j^T$. $j=j+1$. End While Compute $\widehat{A}=U_j^{T}A{U}_j$, $\widehat{B}=U_j^{T}B$, $\widehat{C}=C{U}_j$.

We note that a detailed complexity analysis of the gradient flow method for model reduction of continuous-time LTI systems is presented in [23]. With a similar analysis, it is easy to show that the computational complexity of Algorithm 1 is $\mathcal{O}\left(N(r(kn+\mathcal{N}\left(A\right))+N_{s}n{r}^2+nrm+nrp)\right)$, where k is some fixed integer; $\mathcal{N}\left(A\right)$ denotes the number of nonzero elements; N is the number of iterations; $N_s$ is the maximum number of search steps; $m,p,r$ are the input, output, and reduced dimensions, respectively; and $\mathcal{O}(kn+\mathcal{N}(A\left)\right)$ is the complexity of iterative methods like GMRES for solving linear systems with a coefficient matrix $\lambda A-I_n$. The computational complexity of Algorithms 2 and 3 can be analyzed similarly.

3.2. Solving the Optimization Problem via Sequentially Quadratic Approximation

Sequentially quadratic approximation [41] is one of the most known methods in the optimization field. In [23], Xu and Zeng applied this technique to minimization problem (42), which arises in the $H_2$ model reduction of continuous-time systems. In this subsection, we extend sequentially quadratic approximation for the $h_2$ model reduction of discrete-time systems.

Let $U_j\in {\mathbb{R}}^{n\times r}$, with $U_j^T{U}_j=I_r$, be the j-th projection matrix generated in the iteration. In sequentially quadratic approximation, the key step is to approximate the cost $J\left(U\right)$, given in (42), by the following quadratic function:

$$\widehat{J}\left(U\right)=\mathrm{trace}\left(C^{T}C\left(P+U{\widehat{P}}_j{U}^T-2X_j{U}^T\right)\right),$$

(48)

where ${\widehat{P}}_j$ and $X_j$ are the solutions of Stein Equations (31) and (32) with $U=U_j$, respectively. It is clear that the partial derivative matrix of $\widehat{J}\left(U\right)$ is given by ${\widehat{J}}_U=2C{C}^T\left(U{\widehat{P}}_j-X_j\right)$. By setting ${\widehat{J}}_U=0$, we get ${\widehat{U}}_{j+1}=X_j{\widehat{P}}_j^{-1}$ if ${\widehat{P}}_j$ is invertible. Now, we can define a possible search direction ${\Delta }_j$ by projecting the difference ${\widehat{U}}_{j+1}-U_j$ to the tangent space of $\mathbf{Gr}(r,n)$ at point $\left[U_j\right]$; i.e., ${\Delta }_j$ is defined by

$${\Delta }_j=(I-U_j{U}_j^T)({\widehat{U}}_{j+1}-U_j)=(I-U_j{U}_j^T){\widehat{U}}_{j+1}.$$

In order to generate a gradient-related sequence $\left\{U_j\right\}$, it is proposed in [41] to select the new search $S_{U_j}$ for obtaining $U_{j+1}$ according to the following rule: Given constants $c_1>0$ and $c_2\in (-1,0)$, if

$$\parallel {\Delta }_j\parallel >c_1\mathrm{and}{\displaystyle \frac{⟨\nabla J_{U_j},{\Delta }_j⟩}{\parallel \nabla J_{U_j}\parallel ·\parallel {\Delta }_j\parallel }}<c_2,$$

then the search direction is $S_{U_j}={\Delta }_j$; otherwise, the search direction is the negative gradient direction of $J\left(U\right)$ at $U_j$, i.e., $S_{U_j}=-\nabla J_{U_j}$. With the SVD of the new search direction, $S_{U_j}={\Phi }_j{\Gamma }_j{\Psi }_j^T$, the $(j+1)$-iterate $U_{j+1}$ is generated according to iteration scheme (46) with the step size $t_j={\delta }^l\gamma$ satisfying the Armijo rule in (47).

We summarize the sequentially quadratic approximation method for the $h_2$-optimal model reduction of discrete-time system (1) in Algorithm 2.

Algorithm 2 Sequentially quadratic approximation method

Input:  the original system $\Sigma =(A,B,C)$ with $A\in {\mathbb{R}}^{n\times n},B\in {\mathbb{R}}^{n\times m},C\in {\mathbb{R}}^{p\times n}$, $c,\delta \in (0,1),\gamma >0$, $c_1>0$, $c_2\in (-1,0)$, and the dimension r of the reduced system. Output:  the reduced system $\widehat{\Sigma }=(\widehat{A},\widehat{B},\widehat{C})$ with $\widehat{A}\in {\mathbb{R}}^{r\times r},\widehat{B}\in {\mathbb{R}}^{r\times m},\widehat{C}\in {\mathbb{R}}^{p\times r}$. Choose an initial matrix $U_0\in {\mathbb{R}}^{n\times r}$ such that $U_0^T{U}_0=I_r$. Set ${\widehat{A}}_0=U_0^{T}A{U}_0$, ${\widehat{B}}_0=U_0^{T}B$, $j=0$. While not converge do Compute ${\widehat{P}}_j$, $X_j$, ${\widehat{Q}}_j$, $Y_j$ by solving the Equations (31), (32), (34) and (35) with $U=U_j$, $\widehat{P}={\widehat{P}}_j$, $X=X_j$, $\widehat{Q}={\widehat{Q}}_j$, and $Y=Y_j$. Compute $R_j$ according to (44). Compute the gradient direction $\nabla J_{U_j}=(I_n-U_j{U}_j^T)R_j$. Compute ${\widehat{U}}_{j+1}=X_j{\widehat{P}}_j^{-1}$. Compute ${\Delta }_j=(I-U_j{U}_j^T){\widehat{U}}_{j+1}$. If $\parallel {\Delta }_j\parallel >c_1$ and ${\displaystyle \frac{⟨\nabla J_{U_j},{\Delta }_j⟩}{\parallel \nabla J_{U_j}\parallel ·\parallel {\Delta }_j\parallel }}<c_2$ then –     $S_{U_j}={\Delta }_j$, else –     $S_{U_j}=-\nabla J_{U_j}$. Compute the SVD: $S_{U_j}={\Phi }_j{\Gamma }_j{\Psi }_j^T$. Choose the step size $t_j={\delta }^l\gamma$ so that l is the smallest nonnegative integer satisfying $$J\left(U_j\right)-J\left({\mathbf{U}}_j\left({\delta }^l\gamma \right)\right)\ge c{\delta }^l\gamma ⟨S_{U_j},S_{U_j}⟩.$$ Compute $U_{j+1}=U_j{\Psi }_{j}cos\left(t_j{\Gamma }_j\right){\Psi }_j^T+{\Phi }_{j}sin\left(t_j{\Gamma }_j\right){\Psi }_j^T$. $j=j+1$. End While Compute $\widehat{A}=U_j^{T}A{U}_j$, $\widehat{B}=U_j^{T}B$, $\widehat{C}=C{U}_j$.

Algorithm 3 Two-sided projection method

Input:  the full system $\Sigma =(A,B,C)$ with $A\in {\mathbb{R}}^{n\times n},B\in {\mathbb{R}}^{n\times m},C\in {\mathbb{R}}^{p\times n}$, $c,\delta \in (0,1),\gamma >0$, and the dimension r of the reduced system. Output:  the reduced system $\widehat{\Sigma }=(\widehat{A},\widehat{B},\widehat{C})$ with $\widehat{A}\in {\mathbb{R}}^{r\times r},\widehat{B}\in {\mathbb{R}}^{r\times m},\widehat{C}\in {\mathbb{R}}^{p\times r}$. Choose the initial projection matrices $W_0,V_0$. Set ${\widehat{A}}_0=W_0^{T}A{V}_0$, ${\widehat{B}}_0=W_0^{T}B$, $j=0$. While not converge do Solve the Stein Equations (66) and (67) for ${\widehat{P}}_j$ and $X_j$, respectively. Compute ${\widehat{V}}_{j+1}=X_j{\widehat{P}}_j^{-1}$. Compute the search direction $S_{V_j}=\left(I-V_j{\left(V_j^T{V}_j\right)}^{-1}{V}_j^T\right){\widehat{V}}_{j+1}$. Find the smallest positive number l so that $$J(W_j,V_j)-J(W_j,{\mathbf{V}}_j\left({\delta }^l\gamma \right))\ge c{\delta }^l\gamma ⟨S_{V_j},S_{V_j}⟩,$$ Set $V_{j+1}={\mathbf{V}}_j\left({\delta }^l\gamma \right)=\left(V_j+{\delta }^l\gamma S_{V_j}\right){\left(W_j^T\left(V_j+{\delta }^l\gamma S_{V_j}\right)\right)}^{-1}$. Solve the Stein Equations (68) and (69) for ${\widehat{Q}}_j$ and $Y_j$, respectively. Compute ${\tilde{W}}_{j+1}=-Y_j{\widehat{Q}}_j^{-1}$. Compute the search direction $S_{W_j}=\left(I-W_j{\left(W_j^T{W}_j\right)}^{-1}{W}_j^T\right){\tilde{W}}_{j+1}$. Find the smallest positive number l so that $$J(W_j,V_{j+1})-J({\mathbf{W}}_j\left({\delta }^l\gamma \right),V_{j+1})\ge c{\delta }^l\gamma ⟨S_{W_j},S_{W_j}⟩,$$ Set $W_{j+1}={\mathbf{W}}_j\left({\delta }^l\gamma \right)=\left(W_j+{\delta }^l\gamma S_{W_j}\right){\left(V_{j+1}^T\left(W_j+{\delta }^l\gamma S_{W_j}\right)\right)}^{-1}$. $j=j+1$. End While Compute $\widehat{A}=W_j^{T}A{V}_j$, $\widehat{B}=W_j^{T}B$, $\widehat{C}=C{V}_j$.

4. Two-Sided Projection via Optimization on Grassmann Manifold

Zeng and Lu [24] considered a two-sided projection for $H_2$-optimal model reduction of continuous-time linear systems. The cost is minimized over a Grassmann manifold with two project matrices. In this subsection, we will extend this two-sided projection approach to discrete-time linear system (1).

The two-sided projection based approach naturally involves a pair of biorthogonal matrices $W,V\in {\mathbb{R}}^{n\times r}$, i.e., $W^{T}V=I_r$. The coefficient matrices of the reduced-order systems can be obtained via projections as follows:

$$\widehat{A}=W^{T}AV,\widehat{B}=W^{T}B,\widehat{C}=CV.$$

(49)

In the two-sided projection framework, the cost function for the error system ${\Sigma }_e$ is given by

$$J(W,V)=\parallel H_e{\parallel }_{h_2}^2,$$

(50)

and the minimization problem to be solved is given by the problem

$$\underset{W,V\in {\mathbb{R}}^{n\times r},W^{T}V=I_r}{min}J(W,V).$$

(51)

As shown in [24], this optimal problem (51) can be equivalently formulated as the following minimization problem over a Grassmann manifold with two variable matrices:

$$\underset{\left[W\right],\left[V\right]\in \mathbf{Gr}(r,n),W^{T}V=I_r}{min}J(W,V).$$

(52)

From (26) and (27), we can express the cost $J(W,V)$ by $W,V$ as follows:

$$\begin{array}{ccc}\hfill J(W,V)& =& \mathrm{trace}\left(C^{T}C(P+V\widehat{P}{V}^T-2X{V}^T)\right),\hfill \end{array}$$

(53)

$$\begin{array}{ccc}& =& \mathrm{trace}\left(B{B}^T(Q+W\widehat{Q}{W}^T+2Y{W}^T)\right),\hfill \end{array}$$

(54)

where $P,\widehat{P},Q,\widehat{Q},X,$ and Y satisfy

$$\begin{array}{ccc}\hfill AP{A}^T-P+B{B}^T& =& 0,\hfill \end{array}$$

(55)

$$\begin{array}{ccc}\hfill W^{T}AV\widehat{P}{V}^T{A}^{T}W-\widehat{P}+W^{T}B{B}^{T}W& =& 0,\hfill \end{array}$$

(56)

$$\begin{array}{ccc}\hfill AX{V}^T{A}^{T}W-X+B{B}^{T}W& =& 0,\hfill \end{array}$$

(57)

$$\begin{array}{ccc}\hfill A^{T}QA-Q+C^{T}C& =& 0,\hfill \end{array}$$

(58)

$$\begin{array}{ccc}\hfill V^T{A}^{T}W\widehat{Q}{W}^{T}AV-\widehat{Q}+V^T{C}^{T}CV& =& 0,\hfill \end{array}$$

(59)

$$\begin{array}{ccc}\hfill A^{T}Y{W}^{T}AV-Y-C^{T}CV& =& 0.\hfill \end{array}$$

(60)

We give the expressions of the matrices of all partial derivatives of J with respect to V and W in the following theorem, the proof of which is similar to that of Theorem 1 and is thus omitted.

Theorem 2. 

The matrices $J_V,J_W$ of all partial derivatives of J with respect to $V,W$ can be expressed as

$$\begin{array}{ccc}\hfill J_V& =& 2C^{T}C\left(V\widehat{P}-X\right)+2A^{T}W\left(Y^{T}AX+\widehat{Q}{W}^{T}AV\widehat{P}\right),\hfill \end{array}$$

(61)

$$\begin{array}{ccc}\hfill J_W& =& 2B{B}^T\left(W\widehat{Q}+Y\right)+2AV\left(X^T{A}^{T}Y+\widehat{P}{V}^T{A}^{T}W\widehat{Q}\right),\hfill \end{array}$$

(62)

where $\widehat{P}$, X, $\widehat{Q}$, and Y are the solutions of (56), (57), (59), and (60), respectively.

With the help of the matrices $J_W,J_V$ of all partial derivatives of $J(W,V)$, from [24] (Theorem 1), the gradients $\nabla J_V$ and $\nabla J_W$ of J at points $\left[V\right]$ and $\left[W\right]$ of Grassmann manifold $\mathbf{Gr}(r,n)$ are explicitly written as

$$\begin{array}{ccc}\hfill \nabla J_V& =& J_V-V{\left(V^{T}V\right)}^{-1}{V}^T{J}_V,\hfill \end{array}$$

(63)

$$\begin{array}{ccc}\hfill \nabla J_W& =& J_W-W{\left(W^{T}W\right)}^{-1}{W}^T{J}_W,\hfill \end{array}$$

(64)

respectively, where $J_V,J_W$ are given by (61) and (62), respectively.

So, for the two-sided projection, $V,W$ must satisfy the necessary conditions

$$\nabla J_V=0,\nabla J_W=0,V^{T}W=I.$$

Solving the Optimization Problem via an Alternating Direction Approach

Now we consider applying the alternating direction algorithm with sequentially quadratic approximation of the cost $J(W,V)$ to solve minimization problem (52).

Suppose that for $j=0,1,2,\dots$, $W_j$ and $V_j$ are the known projection matrices. Firstly, we approximate the cost $J(W,V)$, expressed by (53), in the V-direction by the following quadratic function:

$$\widehat{J}\left(V\right)=\mathrm{trace}\left(C^{T}C\left(P+V{\widehat{P}}_j{V}^T-2X_j{V}^T\right)\right),$$

(65)

where ${\widehat{P}}_j$ and $X_j$ are the solutions of the Stein equations

$$\begin{array}{ccc}\hfill {\widehat{A}}_j{\widehat{P}}_j{\widehat{A}}_j^T-{\widehat{P}}_j+{\widehat{B}}_j{\widehat{B}}_j^T& =& 0,\hfill \end{array}$$

(66)

$$\begin{array}{ccc}\hfill A{X}_j{\widehat{A}}_j^T-X_j+B{\widehat{B}}_j^T& =& 0\hfill \end{array}$$

(67)

respectively, with ${\widehat{A}}_j=W_j^{T}A{V}_j$ and ${\widehat{B}}_j=W_j^{T}B$. It is not difficult to see that the partial derivative ${\widehat{J}}_V$ of the quadratic function $\widehat{J}$ can be formulated as

$${\widehat{J}}_V=2C{C}^T\left(V{\widehat{P}}_j-X_j\right).$$

By setting ${\widehat{J}}_V=0$, we can obtain a new matrix ${\widehat{V}}_{j+1}=X_j{\widehat{P}}_j^{-1}$ if ${\widehat{P}}_j$ is invertible. However, we cannot use ${\widehat{V}}_{j+1}$ as a projection matrix since ${\widehat{V}}_{j+1}$ does not usually satisfy $W_j^T{\widehat{V}}_{j+1}=I_r$. In [24], an approach is proposed to construct the $(j+1)$-th iterate $V_{j+1}$ based on the difference ${\widehat{V}}_{j+1}-V_j$. We outline the scheme as follows:

• Firstly, by projecting ${\widehat{V}}_{j+1}-V_j$ onto the tangent space of $\mathbf{Gr}(r,n)$ at point $\left[V_j\right]$, a search direction $S_{V_j}$ is generated by

  $$S_{V_j}=\left(I-V_j{\left(V_j^T{V}_j\right)}^{-1}{V}_j^T\right){\widehat{V}}_{j+1}.$$
• Secondly, we define the search matrix ${\mathbf{V}}_j\left(t\right)$, which should satisfy $W_j^T{\mathbf{V}}_j\left(t\right)=I_r$, by

  $${\mathbf{V}}_j\left(t\right)=\left(V_j+t{S}_{V_j}\right){\left(W_j^T\left(V_j+t{S}_{V_j}\right)\right)}^{-1},$$

  where t denotes the step length parameter.
• Finally, the $(j+1)$-th iterate $V_{j+1}$ is constructed by the inexact line search with the Armijo rule. That is, for some given $c,\delta \in (0,1),\gamma >0$, we find the smallest positive number l so that ${\mathbf{V}}_j\left({\delta }^l\gamma \right)$ satisfies the adequate reduction condition

  $$J(W_j,V_j)-J(W_j,{\mathbf{V}}_j\left({\delta }^l\gamma \right)\ge c{\delta }^l\gamma ⟨S_{V_j},S_{V_j}⟩,$$

  and then set $V_{j+1}={\mathbf{V}}_j\left({\delta }^l\gamma \right)=\left(V_j+{\delta }^l\gamma S_{V_j}\right){\left(W_j^T\left(V_j+{\delta }^l\gamma S_{V_j}\right)\right)}^{-1}$.

After obtaining the projection matrix $V_{j+1}$, the $(j+1)$ iterate $W_{j+1}$ for W is constructed in an analogous way. From (54), the quadratic function $\tilde{J}\left(W\right)$, which approximates $J(W,V)$ at point $(W_j,V_{j+1})$, is defined by

$$\tilde{J}\left(W\right)=\mathrm{trace}\left(B{B}^T(Q+W{\widehat{Q}}_j{W}^T+2Y_j{W}^T)\right)$$

where ${\widehat{Q}}_j$ and $Y_j$ satisfy

$$\begin{array}{ccc}\hfill {\tilde{A}}_j^T{\widehat{Q}}_j{\tilde{A}}_j-{\widehat{Q}}_j+{\widehat{C}}_j^T{\widehat{C}}_j& =& 0,\hfill \end{array}$$

(68)

$$\begin{array}{ccc}\hfill A^T{Y}_j{\tilde{A}}_j-Y_j+C^T{\widehat{C}}_j& =& 0\hfill \end{array}$$

(69)

respectively, with ${\tilde{A}}_j=W_j^{T}A{V}_{j+1}$ and ${\widehat{C}}_j=C{V}_{j+1}$. Similarly, by setting ${\tilde{J}}_W=B{B}^T\left(Y_j+W{\widehat{Q}}_j\right)=0$, we obtain ${\tilde{W}}_{j+1}=-Y_j{\widehat{Q}}_j^{-1}$. Following the steps in the construction of $V_{j+1}$, the new projection matrix $W_{j+1}$ is established as follows:

• Construct the search direction $S_{W_j}$ by

  $$S_{W_j}=\left(I-W_j{\left(W_j^T{W}_j\right)}^{-1}{W}_j^T\right){\tilde{W}}_{j+1}.$$
• Define the search matrix ${\mathbf{W}}_j\left(t\right)$ by

  $${\mathbf{W}}_j\left(t\right)=\left(W_j+t{S}_{W_j}\right){\left(V_{j+1}^T\left(W_j+t{S}_{W_j}\right)\right)}^{-1}.$$
• For some given $c,\delta \in (0,1),\gamma >0$, find the smallest positive number l so that

  $$J(W_j,V_{j+1})-J({\mathbf{W}}_j\left({\delta }^l\gamma \right),V_{j+1})\ge c{\delta }^l\gamma ⟨S_{W_j},S_{W_j}⟩,$$

  and then set $W_{j+1}={\mathbf{W}}_j\left({\delta }^l\gamma \right)=\left(W_j+{\delta }^l\gamma S_{W_j}\right){\left(V_{j+1}^T\left(W_j+{\delta }^l\gamma S_{W_j}\right)\right)}^{-1}$.

In summary, we propose a two-sided projection method based on Grassmann manifold optimization for $h_2$ model reduction of discrete-time system (1), as presented in Algorithm 3.

5. Implementation Issues

In this section, we consider some issues relative to the implementation of the algorithms proposed in this paper, including how to choose the initial projection matrices and the termination criterion and select suitable methods for the Stein equations.

5.1. Initial Projection Matrix Selection

For the algorithms proposed in this paper, the first possible choice is to construct initial projection matrices randomly. For one-sided projection, after generating the $n\times r$ random matrix, we need to compute its QR decomposition to obtain an orthogonal initial matrix $U_0$. For the two-sided projection method, we first construct two $n\times r$ random matrices $W_0,V_0$ and then set $V_0:=V_0{\left(W_0^T{V}_0\right)}^{-1}$ so that the biorthogonal condition $W_0^T{V}_0=I_r$ holds.

Secondly, we can choose the initial projection matrices so that the subspaces spanned by them are the block Krylov subspace or the rational block Krylov subspace:

$$\begin{array}{ccc}& & \mathrm{span}\left\{B,AB,\dots ,\dots ,A^{l-1}B\right\},\hfill \\ & & \mathrm{span}\left\{{({\sigma }_1{I}_n-A)}^{-1}B,{({\sigma }_2{I}_n-A)}^{-1}B,\dots ,\dots ,{({\sigma }_l{I}_n-A)}^{-1}B\right\}.\hfill \end{array}$$

Recall that the first-order necessary conditions for $h_2$ model reduction of a discrete-time system state that the optimal reduced system interpolates the full system at the mirror images with respect to the unit circle of the poles of the reduced system; see, for example, ref. [35]. So the reciprocal of the approximate eigenvalue or the Ritz value of A is the suitable choice of shift parameters ${\sigma }_i$ for the rational block Krylov subspace. In numerical experiments, we will test random selection and the choice of the block Krylov subspace and the rational block Krylov subspace.

5.2. Termination Criterion

As we know, in fact, the algorithms proposed in this paper are developed for solving optimization problems over Grassmann manifolds. So we should stop these algorithms if the norm of the gradient of the cost function J at some approximate solution is small.

For one-sided projection methods such as Algorithms 1 and 2, the cost J is defined based on one matrix variable U and is given in (42). Therefore, the iteration should be naturally terminated if the gradient at point $U_j$ satisfies

$$\parallel \nabla J_{U_j}\parallel <\epsilon ,$$

where $\epsilon$ is a prescribed tolerance and $\nabla J_{U_j}=(I_n-U_j{U}_j^T)R_j$, with $R_j$ given by (44).

For two-sided projection, the cost J has two matrix variables $W,V$. It is known that if $(W,V)$ is the solution of minimization problem (52), the gradients at W and V should satisfy $\nabla J_W=\nabla J_V=0$. So the stopping rule for Algorithm 3 can be defined by

$$\sqrt{\parallel \nabla J_{W_j}{\parallel }^2+{\parallel \nabla J_{V_j}\parallel }^2}<\epsilon ,$$

where $\nabla J_{V_j}$ and $\nabla J_{W_j}$ are computed by using (63) and (64), respectively.

5.3. Solving Stein Equations

In this subsection, we consider the numerical solution of two classes of Stein equations appearing in our algorithms.

The Stein equation of the type shown in (66), called the discrete-time Lyapunov equation, has a small dimension r. This kind of equations can be solved by the standard direct method provided in [42], which is a direct extension of the well-known Bartels–Stewart algorithm [43] for continuous Lyapunov equations. Note that in order to obtain the real solution, the real Schur decomposition [13] should be used. In [44], low-rank methods were proposed for solving discrete-time projected Lyapunov equations. For a comprehensive review of numerical methods for solving matrix equations, we refer to [45].

We now consider how to solve Stein equations like (67), which are also known as discrete-time Sylvester equations. We point out that this kind of matrix equations cannot be solved by a Bartels–Stewart-like algorithm. The reason is that the matrix A has a large dimension n, so computing the real Schur decomposition of A requires $\mathcal{O}\left(n^3\right)$ time complexity.

For simplicity of notation, we drop the index j in (67). This kind of equation is now of the form

$$\begin{array}{ccc}\hfill AX{\widehat{A}}^T-X+B{\widehat{B}}^T& =& 0\hfill \end{array}$$

(70)

with $A\in {\mathbb{R}}^{n\times n}$, $\widehat{A}\in {\mathbb{R}}^{r\times r}$, $X\in {\mathbb{R}}^{n\times r}$, $B\in {\mathbb{R}}^{n\times m}$, and $\widehat{B}\in {\mathbb{R}}^{m\times r}$. Usually, A is sparse, and $\widehat{A}$ is dense. So, this class of matrix equations can be called the sparse–dense discrete-time Sylvester equations. We note that in [46], a method based on Schur decomposition of the dense matrix is proposed for solving sparse–dense continuous-time Sylvester equations, which arise in $H_2$ model order reduction of continuous-time LTI systems.

Here, we will use a variant of the real Bartels–Stewart algorithm to obtain the real solution of (70). Let ${\widehat{A}}^T=Q\Lambda Q^T$ be the real Schur decomposition of ${\widehat{A}}^T$ with

$$\Lambda =\left[\begin{array}{cccc}{\Lambda }_{11}& {\Lambda }_{12}& \dots & {\Lambda }_{1l}\\ 0& {\Lambda }_{22}& \ddots & ⋮\\ ⋮& ⋮& \ddots & ⋮\\ 0& 0& \dots & {\Lambda }_{ll}\end{array}\right],$$

where ${\Lambda }_{ii}$ is a real number or a $2\times 2$ real matrix having a pair of complex conjugate eigenvalues. With this decomposition, Equation (70) is transformed to the equivalent equation

$$\begin{array}{ccc}\hfill A\tilde{X}\Lambda -\tilde{X}+B{\tilde{B}}^T& =& 0,\hfill \end{array}$$

(71)

where $\tilde{X}=XQ$ and ${\tilde{B}}^T={\widehat{B}}^{T}Q$. We partition $\tilde{X}$ and ${\tilde{B}}^T={\widehat{B}}^{T}Q$ as

$$\begin{array}{ccc}\hfill \tilde{X}& =& \left[{\tilde{X}}_1,{\tilde{X}}_2,\dots ,{\tilde{X}}_l\right],\hfill \\ \hfill {\tilde{B}}^T& =& \left[{\tilde{B}}_1^T,{\tilde{B}}_2^T,\dots ,{\tilde{B}}_l^T\right],\hfill \end{array}$$

where ${\tilde{X}}_i$ and ${\tilde{B}}_1^T$ have the same number of columns of ${\Lambda }_{ii}$. From (71) and the above partition, we have

$$A{\tilde{X}}_i{\Lambda }_{ii}-{\tilde{X}}_i=-B{\tilde{B}}_i^T-A\sum _{k=1}^{i-1}{\tilde{X}}_k{\Lambda }_{ki},1\le i\le l$$

(72)

with ${\Lambda }_{0i}=0$.

If ${\Lambda }_{ii}$ is a number, it follows from (72) that

$${\tilde{X}}_i=-{({\Lambda }_{ii}A-I_n)}^{-1}\left(B{\tilde{B}}_i^T+A\sum _{k=1}^{i-1}{\tilde{X}}_k{\Lambda }_{ki}\right).$$

(73)

Now we consider the case where ${\Lambda }_{ii}$ is a $2\times 2$ real matrix. Let ${\lambda }_i\in \mathbb{C}$ be one eigenvalue of ${\Lambda }_{ii}$ with the corresponding eigenvector ${\phi }_i\in {\mathbb{C}}^2$. Then, $({\overline{\lambda }}_i,{\overline{\phi }}_i)$ is also an eigenpair of ${\Lambda }_{ii}$. Thus, we have

$${\Lambda }_{ii}\left[\begin{array}{cc}{\phi }_i& {\overline{\phi }}_i\end{array}\right]=\left[\begin{array}{cc}{\phi }_i& {\overline{\phi }}_i\end{array}\right]\left[\begin{array}{cc}{\lambda }_i& 0\\ 0& {\overline{\lambda }}_i\end{array}\right].$$

(74)

Multiplying (72) by $\left[\begin{array}{cc}{\phi }_i& {\overline{\phi }}_i\end{array}\right]$ the right-hand side and using (74) lead to

$$({\lambda }_{i}A-I_n){\tilde{X}}_i{\phi }_i=-\left(B{\tilde{B}}_i^T+A\sum _{k=1}^{i-1}{\tilde{X}}_k{\Lambda }_{ki}\right){\phi }_i.$$

Define

$${\psi }_i:={\tilde{X}}_i{\phi }_i=-{({\lambda }_{i}A-I_n)}^{-1}\left(B{\tilde{B}}_i^T+A\sum _{k=1}^{i-1}{\tilde{X}}_k{\Lambda }_{ki}\right){\phi }_i.$$

(75)

Then, we have

$$\left[\begin{array}{cc}{\psi }_i& {\overline{\psi }}_i\end{array}\right]={\tilde{X}}_i\left[\begin{array}{cc}{\phi }_i& {\overline{\phi }}_i\end{array}\right],$$

i.e.,

$$\left[\begin{array}{cc}\mathrm{real}\left({\psi }_i\right)& \mathrm{imag}\left({\psi }_i\right)\end{array}\right]\left[\begin{array}{cc}1& 1\\ \mathsf{ȷ}& -\mathsf{ȷ}\end{array}\right]={\tilde{X}}_i\left[\begin{array}{cc}\mathrm{real}\left({\phi }_i\right)& \mathrm{imag}\left({\phi }_i\right)\end{array}\right]\left[\begin{array}{cc}1& 1\\ \mathsf{ȷ}& -\mathsf{ȷ}\end{array}\right],$$

where $\mathrm{real}(·)$ and $\mathrm{imag}(·)$ denote the real and imaginary parts of a complex number or a complex matrix, respectively, and ȷ is the imaginary unit.

So, we obtain a real formula for ${\tilde{X}}_i$:

$${\tilde{X}}_i=\left[\begin{array}{cc}\mathrm{real}\left({\psi }_i\right)& \mathrm{imag}\left({\psi }_i\right)\end{array}\right]{\left[\begin{array}{cc}\mathrm{real}\left({\phi }_i\right)& \mathrm{imag}\left({\phi }_i\right)\end{array}\right]}^{-1}.$$

(76)

We summarize the framework of this method for solving the discrete-time Sylvester Equation (70) as follows:

• Compute the real Schur decomposition ${\widehat{A}}^T=Q\Lambda Q^T$, and set ${\tilde{B}}^T={\widehat{B}}^{T}Q$.
• For $1\le i\le l$, if ${\Lambda }_{ii}$ is a number, compute ${\tilde{X}}_i$ by (73); otherwise, compute one eigenpiar $({\lambda }_i,{\phi }_i)$ of ${\Lambda }_{ii}$, compute ${\psi }_i$ by (75), and then compute ${\tilde{X}}_i$ by (76).
• Finally, compute $X=\tilde{X}{Q}^T$.

We remark that this approach involves complex arithmetic and the storage of complex matrices only in computing ${\psi }_i$ and thus avoids complex arithmetic and the storage of complex matrices as much as possible. Moreover, for a $2\times 2$ block ${\Lambda }_{ii}$, only one system of linear equations needs to be solved. The main computation in this method and thus in the algorithms proposed in this paper for model reduction is the solution of linear systems with a coefficient matrix $\lambda A-I_n$. Thus, sparse direct solvers or Krylov subspace methods like GMRES or QMR [13] can be applied, which makes these model reduction methods suitable for large-scale problems.

6. Numerical Experiments

In this section, we present two examples to illustrate the performance of $h_2$-optimization over Grassmann manifolds for the model order reduction of linear discrete-time systems. We compare Algorithms 1–3 with balanced truncation (BT) and discrete-time IRKA. We note that the step size $t_j={\delta }^l\gamma$ in Algorithms 1–3 is chosen to satisfy the Armijo rules. So, stability is preserved if the full-order system is stable (see [23]). However, IRKA may not necessarily generate stable reduced systems even if the original system is stable. The numerical experiments are carried out in Matlab 2022a on an Intel i5 processor with 2.3 GHz and 16 GB memory. The $h_2$ relative error is defined by

$$\mathrm{rerr}={\displaystyle \frac{\parallel H-\widehat{H}{\parallel }_2}{{\parallel H\parallel }_2}},$$

where H and $\widehat{H}$ are the transfer functions of the full system and the reduced system, respectively. In the numerical examples, we make the following parameter choices: $c=0.5$, $\delta ={10}^{-3}$, $\gamma =3000$, $c_1={10}^{-11}$, and $c_2=-{10}^{-7}$.

6.1. Example 1

In the first numerical example, the linear discrete-time system is obtained from the single-input single-output ISS-1R module (cf. [47]), which is a continuous-time model of the international space station. We discretize this model by using a semi-explicit Euler and a semi-implicit Euler method. The discrete-time model is a system with dimension $n=270$. The numerical results are presented in Table 1. Table 1 shows that of all three algorithms proposed in this paper, the two-sided projection method (Algorithm 3) has the least relative errors for the reduced orders $r=5,10$. Moreover, Algorithm 3 and IRKA have almost the same relative errors and are slightly better than BT. In this numerical example, the algorithms tested in this paper have preserved the stability of the original system.

Table 1. $h_2$ relative errors of Example 1.

Method	$\mathit{r}=5$	$\mathit{r}=10$
Algorithm 1	$5.07\times {10}^{-1}$	$7.62\times {10}^{-2}$
Algorithm 2	$4.79\times {10}^{-1}$	$7.41\times {10}^{-2}$
Algorithm 3	$8.12\times {10}^{-2}$	$3.24\times {10}^{-3}$
BT	$1.22\times {10}^{-1}$	$5.36\times {10}^{-3}$
IRKA	$8.10\times {10}^{-2}$	$3.25\times {10}^{-3}$

6.2. Example 2

In the second numerical example, we discretize the continuous-time CD player model (cf. [47]) by using the same scheme as in the first example to generate a discrete-time system. The discrete-time model is a system with dimension $n=120$ and has 2 inputs and 2 outputs. The $h_2$ relative errors are given in Table 2. It is clear from Table 2 that all three algorithms proposed in this paper are effective. For this example, Algorithm 3 and IRKA have also almost the same relative errors and are slightly better than BT. We point out that Algorithm 3 preserves stability in the reduced system, but IRKA does not preserve stability in this numerical example.

Table 2. $h_2$ relative errors of Example 2.

Method	$\mathit{r}=5$	$\mathit{r}=10$
Algorithm 1	$3.06\times {10}^{-1}$	$3.52\times {10}^{-3}$
Algorithm 2	$2.79\times {10}^{-1}$	$3.41\times {10}^{-3}$
Algorithm 3	$5.17\times {10}^{-2}$	$6.55\times {10}^{-4}$
BT	$5.74\times {10}^{-2}$	$8.29\times {10}^{-4}$
IRKA	$5.17\times {10}^{-2}$	$6.54\times {10}^{-4}$

7. Conclusions

In this paper, we have investigated $h_2$-optimal model reduction over Grassmann manifolds for discrete-time time-invariant systems. We have presented necessary conditions for optimization problems over Grassmann manifolds. We have considered two kinds of projections: one-sided projection and two-sided projection. For one-sided projection, we apply the gradient flow method and the sequentially quadratic approximation method to solve the related optimization problem. For two-sided projection, we apply the strategies of alternating direction iteration and sequentially quadratic approximation to the minimization problem. Numerical examples are presented to demonstrate the effectiveness of the proposed algorithms.