On Some Numerical Methods for Solving Large Differential Nonsymmetric Stein Matrix Equations

Abstract

In this paper, we propose a new numerical method based on the extended block Arnoldi algorithm for solving large-scale differential nonsymmetric Stein matrix equations with low-rank right-hand sides. This algorithm is based on projecting the initial problem on the extended block Krylov subspace to obtain a low-dimensional differential Stein matrix equation. The obtained reduced-order problem is solved by the backward differentiation formula (BDF) method or the Rosenbrock (Ros) method, the obtained solution is used to build the low-rank approximate solution of the original problem. We give some theoretical results and report some numerical experiments.

1. Preliminaries

In the present paper, we are interested in the numerical solution of large-scale differential nonsymmetric Stein matrix equations on the time interval $[t_0,T_f]$ of the form:

$$\left\{\begin{array}{cc}\dot{X}\left(t\right)=AX\left(t\right)B-X\left(t\right)+E{F}^T,t\in [t_0,T_f],\left(DNSE\right);\hfill & \hfill \\ X\left(t_0\right)=X_0,\hfill \end{array}\right.$$

(1)

where A and B are real, sparse and square matrices of size $n\times n$ and $p\times p,$ respectively, and E and F are matrices of size $n\times r$ and $p\times r$, respectively, with $r\ll n,p$. The initial condition is given in a factored form as $X_0=Z_0{\tilde{Z}}_0^T\in {\mathbb{R}}^{n\times p}$ and the matrices A and B are assumed to be large and sparse.

Differential Stein matrix equations play an important role in many problems in control and filtering theory for discrete-time large-scale dynamical systems and other problems; see [1,2,3,4,5,6,7,8]. For solving the large matrix equations during the last years, several projection methods based on Krylov subspaces have been proposed, for example Sylvester matrix equations, differential Sylvester matrix equations, Riccati and others, see, e.g., [8,9,10,11,12,13,14,15,16]. The main idea employed in these methods is to project the initial problem by using extended Krylov subspaces and then apply the Petrov–Galerkin orthogonality condition.

In this work, we present an extended block Krylov method for solving large differential Stein matrix equations. Our method uses the Petrov–Galerkin projection method and the extended block Arnoldi algorithm. The problem (1) is projected onto small extended Krylov subspaces to obtain low-order differential Stein equations that are solved by a BDF method or Ros method. The approximate solution is then given as a product of matrices with low rank.

We recall the extended block Arnoldi algorithm applied to $(A,V)$ where $V\in {\mathbb{R}}^{n\times r}$. The extended block Krylov subspace associated to $(A,V)$ (see [8,9,14,15,17]) is given by:

$$\begin{array}{ccc}\hfill {\mathcal{K}}_m^e(A,V)& =& Range\{V,A^{-1}V,AV,A^{-2}V,A^{2}V,\cdots ,A^{m-1}V,A^{-m}V\})\hfill \\ & =& {\mathbb{K}}_m(A,V)+{\mathbb{K}}_m(A^{-1},A^{-1}V),\hfill \end{array}$$

where

$${\mathbb{K}}_m(A,V)=Range\{V,AV,A^{2}V,\cdots ,A^{m-1}V\}.$$

Algorithm 1 allows us to construct an orthonormal matrix ${\mathcal{V}}_m=\left[V_1,V_2,\dots ,V_m\right]$ that is a basis of the block extended Krylov subspace ${\mathcal{K}}_m^e(A,V)$. Let ${\mathcal{T}}_{m,A}$ the restriction of the matrix A to the block extended Krylov subspace ${\mathcal{K}}_m^e(A,V)$ is given by ${\mathcal{T}}_{m,A}={\mathcal{V}}_m^{T}A{\mathcal{V}}_m$. Then, we have the following relations (see [15])

$$\begin{array}{ccc}\hfill A{\mathcal{V}}_m& =& {\mathcal{V}}_m{\mathcal{T}}_{m,A}+V_{m+1}{T}_{m+1,m}^A{E}_m^T.\hfill \end{array}$$

where $E_m={[0_{2r\times 2(m-1)r},I_{2r}]}^T$ and ${\mathcal{T}}_{m,A}=\left[T_{i,j}^A\right]$.

Algorithm 1: The extended block Arnoldi algorithm (EBA)

1.  Inputs: A an $n\times n$ matrix, V an $n\times r$ matrix and m an integer.

2.  Compute the QR decomposition of $[V,A^{-1}V]$, i.e., $[V,A^{-1}V]=V_1\mathsf{\Lambda }$;

3.  Set ${\mathcal{V}}_0=\left[\right]$;

4.  For $j=1,2,...,m$

(a)  Set $V_j^{\left(1\right)}$: first r columns of $V_j$; $V_j^{\left(2\right)}$: second r columns of $V_j$

(b)  ${\mathcal{V}}_j=\left[{\mathcal{V}}_{j-1},V_j\right]$; ${\widehat{V}}_{j+1}=\left[A{V}_j^{\left(1\right)},A^{-1}{V}_j^{\left(2\right)}\right]$;

(c)  Orthogonalize ${\widehat{V}}_{j+1}$ i.e.,

*  for $i=1,2,\dots ,j$

*  ${\displaystyle H_{i,j}=V_i^T{\widehat{V}}_{j+1};}$

*  ${\displaystyle {\widehat{V}}_{j+1}={\widehat{V}}_{j+1}-V_i{H}_{i,j}}$;

*  End for

(d)  Compute the $QR$ decomposition of ${\widehat{V}}_{j+1}$, i.e., ${\widehat{V}}_{j+1}=V_{j+1}{H}_{j+1,j}$;

5.  End For.

Throughout the paper, we use the following notations. The Frobenius inner product of the matrices X and Y is defined by ${⟨X,Y⟩}_F=tr\left(X^{T}Y\right)$, where $tr\left(Z\right)$ denotes the trace of a square matrix Z. The associated norm is the Frobenius norm denoted by ${\Vert \Vert }_F$. The Kronecker product $A\otimes B=\left[a_{i,j}B\right]\in {\mathbb{R}}^{np\times np}$ where $A=\left[a_{i,j}\right]$. This product satisfies the properties: $(A\otimes B)(C\otimes D)=(AC\otimes BD)$ and ${(A\otimes B)}^T=A^T\otimes B^T$.

The rest of the paper is organized as follows. In Section 2, we derive the low-rank approximate solutions method and give some theoretical results. In Section 3, we apply the backward differentiation formula method and Rosenbrock method for solving reduced order problem. Section 4 is devoted to some numerical examples.

2. Low-Rank Approximate Solutions Method

In this section, we project the large differential equation by using the extended block Krylov subspaces ${\mathcal{K}}_m^e(A,E)$ and ${\mathcal{K}}_m^e(B^T,F)$ to obtain low-rank approximate solutions of Equation (1).

We apply the extended block Arnoldi algorithm (Algorithm 1) to the pairs $(A,E)$ and $(B^T,F)$, respectively, and to obtain orthonormal bases $\left\{V_1,...,V_m\right\}$ and $\left\{W_1,...,W_m\right\}$ and we have

$${\mathcal{T}}_{m,A}={\mathcal{V}}_m^{T}A{\mathcal{V}}_m\mathrm{and}{\mathcal{T}}_{m,B}={\mathcal{W}}_m^T{B}^T{\mathcal{W}}_m,$$

where

$${\mathcal{V}}_m=[V_1,V_2,...,V_m]\mathrm{and}{\mathcal{W}}_m=[W_1,W_2,...,W_m].$$

We then consider approximate solutions of the large differential Stein matrix Equation (1) that have the low-rank form

$$X_m\left(t\right)={\mathcal{V}}_m{Y}_m\left(t\right){\mathcal{W}}_m^T.$$

(2)

The matrix $Y_m\left(t\right)$ is determined from the following Petrov–Galerkin orthogonality condition:

$${\mathcal{V}}_m^T{R}_m\left(t\right){\mathcal{W}}_m=0,t\in [t_0,T_f],$$

(3)

where $R_m\left(t\right)$ is the residual

$$R_m\left(t\right)={\dot{X}}_m\left(t\right)-A{X}_m\left(t\right)B+X_m\left(t\right)-E{F}^T.$$

(4)

Then, from (4) and (3), we obtain the low dimensional differential Stein equation

$$\left\{\begin{array}{cc}{\dot{Y}}_m\left(t\right)={\mathcal{T}}_{m,A}{Y}_m\left(t\right){\mathcal{T}}_{m,B}^T-Y_m\left(t\right)+{\tilde{E}}_m{\tilde{F}}_m^T,\hfill & \\ Y_m\left(t_0\right)={\mathcal{V}}_m{X}_0{\mathcal{W}}_m^T,\hfill \end{array}\right.$$

(5)

where ${\tilde{E}}_m={\mathcal{V}}_m^{T}E$ and ${\tilde{F}}_m={\mathcal{W}}_m^{T}F$.

In the following result, we derive an expression for computation of the norm of the residual R, without having to compute matrix products with the large matrices A and B. This result allows us to reduce the cost in the proposed method when checking if the residual norm is less than some fixed tolerance.

Theorem 1. 

Let $X_m\left(t\right)={\mathcal{V}}_m{Y}_m\left(t\right){\mathcal{W}}_m^T$ be the approximation obtained at step m by the extended block Arnoldi method, and $Y_m\left(t\right)$ the exact solution of low dimensional differential nonsymmetric Stein Equation (5). Then, the residual $R_m\left(t\right)$ associated to $X_m\left(t\right)$ satisfies the relation

$$\parallel R_m{\left(t\right)\parallel }_F^2=\parallel T_{m+1,m}^A{\overline{Y}}_{m,1}\left(t\right){\mathcal{T}}_{m,B}^T{\parallel }_F^2+\parallel T_{m+1,m}^B{\overline{Y}}_{m,2}\left(t\right){\mathcal{T}}_{m,A}^T{\parallel }_F^2+{\parallel T_{m+1,m}^A{Y}_{m,m}\left(t\right){\left(T_{m+1,m}^B\right)}^T\parallel }_F,$$

(6)

where ${\overline{Y}}_{m,1}\left(t\right)$ is the $2r\times 2mr$ matrix corresponding to the last $2r$ rows of $Y_m\left(t\right)$, ${\overline{Y}}_{m,2}\left(t\right)=E_m^T{Y}_m{\left(t\right)}^T$ and $Y_{m,m}\left(t\right)=E_m^T{Y}_m\left(t\right)E_m$.

Proof. 

Using the relations (4) and (2), we have

$$R_m\left(t\right)={\mathcal{V}}_m{\dot{Y}}_m\left(t\right){\mathcal{W}}_m^T-A{\mathcal{V}}_m{Y}_m\left(t\right){\mathcal{W}}_m^{T}B+{\mathcal{V}}_m{Y}_m\left(t\right){\mathcal{W}}_m^T-E{F}^T.$$

By using the following relations

$$\begin{array}{c}\hfill \left\{\begin{array}{cc}A{\mathcal{V}}_m={\mathcal{V}}_{m+1}\left[\begin{array}{c}{\mathcal{T}}_{m,A}\\ T_{m+1,m}^A{E}_m^T\end{array}\right],\hfill & \\ {\mathcal{W}}_m^{T}B=\left[\begin{array}{cc}{\mathcal{T}}_{m,B}^T& E_m{\left(T_{m+1,m}^B\right)}^T\end{array}\right]{\mathcal{W}}_{m+1}^T,\hfill & \\ {\mathcal{V}}_m={\mathcal{V}}_{m+1}\left[\begin{array}{c}{I}_{2rm}\\ 0_{2r\times 2r}\end{array}\right],\hfill & \\ {\mathcal{W}}_m^T=\left[\begin{array}{cc}{I}_{2rm}& 0_{2r\times 2r}\end{array}\right]{\mathcal{W}}_{m+1}^T,\hfill & \end{array}\right.\end{array}$$

we have $R_m\left(t\right)={\mathcal{V}}_{m+1}\left[\begin{array}{c}{I}_{2rm}\\ 0_{2r\times 2r}\end{array}\right]{\dot{Y}}_m\left(t\right)\left[\begin{array}{cc}{I}_{2rm}& 0_{2r\times 2r}\end{array}\right]{\mathcal{W}}_{m+1}^T-{\mathcal{V}}_{m+1}\left[\begin{array}{c}{\mathcal{T}}_{m,A}\\ T_{m+1,m}^A{E}_m^T\end{array}\right]$

$Y_m\left(t\right)\left[\begin{array}{cc}{\mathcal{T}}_{m,B}^T& E_m{\left(T_{m+1,m}^B\right)}^T\end{array}\right]{\mathcal{W}}_{m+1}^T+{\mathcal{V}}_{m+1}\left[\begin{array}{c}{I}_{2rm}\\ 0_{2r\times 2r}\end{array}\right]Y_m\left(t\right)\left[\begin{array}{cc}{I}_{2rm}& 0_{2r\times 2r}\end{array}\right]{\mathcal{W}}_{m+1}^T-E{F}^T$, since $E{F}^T={\mathcal{V}}_{m+1}\left[\begin{array}{cc}{\tilde{E}}_m{\tilde{F}}_m^T& 0_{2mr\times 2r}\\ 0_{2r\times 2mr}& 0_{2r\times 2mr}\end{array}\right]{\mathcal{W}}_{m+1}^T$, so

$$R_m\left(t\right)={\mathcal{V}}_{m+1}\left[\begin{array}{cc}{\dot{Y}}_m\left(t\right)-{\mathcal{T}}_{m,A}{Y}_m\left(t\right){\mathcal{T}}_{m,B}^T+Y_m\left(t\right)-{\tilde{E}}_m{\tilde{F}}_m^T& -{\mathcal{T}}_{m,A}{Y}_m\left(t\right)E_m{\left(T_{m+1,m}^B\right)}^T\\ -T_{m+1,m}^A{E}_m^T{Y}_m\left(t\right){\mathcal{T}}_{m,B}^T& -T_{m+1,m}^A{E}_m^T{Y}_m\left(t\right)E_m{\left(T_{m+1,m}^B\right)}^T\end{array}\right]{\mathcal{W}}_{m+1}^T.$$

Now, as $Y_m\left(t\right)$ exact solution of the low dimensional differential Stein equation

$${\dot{Y}}_m\left(t\right)={\mathcal{T}}_{m,A}{Y}_m\left(t\right){\mathcal{T}}_{m,B}^T-Y_m\left(t\right)+{\tilde{E}}_m{\tilde{F}}_m^T,$$

so

$$R_m\left(t\right)={\mathcal{V}}_{m+1}\left[\begin{array}{cc}0& -{\mathcal{T}}_{m,A}{Y}_m\left(t\right)E_m{\left(T_{m+1,m}^B\right)}^T\\ -T_{m+1,m}^A{E}_m^T{Y}_m\left(t\right){\mathcal{T}}_{m,B}^T& -T_{m+1,m}^A{E}_m^T{Y}_m\left(t\right)E_m{\left(T_{m+1,m}^B\right)}^T\end{array}\right]{\mathcal{W}}_{m+1}^T.$$

and since ${\mathcal{V}}_{m+1}$ and ${\mathcal{W}}_{m+1}$ are orthonormal, we obtain

$$\parallel R_m{\left(t\right)\parallel }_F^2=\parallel T_{m+1,m}^A{\overline{Y}}_{m,1}\left(t\right){\mathcal{T}}_{m,B}^T{\parallel }_F^2+\parallel T_{m+1,m}^B{\overline{Y}}_{m,2}\left(t\right){\mathcal{T}}_{m,A}^T{\parallel }_F^2+{\parallel T_{m+1,m}^A{Y}_{m,m}\left(t\right){\left(T_{m+1,m}^B\right)}^T\parallel }_F.$$

where ${\overline{Y}}_{m,1}\left(t\right)=E_m^T{Y}_m\left(t\right)$, ${\overline{Y}}_{m,2}\left(t\right)=E_m^T{Y}_m{\left(t\right)}^T$ and $Y_{m,m}\left(t\right)=E_m^T{Y}_m\left(t\right)E_m$.    □

The approximate solution $X_m\left(t\right)$ can be given as a product of two matrices of low rank. Consider the singular value decomposition of the $2mr\times 2mr$ matrix:

$$Y_m\left(t\right)={\tilde{Y}}_1\Sigma {\tilde{Y}}_2^T,$$

where $\Sigma$ is the diagonal matrix of the singular values of $Y_m$ sorted in decreasing order. Let $Y_{1,l}$ and $Y_{2,l}$ be the $2mr\times l$ matrices of the first l columns of ${\tilde{Y}}_1$ and ${\tilde{Y}}_2,$ respectively, corresponding to the l singular values of magnitude greater than some tolerance. We obtain the truncated singular value decomposition:

$$Y_m\approx U_{1,l}{\Sigma }_l{U_{2,l}}^T,$$

where ${\Sigma }_l=\mathrm{diag}[{\sigma }_1,\dots ,{\sigma }_l]$. Setting $Z_{1,m}={\mathcal{V}}_m{U}_{1,l}{\Sigma }_l^{1/2}$, and $Z_{2,m}={\mathcal{W}}_m{U}_{2,l}{\Sigma }_l^{1/2},$ it follows that:

$$X_m\approx Z_{1,m}\left(t\right)Z_{2,m}^T\left(t\right).$$

(7)

This is very important for large problems when one does not need to compute and store the approximation $X_m$ at each iteration.

The following result shows that the approximation $X_m\left(t\right)$ is an exact solution of a perturbed differential Stein equation.

Theorem 2. 

Let $X_m\left(t\right)$ be the approximate solution given by (2). Then, we have

$${\dot{X}}_m\left(t\right)=(A-F_{m,A})X_m\left(t\right)(B-F_{m,B})-X_m\left(t\right)+E{F}^T,$$

(8)

where

$$\left\{\begin{array}{cc}{F}_{m,A}=V_{m+1}{T}_{m+1,m}^A{E}_m^T{\mathcal{V}}_m^T,\hfill & \hfill \\ F_{m,B}={\mathcal{W}}_m{E}_m{\left(T_{m+1,m}^B\right)}^T{W}_{m+1}^T.\hfill & \hfill \end{array}\right.$$

Proof. 

By multiplying the (5) left by ${\mathcal{V}}_m$ and right by ${{\mathcal{W}}_m}^T$, we obtain

$${\mathcal{V}}_m{\dot{Y}}_m\left(t\right){{\mathcal{W}}_m}^T={\mathcal{V}}_m{\mathcal{T}}_{m,A}{Y}_m\left(t\right){\mathcal{T}}_{m,B}^T{\mathcal{W}}_m^T-{\mathcal{V}}_m{Y}_m\left(t\right){\mathcal{W}}_m^T+{\mathcal{V}}_m{\tilde{E}}_m{\tilde{F}}_m^T{\mathcal{W}}_m^T.$$

Using relationships

$$\left\{\begin{array}{c}A{\mathcal{V}}_m={\mathcal{V}}_m{\mathcal{T}}_{m,A}+V_{m+1}{T}_{m+1,m}^A{E}_m^T,\hfill \\ {\mathcal{W}}_m^{T}B={\mathcal{T}}_{m,B}^T{\mathcal{W}}_m^T+E_m{\left(T_{m+1,m}^B\right)}^T{W}_{m+1}^T,\hfill \end{array}\right.$$

we find

$$\begin{array}{ccc}\hfill {\dot{X}}_m\left(t\right)& =& [A{\mathcal{V}}_m-V_{m+1}{T}_{m+1,m}^A{E}_m^T]Y_m\left(t\right)[{\mathcal{W}}_m^{T}B-E_m{\left(T_{m+1,m}^B\right)}^T{W}_{m+1}^T]-X_m\left(t\right)+E{F}^T\hfill \\ & =& [A-V_{m+1}(T_{m+1,m}^A{E}_m^T{\mathcal{V}}_m^T]X_m\left(t\right)[B-{\mathcal{W}}_m{E}_m{\left(T_{m+1,m}^B\right)}^T{W}_{m+1}^T]-X_m\left(t\right)+E{F}^T.\hfill \end{array}$$

So

$${\dot{X}}_m\left(t\right)=(A-F_{m,A})X_m\left(t\right)(B-F_{m,B})-X_m\left(t\right)+E{F}^T,$$

where $F_{m,A}=V_{m+1}{T}_{m+1,m}^A{V}_m^T$ and $F_{m,B}=W_m{\left(T_{m+1,m}^B\right)}^T{W}_{m+1}^T$.    □

The next result states that the error ${\mathcal{E}}_m\left(t\right)=X\left(t\right)-X_m\left(t\right)$ satisfies also a differential Stein matrix equation.

Theorem 3. 

Let $X\left(t\right)$, the exact solution of (1), and $X_m\left(t\right)$ be the low-rank approximate solution obtained at step m. The error ${\mathcal{E}}_m\left(t\right)$ satisfies the following differential Stein equation

$${\dot{\mathcal{E}}}_m\left(t\right)=A{\mathcal{E}}_m\left(t\right)B-{\mathcal{E}}_m\left(t\right)-R_m\left(t\right).$$

(9)

Proof. 

According to (1) and (4) we have

$$\begin{array}{ccc}\hfill {\dot{\mathcal{E}}}_m\left(t\right)& =& \dot{X}\left(t\right)-{\dot{X}}_m\left(t\right)\hfill \\ & =& AX\left(t\right)B-X\left(t\right)+E{F}^T-A{X}_m\left(t\right)B+X_m\left(t\right)-E{F}^T-R_m\left(t\right)\hfill \\ & =& A(X\left(t\right)-X_m\left(t\right))B-(X\left(t\right)-X_m\left(t\right))-R_m\left(t\right)\hfill \\ & =& A{\mathcal{E}}_m\left(t\right)B-{\mathcal{E}}_m\left(t\right)-R_m\left(t\right).\hfill \end{array}$$

   □

Next, we give an upper bound for the norm of the error $X-X_m$ by using the 2-logarithmic norm defined by ${\mu }_2\left(A\right)=\frac{1}{2}{\lambda }_{max}(A+A^T)$. The logarithmic norm satisfies the following relation

$$\Vert e^{tA}{\Vert }_2\le e^{{\mu }_2\left(A\right)t};t\ge 0.$$

(10)

Theorem 4. 

At step m of the extended block Arnoldi process, we have the following upper bound for the norm of the error,

$$\begin{array}{ccc}\hfill \parallel {\mathcal{E}}_m{\left(t\right)\parallel }_F& \le & e^{(t-t_0){\mu }_2\left(\mathcal{A}\right)}\parallel {\mathcal{E}}_m\left(t_0\right){\parallel }_F+\underset{\tau \in [t_0,t]}{max}{\parallel R_m\left(\tau \right)\parallel }_F\frac{e^{(t-t_0){\mu }_2\left(\mathcal{A}\right)}-1}{{\mu }_2\left(\mathcal{A}\right)}.\hfill \end{array}$$

Proof. 

We notice that the differential Stein Equation (9) is equivalent to the linear ordinary differential equation

$$\left\{\begin{array}{cc}{\dot{err}}_m\left(t\right)=\mathcal{A}er{r}_m\left(t\right)-b_m\left(t\right),\hfill & \hfill \\ er{r}_0=vec\left({\mathcal{E}}_m\left(t_0\right)\right),\hfill & \hfill \end{array}\right.$$

(11)

where

$$\left\{\begin{array}{cc}\mathcal{A}=B^T\otimes A-I_{np},\hfill & \\ er{r}_m\left(t\right)=vec\left({\mathcal{E}}_m\left(t\right)\right),\hfill & \\ b_m\left(t\right)=vec\left(R_m\left(t\right)\right),\hfill \end{array}\right.$$

Then, the error $er{r}_m\left(t\right)$ can be expressed in the integral form as follows

$$\begin{array}{ccc}\hfill er{r}_m\left(t\right)& =& e^{(t-t_0)\mathcal{A}}er{r}_0-{\int }_{t_0}^t{e}^{(t-\tau )\mathcal{A}}{b}_m\left(\tau \right)d\tau ,t\in [t_0,T_f].\hfill \end{array}$$

By using $\parallel e^{t\mathcal{A}}{\parallel }_2\le e^{{\mu }_2\left(\mathcal{A}\right)t}$, we have

$$\begin{array}{ccc}\hfill \parallel er{r}_m{\left(t\right)\parallel }_2& \le & \parallel e^{(t-t_0)\mathcal{A}}er{r}_0-{\int }_{t_0}^t{e}^{(t-\tau )\mathcal{A}}{b}_m{\left(\tau \right)d\tau \parallel }_2\hfill \\ & \le & \parallel e^{(t-t_0)\mathcal{A}}er{r}_0{\parallel }_2+{\parallel {\int }_{t_0}^t{e}^{(t-\tau )\mathcal{A}}{b}_m\left(\tau \right)d\tau \parallel }_2\hfill \\ & \le & e^{(t-t_0){\mu }_2\left(\mathcal{A}\right)}\parallel er{r}_0{\parallel }_2+{\int }_{t_0}^t{e}^{(t-\tau ){\mu }_2\left(\mathcal{A}\right)}{\parallel b_m\left(\tau \right)\parallel }_{2}d\tau \hfill \\ & \le & e^{(t-t_0){\mu }_2\left(\mathcal{A}\right)}\parallel er{r}_0{\parallel }_2+\underset{\tau \in [t_0,t]}{max}{\parallel b_m\left(\tau \right)\parallel }_2{\int }_{t_0}^t{e}^{(t-\tau ){\mu }_2\left(\mathcal{A}\right)}d\tau \hfill \\ & \le & e^{(t-t_0){\mu }_2\left(\mathcal{A}\right)}\parallel er{r}_0{\parallel }_2+\underset{\tau \in [t_0,t]}{max}{\parallel b_m\left(\tau \right)\parallel }_2\frac{e^{(t-t_0){\mu }_2\left(\mathcal{A}\right)}-1}{{\mu }_2\left(\mathcal{A}\right)}\hfill \\ & \le & e^{(t-t_0){\mu }_2\left(\mathcal{A}\right)}\parallel er{r}_0{\parallel }_2+\underset{\tau \in [t_0,t]}{max}{\parallel vec\left(R_m\left(\tau \right)\right)\parallel }_2\frac{e^{(t-t_0){\mu }_2\left(\mathcal{A}\right)}-1}{{\mu }_2\left(\mathcal{A}\right)}.\hfill \end{array}$$

As $\parallel vec\left({\mathcal{E}}_m\left(t\right)\right){\parallel }_2={\parallel {\mathcal{E}}_m\left(t\right)\parallel }_F$, so

$$\begin{array}{ccc}\hfill \parallel {\mathcal{E}}_m{\left(t\right)\parallel }_F& \le & e^{(t-t_0){\mu }_2\left(\mathcal{A}\right)}\parallel {\mathcal{E}}_m\left(t_0\right){\parallel }_F+\underset{\tau \in [t_0,t]}{max}{\parallel R_m\left(\tau \right)\parallel }_F\frac{e^{(t-t_0){\mu }_2\left(\mathcal{A}\right)}-1}{{\mu }_2\left(\mathcal{A}\right)}.\hfill \end{array}$$

   □

3. Solving the Projected Differential Stein Matrix Equation

3.1. Rosenbrock Method

In this section, we are applying the Ros method [18] for solving the projected differential Stein matrix equation. We can write the low dimensional nonsymmetric differential Stein Equation (5) in the following form

$$\left\{\begin{array}{cc}\dot{Y_m}\left(t\right)=\mathcal{S}\left(Y_m\left(t\right)\right),t\in [t_0,T_f]\hfill & \\ Y_m\left(t_0\right)=Y_m^{\left(0\right)},\hfill \end{array}\right.$$

(12)

where $\mathcal{S}\left(Y_m\right)={\mathcal{T}}_{m,A}{Y}_m{\mathcal{T}}_{m,B}^T-Y_m+{\tilde{E}}_m{\tilde{F}}_m^T$ and $Y_m^{\left(0\right)}={\mathcal{V}}_m{X}_0{\mathcal{W}}_m^T$.

The approximation $Y_{m,k+1}$ of $Y_m\left(t_{k+1}\right)$ obtained at step $k+1$ by Ros method is given by

$$Y_{k+1}=Y_k+\frac{3}{2}{P}_1+\frac{1}{2}{P}_2,$$

(13)

where $P_1$ and $P_2$ solve the following Stein matrix equations

$${\mathsf{\Lambda }}_A{P}_1{\mathsf{\Lambda }}_B^T-P_1+\mathcal{S}\left(Y_k\right)=0,$$

(14)

and

$${\mathsf{\Lambda }}_A{P}_2{\mathsf{\Lambda }}_B^T-P_2+\mathcal{S}(Y_k+P_1)-\frac{2}{h}{P}_1=0,$$

(15)

where

$$\left\{\begin{array}{cc}{\mathsf{\Lambda }}_A=\sqrt{\gamma }{\mathcal{T}}_{m,A}+\sqrt{\frac{1}{h}+1-\gamma }{I}_{size\left({\mathcal{T}}_{m,A}\right)},\hfill & \\ {\mathsf{\Lambda }}_B=\sqrt{\gamma }{\mathcal{T}}_{m,B}+\sqrt{\frac{1}{h}+1-\gamma }{I}_{size\left({\mathcal{T}}_{m,B}\right)},\hfill \end{array}\right.$$

with $\gamma =1/2.$

The Ros algorithm for solving the reduced differential Stein matrix Equation (5) is summarized in Algorithm 2.

Algorithm 2: The 2-Rosenbrock method for solving the reduced NDSE (12)

Input: ${\mathcal{T}}_{m,A},{\mathcal{T}}_{m,B},{\tilde{E}}_m,{\tilde{F}}_m,t_0,T_f$.

1.  Choose h.

2.  ompute: $N=\frac{T_f-t_0}{h}$

3.  Compute: ${\mathsf{\Lambda }}_A$

4.  Compute: ${\mathsf{\Lambda }}_B$

5.  For $k=1:N+1$

(a)  Compute $P_1$ by solving Stein matrix Equation (14).

(b)  Compute $P_2$ by solving Stein matrix Equation (15).

(c)  Compute $Y_{k+1}$ by (13).

6.  End For k.

Output: $Y_{T_f}.$

3.2. Backward Differentiation Formula Method

In this section, we present a BDF method [19] for solving, at each step m of the extended block Arnoldi process, the low dimensional differential Stein matrix Equation (5).

At each time $t_k$, let $Y_{m,k}$ be the approximation of $Y_m\left(t_k\right)$, where $Y_m$ is a solution of (5). Then, the new approximation $Y_{m,k+1}$ of $Y_m\left(t_{k+1}\right)$ obtained at step $k+1$ by BDF method is defined by the implicit relation

$$Y_{m,k+1}=\sum _{i=0}^{p-1}{\alpha }_i{Y}_{m,k-i}+h\beta \mathcal{S}\left(Y_{m,k+1}\right),$$

(16)

where $h=t_{k+1}-t_k$ is the step size, ${\alpha }_i$ and $\beta$ are the coefficients of the BDF method as listed in the Table 1.

Table 1. Coefficients of the p-step BDF method with $p\le 2$.

	p	$\mathit{\beta }$	${\mathit{\alpha }}_0$	${\mathit{\alpha }}_1$
BDF1	1	1	1	0
BDF2	2	$2/3$	$4/3$	$-1/3$

The approximate $Y_{m,k+1}$ solves the following matrix equation

$$-Y_{m,k+1}+h\beta ({\mathcal{T}}_{m,A}{Y}_{m,k+1}{\mathcal{T}}_{m,B}^T-Y_{m,k+1}+{\tilde{E}}_m{\tilde{F}}_m^T)+\sum _{i=0}^{p-1}{\alpha }_i{Y}_{m,k-i}=0.$$

We assume that at each time $t_k$, the approximation $Y_{m,k}$ is factorized as a low-rank product $Y_{m,k}\approx {\tilde{U}}_{m,k}{\tilde{V}}_{m,k}^T$, where ${\tilde{U}}_{m,k}\in {\mathbb{R}}^{n\times m_k}$ and ${\tilde{V}}_{m,k}\in {\mathbb{R}}^{p\times m_k}$, with $m_k\ll n,p$. We define

$$\left\{\begin{array}{c}{\tilde{\mathbb{T}}}_{m,A}=\sqrt{h\beta }{\mathcal{T}}_{m,A}-\sqrt{h\beta }{I}_{2m},\hfill \\ {\tilde{\mathbb{T}}}_{m,B}=\sqrt{h\beta }{{\mathcal{T}}_{m,B}}^T+\sqrt{h\beta }{I}_{2m},\hfill & \\ {\mathbb{E}}_{m,k+1}={[\sqrt{h\beta }{\tilde{E}}_m^T,\sqrt{{\alpha }_0}{\tilde{U}}_{m,k}^T,...,\sqrt{{\alpha }_{p-1}}{\tilde{U}}_{m,k+1-p}^T]}^T,\hfill & \\ {\mathbb{F}}_{m,k+1}={[\sqrt{h\beta }{\tilde{F}}_m^T,\sqrt{{\alpha }_0}{\tilde{V}}_{m,k}^T,...,\sqrt{{\alpha }_{p-1}}{\tilde{V}}_{m,k+1-p}^T]}^T.\hfill \end{array}\right.$$

Then, we obtain the following matrix Stein equation:

$${\tilde{\mathbb{T}}}_{m,A}{Y}_{m,k+1}{\tilde{\mathbb{T}}}_{m,B}-Y_{m,k+1}+{\mathbb{E}}_{m,k}{\mathbb{F}}_{m,k}^T=0.$$

(17)

The low-rank approximate solutions method by extended block Arnoldi algorithm for the large differential nonsymmetric stein matrix equations is summarized as follows in Algorithm 3.

Algorithm 3: The low-rank extended block Arnoldi method for DNSE

Input: $X_0$, choose a tolerance $tol>0$, a maximum number of $m_{max}$ iterations.

1.  For $m=,2,3,...,m_{max}$ do

2.  Update ${\mathcal{V}}_m,{\mathcal{T}}_{m,A}$, by Algorithm 1 (EBA) applied to $(A,E)$

3.  Update ${\mathcal{W}}_m,{\mathcal{T}}_{m,B}$, by Algorithm 1 (EBA) applied to $(B^T,F)$

4.  Solve the low-dimensional problem (5) by BDF method or Ros method.

5.  If $\parallel R_m{\left(t\right)\parallel }_F<tol$, stop.

6.  End If;

7.  End For (m);

8.  Using (7), the approximate solution $X_m$ is given by $X_m\approx Z_{1,m}{Z}_{2,m}^T$.

Output: $X_m$.

4. Numerical Experiments

In this section, we give some numerical examples of large nonsymmetric differential Stein matrix equations. All the experiments were performed on a computer of Intel Core i5 at 1.3 GHz and 4 GB of RAM. The Algorithm 3 were coded in Matlab2014. In all of the examples, the coefficients of the matrices E and F were random values uniformly distributed on $[0,1]$. The stopping criterion used for EBA-BDF method and EBA-Ros was $\parallel \mathcal{R}\left(X_m\right){\parallel }_F<{10}^{-10}$ or a maximum of $m_{max}=40$ iterations was achieved.

4.1. Example 1

In this first example, the matrices A and B are obtained from the centered finite difference discretization of the operators:

$$L_A\left(u\right)=\Delta u+f_1(x,y)\frac{\partial u}{\partial x}+,f_2(x,y)\frac{\partial u}{\partial y}+f(x,y)u,$$

$$L_B\left(u\right)=\Delta u+g_1(x,y)\frac{\partial u}{\partial x}+g_2(x,y)\frac{\partial u}{\partial y}+g(x,y)u,$$

on the unit square $[0,1]\times [0,1]$ with homogeneous Dirichlet boundary conditions. The number of inner grid points in each direction was $n_0$ and $p_0$ for the operators $L_A$ and $L_B$, respectively. The matrices A and B were obtained from the discretization of the operator $L_A$ and $L_B$ with the dimensions $n=n_0^2$ and $p=p_0^2$, respectively. The discretization of the operator $L_A\left(u\right)$ and $L_B\left(u\right)$ yields matrices extracted from the Lyapack package [20] using the command fdm_2d_matrix and denoted as A = fdm($n_0$,’f_1(x,y)’,’f_2(x,y)’,’f(x,y)’). In this example, $n=$ 10,000 and $p=$ 10,000, respectively, and are named as $A=\mathrm{fdm}(n_0,f_1(x,y),$ $f_2(x,y),f(x,y))$ and $B=\mathrm{fdm}(p0,g_1(x,y),g_2(x,y),g(x,y))$ with $f_1(x,y)=-e^{xy}$, $f_2(x,y)=-sin\left(xy\right)$, $f(x,y)=y^2$, $g_1(x,y)=-100e^x$, $g_2(x,y)=-12xy$ and $g(x,y)=\sqrt{x^2+y^2}$. For this experiment, we used $r=2$.

In Figure 1, we plotted the Frobenius norms of the residuals versus the number of iterations for the EBA-BDF and the EBA-Ros method.

Figure 1. EBA-BD F method and EBA-Ros method.

In Table 2, we compared the performances of the EBA-BDF method and the EBA-Ros. For both methods, we listed the residual norms, the maximum number of iteration and the corresponding execution time.

Table 2. Results for Example 1.

Test Problem	Method	Iterations	Residual Norm	Times (s)
$n=8100$, $p=4900$, $r=2$, $h=0.3$	EBA-BDF	5	2.03 × 10${}^{-11}$	0.497
	EBA-Ros	5	$1.40\times {10}^{-11}$	0.854
$n=$ 10,000, $p=4900$, $r=3$, $h=0.2$	EBA-BDF	5	$4.38\times {10}^{-12}$	1.025
	EBA-Ros	5	$5.77\times {10}^{-13}$	1.53
$n=$ 40,000, $p=$ 12,100, $r=4$, $h=0.1$	EBA-BDF	5	$8.27\times {10}^{-12}$	6.80
	EBA-Ros	5	$8.70\times {10}^{-13}$	6.85

4.2. Example 2

For the second set of experiments, we use the matrices $add32$, $pde2961$, and $thermal$ from the Harwell Boeing Collection [21]. The tolerance was set to ${10}^{-8}$ for the stop test on the residual. For the EBA-BDF and EBA-Ros methods, we used a constant timestep $h=0.1$,

In Figure 2, the matrices $A=add32$ and $B=pde2961$ with dimensions $n=4960$ and $p=2961$, respectively, and $r=2$. We plotted the Frobenius norms of the residuals $\parallel R_m\left(T_f\right){\parallel }_F$ at final time $T_f$ versus the number of extended block Arnoldi iterations for the EBA-BDF and EBA-Ros method.

Figure 2. Residual norm vs number m of extended block Arnoldi iterations.

In Figure 3, the matrices $A=\mathtt{fdm}(x+10y^2,\sqrt{2x^2+y^2},y^2-x^2)$ and $B=thermal$ with dimensions $n=$ 10,000 and $p=3456$, respectively, $h=0.2$ and $r=3$. We plotted the Frobenius norms of the residuals $\parallel R_m\left(T_f\right){\parallel }_F$ at final time $T_f$ versus the number of extended block Arnoldi iterations for the EBA-BDF and EBA-Ros method.

Figure 3. Residual norm vs number m of extended block Arnoldi iterations.

In Table 3, we list the Frobenius residual norms at final time $T_f=2$ and the corresponding CPU time for each method.

Table 3. Runtimes in seconds and the residual norms.

Test Problem	Method	CPU Time	Iterations	$\parallel {\mathit{R}}_{\mathit{m}}\left({\mathit{T}}_{\mathit{f}}\right){\parallel }_{\mathit{F}}$
$n=p=2500$$r=2$ and $h=0.2$	EBA-BDF	$11.01$ s	16	$4.57\times {10}^{-9}$
$A=add32$ and $B=pde2961$	EBA-Ros	$11.49$ s	19	$6.59\times {10}^{-9}$
$n=$ 12,100, $p=4900$	EBA-BDF	$10.77$ s	4	$7.75\times {10}^{-9}$
$p=8100$, $h=0.1$ and $r=3$	EBA-Ros	$10.78$ s	5	$6.31\times {10}^{-10}$
$B=thermal$, $n=$ 10,000, $p=3456$ and $r=3$, $h=0.2$	EBA-BDF	$6.20$ s	7	$5.87\times {10}^{-9}$
A = fdm$(x+10y^2,\sqrt{2x^2+y^2},y^2-x^2)$	EBA-Ros	$6.27$ s	8	$2.18\times {10}^{-9}$

The numerical results are promising, showing the effectiveness of the proposed methods.

5. Conclusions

We presented, in this paper, a new iterative method for computing numerical solutions for large-scale differential Stein matrix equations with low-rank right-hand sides. This approach is based on projection onto extended block Krylov subspaces with a Galerkin method. The approximate solutions are given as products of two low-rank matrices and allow for saving memory for large problems. The numerical experiments show that the proposed extended block Krylov-based method is effective for large and sparse matrices.