Efficient Reduction Algorithms for Banded Symmetric Generalized Eigenproblems via Sequentially Semiseparable (SSS) Matrices

Abstract

In this paper, a novel algorithm is proposed for reducing a banded symmetric generalized eigenvalue problem to a banded symmetric standard eigenvalue problem, based on the sequentially semiseparable (SSS) matrix techniques. It is the first time that the SSS matrix techniques are used in such eigenvalue problems. The newly proposed algorithm only requires linear storage cost and $O\left(n^2\right)$ computation cost for matrices with dimension n, and is also potentially good for parallelism. Some experiments have been performed by using Matlab, and the accuracy and stability of algorithm are verified.

1. Introduction

In this paper, we consider how to reduce the following generalized eigenvalue problem (GEP) to a standard eigenvalue problem,

$$AQ=BQ\Lambda ,$$

(1)

with banded Hermitian matrices $A,B\in {\mathbb{C}}^{n\times n}$ and B positive definite. In the real case, A and B would be symmetric instead of Hermitian. The classical way is first to compute the Cholesky factorization of $B=L{L}^H$ with a lower triangular matrix L, and then multiply (1) with $L^{-1}$, and it yields a standard eigenvalue problem,

$$L^{-1}A{L}^{-H}\times L^{H}Q=L^{H}Q\Lambda .$$

(2)

The problem with this approach is that $C:=L^{-1}A{L}^{-H}$ is dense though A and L are banded, since $L^{-1}$ is fully triangular in general. In this paper, we consider how to reduce matrix C to a symmetric banded form efficiently.

The LAPACK library [1] includes some routines for reducing the banded GEP to banded SEP, named XHBGST and XSBGST, which X denotes different precision, S (single precision), D (double precision), C (single complex precision) and Z (double complex precision). We use the real double case to introduce the main process. First, it (DPBSTF) computes a split Cholesky factorization [2] of the real symmetric banded positive definite matrix B, $B=S^{T}S$, where the leading $p\times p$ submatrix of S an upper banded matrix with bandwidth $b_B$ and the lowers $n-p$ rows form a lower banded matrix with bandwidth $b_B$. This factorization is also called the ‘twisted factorization’ [3]. Then, the routine PSBGST updates $A=X^{T}AX$, where $X=S^{-1}Q$ and Q is an orthogonal matrix chosen to preserve the bandwidth of A. The matrix S is treated as a product of elementary matrices:

$$S=S_m{S}_{m-1}\dots S_2{S}_1{S}_{m+1}\dots S_{n-1}{S}_n,$$

where $S_i$ is determined by the i-th row of S. For each value of i, the current matrix A is updated by forming $S_i^{-1}A{S}_i^{-T}$, and the introduced bulge is chased out by applying plane rotations.

Some approaches have been proposed to reduce the generalized eigenvalue problem to the tridiagonal-diagonal form [4,5], which directly reduce B to the diaognal form and A to the tridiagonal form, respectively. In 1973, Crawford [6] proposed a new reduction method for the real symmetric case with $b_A=b_B$. Different from LAPACK, the method is based on a decomposition of the matrices into $b_B\times b_B$ blocks and the bulge is removed immediately by using matrix-matrix multiplications. Lang [7] combined the good features of Crawford’s scheme with LAPACK routines, which proceeds by blocks and can also handle different bandwidths $b_B<b_A$. A distributed parallel version [8] is included in ELPA [9].

In this work we present a new reduction algorithm that is completely different from previous algorithm, and the tool that we use is the sequentially semiseparable matrix (SSS) techniques. The SSS matrix was introduced in [10,11], which is a kind of rank-structured matrices, see the Section 2. Since a symmetric banded matrix can be seen as a special block tridiagonal matrix, L is a block bidiagonal matrix and it is well-known that the inverse of L is a lower triangular SSS matrix, see the Section 2 and [12]. The matrix $C=L^{-1}A{L}^{-T}$ can be proved to be an SSS matrix, see the Section 2. Different from Crawford’s method [6] and LAPACK [1], we compute C explicitly, but express it in SSS form. The computation and storage do not increase much. Like Crawford’s method, the advantage of our approach is that the matrices are partitioned into blocks, almost all operations are (small) matrix-matrix multiplications, and some of these small matrix-matrix multiplications can be computed in parallel by using dynamic modeling. For the task-based implementation of matrix operations, we can leverage the CHAMELEON library [13,14] to implement a parallel version of our algorithm, which is one potential advantage of our algorithm. This will be our future work.

In this paper we reduce the original problem to the block tridiagonalization problem of an SSS matrix. Some fast algorithms for tridiagonalizing a diagonal plus semiseparable matrix was introduced in [15,16] which costs $O\left(n^2\right)$ flops. We generalize the tridiagonalization approach in [15] to the (block) SSS matrix case, and show how to further get its banded form, and the complexity is $O\left(n{r}^2\right)$ flops, where n is the dimension of matrix, and $r=b_A=b_B$. The disadvantage of our algorithm proposed in this work is that it requires the semi-bandwidths of A and B to be equal, $b_A=b_B$. The procedure is shown in Algorithm 1, and it works on the SSS generators of matrix $C=L^{-1}A{L}^{-H}$ and the outputs are also some small matrices. The memory and computation costs are in the same order as the algorithms in LAPACK, and our algorithm is easy to be implemented in parallel.

The following sections of this paper are organized as follows. Section 2 gives a brief introduction to semiseparable and SSS matrices, and fast matrix multiplication of two SSS matrices is also included. Section 3 describes how to express matrix C into an SSS matrix and how to recompress its generators. The banded reduction process for symmetric SSS matrix is shown in Section 4 and the complexity analysis is included. All the performance results are summarized in Section 5. Conclusions are drawn in Section 6.

2. Semiseparable and SSS Matrices

Rank structured matrices have attracted much attention in recent years. In [17], Raf Vandevril, Marc Van Barel, and Nicola, Mastronardi present a comprehensive overview of the mathematical and numerical properites of one class of these matrices: semiseparable matrices, which is the simplest case. The rank-structured matrices include $\mathcal{H}$-matrices [18,19,20,21], ${\mathcal{H}}^2$-matrices [22,23], quasiseparable matrices [24,25], semiseparable matrices [17,26], sequentially semiseparable matrices [11], hierarchically semiseparable matrices [27,28,29], etc. Current machine learning and big data analysis are research hotspots [30], and rank-structured matrix techniques can also be used in these areas [31,32,33,34].

The semiseparable structure is a matrix analog of the semiseparable integral kernels as described by Kailath in [35]. The semiseparable matrix has been referred as the inverse of unreducible tridiagonal matrix, and Green matrix, one-pair matrix, and single-pair matrix, see [36,37,38]. Semiseparable matrices appear in several types of applications, e.g., the field of integral equations, boundary value problems, Gauss-Markov process, time-varying linear systems, statistics, acoustic and electromagnetic scattering theory, rational interpolation, and so on.

The sequentially semiseparable matrices (SSS) matrices exploit the off-diagonal low-rank property: the off-diagonal blocks are represented as product of a sequence low-rank matrices. For an $n\times n$ matrix A with block partitioning

$$A=\left(\begin{array}{cccc}{A}_{11}& A_{12}& \cdots & A_{1N}\\ A_{21}& A_{22}& \cdots & A_{2N}\\ ⋮& ⋮& & ⋮\\ A_{N1}& A_{N2}& \cdots & A_{NN}\end{array}\right),$$

(3)

where $A_{ij}\in {\mathbb{R}}^{m_i\times m_j}$ and $n=m_1+\cdots +m_N$, it can be represented by

$$A_{ij}=\left\{\begin{array}{cc}{A}_{ii}\hfill & \mathrm{if}i=j,\hfill \\ U_i{W}_{i+1}\cdots W_{j-1}{V}_j^T\hfill & \mathrm{if}j>i,\hfill \\ P_i{R}_{i-1}\cdots R_{j+1}{Q}_j^T\hfill & \mathrm{if}j<i.\hfill \end{array}\right.$$

(4)

For a symmetric matrix A, $P_k=V_k$, $R_k=W_k^T$ and $Q_k=U_k$ for each k. The dimensiosn of these generator matrices ${\left\{U_i\right\}}_{i=1}^{N-1}$, ${\left\{V_i\right\}}_{i=2}^N$, ${\left\{W_i\right\}}_{i=2}^{N-1}$, ${\left\{P_i\right\}}_{i=2}^N$, ${\left\{Q_i\right\}}_{i=1}^{N-1}$, ${\left\{R_i\right\}}_{i=2}^{N-1}$ and ${\left\{D_i\right\}}_{i=1}^N$ are shown in

Table 1. Dimensions of the generators of the SSS matrix shown in Equation (4), $k_i$ and $l_i$ are column dimensions of $U_i$ and $P_i$, respectively.

Matrix	$U_i$	$V_i$	$W_i$	$P_i$	$Q_i$	$R_i$
Dimension	$m_i\times k_i$	$m_i\times k_{i-1}$	$k_{i-1}\times k_i$	$m_i\times l_i$	$m_i\times l_{i+1}$	$l_{i+1}\times l_i$

. The empty products are defined to be the identity matrix. For $N=4$, the matrix A has the following form,

$$A=\left[\begin{array}{cccc}{D}_1& U_1{V}_2^T& U_1{W}_2{V}_3^T& U_1{W}_2{W}_3{V}_4^T\\ P_2{Q}_1^T& D_2& U_2{V}_3^T& U_2{W}_3{V}_4^T\\ P_3{R}_2{Q}_1^T& P_3{Q}_2^T& D_3& U_3{V}_4^T\\ P_4{R}_3{R}_2{Q}_1^T& P_4{R}_3{Q}_2^T& P_4{Q}_3^T& D_4\end{array}\right].$$

(5)

Fast Matrix-Matrix Multiplication

A fast algorithm for multiplying an SSS matrix (4) with any given vector or matrix has been presented in [10,39]. This subsection only introduces the case that both A and B are SSS matrices [10]. Let A and B be matrices in SSS form that are conformally partitioned. The forward and backward recursions are defined as

$$\begin{array}{ccc}\hfill G_1=0,& G_{i+1}=Q_i^T\left(A\right)U_i\left(B\right)+R_i\left(A\right)G_i{W}_i\left(B\right),& i=1,\dots ,n-1,\hfill \\ \hfill H_n=0,& H_{i-1}=V_i^T\left(A\right)P_i\left(B\right)+W_i\left(A\right)H_i{R}_i\left(B\right),& i=n,\dots ,2.\hfill \end{array}$$

We have the following theorem.

Theorem 1

(see [40]). The SSS form of matrix $C=AB$ can be computed through the following recursions:

$$\begin{array}{cc}\hfill D_i\left(C\right)& =D_i\left(A\right)D_i\left(B\right)+P_i\left(A\right)G_i{V}_i^T\left(B\right)+U_i\left(A\right)H_i{Q}_i^T\left(B\right),\hfill \\ \hfill P_i\left(C\right)& =\left[\begin{array}{cc}{D}_i\left(A\right)P_i\left(B\right)+U_i\left(A\right)H_i{R}_i\left(B\right)& P_i\left(A\right)\end{array}\right]\hfill \\ \hfill R_i\left(C\right)& =\left[\begin{array}{cc}{R}_i\left(B\right)& \\ Q_i^T\left(A\right)P_i\left(B\right)& R_i\left(A\right)\end{array}\right]\hfill \\ \hfill Q_i\left(C\right)& =\left[\begin{array}{cc}{Q}_i\left(B\right)& D_i^T\left(B\right)Q_i\left(A\right)+V_i\left(B\right)G_i^T{R}_i^T\left(A\right)\end{array}\right]\hfill \\ \hfill U_i\left(C\right)& =\left[\begin{array}{cc}{D}_i\left(A\right)U_i\left(B\right)+P_i\left(A\right)G_i{W}_i\left(B\right)& U_i\left(A\right)\end{array}\right]\hfill \\ \hfill W_i\left(C\right)& =\left[\begin{array}{cc}{W}_i\left(B\right)& \\ V_i^T\left(A\right)U_i\left(B\right)& W_i\left(A\right)\end{array}\right]\hfill \\ \hfill V_i\left(C\right)& =\left[\begin{array}{cc}{V}_i\left(B\right)& D_i^T\left(B\right)V_i\left(A\right)+Q_i\left(B\right)H_i^T{W}_i^T\left(A\right)\end{array}\right].\hfill \end{array}$$

(6)

This algorithm is an order of magnitude faster than the general matrix-matrix multiplication algorithms. Notice that after multiplication the ranks of generators will increase. Dewilde and van der veen [39] present a technique to compress the generators. A simple, efficient and numerically stable method is further proposed in [11] to compress a given SSS representation to a predefined tolerance $\tau$, and this method is further introduced in Section 3.2 for completeness.

3. The Reduction Algorithm

Assume the bandwidth of matrix B is $b_B$, $N=\lceil n/b_B\rceil$, and L can be seen as a lower block bidiagonal matrix and each block is a $b_B\times b_B$ small matrix. Without loss of any generality, we assume $n=N\times b_B$. From Gaussian elimination, we know

$$L=L_1\cdots L_N,$$

(7)

where $L_i$ is an identity matrix except for the $b_B\times b_B$ diagonal block $L_{ii}$ and the $b_B\times b_B$ subdiagonal block $L_{i,i-1}$ or $L_{i+1,i}$ from L. There are two form of L: the ‘row-wise’ and ‘column-wise’ forms. If $N=3$, the row-wise form, i.e., $L_{i,i-1}$ is nonzero, is written as

$$L=\left[\begin{array}{ccc}{L}_{11}& & \\ L_{21}& L_{22}& \\ & L_{32}& L_{33}\end{array}\right]=\left[\begin{array}{ccc}{L}_{11}& & \\ & I& \\ & & I\end{array}\right]\left[\begin{array}{ccc}I& & \\ L_{21}& L_{22}& \\ & & I\end{array}\right]\left[\begin{array}{ccc}I& & \\ & I& \\ & L_{32}& L_{33}\end{array}\right]\equiv L_1{L}_2{L}_3.$$

Then, its inverse yields

$$\begin{array}{cc}\hfill L^{-1}& =\left[\begin{array}{ccc}I& & \\ & I& \\ & -L_{33}^{-1}{L}_{32}& L_{33}^{-1}\end{array}\right]\left[\begin{array}{ccc}I& & \\ -L_{22}^{-1}{L}_{21}& L_{22}^{-1}& \\ & & I\end{array}\right]\left[\begin{array}{ccc}{L}_{11}^{-1}& & \\ & I& \\ & & I\end{array}\right]\hfill \\ & \equiv \left[\begin{array}{ccc}{\tilde{L}}_{11}& & \\ {\tilde{L}}_{21}{\tilde{L}}_{11}& {\tilde{L}}_{22}& \\ {\tilde{L}}_{32}{\tilde{L}}_{21}{\tilde{L}}_{11}& {\tilde{L}}_{32}{\tilde{L}}_{22}& {\tilde{L}}_{33}\end{array}\right],\mathrm{where}{\tilde{L}}_{i,i-1}=-L_{ii}^{-1}{L}_{i,i-1}.\hfill \end{array}$$

If $N=3$ and the column-wise form is written as

$$L=\left[\begin{array}{ccc}{L}_{11}& & \\ L_{21}& L_{22}& \\ & L_{32}& L_{33}\end{array}\right]=\left[\begin{array}{ccc}{L}_{11}& & \\ L_{21}& I& \\ & & I\end{array}\right]\left[\begin{array}{ccc}I& & \\ & L_{22}& \\ & L_{32}& I\end{array}\right]\left[\begin{array}{ccc}I& & \\ & I& \\ & & L_{33}\end{array}\right]\equiv L_1{L}_2{L}_3.$$

Then, the inverse yields

$$\begin{array}{cc}\hfill L^{-1}& =\left[\begin{array}{ccc}I& & \\ & I& \\ & & L_{33}^{-1}\end{array}\right]\left[\begin{array}{ccc}I& & \\ & L_{22}^{-1}& \\ & -L_{32}{L}_{22}^{-1}& I\end{array}\right]\left[\begin{array}{ccc}{L}_{11}^{-1}& & \\ -L_{21}{L}_{11}^{-1}& I& \\ & & I\end{array}\right]\hfill \\ & \equiv \left[\begin{array}{ccc}{\tilde{L}}_{11}& & \\ {\tilde{L}}_{22}{\tilde{L}}_{21}& {\tilde{L}}_{22}& \\ {\tilde{L}}_{33}{\tilde{L}}_{32}{\tilde{L}}_{21}& {\tilde{L}}_{33}{\tilde{L}}_{32}& {\tilde{L}}_{33}\end{array}\right],\mathrm{where}{\tilde{L}}_{i,i-1}=-L_{i,i-1}{L}_{ii}^{-1}.\hfill \end{array}$$

It can be seen that $L^{-1}$ is exactly the sequentially semiseparable (SSS) matrix defined in [11]. The exact formulae for the entries of $L^{-1}$ are given as, see also [12],

$${\left(L^{-1}\right)}_{ij}={(-1)}^{i+j}\left[\prod _{k=i}^{j+1}\left(L_{kk}^{-1}{L}_{k+1,k}\right)\right]L_{jj}^{-1},$$

for $i=2,\cdots ,N$ and $j=1,\cdots ,i-1$. It is easy to see that the off-diagonal blocks of $L^{-1}$ are low-rank, and its rank is $b_B$, see [11,17]. The SSS generators of $L^{-1}$ are

$$D_i\left(L^{-1}\right)=L_{ii}^{-1},P_i=-D_i\left(L^{-1}\right)L_{i,i-1},R_i\left(L^{-1}\right)=P_{i-1}\left(L^{-1}\right),Q_i\left(L^{-1}\right)=D_i\left(L^{-1}\right).$$

The complexity of computing these generators is $N\times (r^3+\frac{2}{3}{r}^3)=O\left(\frac{5}{3}n{r}^2\right)$, where $L_{ii}$ and $L_{i,i-1}$ are upper and lower triangular of dimension $r=b_B$, respectively. To represent $L^{-1}$ in SSS form, we only need $\left\{D_i\right\}$ and $\left\{P_i\right\}$, and their numbers are N and $N-1$, respectively. For more complex operations, we introduce the other two generators $\left\{R_i\right\}$ and $\left\{Q_i\right\}$.

3.1. The SSS Representation of C

In this subsection, we show that matrix $C=L^{-1}A{L}^{-H}$ is also an SSS matrix. It is well-known that $L^{-1}$ is an SSS matrix. If A is a symmetric block tridiagonal matrix, it is also an SSS matrix, see Equation (5), with generators,

$$D_i\left(A\right)=A_{ii},P_i\left(A\right)=V_i\left(A\right)=A_{i+1,i},R_i\left(A\right)=W_i{\left(i\right)}^T=0,Q_i\left(A\right)=U_i\left(A\right)=I.$$

According to Theorem 1, if we multiply $L^{-1}$ with A, the forward and backward recursions, $\left\{G_i\right\}$ and $\left\{H_i\right\}$, are

$$\begin{array}{ccc}\hfill G_1=0,& G_{i+1}=Q_i^T\left(L^{-1}\right)U_i\left(A\right)=Q_i^T\left(L^{-1}\right),& i=1,\dots ,N-1,\hfill \\ \hfill H_n=0,& H_{i-1}=V_i^T\left(L^{-1}\right)P_i\left(A\right)=V_i^T\left(L^{-1}\right)A_{i+1,i}=0,& i=N,\dots ,2.\hfill \end{array}$$

It is because $R_i\left(A\right)=W_i\left(A\right)=0$, and $V_i\left(L^{-1}\right)=0$ (L is lower triangular). Then, the generators of $\overline{C}=L^{-1}A$ are

$$\begin{array}{cc}\hfill D_i\left(\overline{C}\right)& =D_i\left(L^{-1}\right)D_i\left(A\right)+P_i\left(L^{-1}\right)G_i{V}_i^T\left(A\right),\hfill \\ \hfill P_i\left(\overline{C}\right)& =\left[\begin{array}{cc}{D}_i\left(L^{-1}\right)P_i\left(A\right)& P_i\left(L^{-1}\right)\end{array}\right]\hfill \\ \hfill R_i\left(\overline{C}\right)& =\left[\begin{array}{cc}0& \\ Q_i^T\left(L^{-1}\right)P_i\left(A\right)& R_i\left(L^{-1}\right)\end{array}\right]\hfill \\ \hfill Q_i\left(\overline{C}\right)& =\left[\begin{array}{cc}{Q}_i\left(A\right)& D_i^T\left(A\right)Q_i\left(L^{-1}\right)+V_i\left(A\right)G_i^T{R}_i^T\left(L^{-1}\right)\end{array}\right]\hfill \\ \hfill U_i\left(\overline{C}\right)& =\left[\begin{array}{cc}{D}_i\left(L^{-1}\right)& 0\end{array}\right]\equiv D_i\left(L^{-1}\right)\hfill \\ \hfill W_i\left(\overline{C}\right)& =\left[\begin{array}{cc}0& \\ & 0\end{array}\right]\equiv 0\hfill \\ \hfill V_i\left(\overline{C}\right)& =\left[\begin{array}{cc}{V}_i\left(A\right)& 0\end{array}\right]\equiv V_i\left(A\right).\hfill \end{array}$$

(8)

The ranks of $P_i\left(\overline{C}\right),R_i\left(\overline{C}\right)$ and $Q_i\left(\overline{C}\right)$ are all $b_B$. We can use the recompression techniques [10] to compress them into compact form, which is also introduced in Section 3.2. After obtaining the compact form of $\overline{C}$, we can further compute $C=\overline{C}\times L^{-H}$ and express it in SSS form. The forward and backward recursions are computed as

$$\begin{array}{ccc}\hfill {\overline{G}}_1=0,& {\overline{G}}_{i+1}=Q_i^T\left(\overline{C}\right)U_i\left(L^{-H}\right)+R_i\left(\overline{C}\right){\overline{G}}_i{W}_i\left(L^{-H}\right),& i=1,\dots ,N-1,\hfill \\ \hfill {\overline{H}}_n=0,& {\overline{H}}_{i-1}=V_i^T\left(\overline{C}\right)P_i\left(L^{-H}\right)+W_i\left(\overline{C}\right){\overline{H}}_i{R}_i\left(L^{-H}\right)\equiv 0,& i=N,\dots ,2.\hfill \end{array}$$

Since the generators of $L^{-T}$ are the same as those of $L^{-1}$, i.e., $D_i\left(L^{-T}\right)=D_i{\left(L^{-1}\right)}^T$, $V_i\left(L^{-T}\right)=P_i\left(L^{-1}\right),W_i\left(L^{-T}\right)=R_i{\left(L^{-1}\right)}^T$, $U_i\left(L^{-T}\right)=Q_i\left(L^{-1}\right)$, ${\overline{G}}_i$ can be computed as

$${\overline{G}}_1=0,{\overline{G}}_{i+1}=Q_i^T\left(\overline{C}\right)Q_i\left(L^{-1}\right)+R_i\left(\overline{C}\right){\overline{G}}_i{R}_i{\left(L^{-1}\right)}^T,i=1,\dots ,N-1,$$

and the generators of C are computed as

$$\begin{array}{cc}\hfill D_i\left(C\right)& =D_i\left(\overline{C}\right)D_i^T\left(L^{-1}\right)+P_i\left(\overline{C}\right){\overline{G}}_i{P}_i^T\left(L^{-1}\right),\hfill \\ \hfill P_i\left(C\right)& =P_i\left(\overline{C}\right),\hfill \\ \hfill R_i\left(C\right)& =R_i\left(\overline{C}\right),\hfill \\ \hfill Q_i\left(C\right)& =D_i\left(L^{-1}\right)Q_i\left(\overline{C}\right)+P_i\left(L^{-1}\right){\overline{G}}_i^T{R}_i^T\left(\overline{C}\right).\hfill \end{array}$$

(9)

All the generators of C are computed and their ranks are also $b_B$. Since C is symmetric, only the generators of the diagonals and lower triangular part are needed. The generators of C are already in compact form. To summarize, we have the following proposition.

Proposition 1.

Assume that A and B are Hermitian matrices with bandwidth $b_A=b_B$, and B is further positive definite, and its Cholesky factorization is $B=L{L}^H$. Then, the matrix $C=L^{-1}A{L}^{-H}$ is an SSS matrix and the ranks of its off-diagonal generators are all $b_B$.

It is easy to see that the complexity of computing the generators of $\overline{C}$ is $O\left(\frac{31}{3}n{r}^2\right)$ floating point operations (flops), and similarly, the complexity of computing the generators of C from $\overline{C}$ is another $O\left(10n{r}^2\right)$ flops.

3.2. Recompression of C

From Equation (8), we know that the SSS representation of $\overline{C}$ is not compact. We can use the techniques proposed in Section 3.7 of [11] and Section 10.6 of [10], to compress them into compact forms. Furthermore, since $C=L^{-1}A{L}^{-T}$ is symmetric, we only need to compress the generators of the lower triangular part, $\left\{P_i\right\}$, $\left\{R_i\right\}$ and $\left\{Q_i\right\}$. For a symmetric SSS matrix, its generators satisfy $V_i=P_i,U_i=Q_i$ and $W_i=R_i^T$. Therefore, we only need to consider the lower triangular part of $\overline{C}$.

For completeness, we recall the recompression process in [11]. To be consistent with the context of this paper, we introduce the process by using the generators of the lower triangular part. The recompression method is split into two stages in [11]. In this paper, we reverse the stages. In the first stage, the representation is converted into the right proper form; that is, now all the row bases $G_i$ of the Hankel-blocks (The term Hankel-block is taken from [39]. In this paper it denotes the off-diagonal blocks that extend from the diagonal to the southwest corner), where

$$\begin{array}{cc}\hfill G_N& =P_N,\hfill \\ \hfill G_i& =\left(\begin{array}{c}{P}_i\\ G_{i+1}{R}_i\end{array}\right),\mathrm{for}i=N-1,\cdots ,2.\hfill \end{array}$$

will have orthonormal columns. In the second stage, the representation is converted into the left proper form. By left proper form they mean that all the column bases $C_i$ of the Hankel-blocks, where

$$\begin{array}{cc}\hfill C_1& =Q_1^T,\hfill \\ \hfill C_i& =\left(\begin{array}{cc}{R}_i{C}_{i-1}& Q_i^T\end{array}\right),\hfill \end{array}$$

should have orthonormal columns. The second stage recursions will essentially be first-stage recursions in the opposite order. Note that the Hankel-block $H_i=G_{i+1}{C}_i$.

We follow the notation used in [11], and use hats to denote the representation in right proper form. Consider the following recursions:

$$\begin{array}{cc}\hfill \left(\begin{array}{c}{P}_i\\ {\tilde{R}}_i\end{array}\right)& \approx \left(\begin{array}{c}{\hat{P}}_i\\ {\widehat{R}}_i\end{array}\right){\Sigma }_i{F}_i^H,\tau -\mathrm{accurate}\mathrm{SVD},\hfill \\ \hfill {\tilde{R}}_{i-1}& ={\Sigma }_i{F}_i^H{R}_{i-1},\hfill \\ \hfill {\widehat{Q}}_{i-1}& =Q_{i-1}{F}_i{\Sigma }_i^H,\hfill \end{array}$$

(10)

with the understanding that ${\tilde{R}}_N$ and ${\widehat{R}}_N$ are empty matrices. Then it is easy to check that the new row bases

$$\begin{array}{cc}\hfill {\widehat{G}}_n& ={\hat{P}}_n,\hfill \\ \hfill {\widehat{G}}_i& =\left(\begin{array}{c}{\hat{P}}_i\\ {\widehat{G}}_{i+1}{\widehat{R}}_i\end{array}\right),\hfill \end{array}$$

have orthonormal columns and that the hatted sequences form a valid SSS representation for the given matrix. The generators $\{{\hat{P}}_i\}$, $\{{\widehat{R}}_i\}$ and $\{{\widehat{Q}}_i\}$ are all $r\times r$ matrices. For our problem, our main goal is to have a compact form of generators and the orthonormality does not matter much. Therefore, we only need the first stage. We do not introduce the next stage, and the interested readers can refer to Section 3.7 of [11].

In Equation (10), it uses truncated SVD to compute low-rank approximations of generators $\{P_i\}$. For accuracy, we can let $\tau$ be small or zero. We can also use RRQR [41] or interpolative decomposition (ID) [42] to find a low-rank approximation. In our implementation, we used ID and the computed generators $\{{\hat{P}}_i\}$ are not orthonormal.

Complexity

The recompression consists of three steps:

• Compute a low-rank approximation of $\left(\begin{array}{c}{P}_i\\ {\tilde{R}}_i\end{array}\right)$ of dimensions $r\times 2r$ or $2r\times 2r$. If using ID, it costs $2r^3+(N-2)4r^3=O\left(4n{r}^2\right)$ flops.
• Compute ${\tilde{R}}_{i-1}=X_i{R}_{i-1}$, where X is an $r\times 2r$ matrix and $R_{i-1}$ is of dimension $2r\times 2r$. It costs $(N-2)2r\times 2r^2=O\left(4n{r}^2\right)$ flops.
• Compute ${\widehat{Q}}_{i-1}=Q_{i-1}\times X^T$, where $X^T$ is of dimension $2r\times r$ and $Q_{i-1}$ is of dimension $r\times 2r$. It costs $(N-1)\times 2r\times 2r^2=O\left(4n{r}^2\right)$ flops.

We use the fact that ID costs $O\left(mnk\right)$ flops for computing a rank k approximation of a $m\times n$ matrix, Therefore, the recompression of matrix $\overline{C}$ costs $O\left(12n{r}^2\right)$ flops in total. Thus, computing the SSS representation of $C=L^{-1}A{L}^{-T}$ totally costs $(\frac{5}{3}+\frac{31}{3}+10+12)n{r}^2=O\left(34n{r}^2\right)$ flops.

4. Banded Reduction for Symmetric SSS Matrix

We know that matrix C is an SSS matrix and we can use the method in Section 3.2 to get its compact SSS representation. In this section, we introduce how to tridiagonalize C by using its SSS representations.

For clearness, we assume that C is a $4\times 4$ SSS matrix, the same as Equation (5),

$$C=\left[\begin{array}{cccc}{D}_1& U_1{V}_2^T& U_1{W}_2{V}_3^T& U_1{W}_2{W}_3{V}_4^T\\ P_2{Q}_1^T& D_2& U_2{V}_3^T& U_2{W}_3{V}_4^T\\ P_3{R}_2{Q}_1^T& P_3{Q}_2^T& D_3& U_3{V}_4^T\\ P_4{R}_3{R}_2{Q}_1^T& P_4{R}_3{Q}_2^T& P_4{Q}_3^T& D_4\end{array}\right].$$

(11)

Since matrix C is symmetric, we have $P_i=V_i$, $Q_i=U_i$ and $R_i=W_i^T$.

We perform the following steps and see how to convert C into a block tridiagonal form by working on the generators of C.

• Work on the last two block rows. The off-diagonal block is

  $$\left[\begin{array}{cc}{P}_3{R}_2{Q}_1^T& P_3{Q}_2^T\\ P_4{R}_3{R}_2{Q}_1^T& P_4{R}_3{Q}_2^T\end{array}\right]=\left[\begin{array}{c}{P}_3\\ P_4{R}_3\end{array}\right]\odot \left[\begin{array}{cc}{R}_2{Q}_1^T& Q_2^T\\ R_2{Q}_1^T& Q_2^T\end{array}\right],$$

  where ⊙ means to multiply in the same block row. Find an orthogonal matrix $Q\in {\mathbb{C}}^{2r\times 2r}$ such that $Q\times \left[\begin{array}{c}{P}_3\\ P_4{R}_3\end{array}\right]=\left[\begin{array}{c}{\hat{P}}_3\\ 0\end{array}\right]$. Compute $Q\times \left[\begin{array}{cc}{D}_3& Q_3{P}_4^T\\ P_4{Q}_3^T& D_4\end{array}\right]\times Q^T=\left[\begin{array}{cc}{\hat{D}}_3& {\widehat{Q}}_3^T\\ {\widehat{Q}}_3& {\hat{D}}_4\end{array}\right]$, and define ${\widehat{R}}_3=0,{\hat{P}}_4=I$, and update ${\hat{D}}_3,{\widehat{Q}}_3$. Now, we have

  $$C=\left[\begin{array}{cccc}{D}_1& U_1{V}_2^T& U_1{W}_2{\widehat{V}}_3^T& 0\\ P_2{Q}_1^T& D_2& U_2{\widehat{V}}_3^T& 0\\ {\hat{P}}_3{R}_2{Q}_1^T& {\hat{P}}_3{Q}_2^T& {\hat{D}}_3& {\widehat{U}}_3\\ 0& 0& {\widehat{Q}}_3^T& D_4\end{array}\right].$$
• Work on the 2-nd and 3-rd block rows. The off-diagonal block is

  $$\left[\begin{array}{c}{P}_2{Q}_1^T\\ {\hat{P}}_3{R}_2{Q}_1^T\end{array}\right]=\left[\begin{array}{c}{P}_2\\ {\hat{P}}_3{R}_2\end{array}\right]\odot \left[\begin{array}{c}{Q}_1^T\\ Q_1^T\end{array}\right].$$
  We can compute an orthogonal matrix with dimension $2r$ such that $Q\times \left[\begin{array}{c}{P}_2\\ {\hat{P}}_3{R}_2\end{array}\right]=\left[\begin{array}{c}{\hat{P}}_2\\ 0\end{array}\right]$. Compute $Q\times \left[\begin{array}{cc}{D}_2& Q_2{\hat{P}}_3^T\\ {\hat{P}}_3{Q}_2^T& {\hat{D}}_3\end{array}\right]\times Q^T=\left[\begin{array}{cc}{\hat{D}}_2& {\widehat{Q}}_2^T\\ {\widehat{Q}}_2& {\hat{D}}_3\end{array}\right]$, and define ${\widehat{R}}_2=0,{\widehat{P}}_3=I_r$, and update ${\hat{D}}_2,{\widehat{Q}}_2$. Now, we will introduce a bulge at positions $(2,4)$ and $(4,2)$, and C looks like

  $$C=\left[\begin{array}{cccc}{D}_1& U_1{\widehat{V}}_2^T& 0& 0\\ {\hat{P}}_2{Q}_1^T& {\hat{D}}_2& {\widehat{U}}_2& X\\ 0& {\widehat{Q}}_2^T& {\hat{D}}_3& {\widehat{U}}_3\\ 0& X^T& {\widehat{Q}}_3^T& D_4\end{array}\right].$$
  The bulge is computed as $\left[\begin{array}{cc}0& {\widehat{Q}}_3^T\end{array}\right]Q^T=\left[\begin{array}{cc}X& {\widehat{Q}}_3^T\end{array}\right]$, and ${\widehat{Q}}_3$ is updated. We can use the standard chasing algorithm [43] to eliminate the bulge. The bulge chasing process does not affect the top-left part of matrix C which is represented in SSS form. We draw attention to the fact that $P_i=V_i\Rightarrow I_r$, $R_i=W_i^T\Rightarrow 0$, for $i=3,4$, and the block tridiagonal matrix is defined by ${\hat{D}}_i$ and ${\widehat{Q}}_i$ or ${\widehat{U}}_i$. Finally, matrix C has the following form after bulge chasing,

  $$C=\left[\begin{array}{cccc}{D}_1& U_1{\widehat{V}}_2^T& 0& 0\\ {\hat{P}}_2{Q}_1^T& {\hat{D}}_2& {\widehat{U}}_2& 0\\ 0& {\widehat{Q}}_2^T& {\widehat{D}}_3& {\widehat{U}}_3\\ 0& 0& {\widehat{Q}}_3^T& {\hat{D}}_4\end{array}\right].$$
• Define ${\widehat{Q}}_1^T={\hat{P}}_2{Q}_1^T$, ${\hat{D}}_1=D_1$ and ${\hat{P}}_2=I_r.$ Finally, we get the block tridiagonal matrix

  $$C=\left[\begin{array}{cccc}{\hat{D}}_1& {\widehat{U}}_1& 0& 0\\ {\widehat{Q}}_1^T& {\hat{D}}_2& {\widehat{U}}_2& 0\\ 0& {\widehat{Q}}_2^T& {\hat{D}}_3& {\widehat{U}}_3\\ 0& 0& {\widehat{Q}}_3^T& {\hat{D}}_4\end{array}\right].$$

The whole procedure is summarized in Algorithm 1. The procedure starts from the last row ($k=N$) and ends at the second row ($k=2$), i.e., Step 11 computes the off-diagonal block ${\widehat{Q}}_1$ of matrix C above.

Algorithm 1: (Symmetric banded reduction algorithm for symmetric SSS matrix) Assume that C is an $N\times N$ block symmetric SSS matrix, and its generators are $\left\{P_i\right\},\left\{R_i\right\},\left\{Q_i\right\}$ and $\left\{D_i\right\}$, for $i=1,\cdots ,N$ and each generator is a $r\times r$ matrix.

Inputs: generators $\left\{P_i\right\},\left\{R_i\right\},\left\{Q_i\right\}$ and $\left\{D_i\right\}$, for $i=1,\cdots ,N$   Outputs: a block tridiagonal matrix defined in $\left\{D_i\right\}$ and $\left\{Q_i\right\}$. DO$k=N:-1:3$    Compute orthogonal matrix $H_k$ such that $H_k\left[\begin{array}{c}{P}_{k-1}\\ P_k{R}_{k-1}\end{array}\right]=\left[\begin{array}{c}{\hat{P}}_{k-1}\\ 0\end{array}\right]$, and update $P_{k-1}={\hat{P}}_{k-1}$.    Compute $H_k\left[\begin{array}{cc}{D}_{k-1}& Q_{k-1}{P}_k^T\\ P_k{Q}_{k-1}^T& D_k\end{array}\right]H_k^T=\left[\begin{array}{cc}{\hat{D}}_{k-1}& {\widehat{U}}_{k-1}\\ {\widehat{Q}}_{k-1}& {\hat{D}}_k\end{array}\right]$, and define    $R_{k-1}=0,P_k=I_r$, and update $D_{k-1}={\hat{D}}_{k-1},Q_{k-1}={\widehat{Q}}_{k-1}$ and $D_k={\hat{D}}_k$.    if $k<N$    %(it will introduce a bulge.)        Compute the bulge X by computing $H_k\times \left[\begin{array}{c}0\\ Q_k\end{array}\right]=\left[\begin{array}{c}X\\ {\widehat{Q}}_k\end{array}\right],$ and        update $Q_k={\widehat{Q}}_k$;        for $i=k,N-1$           Apply the standard chasing procedure and chase the bulge down;        end for    end if END DO when $k=2$, compute ${\widehat{Q}}_1^T={\hat{P}}_2{Q}_1^T$ and update $Q_1={\widehat{Q}}_1.$

Remark 1.

By further computing the QR factorization of $Q_i=\widehat{Q}{\widehat{R}}_i$ for $i=2,\cdots ,N$, we can get an symmetric banded matrix, and its off-diagonal blocks are ${\widehat{R}}_i$ and its diagonal blocks are ${\hat{D}}_i={\widehat{Q}}_i^T{D}_i{\widehat{Q}}_i.$

Remark 2.

For an $N\times N$ block SSS matrix, the number of $D_i$ is N, the number of $P_i$ is N − 1, $Q_i$ is N − 1, and $R_i$ is N − 2, and the total number of generators is $4(N-1)$. The lower triangular part of C has $\frac{N(N+1)}{2}$ blocks. When $N\ge 6$, the generators require less storage than storing C explicitly as a dense matrix.

Remark 3.

Algorithm 1 is based on eliminating block rows and columns one by one from the bottom. We can get a similar algorithm by reducing the block rows and columns from the top.

Complexity

For a symmetric SSS matrix, we assume that all its generators are $r\times r$ matrices. We follow the steps of Algorithm 1 to estimate its computational cost.

• Step 2: QR factorization of a $2r\times r$ tall matrix costs $2r^2(2r-\frac{r}{3})=\frac{10}{3}{r}^3.$ It totally executes $N-2$ times, and thus it costs $O\left(\frac{10}{3}n{r}^2\right)$ flops.
• Step 3: Computing $P_k{Q}_{k-1}^T$ costs $2r^3$ and the product of $H_k$ and $H_k^T$ with $\left[\begin{array}{cc}{D}_{k-1}& Q_{k-1}{P}_k^T\\ P_k{Q}_{k-1}^T& D_k\end{array}\right]$ costs $2\times 2{\left(2r\right)}^3=32r^3$ flops. Since it executes $N-2$ times, it costs $O\left(34n{r}^2\right)$ flops in total.
• Step 4:
  • Computing the bulge costs $2\times 2r\times 2r\times r=8r^3$. There are $N-2$ times, and it costs $8n{r}^2$ flops in total.
  • Each bulge chasing costs $\frac{10}{3}{r}^3+2\times \left(2r\right)\times \left(2r\right)\times \left(3r\right)+2\times \left(2r\right)\times \left(2r\right)\times \left(2r\right)=(\frac{10}{3}+24+16)r^3=(\frac{10}{3}+40)r^3.$
  • For k-th step, it requires $N-k$ bulge chasing steps. Since ${\sum }_{k=N-1}^3(N-k)={\sum }_{\ell =1}^{N-3}\ell =\frac{(N-3)(N-2)}{2}$, it totally costs $O((\frac{10}{3}+40)\frac{1}{2}(N^2-5N)r^3=O\left(\frac{65}{3}(n^{2}r-5n{r}^2)\right)$ flops.
• Step 11: it costs $2r^3$ flops.

Therefore, the block tridiagonalization of a symmetric SSS matrix totally costs $O(\frac{65}{3}{n}^{2}r+(34+\frac{10}{3}-\frac{325}{3}+8)n{r}^2)=O(\frac{65}{3}{n}^{2}r-\frac{188}{3}n{r}^2)$ flops.

5. Numerical Results

In this section we test the accuracy of the banded matrix obtained by using SSS matrix techniques. We further compare the accuracy of the proposed algorithms in computing the eigenvalues of symmetric banded positive generalized eigenvalue matrices. All the numerical results are obtained by using Matlab 2017(b) on a laptop with 16GB memory.

Example 1.

Assume that A and B are two randomized symmetric banded matrices and B is further positive definite, which are constructed by using the following Matlab codes

• A = rand(n); B = rand(n);
• A = (A + A’)/2; B = (B + B’)/2 + $\alpha \ast$eye(n);
• A = triu(tril(A,r), −r); B = triu(tril(B,r), −r);

where n is the dimension of matrix and r is the semi-bandwidth, α is a constant to make sure B positive, which is 10 in our experiments. We compute matrix $C=L^{-1}A{L}^{-T}$ explicitly and L is the lower Cholesky factor of matrix B. Then, we compute $\parallel C-\hat{C}{\parallel }_F$ where $\hat{C}=Q_S^{\prime }\widehat{T}{Q}_S$, $\widehat{T}$ is the symmetric banded matrix computed by Algorithm 1, and $Q_S$ is accumulated orthogonal matrix in Algorithm 1. The results are shown in

Table 2. Backward errors of the computed banded matrix by Algorithm 1.

	$\mathit{r}=8$	$\mathit{r}=16$	$\mathit{r}=32$
$N=16$	$2.23\times {10}^{-15}$	$4.74\times {10}^{-15}$	$1.39\times {10}^{-14}$
$N=64$	$8.83\times {10}^{-15}$	$1.89\times {10}^{-14}$	$5.82\times {10}^{-14}$
$N=256$	$3.40\times {10}^{-14}$	$7.44\times {10}^{-14}$	$2.27\times {10}^{-13}$
$N=512$	$6.69\times {10}^{-14}$	$1.46\times {10}^{-13}$	-

. For simplicity we assume $n=N\times r$, i.e., n is divisible by r.

We let $N=16,64,256,512$ and $r=8,16,32$, respectively. The backward errors of Algorithm 1 are shown in Table 2, and the times cost by Algorithm 1 are shown in

Table 3. The times in second cost by Algorithm 1.

	$\mathit{r}=8$	$\mathit{r}=16$	$\mathit{r}=32$
$N=16$	$5.33\times {10}^{-2}$	$2.78\times {10}^{-2}$	$4.04\times {10}^{-2}$
$N=64$	$1.34\times {10}^{-1}$	$3.00\times {10}^{-1}$	$2.18\times {10}^0$
$N=256$	$4.87\times {10}^0$	$3.37\times {10}^1$	$1.40\times {10}^2$
$N=512$	$3.91\times {10}^1$	$2.77\times {10}^2$	-

. When $N=512$ and $r=32$, it is out of memory on the computer used for experiments with 16GB memory, and the result is not included in the following tables. The results show that the proposed algorithm is numerically stable.

Example 2.

In this example we compare the accuracy of the computed generalized eigenvalues by the SSS approach. After obtaining the symmetric banded matrix $\widehat{T}$ from Algorithm 1, we call the Matlab routine eig to compute the eigenvalues of $\widehat{T}$, and then compare with the eigenvalues computed directly from A and B (by using eig(A,B,’chol’)). The relative errors are measured as ${max}_{i=1}^n\frac{|{\widehat{\lambda }}_i-{\lambda }_i|}{{\lambda }_i}$, where ${\widehat{\lambda }}_i$ is the eigenvalue computed by using Algorithm 1, and ${\lambda }_i$ is the eigenvalue computed directly from A and B. The maximum errors and maximum relative errors are shown in

Table 4. The maximum errors of eigenvalues computed by Algorithm 1.

	$\mathit{r}=8$	$\mathit{r}=16$	$\mathit{r}=32$
$N=16$	$2.55\times {10}^{-15}$	$4.44\times {10}^{-15}$	$1.73\times {10}^{-14}$
$N=64$	$3.11\times {10}^{-15}$	$9.99\times {10}^{-15}$	$3.71\times {10}^{-14}$
$N=256$	$1.76\times {10}^{-14}$	$1.87\times {10}^{-14}$	$1.51\times {10}^{-13}$
$N=512$	$2.93\times {10}^{-14}$	$7.79\times {10}^{-14}$	-

and

Table 5. The maximum relative errors of eigenvalues computed by Algorithm 1.

	$\mathit{r}=8$	$\mathit{r}=16$	$\mathit{r}=32$
$N=16$	$7.29\times {10}^{-14}$	$6.25\times {10}^{-14}$	$5.15\times {10}^{-14}$
$N=64$	$9.15\times {10}^{-14}$	$3.89\times {10}^{-11}$	$3.80\times {10}^{-13}$
$N=256$	$3.92\times {10}^{-13}$	$5.36\times {10}^{-11}$	$2.73\times {10}^{-12}$
$N=512$	$1.12\times {10}^{-12}$	$1.70\times {10}^{-12}$	-

, respectively. Matrices A and B are constructed as in Example 1, and the meanings of parameters are the same.

6. Conclusions

In this paper, the rank-structured matrix techniques are first used to reduce a banded generalized eigenvalue problem to a banded standard eigenvalue problem, and it is the first time that rank-structured matrix is used for such eigenvalue problems. Note that there are some works for reducing a rank-structured matrix to its tridiagonal or Hessenberg forms [25], which is different from this work. We in this work focus on the symmetric banded matrices which are sparse, not one particular rank-structured matrix such as quasiseparable, semiseparable and so on. In particular, we use the fast algorithms based on the sequentially semiseparable (SSS) matrix to reduce the banded symmetric generalized eigenvalue problems. The whole process of the proposed algorithm is shown in Algorithm 1, and the complexity analysis is also included. Comparing with the classical algorithms in LAPACK, the algorithm proposed in this paper requires the same order of storage and computation cost. The newly proposed algorithm consists of many small matrix-matrix multiplications which can be potentially executed in parallel. We plan to implement our algorithm by leveraging CHAMELEON library [13,14] and even extend it to the distributed parallel computing case by combining some data redistribution techniques such as [44] in near future.