Rigorous Asymptotic Perturbation Bounds for Hermitian Matrix Eigendecompositions

Abstract

In this paper, we present rigorous asymptotic componentwise perturbation bounds for regular Hermitian indefinite matrix eigendecompositions, obtained via the method of splitting operators. The asymptotic bounds are derived from exact nonlinear expressions for the perturbations and allow each entry of every matrix eigenvector to be bounded in the case of distinct eigenvalues. In contrast to the perturbation analysis of the Schur form of a nonsymmetric matrix, the bounds obtained here do not rely on the Kronecker product, which significantly reduces both memory requirements and computational cost. This enables efficient sensitivity analysis of high-order problems. The eigenvector perturbation bounds are further applied to estimate the angles between perturbed and unperturbed one-dimensional invariant subspaces spanned by the corresponding eigenvectors. To reduce conservatism in the case of high-order problems, we propose the use of probabilistic perturbation bounds based on the Markov inequality. The analysis is illustrated by two numerical experiments of order 5000.

1. Introduction

The perturbation analysis of eigenvalues in real symmetric and complex Hermitian matrices is a well-established topic in matrix analysis. It has been extensively presented in depth in foundational works such as those of Kato [1], Wilkinson [2], Parlett [3], Stewart and Sun [4], Bhatia [5], Stewart [6], and Chatelin [7]; in the surveys of Sun [8] and Li [9]; and in numerous research articles, including [10,11,12,13,14,15], and the references therein. The perturbation analysis of Hermitian decompositions is generally simpler than that of nonsymmetric matrix decompositions (see [16] for a survey of nonsymmetric cases). The main objective of such an analysis is to establish perturbation bounds for the eigenvalues and eigenvectors of Hermitian matrices subject to either Hermitian or non-Hermitian perturbations; see [17] for the interesting case of non–Hermitian perturbations.

Depending on the goal, perturbation analysis may be a priori, where one seeks to predict changes in the eigendecomposition before perturbation is applied, or a posteriori, where errors in eigenvalues and eigenvectors are estimated based on computed perturbed quantities [7] (Ch. 4). In this work, we focus on the former; the latter is typically related to the derivation of residual bounds, as discussed by Davis and Kahan [18], Parlett [3] (Ch. 11), Stewart and Sun [4] (Ch. 5), Chatelin [7] (Ch. 4), and Nakatsukasa [19]. Furthermore, perturbation analysis may be asymptotic (local), where the effect of vanishing perturbations is investigated, or global, where guaranteed bounds are sought for sufficiently large perturbations. In most studies, emphasis is placed on eigenvalue perturbations, while eigenvector perturbation analysis is often reduced to sensitivity analysis of the associated invariant subspace. In several cases, this sensitivity analysis is limited to bounding the distance between the unperturbed and perturbed subspaces [13] or estimating the angles between these subspaces [6] (Ch. 4), [4] (Ch. 5), [8,9]. However, in some applications, it is necessary to obtain bounds on the individual elements of specific eigenvectors, not merely on the sensitivity of the corresponding invariant subspace. This necessitates a componentwise perturbation analysis [20] of the whole matrix eigensystem, consisting of eigenvalues and corresponding eigenvectors.

In this paper, we focus on performing such a componentwise perturbation analysis for indefinite Hermitian matrices, allowing us to bound every entry of each corresponding eigenvector. This analysis is carried out only for regular eigenvalue problems, i.e., perturbation problems involving matrices with distinct eigenvalues. Singular problems, corresponding to multiple eigenvalues, require different techniques (see [21,22]). Regular problems, on the other hand, can be addressed in a simple and efficient way using the method of splitting operators [23], which has already been applied to several other matrix perturbation problems. Importantly, the bounds obtained for the eigenvector entries via this method can also be used to derive bounds on the sensitivity of the invariant subspaces spanned by the corresponding eigenvectors.

In principle, perturbation analysis of Hermitian decompositions can be performed using techniques developed for non-Hermitian problems. However, such approaches often require substantially more memory and computational effort. For example, the method for perturbation analysis of the Schur form proposed in [24] requires the construction of large matrices via the Kronecker product. The method presented here avoids the use of such products, enabling the analysis of significantly larger problems.

Consider a Hermitian matrix $A\in {\mathbb{C}}^{n\times n}$. Using a unitary transformation matrix $U\in {\mathbb{C}}^{n\times n}$, the matrix A can be reduced to the diagonal form

$$\mathsf{\Lambda }=U^{H}AU=\mathrm{diag}({\lambda }_1,{\lambda }_2,\dots ,{\lambda }_n),$$

(1)

where ${\lambda }_1,{\lambda }_2,\dots ,{\lambda }_n$ are the eigenvalues of A, while $U=[U_1,U_2,\dots ,U_n]$ is the matrix of the corresponding orthonormal eigenvectors. Note that the eigenvalues of a Hermitian matrix are always real. If A is real, then U is orthogonal [25] (Ch. 8), [3] (Ch. 1). The diagonal decomposition (1) is also referred to as the symmetric Schur decomposition. Without loss of generality, we assume that the eigenvalues of A are ordered such that ${\lambda }_1\ge {\lambda }_2\ge \dots \ge {\lambda }_n$.

If the matrix A is subject to a Hermitian perturbation $\delta A\in {\mathbb{C}}^{n\times n},\delta A^H=\delta A$, then instead of the decomposition (1), we obtain the perturbed diagonal decomposition

$$\tilde{\mathsf{\Lambda }}={\tilde{U}}^H\tilde{A}\tilde{U}=\mathrm{diag}({\tilde{\lambda }}_1,{\tilde{\lambda }}_2,\dots ,{\tilde{\lambda }}_n),$$

(2)

where ${\tilde{\lambda }}_1\ge {\tilde{\lambda }}_2\ge \dots \ge {\tilde{\lambda }}_n$ are the perturbed eigenvalues and $\tilde{U}=U+\delta U=[{\tilde{U}}_1,{\tilde{U}}_2,\dots ,{\tilde{U}}_n]$ is the matrix of the perturbed eigenvectors.

The objective of the perturbation analysis is to determine bounds on the eigenvalue perturbations

$$\delta \mathsf{\Lambda }=\mathrm{diag}\left(\delta {\lambda }_j\right),\delta {\lambda }_j={\tilde{\lambda }}_j-{\lambda }_j,i=1,2,\dots ,n$$

and on the corresponding eigenvector perturbations $\delta U_j={\tilde{U}}_j-U_j$. The perturbation problem for eigendecompositions of Hermitian matrices is regular if and only if the eigenvalues of A are distinct. Another important objective is to determine the sensitivity of the one-dimensional invariant subspaces spanned by the eigenvectors $U_j,j=1,2,\dots ,n$. According to the Wielandt–Hofmann theorem [25] (Ch. 8), the eigenvalues of a Hermitian matrix are always well conditioned, since their perturbations satisfy

$$|{\tilde{\lambda }}_j-{\lambda }_j{|\le \parallel \delta A\parallel }_2.$$

However, the eigenvectors may be highly sensitive to perturbations in A, depending on the separation of the eigenvalues; see, e.g., [12].

According to the method of splitting operators [23], it is possible to derive separate perturbation bounds for $\left|\delta U\right|$ and $\left|\delta \mathsf{\Lambda }\right|$, which enables the construction of tighter perturbation bounds for the eigenvector. To this end, it is convenient to introduce the matrix

$$\delta W=U^H\delta U=\left[\begin{array}{cccc}{U}_1^H\delta U_1& U_1^H\delta U_2& \dots & U_1^H\delta U_n\\ U_2^H\delta U_1& U_2^H\delta U_2& \dots & U_2^H\delta U_n\\ ⋮& ⋮& \ddots & ⋮\\ U_n^H\delta U_1& U_n^H\delta U_2& \dots & U_n^H\delta U_n\end{array}\right]\in {\mathbb{C}}^{n\times n},$$

which has unitary equivalence to the unknown matrix $\delta U$. By estimating the entries of this matrix, sharp bounds on the entries of $\delta U$ can be obtained thanks to the orthogonality of U.

We further introduce the perturbation parameter vector

$$\begin{array}{ccc}\hfill x& =& \mathrm{vec}\left(\mathrm{Low}\right(\delta W\left)\right)\hfill \\ & =& {\left[U_2^H\delta U_1\dots U_n^H\delta U_1|U_3^H\delta U_2\dots U_n^H\delta U_2|\dots |U_n^H\delta U_{n-1}\right]}^T\in {\mathbb{C}}^{\nu },\nu =n(n-1)/2,\hfill \end{array}$$

whose components are the entries of the strictly lower triangular part of $\delta W$. This vector is a convenient tool for bounding various quantities arising in the perturbation analysis of A.

The structure of the paper is as follows. In Section 2, we derive asymptotic bounds on the perturbation parameters, which are then applied to the perturbation analysis of eigenvalues and eigenvectors of a symmetric matrix. The eigenvector perturbation bounds are subsequently used to obtain asymptotic bounds on the perturbations of the invariant subspaces spanned by the eigenvectors. Section 3 briefly presents results concerning the determination of realistic probabilistic bounds on the entries of a random matrix using its Frobenius norm. We describe how such bounds can be incorporated into the derivation of probabilistic perturbation bounds for the eigenvector entries, eigenvalues, and invariant subspaces. Section 4 illustrates the theoretical results by two numerical experiments with matrices of order 5000, demonstrating the performance of the derived bounds. Finally, Section 5 contains concluding remarks.

All computations in this paper were performed using MATLAB^® Version 9.9 (R2020b) [26] with IEEE double-precision arithmetic on a machine equipped with a 12th Gen Intel(R) Core(TM) i5-1240P CPU running at 1.70 GHz and 32 GB of RAM. M-files implementing the perturbation bounds described herein are available from the authors upon request.

2. Asymptotic Perturbation Bounds

2.1. Asymptotic Bounds for the Perturbation Parameters

 Theorem 1.  

Let $A\in {\mathbb{C}}^{n\times n}$ be a Hermitian matrix ($A=A^H$) with distinct eigenvalues ${\lambda }_1,{\lambda }_2,\dots ,{\lambda }_n$, decomposed as $A=U\mathsf{\Lambda }{U}^H$, where U is unitary and Λ is diagonal. Assume that A is perturbed by a Hermitian perturbation $\delta A$ so that

$$\tilde{A}=A+\delta A=\tilde{U}\tilde{\mathsf{\Lambda }}{\tilde{U}}^H.$$

Denote as $F=-U^H\delta AU$ the transformed perturbation matrix, and construct the vector

$$f=\mathrm{vec}\left(\mathrm{Low}\left(F\right)\right)\in {\mathbb{C}}^{\nu },f_{\mathsf{\ell }}=-U_i^H\delta A{U}_j,\mathsf{\ell }=i+(j-1)n-j(j+1)/2,1\le j<i\le n.$$

Then, the perturbation parameters

$$x_{\mathsf{\ell }}=U_i^H\delta U_j,\mathsf{\ell }=i+(j-1)n-j(j+1)/2,1\le j<i\le n$$

satisfy the equation

$$Mx=f+{\Delta }^x,$$

(3)

where $M\in {\mathbb{C}}^{\nu \times \nu }$ is a diagonal matrix of the form

$$M=\left[\begin{array}{cccccccccc}{\mu }_{21}& 0& \dots & 0& 0& 0& \dots & 0& \dots & 0\\ 0& {\mu }_{31}& \dots & 0& 0& 0& \dots & 0& \dots & 0\\ ⋮& ⋮& \ddots & ⋮& ⋮& ⋮& \ddots & ⋮& ⋮& ⋮\\ 0& 0& \dots & {\mu }_{n1}& 0& 0& \dots & 0& \dots & 0\\ 0& 0& \dots & 0& {\mu }_{32}& 0& \dots & 0& \dots & 0\\ 0& 0& \dots & 0& 0& {\mu }_{42}& \dots & 0& \dots & 0\\ ⋮& ⋮& \ddots & ⋮& ⋮& ⋮& \ddots & ⋮& ⋮& ⋮\\ 0& 0& \dots & 0& 0& 0& \dots & {\mu }_{n2}& \dots & 0\\ ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& ⋮& \ddots & ⋮\\ 0& 0& \dots & 0& 0& 0& \dots & 0& \dots & {\mu }_{n,n-1}\end{array}\right]$$

with ${\mu }_{ij}={\lambda }_i-{\lambda }_j$ and ${\Delta }^x\in {\mathbb{C}}^{\nu }$ being a vector containing second–order terms in $\delta U_i,\delta U_j$.

 Proof. 

From (1) and (2), we have

$${\tilde{U}}_i^H(A+\delta A){\tilde{U}}_j=U_i^{H}A{U}_j=0,j=1,2,\dots ,n-1,i=j+1,j+2,\dots ,n.$$

Hence,

$$U_i^{H}A\delta U_j+\delta U_i^{H}A{U}_j+\delta U_i^{H}A\delta U_j=-{\tilde{U}}_i^H\delta A{\tilde{U}}_j.$$

(4)

From the diagonal decomposition (1), we have

$$\begin{array}{ccc}\hfill U_i^{H}A& =& {\lambda }_i{U}_i^H,i=1,2,\dots ,n,\hfill \\ \hfill A{U}_j& =& {\lambda }_j{U}_j,j=1,2,\dots ,n.\hfill \end{array}$$

(5)

Substituting these expressions into (4) gives

$${\lambda }_i{U}_i^H\delta U_j+{\lambda }_j\delta U_i^H{U}_j+\delta U_i^{H}A\delta U_j=-{\tilde{U}}_i^H\delta A{\tilde{U}}_j.$$

(6)

Since the matrices U and $\tilde{U}$ are unitary, it follows that

$$U^H\delta U=-\delta U^{H}U-\delta U^H\delta U,$$

and therefore,

$$\delta U_i^H{U}_j=-U_i^H\delta U_j-\delta U_i^H\delta U_j,i=1,2,\dots ,n-1,j=i,i+1,\dots ,n.$$

(7)

Substituting this expression for $\delta U_i^H{U}_j$ into (6), we obtain

$$({\lambda }_i-{\lambda }_j)U_i^H\delta U_j={\lambda }_j\delta U_i^H\delta U_j-\delta U_i^{H}A\delta U_j-{\tilde{U}}_i^H\delta A{\tilde{U}}_j.$$

(8)

Equivalently,

$$\begin{array}{ccc}\hfill ({\lambda }_i-{\lambda }_j)U_i^H\delta U_j& =& {\lambda }_j\delta U_i^H\delta U_j-\delta U_i^{H}A\delta U_j-U_i^H\delta A{U}_j-U_i^H\delta A\delta U_j\hfill \\ & & -\delta U_i^H\delta A{U}_j-\delta U_i^H\delta A\delta U_j.\hfill \end{array}$$

(9)

Expression (9) can be rewritten as a system of $\nu =n(n-1)/2$ nonlinear algebraic equations for the unknown quantities

$$x_{\mathsf{\ell }}=U_i^H\delta U_j,\mathsf{\ell }=i+(j-1)n-j(j+1)/2,1\le j<i\le n$$

in the form (3), where the component ${\Delta }_{\mathsf{\ell }}^x,\mathsf{\ell }=i+(j-1)n-j(j+1)/2$ of the nonlinear term ${\Delta }^x$ is given by

$$\begin{array}{ccc}\hfill {\Delta }_{\mathsf{\ell }}^x& =& {\lambda }_j\delta U_i^H\delta U_j-\delta U_i^{H}A\delta U_j-U_i^H\delta A\delta U_j-\delta U_i^H\delta A{U}_j\hfill \\ & & -\delta U_i^H\delta A\delta U_j.\hfill \end{array}$$

(10)

□

We note that solving the system of Equations (3) does not require the explicit formation of the matrix M. In particular, the asymptotic bound $x_{\mathsf{\ell }}^{lin}$ in the inequality

$$|x_{\mathsf{\ell }}|⪯x_{\mathsf{\ell }}^{lin},\mathsf{\ell }=1,2,\dots ,\nu$$

can be obtained directly by using the expression

$$x_{\mathsf{\ell }}^{lin}={\parallel \delta A\parallel }_F/|{\lambda }_i-{\lambda }_j|,\mathsf{\ell }=i+(j-1)n-j(j+1)/2,1\le j<i\le n.$$

(11)

 Example 1.

Consider the $5\times 5$ matrix

$$A=\left[\begin{array}{ccccc}\hfill 2.401616& \hfill 1.198384& \hfill -0.398384& \hfill 0.402384& \hfill -0.398424\\ \hfill 1.198384& \hfill 2.201616& \hfill -0.001616& \hfill -0.002384& \hfill -0.001576\\ \hfill -0.398384& \hfill -0.001616& \hfill 1.801616& \hfill -0.797616& \hfill 0.801576\\ \hfill 0.402384& \hfill -0.002384& \hfill -0.797616& \hfill 1.803616& \hfill -0.797576\\ \hfill -0.398424& \hfill -0.001576& \hfill 0.801576& \hfill -0.797576& \hfill 1.801636\end{array}\right].$$

The eigenvalues of this matrix are

$${\lambda }_1=4.0,{\lambda }_2=3.0,{\lambda }_3=1.01,{\lambda }_4=1.0001,{\lambda }_5=1.0.$$

Note the closeness of the last three eigenvalues, which suggests that the corresponding eigenvectors may be ill conditioned.

The chosen perturbation matrix is

$$\delta A={10}^{-9}\times (\delta A_0+\delta A_0^T),$$

where $\delta A_0$ is a matrix whose elements are random normally distributed numbers.

For this perturbation problem, the matrix $M^{-1}$, which determines the perturbation parameter vector x in (3), has a 2–norm equal to $1\times {10}^4$, confirming that the problem is relatively ill conditioned.

The exact perturbation parameters $x_{\mathsf{\ell }},\mathsf{\ell }=1,2,\dots ,\nu$ and their asymptotic approximations $x_{\mathsf{\ell }}^{lin}$, computed using (11), are shown to eight decimal digits in

Table 1. Exact perturbation parameters $x_{\mathsf{\ell }}$ related to the matrix $\delta W$ and their linear estimates for ${\parallel \delta A\parallel }_F=1.0983610\times {10}^{-8}$.

${\mathit{x}}_{\mathit{\ell }}={\mathit{U}}_{\mathit{i}}^{\mathit{T}}\mathit{\delta }{\mathit{U}}_{\mathit{j}}$	$|{\mathit{x}}_{\mathit{\ell }}|$	${\mathit{x}}_{\mathit{\ell }}^{\mathit{lin}}$	$|\mathit{\Delta }{\mathit{x}}_{\mathit{\ell }}|$
$x_1=U_2^T\delta U_1$	$7.6707043\times {10}^{-11}$	$1.0983610\times {10}^{-8}$	$4.7003350\times {10}^{-19}$
$x_2=U_3^T\delta U_1$	$6.0721878\times {10}^{-10}$	$3.6734482\times {10}^{-9}$	$4.2499607\times {10}^{-19}$
$x_3=U_4^T\delta U_1$	$6.9954664\times {10}^{-10}$	$3.6612033\times {10}^{-9}$	$2.0826081\times {10}^{-19}$
$x_4=U_5^T\delta U_1$	$2.5735541\times {10}^{-10}$	$3.6613255\times {10}^{-9}$	$1.0760803\times {10}^{-18}$
$x_5=U_3^T\delta U_2$	$1.1842760\times {10}^{-9}$	$5.5194023\times {10}^{-9}$	$5.8765348\times {10}^{-18}$
$x_6=U_4^T\delta U_2$	$9.0330987\times {10}^{-10}$	$5.4918052\times {10}^{-9}$	$7.9079719\times {10}^{-18}$
$x_7=U_5^T\delta U_2$	$8.5971180\times {10}^{-10}$	$5.4920797\times {10}^{-9}$	$8.2028102\times {10}^{-18}$
$x_8=U_4^T\delta U_3$	$1.4635852\times {10}^{-7}$	$1.0983611\times {10}^{-6}$	$1.7153442\times {10}^{-15}$
$x_9=U_3^T\delta U_3$	$4.8612969\times {10}^{-7}$	$1.1094556\times {10}^{-6}$	$3.5872417\times {10}^{-15}$
$x_{10}=U_5^T\delta U_4$	$2.3352890\times {10}^{-5}$	$1.0983610\times {10}^{-4}$	$9.1061463\times {10}^{-14}$

. The magnitudes of the corresponding elements of the two vectors show good agreement. The discrepancies between the values of $x_{\mathsf{\ell }}^{lin}$ and $x_{\mathsf{\ell }}$ arise from bounding the entries of the vector f with ${\parallel \delta A\parallel }_F$ and neglecting the nonlinear term ${\delta }^x$.

2.2. Asymptotic Componentwise Eigenvector Bounds

 Theorem 2.  

Under the assumptions of Theorem 1, a strict asymptotic bound on the perturbation of each eigenvector $U_j,j=1,2,\dots ,n$ of A under a perturbation $\delta A$ is given by

$$|{\tilde{U}}_j-U_j|⪯\delta U_j^{lin}=\left|U\right|W_j^{lin},$$

where

$$\begin{array}{ccc}\hfill \delta W^{lin}& =& \left[\begin{array}{cccccc}0& x_1^{lin}& x_2^{lin}& \dots & x_{n-2}^{lin}& x_{n-1}^{lin}\\ x_1^{lin}& 0& x_n^{lin}& \dots & x_{2n-4}^{lin}& x_{2n-3}^{lin}\\ x_2^{lin}& x_n^{lin}& 0& \dots & x_{3n-7}^{lin}& x_{3n-6}^{lin}\\ ⋮& ⋮& ⋮& ⋮& ⋮& ⋮\\ x_{n-2}^{lin}& x_{2n-4}^{lin}& x_{3n-7}^{lin}& \dots & 0& x_{\nu }^{lin}\\ x_{n-1}^{lin}& x_{2n-3}^{lin}& x_{3n-6}^{lin}& \dots & x_{\nu }^{lin}& 0\end{array}\right]\in {\mathbb{C}}^{n\times n},\hfill \end{array}$$

and the elements of the vector $x^{lin}={[x_1^{lin},x_2^{lin},\dots ,x_{\nu }^{lin}]}^T$ are given by

$$x_{\mathsf{\ell }}^{lin}={\parallel \delta A\parallel }_F/|{\lambda }_i-{\lambda }_j|,\mathsf{\ell }=i+(j-1)n-j(j+1)/2,1\le j<i\le n.$$

 Proof. 

Consider the auxiliary matrix introduced above,

$$\delta W=U^H\delta U:=[\delta W_1,\delta W_2,\dots ,\delta W_n],\delta W_j\in {\mathbb{C}}^n.$$

As noted earlier, the strictly lower triangular part of this matrix contains elements of the form

$$U_i^H\delta U_j,j=1,2,\dots ,n-1,i=j+1,j+1,\dots ,n,$$

which can be replaced by the corresponding elements $x_{\mathsf{\ell }},\mathsf{\ell }=i+(j-1)n-j(j+1)/2$ of the vector x. The strictly upper triangular part of $\delta W$ consists of elements

$$U_i^H\delta U_j,i=1,2,\dots ,n-1,j=i+1,i+2,\dots ,n,$$

which, according to the unitary condition (7), can be represented as

$$U_i^H\delta U_j=\delta U_i^H{U}_j-\delta U_i^H\delta U_j$$

or

$$U_i^H\delta U_j=-\overline{U_j^H\delta U_i}-\delta U_i^H\delta U_j.$$

(12)

Here, the term $\overline{U_j^H\delta U_i}$, for $j>i$ is the conjugate of the element $x_{\mathsf{\ell }}$. Hence, the matrix $\delta W$ can be written as

$$\delta W=\delta V-\delta D-\delta Y,$$

(13)

where

$$\begin{array}{ccc}\hfill \delta V& =& \left[\begin{array}{ccccc}0& -{\overline{x}}_1& -{\overline{x}}_2& \dots & -{\overline{x}}_{n-1}\\ x_1& 0& -{\overline{x}}_n& \dots & -{\overline{x}}_{2n-3}\\ x_2& x_n& 0& \dots & -{\overline{x}}_{3n-6}\\ ⋮& ⋮& ⋮& \ddots & ⋮\\ x_{n-1}& x_{2n-3}& x_{3n-6}& \dots & 0\end{array}\right]\in {\mathbb{C}}^{n\times n},\hfill \end{array}$$

and the matrices

$$\delta D=\left[\begin{array}{cccc}\delta U_1^H\delta U_1/2& 0& \dots & 0\\ 0& \delta U_2^H\delta U_2/2& \dots & 0\\ ⋮& ⋮& \ddots & ⋮\\ 0& 0& \dots & \delta U_n^H\delta U_n/2\end{array}\right]\in {\mathbb{C}}^{n\times n},$$

$$\delta Y=\left[\begin{array}{ccccc}0& \delta U_1^H\delta U_2& \delta U_1^H\delta U_3& \dots & \delta U_1^H\delta U_n\\ 0& 0& \delta U_2^H\delta U_3& \dots & \delta U_2^H\delta U_n\\ 0& 0& 0& \dots & \delta U_3^H\delta U_n\\ ⋮& ⋮& ⋮& \ddots & ⋮\\ 0& 0& 0& \dots & \delta U_{n-1}^H\delta U_n\end{array}\right]\in {\mathbb{C}}^{n\times n}$$

contain second-order terms in $\delta U_j,j=1,2,\dots ,n$. For definiteness, we assume that $\tilde{U}$ is chosen such that the diagonal elements of $\delta D$ are real and

$$U_i^H\delta U_i+\overline{U_i^H\delta U_i}=2U_i^H\delta U_i=-\delta U_i^H\delta U_j,$$

since condition (12) does not impose any restriction on the imaginary part of $U_i^H\delta U_i$.

According to (13), the matrix $\left|\delta W\right|$ can be estimated as

$$\left|\delta W\right|⪯\left|\delta V\right|+{\delta }^W,$$

(14)

where

$${\delta }^W=\left|\delta D\right|+\left|\delta Y\right|$$

contains second-order terms in the elements of x. Thus, an asymptotic (linear) approximation of the matrix $\left|\delta U\right|$ can be obtained as

$$|\delta U^{lin}|⪯|U\left|\right|U^H\delta U|=|U|\delta W^{lin}.$$

(15)

□

Note that these bounds are also valid for real symmetric matrices.

 Example 2.  

For the same matrix A and perturbation $\delta A$ as in Example 1, the absolute values of the exact changes of the entries of the matrix U and their linear approximations, obtained using (15), are given, respectively, by

$$\left|\delta U\right|={10}^{-4}\times \left[\begin{array}{ccccc}\hfill 0.0000045& \hfill 0.0000028& \hfill 0.0025383& \hfill 0.0928278& \hfill 0.0953600\\ \hfill 0.0000037& \hfill 0.0000020& \hfill 0.0025204& \hfill 0.0928299& \hfill 0.0953488\\ \hfill 0.0000025& \hfill 0.0000070& \hfill 0.0010687& \hfill 0.0928181& \hfill 0.1381715\\ \hfill 0.0000016& \hfill 0.0000139& \hfill 0.0025323& \hfill 0.0942843& \hfill 0.0904961\\ \hfill 0.0000071& \hfill 0.0000066& \hfill 0.0023290& \hfill 0.1407080& \hfill 0.0953601\end{array}\right]$$

and

$$\delta U^{lin}={10}^{-4}\times \left[\begin{array}{ccccc}\hfill 0.0000879& \hfill 0.0001319& \hfill 0.0088754& \hfill 0.4437818& \hfill 0.4438262\\ \hfill 0.0001099& \hfill 0.0001099& \hfill 0.0088791& \hfill 0.4437855& \hfill 0.4438298\\ \hfill 0.0000952& \hfill 0.0001209& \hfill 0.0110648& \hfill 0.4437745& \hfill 0.6634911\\ \hfill 0.0000953& \hfill 0.0001210& \hfill 0.0088680& \hfill 0.4459712& \hfill 0.4460378\\ \hfill 0.0000952& \hfill 0.0001209& \hfill 0.0110869& \hfill 0.6634467& \hfill 0.4438189\end{array}\right].$$

For all $i,j=1,2,\dots ,5$, it is verified that $|\delta U_{i,j}|<\delta U_{i,j}^{lin}$.

2.3. Eigenvalue Sensitivity

For the changes in the elements of the perturbed diagonal matrix $\tilde{\mathsf{\Lambda }}$, we have

$$\delta {\lambda }_i={\tilde{\lambda }}_i-{\lambda }_i={\tilde{U}}_i^H(A+\delta A){\tilde{U}}_i-U_i^{H}A{U}_i,i=1,2,\dots ,n.$$

Hence,

$$\begin{array}{ccc}\hfill \delta {\lambda }_i& =& U_i^{H}A\delta U_i+\delta U_i^{H}A{U}_i+\delta U_i^{H}A\delta U_i+U_i^H\delta A{U}_j\hfill \end{array}$$

(16)

$$\begin{array}{ccc}& & +U_i^H\delta A\delta U_i+\delta U_i^H\delta A{U}_i+\delta U_i^H\delta A\delta U_i.\hfill \end{array}$$

(17)

Thus, we obtain

$$\delta {\lambda }_i=U_i^H\delta A{U}_i+{\delta }_i^d,i=1,2,\dots ,n,$$

(18)

where, based on (5),

$${\delta }_i^d={\lambda }_i(U_i^H\delta U_i+\delta U_i^H{U}_i)+\delta U_i^{H}A\delta U_i+U_i^H\delta A\delta U_i+\delta U_i^H\delta A{U}_i+\delta U_i^H\delta A\delta U_i.$$

According to (7), it holds that

$$\delta U_i^H{U}_j+U_i^H\delta U_j=-\delta U_i^H\delta U_j,$$

which gives

$${\delta }_i^d=-{\lambda }_i\delta U_i^H\delta U_i+\delta U_i^{H}A\delta U_i+U_i^H\delta A\delta U_i+\delta U_i^H\delta A{U}_i+\delta U_i^H\delta A\delta U_i.$$

(19)

The quantity ${\delta }_i^d$ contains second-order terms in $\delta U_i$.

If the perturbation $\delta A$ is known, expression (19) makes it possible to bound ${\delta }_i^d$ using the eigenvector bound (15).

In the asymptotic eigenvalue analysis, higher-order terms are neglected, and one has

$$\delta {\lambda }_i^{lin}=U_i^H\delta A{U}_i,i=1,2,\dots ,n.$$

Hence,

$$|\delta {\lambda }_i|\le \delta {\lambda }^{lin}={\parallel \delta A\parallel }_2,i=1,2,\dots ,n,$$

which reduces to the well-known corollary of the Wielandt–Hoffman theorem [25] (Ch. 8).

2.4. Sensitivity of One-Dimensional Invariant Subspaces

The estimate of the eigenvector perturbation $\delta U_j$ can be used to derive a bound on the sensitivity of one-dimensional (simple) invariant subspace associated with the eigenvector $U_j$.

Consider the one-dimensional invariant subspace ${\mathcal{X}}_j=\mathcal{R}\left(U_j\right),j=1,2,\dots ,n$. The sensitivity of this subspace is measured by the angle $\mathsf{\Theta }$ between the perturbed and unperturbed invariant subspaces. Since

$${\tilde{U}}_j=U_j+\delta U_j,$$

we have that [6,27] (Ch. 4),

$$sin\left(\mathsf{\Theta }({\tilde{\mathcal{X}}}_j,{\mathcal{X}}_j)\right)=\parallel {U_j^{\perp }}^H{\tilde{U}}_j{\parallel }_2={\parallel {U_j^{\perp }}^H\delta U_j\parallel }_2,$$

(20)

where

$$U_j^{\perp }=\left[U_1,U_2,\dots ,U_{j-1},U_{j+1},\dots ,U_n\right]\in {\mathbb{C}}^{n\times (n-1)}$$

is the orthogonal complement of $U_j$, while ${U_j^{\perp }}^H{U}_j=0_{(n-1)\times 1}$. Note that, in (20), we prefer to use the expression involving $sin\left(\mathsf{\Theta }\right)$ rather than $cos\left(\mathsf{\Theta }\right)$, since, for small $\mathsf{\Theta }$, the sine is well conditioned and the cosine is not.

Equation (20) shows that the sensitivity of the one-dimensional invariant subspace $\mathcal{X}$ is directly related to the perturbation parameters $x_{\mathsf{\ell }}=U_i^H\delta U_j$. Consequently, if bounds on the perturbation parameters are known, the sensitivity of all invariant subspaces can be estimated. More specifically, we have

$${U_1^{\perp }}^H\delta U_1=\left[\begin{array}{c}{x}_1\\ x_2\\ ⋮\\ x_{n-1}\end{array}\right],{U_2^{\perp }}^H\delta U_2=\left[\begin{array}{c}-{\overline{x}}_1\\ x_n\\ ⋮\\ x_{2n-3}\end{array}\right],\dots ,{U_n^{\perp }}^H\delta U_n=\left[\begin{array}{c}-{\overline{x}}_{n-1}\\ -{\overline{x}}_{2n-3}\\ -{\overline{x}}_{3n-6}\\ ⋮\\ -{\overline{x}}_{\nu }\end{array}\right].$$

Hence, the angle between the jth perturbed and unperturbed invariant subspace can be approximated with the following asymptotic estimate:

$$\mathsf{\Theta }({\tilde{\mathcal{X}}}_j,{\mathcal{X}}_j)\le {\mathsf{\Theta }}^{lin}({\tilde{\mathcal{X}}}_j,{\mathcal{X}}_j)=arcsin(\parallel \delta W^{lin}(1:n,j){\parallel }_2),j=1,2,\dots ,n,$$

(21)

where ${\mathsf{\Theta }}^{lin}$ is the asymptotic estimate of $\mathsf{\Theta }$ obtained using the perturbation parameters.

We note that this estimate produces results nearly identical to the linear bounds derived in [8,13,28].

 Example 3.  

For the same matrix and perturbation as in the previous examples, the exact angles between the perturbed and unperturbed one-dimensional invariant subspaces, together with their linear estimates, are shown in

Table 2. Exact angles between perturbed and unperturbed invariant subspaces and their linear estimates.

j	$\mathbf{\Theta }$	${\mathbf{\Theta }}^{\mathit{lin}}$
1	$9.6446662\times {10}^{-10}$	$1.2686356\times {10}^{-8}$
2	$1.7214722\times {10}^{-9}$	$1.4540507\times {10}^{-8}$
3	$5.0768557\times {10}^{-7}$	$1.5611959\times {10}^{-6}$
4	$2.3353348\times {10}^{-5}$	$1.0984160\times {10}^{-4}$
5	$2.3357949\times {10}^{-5}$	$1.0984171\times {10}^{-4}$

.

3. Probabilistic Asymptotic Bounds

Numerical experiments with symmetric matrices indicate that the estimates obtained using Theorem 2 may become overly conservative as the matrix order increases. For instance, when the order of the matrix is 2000, the ratio between the bound (15) and the actual values of the entries of $\left|\delta U\right|$ may grow as large as ${10}^5$, rendering the computed bound useless. A further reduction in the perturbation bounds can be achieved by introducing probabilistic perturbation bounds, which, for large n, allow one to obtain significantly tighter estimates with high probability. To this end, we employ the probabilistic matrix bounds proposed in [29,30], which are based on the Markov inequality [31] (Section 5-4).

We briefly outline the essence of the approach presented in [29]. The goal is to reduce the magnitude of entries of an estimate $\Delta A$ of the matrix perturbation $\delta A$ at the expense of allowing some entries of $|\Delta A|$ to be smaller then the corresponding entries of $\left|\delta A\right|$. We obtain the following result.

 Theorem 3.

Let $\Delta A=[\Delta a_{ij}]$ be an estimate of the $m\times n$ random perturbation $\delta A$, and let $\mathcal{P}\left\{\right|\delta a_{ij}|<\Delta a_{ij}\}$ denote the probability that $|\delta a_{ij}|<\Delta a_{ij}$. If the entries of $\Delta A$ are chosen as

$$\Delta a_{ij}={\displaystyle \frac{{\parallel \delta A\parallel }_F}{\mathsf{\Xi }}},$$

where

$$\mathsf{\Xi }=(1-{\mathcal{P}}^{ref})\sqrt{mn},$$

(22)

and $0<{\mathcal{P}}^{ref}<1$ is a desired probability, then

$$\mathcal{P}\left\{\right|\delta a_{ij}|<\Delta a_{ij}\}\ge {\mathcal{P}}^{ref},i=1,2,\dots ,m,j=1,2,\dots ,n.$$

If the number $mn$ is sufficiently large that $\mathsf{\Xi }>1$, Theorem 3 allows the mean value of the bound $\Delta A$ and, hence, the magnitude of its entries to be reduced by the scaling factor $\mathsf{\Xi }$, provided that the desired probability ${\mathcal{P}}^{ref}$ is set to be less than one. Note that the probability bound produced by the Markov inequality is conservative; in practice, the actual results are usually much closer than those predicted by ${\mathcal{P}}^{ref}$. This is because the Markov inequality is valid for the worst possible distribution of the random variables, whereas in our case we do not impose any restriction on the probability distribution of the entries of $\delta A$.

According to Theorem 3, the use of the scaling factor (22) guarantees that the inequality

$$|\delta a_{ij}|<\Delta a_{ij}$$

holds for each i and j with probability no less than ${\mathcal{P}}^{ref}$.

Using Theorem 3, a probability bound on the perturbation parameters $|x_{\mathsf{\ell }}|,\mathsf{\ell }=1,2,\dots ,\nu$ can be derived as follows.

 Theorem 4.

If the asymptotic estimate of the perturbation parameter vector x is chosen as

$$x_{\mathsf{\ell }}^{est}={\displaystyle \frac{{\parallel \delta A\parallel }_F}{\mathsf{\Xi }|{\lambda }_i-{\lambda }_j|}},\mathsf{\ell }=i+(j-1)n-j(j+1)/2,1\le j<i\le n,$$

(23)

where Ξ is determined according to

$$\mathsf{\Xi }=n(1-{\mathcal{P}}^{ref}),$$

(24)

then

$$\mathcal{P}\left\{\right|x_{\mathsf{\ell }}|\le \parallel x^{est}{\parallel }_2\}\le {\mathcal{P}}^{ref}.$$

(25)

The inequality (25) shows that a probability estimate of the component $|x_{\mathsf{\ell }}|$ can be obtained by replacing the perturbation norm ${\parallel \delta A\parallel }_F$ in the linear estimate (11) with the probability estimate ${\parallel \delta A\parallel }_F/\mathsf{\Xi }$, where the scaling factor $\mathsf{\Xi }$ is chosen according to (22) for a specified probability ${\mathcal{P}}^{ref}$.

The probabilistic perturbation bounds on the elements of the vector x make it possible to find probabilistic bounds on the elements of $\delta U$.

 Theorem 5.

An asymptotic bound with probability ${\mathcal{P}}^{ref}$ on the perturbation of each eigenvector $U_j,j=1,2,\dots ,n$ of A under a perturbation $\delta A$ is given by

$$|{\tilde{U}}_j-U_j|⪯|\delta U_j^{est}|=|U|W_j^{est},$$

where

$$\begin{array}{ccc}\hfill \delta W^{est}& =& \left[\begin{array}{cccccc}0& x_1^{est}& x_2^{est}& \dots & x_{n-2}^{est}& x_{n-1}^{est}\\ x_1^{est}& 0& x_n^{est}& \dots & x_{2n-4}^{est}& x_{2n-3}^{est}\\ x_2^{est}& x_n^{est}& 0& \dots & x_{3n-7}^{est}& x_{3n-6}^{est}\\ ⋮& ⋮& ⋮& \ddots & ⋮& ⋮\\ x_{n-2}^{est}& x_{2n-4}^{est}& x_{3n-7}^{est}& \dots & 0& x_{\nu }^{est}\\ x_{n-1}^{est}& x_{2n-3}^{est}& x_{3n-6}^{est}& \dots & x_{\nu }^{est}& 0\end{array}\right]\in {\mathbb{C}}^{n\times n},\hfill \end{array}$$

and the elements of the vector $x^{est}={[x_1^{est},x_2^{est},\dots ,x_{\nu }^{est}]}^T$ are determined from

$$x_{\mathsf{\ell }}^{est}={\displaystyle \frac{{\parallel \delta A\parallel }_F}{\mathsf{\Xi }|{\lambda }_i-{\lambda }_j|}},\mathsf{\ell }=i+(j-1)n-j(j+1)/2,1\le j<i\le n.$$

The probabilistic bound on $\delta A$ then yields the following asymptotic perturbation bounds on the eigenvalues of A:

$$|\delta {\lambda }_i|\le \delta {\lambda }_i^{est}={\parallel \delta A\parallel }_F/\mathsf{\Xi },i=1,2,\dots ,n.$$

Finally, we find that the angle between the jth perturbed and unperturbed one-dimensional invariant subspaces satisfies the probabilistic asymptotic estimate

$${\mathsf{\Theta }}_j({\tilde{\mathcal{X}}}_j,{\mathcal{X}}_j)\le {\mathsf{\Theta }}_j^{est}({\tilde{\mathcal{X}}}_j,{\mathcal{X}}_j)=arcsin(\parallel \delta W^{est}(1:n,j){\parallel }_2),j=1,2,\dots ,n.$$

(26)

4. Numerical Experiments

In this section, we present two numerical experiments illustrating the perturbation bounds for eigendecompositions of matrices of order 5000. The matrices are constructed in the form $A=VD{V}^T$, where V is orthogonal and D is a diagonal matrix containing the prescribed eigenvalues. The perturbation $\delta A$ is taken as ${10}^{-9}\times (\delta A_0+\delta A_0^T)$, where $\delta A_0$ is a matrix with normally distributed random entries.

 Example 4.

In this examples, the eigenvalues of A are distributed uniformly from 1 to 5.999 with an increment of 0.001, which results in $M^{-1}$ having a 2-norm equal to ${10}^3$.

In Figure 1, we display the entries of the exact perturbation $|\delta U=\tilde{U}-U|$, together with the entries of the asymptotic $\delta U^{lin}$ and the probabilistic bound $\delta U^{est}$. The desired lower bound probability is set to $P^{ref}=0.8$, and the scaling factor for finding the probabilistic estimates is 1000. As a result, the actual probability is equal to $100\%$, since all entries of $\delta U^{est}$ exceed the corresponding entries of $\left|\delta U\right|$. As already mentioned, this phenomenon is due to the conservatism of the Markov inequality. The eigenvalue perturbations $\delta {\lambda }_i={\tilde{\lambda }}_i-{\lambda }_i$, together with their asymptotic estimate $\delta {\lambda }^{lin}=2.8287199\times {10}^{-6}$ and probability estimate $\delta {\lambda }^{est}=7.0717998\times {10}^{-9}$, are shown in Figure 2. There is no eigenvalue whose probability perturbation bound is smaller than the actual $\delta \lambda$, which yields $100\%$ accuracy instead of the reference value of $80\%$. Due to the equal spacing between the eigenvalues, the sensitivity of all invariant subspaces is the same in this case (Figure 3).

Clearly, the probabilistic estimates are much closer to the actual perturbations than the linear deterministic bounds are.

 Example 5.

In this example, the eigenvalues of A are set to ${\lambda }_i=1+500/i$ and are thus distributed between $1.1$ and 501. The norm of the matrix $M^{-1}$ is equal to $4.999\times {10}^4$. The eigenvalues of A are shown in Figure 4.

In Figure 5, Figure 6 and Figure 7, we present the matrix $|{\delta }_U|$, the eigenvalue perturbations, and the angles between the perturbed and unperturbed invariant subspaces, respectively, as well as their asymptotic and probabilistic bounds for $P^{ref}=80\%$. Due to the decreasing separation between the eigenvalues, the sensitivity of the invariant subspace increases with the index i. As in the previous example, the actual probabilistic bounds for the eigenvector matrix and for the eigenvalues again reach $100\%$.

Note that in both examples, the linear estimates correctly reflect the changes in the corresponding exact quantities.

5. Conclusions

In this paper, we have presented strict asymptotic perturbation bounds for the eigenvalues, eigenvectors, and invariant subspaces of symmetric and Hermitian matrices. The use of the splitting operator method enables a unified treatment with the perturbation analysis of the Schur form [24] and the generalized Schur form [32]. Since the deterministic bounds derived may become conservative for high-order matrices, we have introduced probabilistic bounds based on the Markov inequality. These probabilistic bounds are easily obtained from the asymptotic bounds and allow a substantial reduction in the perturbation estimates with a guaranteed probability that is close to one in practice. As shown by the examples, an appealing aspect of the new bounds for symmetric matrices is that they can be determined with low requirements in terms of memory and volume of computational effort.