Small-Deviation Inequalities for Sums of Random Matrices

Abstract

Random matrices have played an important role in many fields including machine learning, quantum information theory, and optimization. One of the main research focuses is on the deviation inequalities for eigenvalues of random matrices. Although there are intensive studies on the large-deviation inequalities for random matrices, only a few works discuss the small-deviation behavior of random matrices. In this paper, we present the small-deviation inequalities for the largest eigenvalues of sums of random matrices. Since the resulting inequalities are independent of the matrix dimension, they are applicable to high-dimensional and even the infinite-dimensional cases.

1. Introduction

Random matrices have been widely used in many problems, e.g., compressed sensing [1], high-dimensional data analysis [2], matrix approximation [3,4] and dimension reduction [5]. In the literature, one of main research issues is to study the deviation behavior of eigenvalues (or singular values) of random matrices.

In general, there are two types of deviation results studied in probability theory: one is the large-deviation inequality that describes the behavior of the probability $\mathbb{P}(|x|>t)$ for large t; and the other is the small-deviation (or small-ball) inequality that controls the probability $\mathbb{P}(|x|<ϵ)$ for small $ϵ$.

The early large-deviation inequalities for sums of random matrices can be dated back to the work of Ahlswede and Winter [6]. Tropp [7] improved their results and developed a user-friendly framework to obtain the large-deviation inequalities for sums of random matrices. To overcome the limitation of the matrix-dimension dependence, Hsu et al. [8] and Minsker [9] introduced the concepts of intrinsic dimension and effective dimension to tighten large-deviation inequalities, respectively. Moreover, Zhang et al. [10] applied a diagonalization method to obtain the dimension-free large-deviation random for largest singular value of sums of random matrices, while it remains a challenge to select the auxiliary matrices and functions. In the scenario of a single random matrix, Ledoux [11] studied the largest eigenvalues of Gaussian unitary ensemble matrices and Vershynin [12] studied the singular values of the sub-Gaussian and sub-exponential matrices.

Small-deviation problems stemmed from some practical applications, e.g., approximation problems [13], Brownian pursuit problems [14], quantization problem [15], and convex geometry [16]. For more details, we refer to the bibliography maintained by Lifshitz [17]. There have been some works on the small-deviation inequalities for the specific types of random matrices. Aubrun [18] obtained the small-deviation inequalities for the largest eigenvalue of a single Gaussian unitary ensemble matrix. Rudelson and Vershynin [19] presented the small-deviation inequalities for the smallest singular value of the random matrix with independent entries. Volodko [20] estimated the small-deviation probability of the determinant of the matrix $\mathbf{B}{\mathbf{B}}^T$, where $\mathbf{B}$ is a $d\times \infty$ random matrix whose entries obey a centered joint Gaussian distribution. To the best of our knowledge, there are few works on the small-deviation inequalities for sums of random matrices.

1.1. Related Works

Let $\{{\mathbf{X}}_1,{\mathbf{X}}_2,\cdots ,{\mathbf{X}}_K\}\subset {\mathbb{C}}^{d\times d}$ be a finite sequence of independent random Hermitian matrices. It follows from Markov’s inequality that

$$\mathbb{P}\left\{{\lambda }_{max}\left(\sum _k{\mathbf{X}}_k\right)\ge t\right\}\le \underset{\theta >0}{inf}\left\{{\mathrm{e}}^{-\theta t}·\mathrm{tr}\left(\mathbb{E}{\mathrm{e}}^{\theta {\sum }_k{\mathbf{X}}_k}\right)\right\},$$

where ${\lambda }_{max}$ denotes the largest eigenvalue and $\mathbb{E}$ stands for the expectation operation. By using Golden-Thompson inequality, Ahlswede and Winter [6] bounded the trace of the matrix moment generating function (mgf) in the following way:

$$\begin{array}{c}\hfill \mathrm{tr}\left(\mathbb{E}{\mathrm{e}}^{\theta {\sum }_k{\mathbf{X}}_k}\right)\le \mathrm{tr}(\mathbf{I})·\left[\prod _k{\lambda }_{max}\left(\mathbb{E}{\mathrm{e}}^{\theta {\mathbf{X}}_k}\right)\right]=d·exp\left(\sum _k{\lambda }_{max}\left(log\mathbb{E}{\mathrm{e}}^{\theta {\mathbf{X}}_k}\right)\right),\end{array}$$

(1)

where $\mathrm{tr}(\mathbf{A})$ stands for the trace of the matrix $\mathbf{A}$. By applying Lieb’s concavity theorem, Tropp [7] achieved a tighter matrix mgf bound than the above one:

$$\mathrm{tr}\left(\mathbb{E}{\mathrm{e}}^{\theta {\sum }_k{\mathbf{X}}_k}\right)\le d·exp\left({\lambda }_{max}\left(\sum _{k}log\mathbb{E}{\mathrm{e}}^{\theta {\mathbf{X}}_k}\right)\right),$$

(2)

where “the eigenvalue of sum of matrices” is smaller than “the sum of eigenvalues of matrices” in the right-hand side of (1). However, there remains a shortcoming that the result (2) is dependent with the matrix dimension d, and its right-hand side will become loose for high-dimensional matrices.

To overcome the shortcoming, Hsu et al. [8] employed the intrinsic dimension $\frac{\mathrm{tr}(\mathbf{X})}{{\lambda }_{max}(\mathbf{X})}$ to replace the ambient dimension d in the case of real symmetric matrices. Minsker [9] provided a dimension-free version of Bernstein’s inequality for sequences of independent random matrices. Zhang et al. [10] introduced a diagonalization method to obtain the tail bounds for the largest singular values of the sum of random matrices. Although their bounds are independent of the matrix dimension and overcame the aforementioned first shortcoming, there remains a challenge to select the appropriate parameters to obtain the tighter bounds.

There are also some small-deviation results on one single random matrix. Edelman [21] presented the small-deviation behavior of the smallest singular value of a Gaussian matrix:

$$\underset{d\to \infty }{lim}\mathbb{P}\left\{s_{min}(\mathbf{A})\le \frac{ϵ}{\sqrt{d}}\right\}=1-exp\left(-ϵ-\frac{{ϵ}^2}{2}\right),$$

where $\mathbf{A}$ is a $d\times d$ random matrix whose entries are independent standard normal random variables. Rudelson and Vershynin [22] studied the small-deviation bound of the smallest singular value of a sub-Gaussian matrix:

$$\mathbb{P}\left\{s_{min}(\mathbf{B})\le \frac{ϵ}{\sqrt{d}}\right\}\le C·ϵ+c^d,$$

where $C>0$, $c\in (0,1)$ only depends on the sub-Gaussian moment of its entries and $\mathbf{B}$ is a $d\times d$ random matrix whose entries are i.i.d. sub-Gaussian random variables with zero mean and unit variance. However, to the best of our knowledge, there is little work on the small-deviation inequalities for sums of random matrices.

1.2. Overview of Main Results

In this paper, we present the small-deviation inequalities for the largest eigenvalue for sums of independent random Hermitian matrices, i.e., the upper bound of

$$\mathbb{P}\left\{{\lambda }_{max}\left(\sum _k{\mathbf{X}}_k\right)\le ϵ\right\}.$$

In particular, we first present some basic small-deviation results of random matrices. We then obtain several types of small-deviation inequalities for the largest eigenvalue of sums of independent random positive semi-definite (PSD) matrices. In contrast to the large-deviation inequalities for random matrices, the resulting small-deviation inequalities are independent of the matrix dimension d and thus our finding are applicable to the high-dimensional and even infinite-dimensional cases.

The rest of this paper is organized as follows. In Section 2, we introduce some useful notations and then give some basic results on small-deviation inequalities for random matrices. The small-deviation results for sums of random PSD matrices are presented in Section 3. The last section concludes the paper.

2. Basic Small-Deviation Inequalities for Random Matrices

In this section, we first introduce the necessary notations and then present some basic small-deviation results of random matrices.

2.1. Necessary Notations

Given a Hermitian matrix $\mathbf{A}$, denote ${\lambda }_{max}(\mathbf{A})$ and ${\lambda }_{min}(\mathbf{A})$ as the largest and the smallest eigenvalues of $\mathbf{A}$, respectively. Denote $\mathrm{tr}(\mathbf{A})$ and $\parallel \mathbf{A}\parallel$ as the trace and the spectral norm of $\mathbf{A}$, respectively. Let $\mathbf{I}$ be the identity matrix, $\mathbf{U}$ be the unitary matrix and ${\mathbf{U}}^{*}$ stand for the Hermitian adjoint of $\mathbf{U}$.

By the spectral mapping theorem, given a real-value function $f:\mathbb{R}\to \mathbb{R}$, then

$$f(\mathbf{A})=\mathbf{U}·f(\mathsf{\Lambda })·{\mathbf{U}}^{*},$$

where $\mathbf{A}=\mathbf{U}\mathsf{\Lambda }{\mathbf{U}}^{*}$ is a diagonalization of $\mathbf{A}$. If $f(a)\le g(a)$ for $a\in I$ when the eigenvalues of $\mathbf{A}$ lie in I, then there holds that $f(\mathbf{A})⪯g(\mathbf{A})$, where the semi-definite partial order ⪯ is defined as follows:

$$\mathbf{A}⪯\mathbf{H}\iff \mathbf{H}-\mathbf{A} \mathrm{is} \mathrm{positive} \mathrm{semi}-\mathrm{definite}.$$

2.2. Basic Small-Deviation Inequalities for Random Matrices

Subsequently, we produce the small-deviation inequalities for random matrices. First, we consider a small-deviation bound for one single matrix:

Lemma 1. 

Let $\mathit{Y}$ be a random Hermitian matrix. Then for any $ϵ>0$,

$$\mathbb{P}\left\{{\lambda }_{max}(\mathit{Y})\le ϵ\right\}\le \underset{\theta >0}{inf}\left\{\frac{1}{d}·{\mathrm{e}}^{\theta ϵ}·\mathbb{E}\mathrm{tr}\left({\mathrm{e}}^{-\theta \mathit{Y}}\right)\right\}.$$

Proof. 

For any $\theta >0$, we have

$$\begin{array}{cc}\hfill \mathbb{P}\left\{{\lambda }_{max}(\mathbf{Y})\le ϵ\right\}& =\mathbb{P}\{{\mathrm{e}}^{-{\lambda }_{max}(\theta \mathbf{Y})}\ge {\mathrm{e}}^{-\theta ϵ}\}\hfill \\ & \le \mathbb{E}{\mathrm{e}}^{-{\lambda }_{max}(\theta \mathbf{Y})}·{\mathrm{e}}^{\theta ϵ}[\mathrm{by} \mathrm{Markov}’\mathrm{s} \mathrm{inequality}]\hfill \\ & =\mathbb{E}{\mathrm{e}}^{{\lambda }_{min}(-\theta \mathbf{Y})}·{\mathrm{e}}^{\theta ϵ}[\mathrm{since}-{\lambda }_{max}(\mathbf{A})={\lambda }_{min}(-\mathbf{A})]\hfill \\ & =\mathbb{E}{\lambda }_{min}({\mathrm{e}}^{-\theta \mathbf{Y}})·{\mathrm{e}}^{\theta ϵ}[\mathrm{by} \mathrm{Spectral} \mathrm{mapping} \mathrm{theorem}]\hfill \\ & \le \frac{1}{d}·{\mathrm{e}}^{\theta ϵ}·\mathbb{E}\mathrm{tr}\left({\mathrm{e}}^{-\theta \mathbf{Y}}\right).\hfill \end{array}$$

The last inequality holds because the minimum eigenvalue of a positive definite (pd) matrix is dominated by the $\mathrm{tr}(·)/d$. Since this inequality holds for any $\theta >0$, taking an infimum over $\theta >0$ completes the proof. □

Then, by using the subadditivity of the matrix cumulant generating function (see [7], Lemma 3.4), we obtain the small-deviation bound for sums of random matrices:

Theorem 1. 

Let $\{{\mathit{X}}_1,{\mathit{X}}_2,\cdots ,{\mathit{X}}_K\}$ be a finite sequence of independent random Hermitian matrices. Then for any $ϵ>0$,

$$\mathbb{P}\left\{{\lambda }_{max}\left(\sum _k{\mathit{X}}_k\right)\le ϵ\right\}\le \underset{\theta >0}{inf}\left\{{\mathrm{e}}^{\theta ϵ}·exp\left({\lambda }_{max}\left(\sum _{k}log\mathbb{E}{\mathrm{e}}^{-\theta {\mathbf{X}}_k}\right)\right)\right\}.$$

(3)

Proof. 

By combining Lemma 1 and Lemma 3.4 of [7], we have for any $\theta >0$,

$$\begin{array}{cc}\hfill \mathbb{P}\left\{{\lambda }_{max}\left(\sum _k{\mathbf{X}}_k\right)\le ϵ\right\}& \le \frac{1}{d}·{\mathrm{e}}^{\theta ϵ}·\mathbb{E}\mathrm{tr}\left({\mathrm{e}}^{-\theta {\sum }_k{\mathbf{X}}_k}\right)\hfill \\ & \le \frac{1}{d}·{\mathrm{e}}^{\theta ϵ}·\mathrm{tr}\left(exp\left(\sum _{k}log\mathbb{E}{\mathrm{e}}^{-\theta {\mathbf{X}}_k}\right)\right)\hfill \\ & \le \frac{1}{d}·{\mathrm{e}}^{\theta ϵ}·d·{\lambda }_{max}\left(exp\left(\sum _{k}log\mathbb{E}{\mathrm{e}}^{-\theta {\mathbf{X}}_k}\right)\right)\hfill \\ & ={\mathrm{e}}^{\theta ϵ}·exp\left({\lambda }_{max}\left(\sum _{k}log\mathbb{E}{\mathrm{e}}^{-\theta {\mathbf{X}}_k}\right)\right).\hfill \end{array}$$

Taking the infimum over $\theta >0$ completes the proof. □

Please note that the above small-deviation bound is independent of the matrix dimension d, and thus it is applicable to the scenarios of high-dimensional and even infinite-dimensional matrices. In addition, we also derive the following small-deviation bounds for sums of random matrices.

Corollary 1. 

Let $\{{\mathit{X}}_1,{\mathit{X}}_2,\cdots ,{\mathit{X}}_K\}$ be a sequence of independent random Hermitian matrices. Assume that there is a function $g(\theta )$ and a sequence $\left\{{\mathit{A}}_k\right\}$ of fixed Hermitian matrices such that

$$\mathbb{E}{\mathrm{e}}^{-\theta {\mathit{X}}_k}⪯{\mathrm{e}}^{g(\theta )·{\mathit{A}}_k},\forall \theta >0.$$

(4)

1. 

Define the scalar parameter

$${\eta }_1:={\lambda }_{max}\left(\sum _k{\mathit{A}}_k\right).$$

If $g(\theta )>0$, then for any $ϵ>0$,

$$\mathbb{P}\left\{{\lambda }_{max}\left(\sum _k{\mathit{X}}_k\right)\le ϵ\right\}\le \underset{\theta >0}{inf}\left\{exp\left(\theta ϵ+g(\theta )·{\eta }_1\right)\right\}.$$

(5)

2. 

Define the scalar parameter

$${\eta }_2:={\lambda }_{min}\left(\sum _k{\mathit{A}}_k\right).$$

If $g(\theta )<0$, then for any $ϵ>0$,

$$\mathbb{P}\left\{{\lambda }_{max}\left(\sum _k{\mathit{X}}_k\right)\le ϵ\right\}\le \underset{\theta >0}{inf}\left\{exp\left(\theta ϵ+g(\theta )·{\eta }_2\right)\right\}.$$

(6)

Proof. 

It follows from (4) that

$$log\mathbb{E}{\mathrm{e}}^{-\theta {\mathbf{X}}_k}⪯g(\theta )·{\mathbf{A}}_k,$$

and substituting it into Theorem 1 leads to the result (5). Then, the fact that ${\lambda }_{max}(-\mathbf{X})=-{\lambda }_{min}(\mathbf{X})$ for any Hermitian matrix $\mathbf{X}$ leads to the result (6). This completes the proof. □

By using the logarithm operation, we then obtain another small-deviation bound for sums of random matrices:

Corollary 2. 

Let $\{{\mathit{X}}_1,{\mathit{X}}_2,\cdots ,{\mathit{X}}_K\}$ be a sequence of independent random Hermitian matrices. Then for any $ϵ>0$,

$$\mathbb{P}\left\{{\lambda }_{max}\left(\sum _k{\mathit{X}}_k\right)\le ϵ\right\}\le \underset{\theta >0}{inf}exp\left(\theta ϵ+K·log{\lambda }_{max}\left(\frac{1}{K}\sum _{k=1}^K\mathbb{E}{\mathrm{e}}^{-\theta {\mathit{X}}_k}\right)\right).$$

Proof. 

Since the matrix logarithm is operator concave, for each $\theta >0$, we have

$$\sum _{i=1}^{K}log\mathbb{E}{\mathrm{e}}^{-\theta {\mathbf{X}}_k}=K·\frac{1}{K}\sum _{i=1}^{K}log\mathbb{E}{\mathrm{e}}^{-\theta {\mathbf{X}}_k}⪯K·log\left(\frac{1}{K}\sum _{i=1}^K\mathbb{E}{\mathrm{e}}^{-\theta {\mathbf{X}}_k}\right).$$

According to (3), we then arrive at

$$\mathbb{P}\left\{{\lambda }_{max}\left(\sum _k{\mathbf{X}}_k\right)\le ϵ\right\}\le \frac{1}{d}·{\mathrm{e}}^{\theta ϵ}·\mathrm{tr}exp\left(K·log\left(\frac{1}{K}\sum _{i=1}^K\mathbb{E}{\mathrm{e}}^{-\theta {\mathbf{X}}_k}\right)\right).$$

Since the trace of a matrix can be bounded by d times of its maximum eigenvalue, taking the infimum over $\theta >0$ completes the proof. □

The following presents the relationship between one random PSD matrix and a sum of PSD random matrices.

Lemma 2. 

Let $\{{\mathit{X}}_1,{\mathit{X}}_2,\cdots ,{\mathit{X}}_K\}$ be a sequence of independent random Hermitian PSD matrices. Then for all k and any $ϵ>0$,

$$\mathbb{P}\left\{{\lambda }_{max}\left(\sum _k{\mathit{X}}_k\right)\le ϵ\right\}\le \prod _k\mathbb{P}\left\{{\lambda }_{max}({\mathit{X}}_k)\le ϵ\right\}\le \mathbb{P}\left\{{\lambda }_{max}({\mathit{X}}_k)\le ϵ\right\}.$$

Proof. 

Since $\{{\mathbf{X}}_1,{\mathbf{X}}_2,\cdots ,{\mathbf{X}}_K\}$ are PSD, we have

$$\begin{array}{cc}\hfill \mathbb{P}\left\{{\lambda }_{max}\left(\sum _k{\mathbf{X}}_k\right)\le ϵ\right\}& \le \mathbb{P}\left\{max\left({\lambda }_{max}({\mathbf{X}}_1),\cdots ,{\lambda }_{max}({\mathbf{X}}_K)\right)\le ϵ\right\}\hfill \\ & =\prod _k\mathbb{P}\left\{{\lambda }_{max}({\mathbf{X}}_k)\le ϵ\right\}\hfill \\ & \le \mathbb{P}\left\{{\lambda }_{max}({\mathbf{X}}_k)\le ϵ\right\}.\hfill \end{array}$$

The last inequality holds for any $k=1,2\cdots ,K$. This completes the proof. □

This lemma shows that the small-deviation probability for sums of random matrices can be bounded by using the small-deviation probability for one single matrix. This fact suggests that the small-deviation bound could be independent of the size of matrix sequence, while this phenomenon will not arise in the large-deviation scenario.

3. Small-Deviation Inequalities for Positive Semi-Definite Random Matrices

In this section, we present several types of small-deviation inequalities for the largest eigenvalue of sums of independent random PSD matrices. Similar to the scalar version of small-deviation inequalities, there remains a challenge to bound the term $\mathbb{E}{\mathrm{e}}^{-\theta {\mathbf{X}}_k}$. Here, we adapt some methods to handle this issue.

First, we introduce the negative moment estimate for the largest eigenvalue to derive a small-deviation inequality for sums of random matrices:

Theorem 2. 

Let $\{{\mathit{X}}_1,{\mathit{X}}_2,\cdots ,{\mathit{X}}_K\}$ be a sequence of independent random Hermitian PSD matrices. Given a $p>0$, if there exists a positive constant $C_p$ such that

$${\left[{\lambda }_{max}\left(\sum _k{\mathbb{E}\mathit{X}}_k\right)\right]}^{-p}<C_p,$$

then there holds that for any $ϵ>0$,

$$\mathbb{P}\left\{{\lambda }_{max}\left(\sum _k{\mathit{X}}_k\right)\le ϵ\right\}\le C_p{ϵ}^p.$$

Proof. 

It follows from Jensen’s inequality that

$$\mathbb{E}{\left({\lambda }_{max}\left(\sum _k{\mathbf{X}}_k\right)\right)}^{-p}\le {\left(\mathbb{E}{\lambda }_{max}\left(\sum _k{\mathbf{X}}_k\right)\right)}^{-p}\le {\left[{\lambda }_{max}\left(\sum _k{\mathbb{E}\mathbf{X}}_k\right)\right]}^{-p}<C_p.$$

Then, the Markov’s inequality yields

$$\begin{array}{cc}\hfill \mathbb{P}\left\{{\lambda }_{max}\left(\sum _k{\mathbf{X}}_k\right)\le ϵ\right\}& =\mathbb{P}\left\{{\left[{\lambda }_{max}\left(\sum _k{\mathbf{X}}_k\right)\right]}^{-p}\ge {ϵ}^{-p}\right\}\hfill \\ & \le {ϵ}^p·\mathbb{E}{\left({\lambda }_{max}\left(\sum _k{\mathbf{X}}_k\right)\right)}^{-p}\le C_p{ϵ}^p.\hfill \end{array}$$

This completes the proof. □

In this theorem, we impose an assumption that the negative moment of ${\lambda }_{max}\left({\sum }_k{\mathbf{X}}_k\right)$ is bounded. In general, this assumption is mild, and can be satisfied in most cases. The following small-deviation results are derived under that condition that the eigenvalues of the matrices $\left\{{\mathbf{X}}_k\right\}$ are bounded:

Theorem 3. 

Let $\{{\mathit{X}}_1,{\mathit{X}}_2,\cdots ,{\mathit{X}}_K\}$ be a sequence of independent random Hermitian PSD matrices such that ${\lambda }_{max}({\mathit{X}}_k)\le L$ ($\forall k=1,2,\cdots ,K$) almost surely. Then for any $ϵ>0$,

$$\mathbb{P}\left\{{\lambda }_{max}\left(\sum _k{\mathit{X}}_k\right)\le ϵ\right\}\le {\left(\frac{\mu }{ϵ}\right)}^{ϵ/L}·exp\left(\frac{ϵ-\mu }{L}\right),$$

(7)

where

$$\mu :={\lambda }_{min}\left(\sum _k\mathbb{E}{\mathit{X}}_k\right).$$

Furthermore, there holds that for any $ϵ>0$,

$$\mathbb{P}\left\{{\lambda }_{max}\left(\sum _k{\mathit{X}}_k\right)\le ϵ\right\}\le {\left(\frac{1}{ϵ}\right)}^{K\epsilon /L}·{\left(\prod _{k=1}^K{\mu }_k\right)}^{ϵ/L}·exp\left(\frac{Kϵ-{\sum }_k{\mu }_k}{L}\right),$$

(8)

where

$${\mu }_k={\lambda }_{min}(\mathbb{E}{\mathit{X}}_k).$$

Proof. 

For any $\theta >0$ and $x\in [0,L]$, there holds that

$${\mathrm{e}}^{-\theta x}\le 1+\frac{{\mathrm{e}}^{-\theta L}-1}{L}·x\le exp\left(\frac{{\mathrm{e}}^{-\theta L}-1}{L}·x\right).$$

According to transfer rule, we have,

$$log\mathbb{E}{\mathrm{e}}^{-\theta {\mathbf{X}}_k}⪯\frac{{\mathrm{e}}^{-\theta L}-1}{L}\mathbb{E}{\mathbf{X}}_k.$$

(9)

By substituting (9) into Corollary 1, we then have,

$$\begin{array}{cc}\hfill \mathbb{P}\left\{{\lambda }_{max}\left(\sum _k{\mathbf{X}}_k\right)\le ϵ\right\}& \le \underset{\theta >0}{inf}{\mathrm{e}}^{\theta ϵ}·exp\left({\lambda }_{max}\left(\sum _k\frac{{\mathrm{e}}^{-\theta L}-1}{L}\mathbb{E}{\mathbf{X}}_k\right)\right)\hfill \\ & =\underset{\theta >0}{inf}exp\left(\theta ϵ+{\lambda }_{max}\left(\frac{{\mathrm{e}}^{-\theta L}-1}{L}\sum _k\mathbb{E}{\mathbf{X}}_k\right)\right)\hfill \\ & =\underset{\theta >0}{inf}exp\left(\theta ϵ+\frac{{\mathrm{e}}^{-\theta L}-1}{L}·{\lambda }_{min}\left(\sum _k\mathbb{E}{\mathbf{X}}_k\right)\right)\hfill \\ & =\underset{\theta >0}{inf}exp\left(\theta ϵ+\frac{{\mathrm{e}}^{-\theta L}-1}{L}·\mu \right),\hfill \end{array}$$

since $\frac{{\mathrm{e}}^{-\theta L}-1}{L}<0$. The infimum is achieved at $\theta =\frac{1}{L}log(\frac{\mu }{ϵ})$, which leads to the result of (7).

Moreover, the combination of Lemma 1 and (9) leads to

$$\mathbb{P}\left\{{\lambda }_{max}({\mathbf{X}}_k)\le ϵ\right\}\le {\left(\frac{{\mu }_k}{ϵ}\right)}^{ϵ/L}·exp\left(\frac{ϵ-{\mu }_k}{L}\right).$$

Then, the result (8) is derived from Lemma 2. This completes the proof. □

Actually, the above results are derived from the geometric point of view, where the term ${\mathrm{e}}^{-\theta x}$ is bounded by the linear function $1+\frac{{\mathrm{e}}^{-\theta L}-1}{L}$ for any $x\in [0,L]$. Finally, we study the small-deviation inequalities for random matrix series ${\sum }_k{x}_k{\mathbf{A}}_k$, which is a sum of fixed Hermitian pd matrices ${\mathbf{A}}_k$ weighted by random variables $x_k$.

Theorem 4. 

Let $\{{\mathit{A}}_1,{\mathit{A}}_2,\cdots ,{\mathit{A}}_K\}$ be a sequence of fixed Hermitian pd matrices, and $\{x_1,x_2,\cdots ,x_K\}$ be a finite sequence of independent variables. If there exist the constants $C>0$ and $\alpha >0$ such that for all $\theta >0$,

$$\mathbb{E}{\mathrm{e}}^{-\theta x_k}\le C·{\theta }^{-\alpha },$$

(10)

then there holds that for any $0<ϵ<\frac{K\alpha }{\mathrm{e}}·\sqrt[\alpha ]{\frac{K}{C\nu }}$,

$$\mathbb{P}\left\{{\lambda }_{max}\left(\sum _k{x}_k{\mathit{A}}_k\right)\le ϵ\right\}\le {\left(\frac{\mathrm{e}ϵ}{K\alpha }\right)}^{\alpha K}·{\left(\frac{C\nu }{K}\right)}^K,$$

(11)

where

$$\nu ={\lambda }_{max}\left(\sum _k{{\mathit{A}}_k}^{-\alpha }\right).$$

Furthermore, for any $0<ϵ<(\alpha /\mathrm{e})·C^{-\frac{1}{\alpha }}·{\left({\prod }_{k=1}^K{\nu }_k\right)}^{-\frac{1}{\alpha K}}$,

$$\mathbb{P}\left\{{\lambda }_{max}\left(\sum _k{x}_k{\mathit{A}}_k\right)\le ϵ\right\}\le \left(\prod _{k=1}^K{\nu }_k\right)·C^K·{\left(\frac{\mathrm{e}ϵ}{\alpha }\right)}^{K\alpha },$$

(12)

where

$${\nu }_k={\lambda }_{max}({{\mathit{A}}_k}^{-\alpha }).$$

Proof. 

According to transfer rule, we have,

$$\mathbb{E}{\mathrm{e}}^{-\theta x_k{\mathbf{A}}_k}⪯C·{(\theta {\mathbf{A}}_k)}^{-\alpha }.$$

(13)

By substituting (13) into Corollary 2, we then have,

$$\begin{array}{cc}\hfill \mathbb{P}\left\{{\lambda }_{max}\left(\sum _k{x}_k{\mathbf{A}}_k\right)\le ϵ\right\}& \le \underset{\theta >0}{inf}exp\left(\theta ϵ+K·log{\lambda }_{max}\left(\frac{1}{K}\sum _{k=1}^{K}C·{(\theta {\mathbf{A}}_k)}^{-\alpha }\right)\right)\hfill \\ & =\underset{\theta >0}{inf}exp\left(\theta ϵ+K·log{\lambda }_{max}\left(\frac{C{\theta }^{-\alpha }}{K}\sum _{k=1}^K{\mathbf{A}}_k^{-\alpha }\right)\right)\hfill \\ & =\underset{\theta >0}{inf}exp\left(\theta ϵ+K·log\frac{C\nu }{K{\theta }^{\alpha }}\right).\hfill \end{array}$$

The infimum will be attained at $\theta =\frac{\alpha K}{ϵ}$, and it leads to the result of (11). Moreover, the combination of Lemma 1 and (13) leads to

$$\mathbb{P}\left\{{\lambda }_{max}(x_k{\mathbf{A}}_k)\le ϵ\right\}\le C·{\nu }_k·{\left(\frac{\mathrm{e}ϵ}{\alpha }\right)}^{\alpha }.$$

Then, the result (12) is resulted from Lemma 2. This completes the proof. □

The above results hold under the condition (10) that $\mathbb{E}{\mathrm{e}}^{-\theta x_k}$ has a power-type upper bound of $C·{\theta }^{-\alpha }$. This condition is mild, and we refer to [23] for the details. Moreover, to keep the results (11) and (12) non-trivial, their right-hand sides should be less than one, and thus we arrive at $ϵ<\frac{K\alpha }{\mathrm{e}}·\sqrt[\alpha ]{\frac{K}{C\nu }}$ and $ϵ<(\alpha /\mathrm{e})·C^{-\frac{1}{\alpha }}·{\left({\prod }_{k=1}^K{\nu }_k\right)}^{-\frac{1}{\alpha K}}$, respectively.

4. Conclusions

In this paper, we present the small-deviation inequalities for the largest eigenvalues of sums of random matrices. In particular, we first give some basic results on small-deviation inequalities for random matrices. We then study the small-deviation inequalities for sums of independent random PSD matrices. In contrast to the large-deviation inequalities for random matrices, our results are independent of the matrix dimension d and thus can be applicable to the scenarios of high-dimensional and even infinite-dimensional matrices. In addition, by using the Hermitian dilation (see Section 2.6 of [7]), our small-deviation results can also be extended to the scenario of non-Hermitian random matrices.