Small-Deviation Inequalities for Sums of Random Matrices

Random matrices have played an important role in many fields including machine learning, quantum information theory and optimization. One of the main research focuses is on the deviation inequalities for eigenvalues of random matrices. Although there are intensive studies on the large-deviation inequalities for random matrices, only a few of works discuss the small-deviation behavior of random matrices. In this paper, we present the small-deviation inequalities for the largest eigenvalues of sums of random matrices. Since the resulting inequalities are independent of the matrix dimension, they are applicable to the high-dimensional and even the infinite-dimensional cases.


Introduction
Random matrices have been widely used in many problems, for example, the compressed sensing (Chandrasekaran et al., 2012), the high-dimensional data analysis (Bühlmann and Van De Geer, 2011), the matrix approximation (Halko et al., 2011;Gittens and Mahoney, 2016) and the dimension reduction (Clarkson and Woodruff, 2013).In the literature, one of main research issues is to study the deviation behavior of the eigenvalues (or singular values) of random matrices.
In general, there are two types of deviation results studied in probability theory: one is the large-deviation inequality that describes the behavior of the probability P(|x| > t) for large t; and the other is the small-deviation (or small-ball) inequality that controls the probability P(|x| < ǫ) for small ǫ.
The early large-deviation inequalities for sums of random matrices can be dated back to the work of Ahlswede and Winter (2002).Tropp (2012) improved their results and developed a user-friendly framework to obtain the large-deviation inequalities for sums of random matrices.To overcome the limitation of the matrix-dimension dependence, Hsu et al. (2012) and Minsker (2017) introduced the concepts of intrinsic dimension and effective dimension to tighten the large-deviation inequalities, respectively.Moreover, Zhang et al. (2017) applied a diagonalization method to obtain the dimension-free large-deviation random for largest singular value of sums of random matrices, while it remains a challenge to select the auxiliary matrices and functions.In the scenario of single random matrix, Ledoux (2007) studied the largest eigenvalues of Gaussian unitary ensemble matrices and Vershynin (2010) studied the singular values of the sub-Gaussian and sub-exponential matrices.
Small-deviation problems were stemmed from some practical applications, e.g., approximation problem (Li et al., 1999), Brownian pursuit problems (Li and Shao, 2001), quantization problem (Dereich et al., 2003) and convex geometry (Klartag and Vershynin, 2007).For more details, we refer to the bibliography maintained by Lifshits (2006).There have been some works on the small-deviation inequalities for the specific types of random matrices.Aubrun (2005) obtained the small-deviation inequalities for the largest eigenvalue of a single Gaussian unitary ensemble matrix.Rudelson and Vershynin (2010) presented the small-deviation inequalities for the smallest singular value of the random matrix with independent entries.Volodko (2014) estimated the small-deviation probability of the determinant of the matrix BB T , where B is a d ×∞ random matrix whose entries obey a centered joint Gaussian distribution.To the best of our knowledge, there are few works on the small-deviation inequalities for sums of random matrices.

Related Works
Let {X 1 , X 2 , • • • , X K } ⊂ C d×d be a finite sequence of independent random Hermitian matrices.It follows from Markov's inequality that where λ max denotes the largest eigenvalue.By using Golden-Thompson inequality, Ahlswede and Winter (2002) bounded the trace of the matrix moment generating function (mgf) in the following way: where tr(A) stands for the trace of the matrix A. By applying Lieb's concavity theorem, Tropp (2012) achieved a tighter matrix mgf bound than the above one: tr where "the eigenvalue of sum of matrices" is smaller than "the sum of eigenvalues of matrices" in the right-hand side of (1).However, there still remains a shortcoming that the result (2) is dependent with the matrix dimension d, and its right-hand side will become loose for highdimensional matrices.
To overcome the shortcoming, Hsu et al. (2012) employed the intrinsic dimension tr(X) λmax(X) to replace the ambient dimension d in the case of real symmetric matrices.Minsker (2017) provided a dimension-free version of Bernstein's inequality for sequences of independent random matrices.Zhang et al. (2017) introduced a diagonalization method to obtain the tail bounds for LSV of the sum of random matrices.Although their bounds are independent of the matrix dimension and overcame the aforementioned first shortcoming, there still remains a challenge to select the appropriate parameters to obtain the tighter bounds.
There are also some small-deviation results on one single random matrix.Edelman (1988) presented the small-deviation behavior of the smallest singular value of a Gaussian matrix: where A is a d × d random matrix whose entries are independent standard normal random variables.Rudelson and Vershynin (2008) studied the the small-deviation bound of the smallest singular value of a sub-gaussian matrix: where C > 0, c ∈ (0, 1) is only depend on the sub-gaussian moment of its entries and B is a d × d random matrix whose entries are i.i.d.sub-gaussian random variables with zero mean and unit variance.However, to the best of our knowledge, there is few work on the small-deviation inequalities for sums of random matrices.

Overview of Main Results
In this paper, we present the small-deviation inequalities for the largest eigenvalue for sums of independent random Hermitian matrices, that is, the upper bound of In particular, we first present some basic small-deviation results of random matrices.We then obtain several types of small-deviation inequalities for the largest eigenvalue of sums of independent random positive semi-definite (psd) matrices.Different from the large-deviation inequalities for random matrices, the resulting small-deviation inequalities are independent of the matrix dimension d and thus our finding are applicable to the high-dimensional and even infinite-dimensional cases.
The rest of this paper is organized as follows.In Section 2, we introduce some useful notations and then give some basic results on small-deviation inequalities for random matrices.The smalldeviation results for sums of random psd matrices are presented in Section 3. The last section concludes the paper.

Basic Small-Deviation Inequalities for Random Matrices
In this section, we first introduce the necessary notations and then present some basic smalldeviation results of random matrices.

Necessary Notations
Given a Hermitian matrix A, denote λ max (A) and λ min (A) as the largest and the smallest eigenvalues of A, respectively.Denote tr(A) and A as the trace and the spectral norm of A, respectively.Let I be the identity matrix, U be the unitary matrix and U * stand for the Hermitian adjoint of U.
By the spectral mapping theorem, given a real-value function f : R → R, then for a ∈ I when the eigenvalues of A lie in I, then there holds that f (A) g(A).

Basic Small-Deviation Inequalities for Random Matrices
Subsequently, we come up with the small-deviation inequalities for random matrices.First, we consider a small-deviation bound for one single matrix: Lemma 2.1 Let Y be a random Hermitian matrix.Then for any ǫ > 0, Proof: For any θ > 0, we have The last inequality holds because the minimum eigenvalue of a positive definite (pd) matrix is dominated by the tr(•)/d.Since this inequality holds for any θ > 0, taking an infimum over θ > 0 completes the proof.Then, by using the subadditivity of the matrix cumulant generating function (see Tropp, 2012, Lemma 3.4), we obtain the small-deviation bound for sums of random matrices: be a finite sequence of independent random Hermitian matrices.Then for any ǫ > 0, Proof: By combining Lemma 2.1 and Lemma 3.4 of (Tropp, 2012), we have for any θ > 0, Taking the infimum over θ > 0 completes the proof.Note that the above small-deviation bound is independent of the matrix dimension d, and thus it is applicable to the scenarios of high-dimensional and even infinite-dimensional matrices.In addition, we also derive the following small-deviation bounds for sums of random matrices.
be a sequence of independent random Hermitian matrices.Assume that there are a function g(θ) and a sequence {A k } of fixed Hermitian matrices such that If g(θ) > 0, then for any ǫ > 0, (ii) Define the scalar parameter If g(θ) < 0, then for any ǫ > 0, and substituting it into Theorem 2.1 leads to the result (5).Then, the fact λ max (−X) = −λ min (X) leads to the result (6).This completes the proof.By using the logarithm operation, we then obtain another small-deviation bound for sums of random matrices: be a sequence of independent random Hermitian matrices.Then for any ǫ > 0, Proof: Since the matrix logarithm is operator concave, for each θ > 0, we have According to (3), we then arrive at Since the trace of a matrix can be bounded by d times of its maximum eigenvalue, taking the infimum over θ > 0 completes the proof.
The following presents the relationship between one random psd matrix and a sum of psd random matrices.
be a sequence of independent random Hermitian psd matrices.Then for any ǫ > 0, The last inequality holds for any k = 1, 2 • • • , K.This completes the proof.
This lemma shows that the small-deviation probability for sums of random matrices can be bounded by using the small-deviation probability for one single matrix.This fact suggests that the small-deviation bound could be independent of the size of matrix sequence, while this phenomenon will not arise in the large-deviation scenario.
3 Small-deviation Inequalities for Positive Semi-Definite Random Matrices In this section, we present several types of small-deviation inequalities for the largest eigenvalue of sums of independent random psd matrices.Similar to the scalar version of small-deviation inequalities, there remains a challenge to bound the term Ee −θX k .Here, we adapt some methods to handle this issue.First, we introduce the negative moment estimate for the largest eigenvalue to derive a smalldeviation inequality for sums of random matrices: be a sequence of independent random Hermitian psd matrices.Given a p > 0, if there exists a positive constant C p such that then there holds that for any ǫ > 0, Then, the Markov's inequality yields This completes the proof.
In this theorem, we impose an assumption that the negative moment of λ max k X k is bounded.In general, this assumption is mild and can be satisfied in most cases.The following small-deviation results are derived under that condition that the eigenvalues of the matrices {X k } are bounded: where Furthermore, there holds that for any ǫ > 0, where Proof: For any θ > 0 and x ∈ [0, L], there holds that According to transfer rule, we have, By substituting (9) into the Corollary 2.1, we then have for any θ > 0, The infimum is achieved at θ = 1 L log( µ ǫ ), which leads to the result of (7).Moreover, the combination of Lemma 2.1 and (9) leads to Then, the result (8) is derived from Lemma 2.2.This completes the proof.Actually, the above results are derived from the geometric point of view, where the term e −θx is bounded by the linear function 1 + e −θL −1 L for any x ∈ [0, L].Finally, we study the smalldeviation inequalities for random matrix series k x k A k , which is a sum of fixed Hermitian psd matrices A k weighted by random variables x k .
then there holds that for any 0 < ǫ < Kα e • α K Cν , where where Proof: According to transfer rule, we have, By substituting (13) into the Corollary 2.2, we then have for any θ > 0, The infimum will be attained at θ = αK ǫ , and it leads to the result of (11).Moreover, the combination of Lemma 2.1 and (13) leads to Then, the result ( 12) is resulted from Lemma 2.2.This completes the proof.The above results hold under the condition (10) that E e −θx k has a power-type upper bound of C • θ −α .This condition is mild and we refer to Li (2012) for the details.Moreover, to keep the results ( 11) and ( 12

Conclusion
In this paper, we present the small-deviation inequalities for the largest eigenvalues of sums of random matrices.In particular, we first give some basic results on small-deviation inequalities for random matrices.We then study the small-deviation inequalities for sums of independent random psd matrices.Different from the large-deviation inequalities for random matrices, our results are independent of the matrix dimension d and thus can be applicable to the scenarios of high-dimensional and even infinite-dimensional matrices.In addition, by using the Hermitian dilation (see Tropp, 2012, Section 2.6), our small-deviation results can also be extended to the scenario of non-Hermitian random matrices.