Editorial: New Advances in High-Dimensional and Non-Asymptotic Statistics

1. Introduction

This editorial paper reviews the Special Issue “New Advances in High-Dimensional and Non-asymptotic Statistics” and summarizes the ten collected papers. For high-dimensional data analysis, we begin by discussing the challenges posed by datasets where the number of covariates p can be comparable to or much larger than the sample size n. Traditional statistical methods often fail in such scenarios, necessitating new approaches like regularization (e.g., Lasso) for variable selection and parameter estimation. The theoretical underpinnings of these methods are explored, including oracle inequalities that provide non-asymptotic bounds on estimation and prediction errors. Next, we resort to functional data analysis (FDA), where data points are functions rather than scalars or vectors. Key concepts like the Karhunen–Loeve expansion and Mercer’s theorem for covariance operators are discussed, along with challenges related to discrete and noisy observations of functional data, differentiating between dense and sparse sampling regimes. It also introduces the extension of these ideas to unstructured data residing in general metric spaces. The significance lies in addressing high- and infinite-dimensional and complex data scenarios increasingly common in modern scientific research, facilitating the development of non-asymptotic statistical inference and high-dimensional probability.

2. Background

In recent years, high-throughput and non-parametric complex datasets have become very common in fields such as genomics, signal processing, neuroscience, and other scientific areas. For regression problems involving complex datasets, we have encountered statistical models with the following characteristics: both the number of covariates p and the sample size n are increasing, and it is assumed that p is a function of n, i.e., $p=p\left(n\right)$. One scenario is when p grows slowly with n and satisfies $p<n$, allowing the maximum likelihood and moment estimators to possess consistency properties. This situation has been extensively studied, as shown in studies by Portnoy [1], He and Shao [2], Sur et al. [3], Zhang [4], Montanari et al. [5], and Li et al. [6]. Another scenario is in large-scale data, where p might be thousands of covariates and much larger than n, but in reality, only a few important predictors are active. For example, in microarray data analysis in genomics, p can be in the millions, while the number of observations n is only a few hundred or even tens [7]. This phenomenon has motivated modern statisticians to focus more on research in high-dimensional statistics.

For high-dimensional data, the important variables are sparse between all of them, which means that the number of important variables s is much smaller than p, i.e., $s\ll p$. The main challenge is that directly using traditional low-dimensional statistical inference and computational methods leads to high complexity. Fortunately, regularization (or penalization) optimization methods can yield sparse parameter estimation, thereby enabling efficient variable selection. In high-dimensional data, typically only a small subset of features has a significant impact on the target variable, i.e., sparsity. A well-known regularization method is Lasso, which is a shrinkage estimation method that adds an $l_1$ penalty to the least squares loss of linear models, and it was introduced by Tibshirani [8] to address estimation problems in high-dimensional linear models. Further research can be found in a monograph by Buhlmann and van de Geer [9] and a review paper by Zhang and Chen [10].

It is well-known that classical probability theory laid the mathematical foundation for traditional statistics. Cutting-edge statistical methods, keeping pace with the times, have also spawned new probability problems, which in turn have promoted the development of the probability field. For example, when $p=o\left(n\right)$, research methods for weak convergence properties in statistics differ from the central limit theorem for sums of fixed-dimensional random variables. The earliest research in this area was by Portnoy [1]. In recent years, the rapid development of cutting-edge statistical fields, such as high-dimensional covariance estimation, has in turn promoted the development of probability theory and even pure mathematics, including random matrices, large deviation inequalities, and geometric functional analysis; see Vershynin [11] and Tropp [12] for more details. These studies of probability theory related to high-dimensional statistics are collectively referred to as high-dimensional probability.

If the dimensionality of the model is particularly high (even larger than the sample size), asymptotic results for the error between model estimates and true parameters can be intractable or impossible to handle with existing techniques. Most estimation errors in traditional statistical inference are based on asymptotic errors as the sample size n tends to infinity. J. Friedman [13] introduced the term Asymptopia (merging “asymptotics” and “utopia”) to critique a common pitfall in statistical theory, namely the allure of asymptotic optimality results that, while mathematically appealing, frequently bear little practical relevance for real-world performance achievable with finite data. The rise of large language models and deep learning provides compelling empirical evidence for this critique, demonstrating how complex models can deliver exceptional performance in practical finite-sample regimes, thereby challenging the validity or utility of some theoretically derived asymptotic guarantees. For generalized linear models with “large p, small n” datasets, van de Geer [14] and Zhang and Jia [15] used the first-order conditions (also known as KKT conditions) of the Lasso optimization problem combined with techniques from sub-Gaussian concentration inequalities in probability theory to provide non-asymptotic optimal error theoretical results for Lasso estimators. When n and p are fixed, sub-Gaussian concentration and exponential tail probability inequalities (also known as large deviation inequalities) play a crucial role in deriving non-asymptotic upper bounds for errors in high-dimensional estimators; see Wainwright [16] for more details. Concurrently, the non-asymptotic properties of high-dimensional variable selection problems have driven recent probability theory research with a machine learning and AI background, as shown by Vershynin [11], Zhou et al. [17], and Zhang and Huang [18] for details.

The data variables described above are Euclidean variables that have certain limitations in practice. With the development of natural sciences and engineering applications, continuous curves or image data are observed and collected by various scientific instruments [19]. As a branch of modern statistics, functional data analysis (FDA) focuses on analyzing complex data in spaces of continuous curves, surfaces, or other forms that continuously vary. In the FDA framework, each sample element is considered a function in some abstract and infinite-dimensional space (e.g., $L_2$, RKHS, Sobolev spaces), see [20]. Due to mathematical difficulties, the theoretical analysis of FDA presents certain challenges. In applications, observed time-varying functional data may consist of dense or sparse time sampling points, and the number of sampling points may be larger than the sample size. Although discretely sampled functional curves are very similar to traditional multivariate data, the correlation between some adjacent sampling points can be very high, and we cannot arbitrarily permute these sampling points. Furthermore, the empirical version of the covariance function (operator) for functional data provides a theoretical basis for various non-parametric estimations and inferences related to functional data. The covariance function is a function on a two-dimensional continuous domain, representing an infinite-dimensional generalization of the covariance matrix in multivariate data. For analyzing the characterization and convergence properties of the eigenvalues of the covariance operator, the required mathematical tools include linear functional analysis, operator perturbation theory [21], and probability in abstract spaces; see Chapter 7 of Hsing and Eubank [22] for more information. Compared to statistical theory in Euclidean spaces, the technical details of research on the convergence rates of various function estimates in functional data models are very complex, see [23,24,25].

Functional data in $L_2$ space can also be generalized to complex functional data composed of random elements (or random objects) in a metric space $(Q,d)$. Big data generally comes in two types, namely structured data and unstructured data. Structured data is highly organized and neatly formatted, usually stored in tabular form. Unstructured data generally refers to data that is difficult to structure, such as images, documents, videos, emails, social media, and websites; all these can be considered random elements in a general metric space. As a mature type of data, structured data analysis has been relatively thoroughly studied in terms of statistical applications and theory. The statistical analysis of unstructured data is an emerging field of significant scientific importance and AI applied value. For example, the recent literature includes a study by Tavakoli et al. [26] on spatial text data analysis; Petersen and Müller [27] on studying brain functional connectivity in patients with mild cognitive impairment and Alzheimer’s disease based on Wasserstein covariance; and Chen et al. [28] on high-dimensional recurrent event data analysis using a low-dimensional dynamic factor model.

3. Current Research Status

The current research status related to this Special Issue can be divided into two parts as follows: (1) a theoretical review of high-dimensional penalized regression and (2) a literature review of functional data and functional regression models.

3.1. High-Dimensional Statistics

In machine learning and AI [17], sub-Gaussian distribution is a common data assumption, originating from a paper on Fourier analysis by the French mathematician J. Kahane in the 1960s. Subsequently, the sub-Gaussian distribution family has been frequently used in information theory [29], probability theory [30], and mathematical statistics [7,9]. High-dimensional statistical theories often rely on sub-Gaussian assumptions in data to obtain the minimax convergence rates (Chapter 15 in [16]). A zero-mean random variable $X\in \mathbb{R}$ is called sub-Gaussian if there exists a sub-Gaussian variance proxy ${\sigma }^2$ such that its moment-generating function satisfies $\mathrm{E}\left[e^{tX}\right]\le e^{t^2{\sigma }^2/2}$ for all $t\in \mathbb{R}$. We denote a sub-Gaussian random variable as $X\sim \mathrm{subG}\left({\sigma }^2\right)$. From the definition of sub-Gaussianity and Chernoff’s inequality, we can obtain $P\left(\right|X|\ge t)\le 2\exp \left\{-{\displaystyle \frac{t^2}{2{\sigma }^2}}\right\}$. Note that ${\sigma }^2$ in the sub-Gaussian definition is any value satisfying the inequality. If a sub-Gaussian parameter ${\sigma }_0^2$ is found, then any number greater than ${\sigma }_0^2$ can also serve as a sub-Gaussian parameter, so the parameter in this definition is not identifiable. Buldygin and Kozachenko [30] defined the optimal sub-Gaussian parameter as

$${\sigma }_{\mathrm{opt}}^2\left(X\right):=\inf \{{\sigma }^2\ge 0:\mathrm{E}\left[e^{tX}\right]\le e^{t^2{\sigma }^2/2},\forall t\in \mathbb{R}\}.$$

The above ${\sigma }_{\mathrm{opt}}^2\left(X\right)$ is called the optimal variance proxy. From the definition of ${\sigma }_{\mathrm{opt}}^2\left(X\right)$ and sub-Gaussianity, we have

$${\sigma }_{\mathrm{opt}}^2\left(X\right)\ge \mathrm{Var}\left(X\right)\left(\mathrm{the}\mathrm{optimal}\mathrm{variance}\mathrm{proxy}\mathrm{is}\mathrm{not}\mathrm{less}\mathrm{than}\mathrm{the}\mathrm{variance}\right).$$

When ${\sigma }_{\mathrm{opt}}^2\left(X\right)=\mathrm{Var}\left(X\right)$, the variable is called strictly sub-Gaussian. For example, Gaussian, symmetric Bernoulli, and uniform distributions are all strictly sub-Gaussian. The estimator for ${\sigma }_{\mathrm{opt}}^2\left(X\right)$ is an unresolved issue that can inspire much follow-up research in statistical machine learning, such as non-asymptotic interval estimation [31,32] and hypothesis testing requiring sub-Gaussian parameter estimates [33].

For the theory, Knight and Fu [34] proved that Lasso estimators are not asymptotically normal, and they argued that the exact and limiting distributions of Lasso estimators are difficult to derive and do not have a clear form. To avoid this trouble, a popular subsequent method is to derive non-asymptotic error bounds (oracle inequalities) for the desired high-dimensional estimators, defined as items 1 and 2 below. For Lasso-type estimators in growing dimension models $p=n^{\alpha },(\alpha \le 1)$ without restrictive sparsity or eigenvalue conditions, Greenshtein and Ritov [35] provided an oracle inequality for prediction error. In classical consistency analysis, the model size p is fixed and the sample size n tends to infinity. However, when both p and n tend to infinity, a common technique is to use non-asymptotic error upper bounds to derive the consistency of high-dimensional estimators. Let ${\beta }^{*}$ be the true regression coefficient in a high-dimensional regression model. Assume ${\{Y_i,X_i\}}_{i=1}^n\stackrel{\mathrm{i}.\mathrm{i}.\mathrm{d}.}{\sim }(X,Y)$ satisfies $\mathrm{E}\left(Y_i\right|X_i)=f\left(X_i^T{\beta }^{*}\right)$, where $X_i$ is a p-dimensional covariate, $Y_i\in \mathbb{R}$ is a Euclidean response variable, and f is a given function. Theoretical properties of interest in high-dimensional regression include (a) the fluctuation behavior of the penalized estimator $\widehat{\beta }$ and (b) whether $\widehat{\beta }$ is consistent in some sense of distance under certain growth conditions for p. For $0<q\le \infty$, we write ${\parallel \beta \parallel }_q:=({\sum }_{i=1}^p|{\beta }_i{{|}^q)}^{1/q}$ as the ${\ell }_q$-norm of a p-dimensional vector $\beta$. Generally, we are interested in two types of non-asymptotic error upper bounds for penalized estimators:

• Prediction error (predictive consistency): $\widehat{\beta }$ performs well on a test sample X, with a prediction error satisfying

  $${\mathrm{E}}_X[{(X^T(\widehat{\beta }-{\beta }^{*}))}^2]\le R_n^{\mathrm{pre}}({\beta }^{*})(\mathrm{or}\mathrm{its}\mathrm{empirical}\mathrm{version}),$$

  where $R_n^{\mathrm{pre}}({\beta }^{*})$ is a non-asymptotic convergence function (converging to 0 with n).
• ${\ell }_q$-estimation error (or other norms): $\widehat{\beta }$ approximates the true parameter ${\beta }^{*}$. With high probability,

  $${\parallel \widehat{\beta }-{\beta }^{*}\parallel }_q\le R_n^{\mathrm{est}}({\beta }^{*}),$$

  where $R_n^{\mathrm{est}}({\beta }^{*})$ is a non-asymptotic convergence function.

Both Types 1 and 2 are called oracle inequalities. Consider the Lasso penalized linear model $Y=X{\beta }^{*}+\epsilon$, with $\mathrm{Var}(\epsilon |X)={\sigma }^2{I}_p$. Under assumptions of sub-Gaussian response variables, sparsity $s:=\parallel {\beta }^{*}{\parallel }_0$, and a restricted eigenvalue $\gamma >0$ for the design matrix, it can be shown that there exist Type 1 and Type 2 oracle inequalities [10,36]:

$$\parallel \widehat{\beta }-{\beta }^{*}{\parallel }_1\le {\displaystyle \frac{12A\sigma }{\gamma }}\sqrt{{\displaystyle \frac{s\log p}{n}}},{\displaystyle \frac{1}{n}}\parallel X(\widehat{\beta }-{\beta }^{*}){\parallel }_2^2\le {\displaystyle \frac{9A\sigma s\log p}{\gamma n}}$$

with probability $1-2p^{1-{\displaystyle \frac{A^2}{8}}}$ (here, a constant $A>2\sqrt{2}$). Blazere et al. [37] and Wainwright [16] (Corollary 9.26), respectively, provided two versions of oracle inequalities for $l_1$-estimation error in generalized linear models under restricted eigenvalue conditions. Proving oracle inequalities requires more probability inequality techniques and is usually more complex than identities in asymptotic analysis; its ideas and techniques originate from proof techniques in non-parametric penalized regression [38]. Given n and p, oracle inequalities can provide deep insights into the non-asymptotic fluctuation properties of estimators. In recent years, oracle inequalities have made significant progress in various complex regression models in statistics and machine learning, not limited to linear models and generalized linear models. There is also research on Cox models with dependent structures and Ising models, which require some exponential-type concentration inequalities for martingales and Markov chains as theoretical foundations; see Wei et al. [39] and Xiao et al. [40]’s studies. In complex data scenarios, Han et al. [41] proposed an online inference framework for high-dimensional semiparametric single-index models with unspecified link functions, focusing on regression parameter estimation. Fu et al. [42,43] introduced modern multiclass support vector machines and multinomial logistic regression models for high-dimensional and divergent settings, particularly involving scenarios where both the number of features and the number of classes increase simultaneously. Zhang et al. [44] studied the problem of estimating the covariance matrix in large-dimension and small-sample-size scenarios. Tian and Qin [45] proposed a theory and method for estimating high-dimensional Toeplitz sparse block precision matrices and applied it to interval-valued time series modeling. For the prediction error, one can obtain predictive consistency without making too many assumptions about the model; see Zhuang and Lederer [46] for more. Lasso estimators are not asymptotically normal, but debiased Lasso estimators obtained by correcting Lasso estimators have asymptotic normality properties and can also be used to control the false discovery rate (FDR) in multiple hypothesis testing, see Javanmard and Javadi [47] and Han et al. [48] for more FDR methods.

3.2. Infinite-Dimensional Statistics

When the dimensionality of Euclidean space data tends to infinity, they can be viewed as infinite-dimensional $L_2$ space random process data (also called functional data). This is based on the isomorphism between $L_2$ space and $l_2$ space. Assume there is a population random process ${\left\{X(s)\right\}}_{s\in [0,1]}$. Functional data are typically a set of independent and identically distributed random processes ${\left\{X_i(s)\right\}}_{i=1}^n\stackrel{\mathrm{i}.\mathrm{i}.\mathrm{d}.}{\sim }X(s)$. Using the idea of principal component analysis, the infinite-dimensional functional curve $X(s)$ can be reduced to a linear combination of a series of basis functions. Let $X(s)$ be a square-integrable random process with a mean function $\mu (s):=\mathrm{E}[X(s)]$ and a covariance function $K(s,t)=\mathrm{E}\{[X(s)-\mu (s)][X(t)-\mu (t)]\}$ existing. The covariance function satisfies the spectral decomposition theorem (Mercer’s theorem):

$$K(s,t)=\sum _{j=1}^{\infty }{\lambda }_j{\varphi }_j(s){\varphi }_j(t),$$

where ${\left\{{\lambda }_j\right\}}_{j\ge 1}$ are non-negative eigenvalues satisfying ${\sum }_{j=1}^{\infty }{\lambda }_j<\infty$ and are non-increasing, and each ${\varphi }_j(t)$ is the corresponding eigenfunction. With Mercer’s theorem, $X(s)$ has the following Karhunen–Loeve expansion:

$$X(s)=\mu (s)+\sum _{j=1}^{\infty }{\xi }_j{\varphi }_j(s),$$

where ${\xi }_j={\int }_0^1[X(s)-\mu (s)]{\varphi }_j(s)ds$ are called functional principal component scores, which are zero-mean and have $\mathrm{Cov}({\xi }_j,{\xi }_k)={\lambda }_j{\delta }_{jk}$ (where ${\delta }_{jk}=1$ if $j=k$ and 0 otherwise). The Karhunen–Loeve expansion maps the $L_2$ space random process to an $l_2$ space sequence $\left\{{\xi }_j\right\}$. Thus, models including functional covariates $X(s)$ can be viewed as approximations of covariates $\left\{{\xi }_j\right\}$ of increasing dimension [$p=n^{\alpha },(0<\alpha \le 1)$]. For real data, an estimated version of the covariance operator is obtained. The Karhunen–Loeve expansion requires some leading terms with large variance (representing large information content), and the number of selected basis functions needs to change with n. This can also balance the variance brought by the data and the bias caused by model approximation.

For example, in scalar-on-function regression, we get

$$Y={\int }_0^1{\beta }^{*}(s)X(s)ds+\epsilon ,$$

where $\epsilon$ is Gaussian and ${\beta }^{*}(s)$ is the unknown slope function. For ease of theoretical derivation, we first assume that $X(s)$ is fully observed with respect to time s. When ${\beta }^{*}(t)$ satisfies certain smoothness conditions and $X(s)$ satisfies some moment conditions (eigenvalue decay conditions corresponding to Mercer’s theorem), Hall and Horowitz [49] rigorously derived the optimal convergence rate for the estimated slope function $\widehat{\beta }(s)$ based on functional principal component analysis (FPCA). Dou et al. [50] further extended this to generalized functional regression models, where the response variable belongs to the exponential family. For the sample curve i, assume that the response variable $Y_i$ conditional on the random process $\{X_i(s):s\in [0,1]\}$, which follows a density function $f_{Y_i}(y_i)\propto \exp \{{\theta }_i{y}_i-\psi ({\theta }_i)\}$ of an exponential family [51]:

$${\theta }_i={\int }_0^1{X}_i(t)\beta (t)dt=:\langle \beta ,X_i\rangle =\sum _{k=1}^{\infty }{\beta }_k\langle X_i,{\varphi }_k\rangle =:\sum _{k=1}^{\infty }{\beta }_k{\xi }_{ik},$$

where $\beta (s)={\sum }_{k=1}^{\infty }{\beta }_k{\varphi }_k(t)\in L^2[0,1]$ is the expansion of the unknown slope function with the Fourier coefficients $\left\{{\beta }_k\right\}$. Without the loss of generality, assume $\mu (s)=0$. From the fully observed ${\{X_i(s):s\in [0,1]\}}_{i=1}^n$, the empirical version of the covariance function and its corresponding spectral decomposition are obtained:

$$\widehat{K}(s,t)={\displaystyle \frac{1}{n}}\sum _{i=1}^n{X}_i(s)X_i(t)=\sum _{j=1}^{\infty }{\widehat{\lambda }}_j{\widehat{\varphi }}_j(s){\widehat{\varphi }}_j(t),$$

where ${\widehat{\lambda }}_1\ge {\widehat{\lambda }}_2\ge \dots \ge 0$ are the non-increasing eigenvalues of the covariance operator $\widehat{K}$. Define the first m estimated principal component score vector ${\widehat{\xi }}_i:={({\widehat{\xi }}_{i,1},\dots ,{\widehat{\xi }}_{i,m})}^T$, where ${\widehat{\xi }}_{ik}=\langle X_i,{\widehat{\varphi }}_k\rangle$ and $m:=m_n\to \infty$ as $n\to \infty$. Note that the true log-likelihood function regarding $\theta$ is ${\sum }_{i=1}^n({\theta }_i{Y}_i-\psi ({\theta }_i))$. To approximate the true parameter, which is an infinite-dimensional likelihood function, consider a truncated version which is a likelihood function $L_n(\gamma )$ with an increasing number of parameters satisfying

$$L_n(\gamma )=\sum _{i=1}^n({\xi }_i^T\gamma )Y_i-\psi ({\xi }_i^T\gamma )\approx \sum _{i=1}^n({\theta }_i{Y}_i-\psi ({\theta }_i)).$$

Maximizing the above truncated likelihood function in the space ${\mathbb{R}}^m$ yields the truncated maximum likelihood estimate $\widehat{\gamma }={({\widehat{\beta }}_1,\dots ,{\widehat{\beta }}_m)}^T$. Then, define the estimated version of the slope function as

$$\widehat{\beta }(t)=\sum _{k=1}^m{\widehat{\beta }}_k{\widehat{\varphi }}_k(t).$$

The handling of functional covariates as stochastic processes presents a major challenge in functional regression. Crucially, a functional covariate comprises infinitely many highly correlated predictors across its domain (observed discretely). This correlation structure is defined via the covariance operator. Estimating the slope function involves solving ill-posed inverse problems. To manage infinite dimensionality, one imposes regularity conditions on the hypothesized slope function space, ensuring tractable finite approximations. However, the convergence rates of slope estimators depend fundamentally on assumptions about the covariance operator’s eigenvalue decay and the slope function’s restricted space. Consequently, convergence rates are inherently nonparametric. State-of-the-art methods, such as FPCA using optimally truncated principal components and RKHS [52] through optimally tuned kernel ridge regression, yield optimal nonparametric procedures. Their convergence rates are critically dependent on the smoothness of both the slope function and the covariance operator.

Usually, we assume that the Fourier coefficients $\left\{{\beta }_k\right\}$ of the slope function and the population eigenvalues $\left\{{\lambda }_j\right\}$ satisfy the decay conditions

$$|{\beta }_k|\le C{k}^{-b},R{k}^{-a}\ge {\lambda }_j\ge {\lambda }_{j+1}+{(R/j)}^{a-1},$$

where, given the principal component variance decay constant $a>1$, the slope decay constant b satisfies the signal detection threshold condition $b>(a+3)/2$. If the number of truncated principal component scores satisfies $m\asymp n^{1/(a+2b)}$, Dou et al. [50] proved that the $L_2$ error of $\widehat{\beta }(s)$ achieves the nonparametric optimal convergence rate.

$${\int }_0^1{[\widehat{\beta }(s)-\beta (s)]}^{2}ds=O_p\left(n^{-{\displaystyle \frac{2b-1}{a+2b}}}\right).$$

Unfortunately, due to instrument measurement limitations, actual functional curve data cannot be fully observed. In reality, sampling times are at discrete points. In practice, we usually observe n sample curves ${\{X_i(s):s\in [0,1]\}}_{i=1}^n$ at N discrete time points, and instrument measurements may have measurement errors. The actual measured values are represented by the following formula:

$$X_{ij}=X_i(T_j)+{\epsilon }_{ij},j=1,\dots ,N;i=1,2,\dots ,n,$$

where $\left\{{\epsilon }_{ij}\right\}\stackrel{\mathrm{iid}}{\sim }\mathcal{E}$ (where $\mathcal{E}$ denotes the distribution of $\epsilon$) are independent and identically distributed random variables satisfying $\mathrm{E}[\epsilon ]=0$ and $\mathrm{Var}(\epsilon )={\sigma }_{\epsilon }^2$. $\left\{T_j\right\}$ can include fixed or random sampling time points. For theoretical convenience, Cai and Yuan [53] assumed that the observation times $\left\{T_j\right\}$ are distributed independently and identically and follow a uniform distribution on $[0,1]$. In the literature on discretely observed functional data, it is usually assumed that $N=n^{\alpha }$, where $\alpha$ is defined as the sampling rate. If sampling is costly or objective conditions do not allow for it, curve sampling is sparse, which requires us to first smooth the discrete observation curves. There are two typical smoothing methods. The first is pre-smoothing each curve, which is suitable for sufficiently dense functional data. Specifically, when $\alpha \ge 5/4$, the mean and covariance function estimates based on pre-smoothed curves using kernel density estimation achieve the parametric convergence rate $\sqrt{n}$. This type of sampling is called “ultra-dense” [54]. The second method is for sparse functional data. Due to under-sampling, an effective smoothing method is to pool the n population curve samples together to estimate the mean and covariance functions. The pooling method usually uses local constant (or local linear) regression. With the recovery of mean and covariance function curves, eigenvalues and eigenfunctions can be further estimated via Mercer’s theorem, as described by Yao et al. [55]. Cai and Yuan [53] and Zhang and Wang [56] proved that when $\alpha \ge 1/4$, estimates of mean and covariance functions using the pooling method can achieve the parametric convergence rate $\sqrt{n}$. Compared to pre-smoothing methods that may lose population information, the pooling method for n population curves requires relatively fewer samples to achieve parametric convergence rates. This type of theory is significant, as it theoretically demonstrates that the pooling method is superior to pre-smoothing. Based on the above theory, a sampling rate $\alpha \ge 1/4$ is called “dense” and a sampling rate $\alpha <1/4$ is called “sparse”. Recent advancements and the theory of functional principal component analysis for discretely observed data and functional linear regressions can be founded in [57,58].

4. Summary of the Special Issue Papers

4.1. Estimation for Partial Functional Multiplicative Regression Model by Liu et al. [59]

The increasing prevalence of functional data, encompassing entities like curves and manifolds, motivates this research. Multiplicative regression is apt for data featuring positive outcomes. This paper focuses on estimating parameters within the partial functional multiplicative regression model (PFMRM). Two criteria, the least absolute relative error (LARE) and least product relative error (LPRE), are employed. An approximation of the functional predictor and its associated slope function is achieved using functional principal component basis functions. Subject to standard regularity assumptions, the study establishes the convergence rate for the slope function estimator and demonstrates the asymptotic normality for the parametric slope vector estimator using both methodologies. The performance of these proposed techniques is evaluated through Monte Carlo simulations, and their practical utility is illustrated with an analysis of the Tecator dataset.

4.2. Sharper Concentration Inequalities for Median-of-Mean Processes by Teng et al. [60]

The median-of-mean (MoM) estimation technique provides a robust statistical tool for managing datasets affected by contamination. This study introduces a variance-aware MoM estimation approach that utilizes binomial distribution tail probabilities. Under moderate conditions, the derived bound for this method proves to be more stringent than the conventional Hoeffding bound. This enhanced method is subsequently applied to explore the concentration properties of variance-dependent MoM empirical processes and the sub-Gaussian intrinsic moment norm. Furthermore, the paper presents a bound for the variance-dependent MoM estimator when applied to distribution-free contaminated data.

4.3. On a Low-Rank Matrix Single-Index Model by Mai [61] (One Citation)

This investigation offers a theoretical analysis of a single-index model featuring a low-rank matrix. While recently introduced within biostatistics, the model’s theoretical underpinnings for the simultaneous estimation of its link function and coefficient matrix remain underexplored. Utilizing the PAC–Bayesian bounds technique, this study provides a comprehensive theoretical framework for this joint estimation task. The results afford a more profound comprehension of the model’s characteristics and its prospective utility across various domains.

4.4. Generalizations of the Kantorovich and Wielandt Inequalities with Applications to Statistics by Zhang et al. [62]

Leveraging the characteristics of positive definite matrices, mathematical expectations, and positive linear functionals within matrix spaces, this study derives both the Kantorovich and Wielandt inequalities for such matrices and associated random variables. Several novel Kantorovich-type inequalities are presented, addressing ordinary matrix products, Hadamard products, and the mathematical expectations of random variables. Additionally, the study explores various interesting unified and generalized forms of the Wielandt inequality applicable to positive definite matrices. These established inequalities are subsequently used to formulate an inequality concerning different correlation coefficients and to examine applications in assessing the relative efficiency of parameter estimation within linear statistical models.

4.5. Optimal Non-Asymptotic Bounds for the Sparse $\beta$ Model by Yang et al. [63]

This article examines the sparse $\beta$ model, incorporating an ${\ell }_1$ penalty, a prominent area in network data modeling for both statistical and social network analysis. A refined algorithm is introduced for parameter estimation within this proposed model. The algorithm’s efficacy is demonstrated by its connection to the proximal gradient descent method, a consequence of the loss function’s convexity. The study investigates estimation consistency and derives an optimal bound for the proposed estimator. Empirical support for the methodology’s effectiveness is provided through carefully designed simulation studies. These findings underscore the potential of this methodology to contribute to advanced network data analysis.

4.6. Non-Asymptotic Bounds of AIPW Estimators for Means with Missingness at Random by Wang and Deng [64] (One Citation)

Augmented inverse probability weighting (AIPW) is recognized for its double robustness in contexts of missing data and causal inference, ensuring consistent estimation if either the propensity score or outcome regression model is correctly specified. A key feature of AIPW is its capacity to achieve first-order equivalence with an oracle estimator (where nuisance parameters are known), even when fitted models do not converge at the parametric $\sqrt{n}$ rate. This research delves into the non-asymptotic characteristics of the AIPW estimator for inferring the population mean under missingness at random. Inferences for mean outcomes in both observed and unobserved groups are also explored.

4.7. Group Logistic Regression Models with ${\ell }_{p,q}$ Regularization by Zhang et al. [65] (Nine Citations)

This study introduces a logistic regression framework employing ${\ell }_{p,q}$ regularization designed to yield group sparse solutions. Such a model is applicable to variable selection tasks where features exhibit sparse group structures. In high-dimensional data settings, solutions to practical problems often manifest group sparsity, necessitating the study of such models. The proposed model is characterized from theoretical, algorithmic, and numerical standpoints. Theoretically, by incorporating the group-restricted eigenvalue condition, an oracle inequality—a crucial property for variable selection—is established. A global recovery bound for the logistic regression model with ${\ell }_{p,q}$ regularization is also derived. Algorithmically, the alternating direction method of multipliers (ADMM) is adapted for the model solution, with effective methods for its subproblems. Numerically, experiments on simulated and real-world factor stock selection data demonstrate the model’s efficacy in variable selection and prediction, utilizing the presented ADMM algorithm.

4.8. Heterogeneous Overdispersed Count Data Regressions via Double-Penalized Estimations by Li et al. [66] (Five Citations)

High-dimensional negative binomial regression (NBR) for count data has gained considerable traction in diverse scientific fields. However, a common assumption in many studies is a constant dispersion parameter, which may not hold in practical scenarios. This investigation addresses variable selection and dispersion estimation in heterogeneous NBR models, where the dispersion parameter is treated as a function. Specifically, a double-regression framework is proposed, with a double ${\ell }_1$ penalty applied to both regression components. Under restricted eigenvalue conditions and using concentration inequalities for empirical processes, oracle inequalities for the Lasso estimators of the two partial regression coefficients are established for the first time. These oracle inequalities further provide theoretical guarantees for consistency and convergence rates of the estimators, facilitating subsequent statistical inference. The effectiveness of these novel methods is substantiated through simulation studies and a real data analysis.

4.9. Representation Theorem and Functional CLT for RKHS-Based Function-on-Function Regressions by Huang et al. [67]

This study explores a nonparametric varying coefficient regression framework for modeling and estimating regression effects between two functionally correlated datasets. Modern biomedical research often involves measuring multiple patient features over time or at discrete intervals to understand biological mechanisms; conventional statistical models may yield biased estimates if they fail to adequately incorporate interventions and their dynamic responses. A shared parameter change point function-on-function regression model is introduced to assess pre- and post-intervention temporal trends, alongside a likelihood-based estimation method for intervention effects and other parameters. New methodologies are also developed for estimating and testing regression parameters for functional data using reproducing kernel Hilbert spaces (RKHSs). These regression parameter estimators are derived in closed form, avoiding large matrix inversions, thereby enhancing computational efficiency and applicability. By establishing a representation theorem and a functional central limit theorem, the asymptotic properties of these estimators are determined, and corresponding hypothesis tests are formulated. The method’s statistical properties and application are illustrated using an immunotherapy clinical trial for advanced myeloma and through simulation studies.

4.10. Sharper Sub-Weibull Concentrations by Zhang and Wei [68] (Thirty-Two Citations)

Constant-specified and exponential concentration inequalities are fundamental to the finite-sample theory prevalent in machine learning and high-dimensional statistics. This study derives more precise, constant-specified concentration inequalities for sums of independent sub-Weibull random variables. These new bounds exhibit a mixed-tail behavior, with sub-Gaussian for small deviations and sub-Weibull for larger deviations from the mean, thereby improving upon existing bounds by offering sharper constants. A novel sub-Weibull parameter is introduced, facilitating the recovery of tight concentration inequalities for random variables or vectors. In statistical applications, an ${\ell }_2$ error bound is provided for estimated coefficients in negative binomial regressions for scenarios with heavy-tailed, sparse sub-Weibull covariates, a new contribution for such regressions. Within random matrix theory, non-asymptotic versions of the Bai-Yin theorem are developed for sub-Weibull entries, featuring exponential tail bounds. Finally, the paper demonstrates a sub-Weibull confidence region for log-truncated Z-estimators without requiring second-moment conditions.

Citation counts were obtained from Google Scholar on 28 June 2025.