Sparse Semi-Functional Partial Linear Single-Index Regression ^†

Abstract

The variable selection problem is studied in the sparse semi-functional partial linear model, with single-index type influence of the functional covariate in the response. The penalized least squares procedure is employed for this task. Some properties of the resultant estimators are derived: the existence (and rate of convergence) of a consistent estimator for the parameters in the linear part and an oracle property for the variable selection method. Finally, a real data application illustrates the good performance of our procedure.

1. Introduction

In many real problems, to predict the value of a random variable, observations of many other variables are available. However, in many cases, it is unknown which of them (very few) have a real influence in the response. In this practical framework, we need procedures able to select the relevant variables to avoid high-dimensionality problems. Reducing the complexity of the model becomes even more crucial when regression involves a functional variable too (data are functions, images...). Therefore, the main goal is the simplification of the model, which makes easier both its estimation and interpretation, without losing its predictive efficiency.

These practical problems have motived the peak of semiparametric models in the functional regression, together with the variable selection procedures. In [1] the penalized least squares method for estimation and variable selection is studied for the partial linear model with functional covariate. In this model, the real variables have a linear effect (involving interpretable coefficients that are the parameters) in the response, while the infinite-dimensional covariate has a nonlinear (nonparametric) influence. However, in real data applications, it would be interesting having parameters related to the functional variable to derive practical interpretations. This is one of the advantages of the semi-functional partial linear single-index model (SFPLSIM): the real covariates also affect in a linear way to the response, but the infinite-dimensional covariate influences it trough a projection in an unknown direction, after applying a nonlinear link function. This direction of projection behaves like a function-parameter that could have interesting interpretations. Some theoretical properties related to the nonparametric estimation of the functional single-index model are given in [2]. In this paper, we will study the sparse SFPLSIM, focusing in the variable selection problem. For this purpose, we will use the penalized least squares procedure for estimating the parameters of the lineal components and, simultaneously, selecting the relevant covariates. The properties of the estimators will be analysed from a theoretical point of view: we will set its convergence rates and the consistency for selecting the model. These results will be illustrated through a real data application.

2. The Model

The SFPLSIM is defined by the relationship

$$Y_i=X_{i1}{\beta }_{01}+\cdots +X_{i{p}_n}{\beta }_{0p_n}+m\left(⟨{\theta }_0,{\mathcal{X}}_i⟩\right)+{\epsilon }_i,\forall i=1,\cdots ,n,$$

(1)

where $Y_i$ denotes a scalar response, $X_{i1},\cdots ,X_{i{p}_n}$ are random covariates taking values in $\mathbb{R}$ and ${\mathcal{X}}_i$ is a functional random covariate valued in a separable Hilbert space $\mathcal{H}$ with inner product $⟨·,·⟩$. ${\mathbf{\beta }}_0={({\beta }_{01},\cdots ,{\beta }_{0p_n})}^{\top }\in {\mathbb{R}}^{p_n}$, ${\theta }_0\in \mathcal{H}$ and $m(·)$ are a vector of unknown real parameters, an unknown functional direction and an unknown smooth real-valued function, respectively. Finally, ${\epsilon }_i$ is the random error, which verifies $\mathbb{E}\left({\epsilon }_i|X_{i1},\cdots ,X_{i{p}_n},{\mathcal{X}}_i\right)=0.$

3. The Penalized Least-Squares Estimators

For the purpose of simultaneously estimating $\beta$-parameters and selecting relevant X-covariates in the SFPLSIM (1), we will apply the penalized least-squares approach. For that, in a first step we transform the SFPLSIM in a linear model by extracting from $Y_i$ and $X_{ij}$ ($j=1,\dots ,p_n$) the effect of the functional covariate ${\mathcal{X}}_i$ when is projected on the direction ${\theta }_0$. Specifically, denoting by ${\mathit{X}}_i={\left(X_{i1},X_{i2},\cdots ,X_{i{p}_n}\right)}^{\top },\mathit{X}={\left({\mathit{X}}_1,\cdots ,{\mathit{X}}_n\right)}^{\top }\mathrm{and}\mathit{Y}={\left(Y_1,\cdots ,Y_n\right)}^{\top }$, the fact that

$$Y_i-\mathbb{E}\left(Y_i|⟨{\theta }_0,{\mathcal{X}}_i⟩\right)={\left({\mathit{X}}_i-\mathbb{E}\left({\mathbf{X}}_i|⟨{\theta }_0,{\mathcal{X}}_i⟩\right)\right)}^{\top }{\mathbf{\beta }}_0+{\epsilon }_i,\forall i=1,\cdots ,n,$$

(2)

allows to consider the following approximate linear model (see Appendix A for understanding the notation):

$${\tilde{\mathit{Y}}}_{{\theta }_0}\approx {\tilde{\mathit{X}}}_{{\theta }_0}{\mathbf{\beta }}_0+\epsilon ,$$

(3)

where $\epsilon ={\left({\epsilon }_1,\cdots ,{\epsilon }_n\right)}^{\top }$. Then, in a second step, the penalized least-squares approach is applied to model (3). Specifically, ${\mathbf{\beta }}_0$ and ${\theta }_0$ are estimated by considering a minimizer, $({\widehat{\mathbf{\beta }}}_0,{\widehat{\theta }}_0)$, of the penalized profile least-squares function

$$\mathcal{Q}\left(\mathbf{\beta },\theta \right)=\frac{1}{2}{\left({\tilde{\mathit{Y}}}_{\theta }-{\tilde{\mathit{X}}}_{\theta }\beta \right)}^{\top }\left({\tilde{\mathit{Y}}}_{\theta }-{\tilde{\mathit{X}}}_{\theta }\mathbf{\beta }\right)+n\sum _{j=1}^{p_n}{\mathcal{P}}_{{\lambda }_{j_n}}\left(|{\beta }_j|\right),$$

where $\mathbf{\beta }={({\beta }_1,\dots ,{\beta }_{p_n})}^{\top }$, ${\mathcal{P}}_{{\lambda }_{j_n}}\left(·\right)$ is a penalty function and ${\lambda }_{j_n}>0$ is a tuning parameter. Note that, simultaneously to the parameter estimation, the previous procedure can be considered as a variable selection method: if ${\widehat{\mathbf{\beta }}}_{0j}$ is a non-null component of ${\widehat{\mathbf{\beta }}}_0$, then $X_j$ is selected as an influential variable.

From now on, we will denote $J_n=\{1,\cdots ,p_n\}$ and $S_n\subset J_n$ such that ${\beta }_{0j}\ne 0$ for $j\in S_n$ and ${\beta }_{0j}=0$ for $j\in S_n^c=J_n/S_n$. In addition $s_n$ will mean card $\left(S_n\right)$ and we will assume that $S_n=\{1,\cdots ,s_n\}$.

4. Asymptotic Theory

In this paper, the existence of the penalized estimator is established as well as the corresponding rates of convergence. In particular, under some assumptions, we proved that there exists a local minimizer $\left({\widehat{\mathbf{\beta }}}_0,{\widehat{\theta }}_0\right)$ of $\mathcal{Q}\left(\mathbf{\beta },\theta \right)$ such that

$$\begin{array}{c}\hfill ∥{\widehat{\mathbf{\beta }}}_0-{\mathbf{\beta }}_0∥=O_p\left(\sqrt{s_n}\left(n^{-1/2}+{\delta }_n\right)\right)\mathrm{where}{\delta }_n=\underset{j\in S_n}{max}\left\{\left|{\mathcal{P}}_{{\lambda }_{j_n}}^{\prime }\left(|{\beta }_{0j}|\right)\right|\right\}.\end{array}$$

(4)

Furthermore, the selected set of variables, ${\widehat{S}}_n=\{j\in J_n;{\widehat{\beta }}_{0j}\ne 0\}$, works as well (at least asymptotically) as it would do if the true set of relevant variables $S_n$ was known. Specifically, $\mathbb{P}({\widehat{S}}_n=S_n)\to 1\mathrm{as}n\to \infty$.

An application to real data is included, which shows the good performance of the presented method in terms of error of prediction.