Smooth kNN Local Linear Estimation of the Conditional Distribution Function

Abstract

Previous works were dedicated to the functional k-Nearest Neighbors (kNN) and the local linearity method estimations of a regression operator. In this paper, a sequence pair of ${(X_i,Y_i)}_{i=1,\dots ,n}$ of functional mixing observations are considered. We treat the local linear estimation of the cumulative function of $Y_i$ given functional input variable $X_i$. Precisely, we combine the kNN method with the local linear algorithm to construct a new and fast efficiency estimator of the conditional distribution function. The main purpose of this paper is to prove the strong convergence of the constructed estimator under mixing conditions. An application to the functional times series prediction is used to compare our proposed estimator with the existing competitive estimators, and show its efficiency and superiority.

1. Introduction

In the last decade, the local linearity method (LLM) estimation has become an interesting growing method in nonparametric Functional Data modeling (NPFDM). This topic’s motivation is the superiority of LLM-estimation over the method of the classical kernel weighting method (CKM). Especially, the CKM has a large bias compared to the LLM-estimation (see Fan and Gijbels [1] for a uni-dimensional framework, and Baìllo and Grané [2] for NPFDM setup). Baìllo and Grané [1] used the LLM-algorithm to estimate the hilibertian conditional expectation operator. A generalized LLM-estimation of this nonparametric operator was obtained by Barrientos et al. [3]. They treated the case of Banach explanatory variable. Berlinet et al. [4] built an alternative LLM-estimator of the functional conditional expectation by inverting the local variance-covariance matrix of the functional variable. The asymptotic distribution of the LLM-estimator proposed by Barrientos et al. [3] was obtained by Zhou and Lin [5].

Furthermore, the LLM-estimation of the conditional cumulative distribution function (CCDF) was investigated by Laksaci et al. [6], who established the almost consistency rate for an LLM-estimator of the CCDF-model when the observations have spatio-functional structure. All these previous studies have utilized the kernel local linearity method; however, this paper focuses on CCDF-estimation with a new weighting approach obtained by mixing the local linear fitting to the kNN method.

1.1. Related Works

Recall that the estimation by kNN method has more advantages than the Nadaraya-Watson algorithm. It is an attractive procedure of estimation which is more acclimated to the functional formation of underlying data (see Burba et al. [7] for more motivations on this approach). Notice also that the method of kNN, in the nonparametric functional statistic, has been studied by many researchers (see, for instance, Laloë [8], Kudraszow and Vieu [9] for previous works and Kara et al. [10] for the uniform consistency on the number of neighbors). On the other hand, the kNN estimation under the local linear approach was recently developed by Chikr-Elmezouar et al. [11]. They constructed an estimator of the conditional density by combining the ideas of the local linear method to the kNN weighting techniques. They proved the almost complete consistency of the constructed estimator when the observations are independent identically distributed. Considering the same situation for functional observations in the case of independent identically distributed, Bachir et al. [12] have studied the estimation of the M-regression function. Their estimator was obtained using the kNN approach over the Nadaraya-Watson method. They established the convergence rate of the uniform consistency on the number of the neighbor of the constructed estimator. As an alternative model to the robust regression, Laksaci et al. [13] have treated the kNN estimation of the quantile regression. They stated the property of the built estimator under the independence structure. We refer to Rachedi et al. [14] for the functional regression when the response variable is observed with missing data at random. More recently, Almanjahie et al. [15] have studied the computational aspects of the kNN estimation of some nonparametric functional models, including the conditional density, the regression operator, and the conditional cumulative function. They examined the feasibility of some selector algorithms to choose the best bandwidth parameter in nonparametric functional data analysis.

1.2. Contribution

While the previous works are dedicated to the functional kNN-estimation of the regression operator using the CKM-method, we consider, in this contribution, the kNN estimation problem of the CCDF using the LLM-smoothing. Precisely, we benefit from the attractive features of both the kNN weighting and LLM-fitting by combining the two algorithms to provide a fast efficiency estimator for the CCDF. On the one hand, it is well known that the main reason behind the implementation of the kNN method is its feature to select an attractive smoothing parameter. Specifically, the kNN method permits the selection of a bandwidth parameter more adapted to the local structure of the data. Moreover, this estimator can be updated to any new observations. Such consideration is essential in functional statistics where the asymptotic properties are strongly dependent on the behavior of the local structure. For the latter reason, the kNN method is better than the classical kernel method. However, the difficulty in the kNN smoothing is the fact that the bandwidth parameter is a random variable, unlike the kernel method in which the smoothing parameter is a deterministic scalar. So, the study of the asymptotic properties of this estimator is complicated, and it requires some additional tools and techniques.

On the other hand, it is well documented that the local linear approach is an alternative method to the usual Nadaraya-Watson technique. As discussed in the first paragraph, the estimation by local linear method permits to improve the asymptotic property of the kernel estimator by reducing the bias term. Thus, with this combined approach, we construct an estimator of the CCDF and state their consistency by establishing the almost complete convergence rate. The second novelty of this contribution is establishing the asymptotic results of the estimator when the observations are correlated as mixing functional time series.

1.3. Organization

This paper is structured as follows. Our Methodology describing the kNN-LLM estimator, as well as the functional time series framework, is presented in Section 2. The main asymptotic results with their proofs are also presented in Section 3. Section 4 is devoted to some comments allowing to reveal the very merits of the proposed approach. The performance of the constructed estimator in temperature prediction, compared to existing estimator, using real data is conducted in Section 5. Our conclusion is presented in Section 6.

2. Methodology

2.1. CCDF-Model and Its kNN-LLM Estimator

Consider $(X_1,Y_1),(X_2,Y_2),(X_3,Y_3)\dots ,(X_n,Y_n)$ be stationary sequence of random vector $(X,Y)$ valued in $\mathcal{F}\times \mathrm{I}\mathrm{R}$, where $\mathcal{F}$ is a separable metric space has a metric d. Let $N_x$ be the neighborhood of fixed curve $x\in \mathcal{F}$, for which we suppose that the conditional cumulative distribution function (CCDF) $F(·|x)$ has a continuous conditional density $f(·,x)$. Usually, the LLM-estimator of CCDF is built by treating the function $F(·|x)$ as a conditional expectation, i.e.,

$$\mathrm{I}\mathrm{E}[H\left({\ell }^{-1}(y-Y_i)\right)|X_i=x]\to F(y|x)\mathrm{as}\ell \to 0,$$

where H is the cumulative distribution function, and $({\ell }_n=\ell )$ is a positive real sequence. In fact, this idea was proposed, first, by Fan and Gijbels [1] in nonfunctional setup. In our functional context, we consider an alternative estimator to that proposed by Almanjahie et al. [16]. It is obtained by approximating $F(y|x)$ locally in $N_x$ by

$$\forall x_0\in N_x,F(y|x_0)=a_{yx}+b_{yx}d(x_0,x)+o\left(d(x,x_0)\right).$$

(1)

So, the kNN-LLM estimator is constructed by estimating the operators $a_{yx}$ and $b_{yx}$ of the formula in (1) as

$$Mi{n}_{(a,b)\in \mathrm{I}{\mathrm{R}}^2}\sum _{i=1}^n{\left(H\left({\ell }_l^{-1}(y-Y_i)\right)-a-bd(X_i,x)\right)}^{2}Ker\left(\frac{d(x,X_i)}{{\mathrm{I}\mathrm{H}}_k}\right),$$

where $Ker(·)$ is a kernel function, ${\ell }_l=min\{\ell \in {\mathrm{I}\mathrm{R}}^{+},\mathrm{satisfies}{\sum }_{i=1}^{n}1{\mathrm{I}}_{(y-\ell ,y+\ell )}\left(Y_i\right)=l\}$, and ${\mathrm{I}\mathrm{H}}_k=min\left\{h\in {\mathrm{I}\mathrm{R}}^{+}\mathrm{such}\mathrm{that}{\sum }_{i=1}^{n}1{\mathrm{I}}_{B(x,h)}\left(X_i\right)=k\right\}$. Then, we prove, later, that the smooth kNN-LLM of the CCDF $F(y|x)$ is explicited by

$$\tilde{F}(y|x)=\frac{{\sum }_{i,j=1}^n{\beta }_{ij}H\left({\ell }_l^{-1}(y-Y_i)\right)}{{\sum }_{i,j=1}^n{\beta }_{ij}},$$

where

$${\beta }_{ij}=d(X_i,x)\left(d(X_i,x)-d(X_j,x)\right)\times Ker\left({\mathrm{I}\mathrm{H}}_k^{-1}d(x,X_i)\right)Ker\left({\mathrm{I}\mathrm{H}}_k^{-1}d(x,X_j)\right).$$

2.2. Functional Time Series Framework

We study the behavior of asymptotic property of the LLM-estimator based on CCDF $F(·,x)$ when the data is observed as functional time series (FTS). It should be noticed that the functional time series analysis has been widely developed in the area of functional statistics; see Ferraty and Vieu [17] for an important discussions. In this paper, we carry out our functional time series framework by assuming that the sequence ${(Z_i=(X_i,Y_i))}_i$ is algebraic $\alpha$-mixing has coefficient of mixing $\alpha \left(n\right)\to 0$ such that

$$\mathrm{there} \mathrm{exists}a>2,\mathrm{such} \mathrm{that}\sum _n{n}^a\alpha \left(n\right)<\infty .$$

(2)

As for all asymptotic results, in nonparametric functional statistic, we need to control the local concentration of both marginal and joint distributions of the functional observations. Indeed, for the mariginal distribution, we assume that

$${\varphi }_x\left(h\right):=\mathrm{I}\mathrm{P}(X\in Ba(x,h))>0,\mathrm{for} \mathrm{any}h>0,$$

(3)

such that

$$\exists 0<c<1<c^{\prime }<\infty \mathrm{satisfies}\underset{r\to 0}{lim}\frac{{\varphi }_x\left(rc\right)}{{\varphi }_x\left(r\right)}<1<\underset{r\to 0}{lim}\frac{{\varphi }_x\left(r{c}^{\prime }\right)}{{\varphi }_x\left(r\right)}.$$

(4)

For the joint distribution, we assume that

$${\mathsf{\Psi }}_x\left(r\right)=\underset{i\ne j}{sup}\mathrm{I}\mathrm{P}\left[(X_i,X_j)\in Ba(x,r)\times Ba(x,r)\right]>0,\mathrm{for} \mathrm{any}r>0,$$

(5)

where $Ba(x,h):=\left\{z\in \mathcal{F}\mathrm{satisfies}d(z,x)<r\right\}$ refers to a ball with a center x and a radius h.

The challenging aim of the paper is to establish the convergence rate of the kNN-LLM estimator without independence assumption. Of course, this general consideration requires different tools to those used for the independent situation. In the rest of this section, we give the necessary background to handle this situation.

Lemma 1

(see Ferraty and Vieu [17]). Let ${\left(Z_i\right)}_{i\in \mathrm{I}\mathrm{N}}$ be an algebraic α-mixing process which is identically distributed.

1. 

If there exist $p>2$ and $M>0$ such that, for all $t>M,\mathrm{I}\mathrm{P}(|Z_1|>t)\le t^{-p}$, then, for all $r\ge 1$, $ϵ>0$, we have

$$\mathrm{I}\mathrm{P}\left(\left|{\displaystyle \sum _{i=1}^n{Z}_i}\right|>ϵ\right)\le C\left[{\left({\displaystyle 1+\frac{{ϵ}^2}{r{s}_n^2}}\right)}^{-r/2}+n{r}^{-1}{\left({\displaystyle \frac{r}{ϵ}}\right)}^{(a+1)p/(a+p)}\right].$$

2. 

If there exists $M<\infty$ such that $|Z_1|\le M$, then, for all $r\ge 1$ and $C<\infty$:

$$\mathrm{I}\mathrm{P}\left(\left|{\displaystyle \sum _{i=1}^n{Z}_i}\right|>ϵ\right)\le C\left[{\left({\displaystyle 1+\frac{{ϵ}^2}{r{s}_n^2}}\right)}^{-r/2}+n{r}^{-1}{\left({\displaystyle \frac{r}{ϵ}}\right)}^{a+1}\right],$$

where ${\displaystyle s_n^2=\sum _{i=1}^n\sum _{j=1}^n|cov(Z_i,Z_j)|}$.

3. Results: The Asymptotic Properties of the kNN-LLM Estimator of CCDF

Now, we prove the a.co. convergence of the estimator $\tilde{F}(y|x)$ toward $F(y|x)$. Firstly, let us point out that the condition (4) ensures the existence of $(\alpha ,\beta )\in {(0,1)}^2$ that satisfies

$${\varphi }_x^{-1}\left(\frac{k}{\alpha n}\right)\le C{\varphi }_x^{-1}\left(\frac{\beta k}{n}\right).$$

So, in the remainder of this paper, we put ${\mathrm{I}\mathrm{H}}_k^r={\varphi }_x^{-1}\left(k/\alpha n\right)$, ${\mathrm{I}\mathrm{H}}_k^l={\varphi }_x^{-1}\left(\beta k/n\right)$, ${\ell }^r=l/\alpha n$ and ${\ell }^l=\alpha l/n$. Next, to establish the convergence rate of our estimator, we set the following conditions.

The kernel $Ker$ is differentiable function and has support $[0,1]$. Moreover, its first derivative $Ke{r}^{\prime }$ exists and ∃ C and $C^{\prime }$ satisfy

$$-\infty <C^{\prime }<Ke{r}^{\prime }\left(t\right)<C<0\mathrm{for}0\le t\le 1.$$

(6)

and for $h={\mathrm{I}\mathrm{H}}_k^l\mathrm{or}{\mathrm{I}\mathrm{H}}_k^r$,

$$\exists n_0\exists C>0\mathrm{such} \mathrm{that}\forall n>n_0,-\frac{1}{{\varphi }_x\left(h\right)}{\int }_{-1}^1{\varphi }_x\left(zh\right)\frac{\partial }{\partial z}\left(z^{2}Ker\left(z\right)\right)dz>C>0.$$

The conditional distribution function satisfies, for all $(x_1,x_2)\in {\mathcal{N}}_x\times {\mathcal{N}}_x$ and $(y_1,y_2)\in {\mathrm{I}\mathrm{R}}^2$,

$$\begin{array}{c}\hfill |F(y_1|x_1)-F(y_2|x_2)|\le C\left(d{(x_1,x_2)}^{b_1}+{|y_1-y_2|}^{b_2}\right)b_1>0,b_2>0.\end{array}$$

(7)

The sequence

$${\tau }_n=\underset{j=0,1;h={\mathrm{I}\mathrm{H}}_k^l,\mathrm{or}{\mathrm{I}\mathrm{H}}_k^r}{max}Var\left[\frac{1}{n{\varphi }_x\left(h\right)}\sum _{i=1}^n{d}^j(x,X_i)Ker\left(h^{-1}d(x,X_i)\right)\right],$$

satisfies that

$$\sum _{n}n{\tau }_n^{-(a+1)}n{log}^{\left(\right(a-3)/2)}n<\infty .$$

(8)

Remark 1.

The conditions (6)–(8) are quite weak in time series analysis. Indeed, we point out that most of the conditions are similar to the previous works in functional time series. On the other hand, similar conditions to (6) and (8) can be also found in Ferraty and Vieu [17]. A deeper and clear discussion on the generality of our framework is given in the next Section.

Theorem 1.

Using the approximation condition in (1) and under the conditions (6)–(8), if the derivative of the function H exists and is an increasing function satisfies

$${\displaystyle {\int |t|}^2{H}^{\prime }\left(t\right)dt<\infty ,}$$

then,

$$|\tilde{F}(y|x)-F(y|x)|=O{\left(\frac{l}{n}\right)}^{b_2}+O\left({\varphi }_x^{-1}{\left(\frac{k}{n}\right)}^{b_1}\right)+O_{a.co.}\left(\sqrt{\frac{{\tau }_{n}logn}{n^2}}\right).$$

Proof of Theorem 1. 

The details of the proof is given in the Supplementary File. It is based on the following Lemmas. □

Lemma 2.

Using the same conditions of Theorem 1, we have

$$|S_{j,n}-\mathrm{I}\mathrm{E}\left[S_{j,n}\right]|=O_{a.co.}\left(\sqrt{\frac{{\tau }_{n}logn}{n^2}}\right),$$

where

$$S_{j,n}=\frac{1}{n{h}^j{\varphi }_x\left(h\right)}\sum _{i=1}^n{d}^j(x,X_i)Ker\left(h^{-1}d(x,X_i)\right);h={\mathrm{I}\mathrm{H}}_k^l,or{\mathrm{I}\mathrm{H}}_k^{r}andj=1,2.$$

Lemma 3.

Using the same conditions of Theorem 1, we have

$$e_{j,n}-\mathrm{I}\mathrm{E}\left[e_{j,n}\right]=O_{a.co.}\left(\sqrt{\frac{{\tau }_{n}logn}{n^2}}\right),$$

where,

$$e_{j,n}=\frac{1}{n{h}^j{\varphi }_x\left(h\right)}\sum _{i=1}^n{d}^j(x,X_i)Ker\left(h^{-1}d(x,X_i)\right)H\left(\ell -1(y-Y_i)\right);ell={\ell }^l,or{\ell }^r,andj=0,1.$$

Lemma 4

(see Laksaci et al. [6]). Under the conditions (1), (3), and (7), we have

$$\mathrm{I}\mathrm{E}\left[e_{j,n}\right]=O{\left(\frac{l}{n}\right)}^{b_2}+O\left({\varphi }_x^{-1}{\left(\frac{k}{n}\right)}^{b_1}\right).$$

In the following, we will show the mathematical proofs of the above intermediate results. When no ambiguity is possible, C and $C^{\prime }$ will be used to denote some strictly positive generic constants with $K_i(h,x)=Ker\left(h^{-1}d(X_i,x)\right)H_i(\ell ,y)=H\left({\ell }^{-1}(y-Y_i)\right)$.

Proof of Theorem 2. 

The proof is based on the exponential inequality of the Fuck-Nagaev (see Lemma 1) on

$$Z_i^j=\frac{1}{n{h}^j{\varphi }_x\left(h\right)}\left(d^j(x,X_i)Ker\left(h^{-1}d(x,X_i)\right)-\mathrm{I}\mathrm{E}\left[d^j(x,X_i)Ker\left(h^{-1}d(x,X_i)\right)\right]\right).$$

Since

$$S_{j,n}=\sum _{i=1}^n{Z}_i^j,$$

it follows that, for all $\epsilon ,r>0$, we obtain

$$\begin{array}{ccc}\hfill \mathrm{I}\mathrm{P}\left[\left|S_{j,n}-\mathrm{I}\mathrm{E}\left[S_{j,n}\right]\right|>\epsilon \sqrt{n^{-j}{\tau }_{n}logn}\right]& \le & C(A_1\left(x\right)+A_2\left(x\right)),\hfill \end{array}$$

(9)

where

$$A_1\left(x\right)={\left({\displaystyle 1+\frac{{\epsilon }^2{n}^{-2}{\tau }_{n}logn}{r{s}_n^2}}\right)}^{-r/2}{s}_n^2=Var\left(S_{j,n}\right)\mathrm{and}{A}_2\left(x\right)=n{r}^{-1}{\left({\displaystyle \frac{r}{\epsilon \sqrt{n^{-2}{\tau }_{n}logn}}}\right)}^{a+1}.$$

Set $r=C{(logn)}^2$ to conclude that

$$\begin{array}{c}\hfill A_2\left(x\right)\le C{n}^{-a}{\tau }_n^{(a+1)/2}{(logn)}^{(a-3)/2},\end{array}$$

(10)

and use (9) to obtain

$$\begin{array}{ccc}\hfill A_1\left(x\right)& \le & C{\left(1+\frac{\epsilon }{logn}\right)}^{-{(logn)}^2/2}\le e^{-({\epsilon }^{2}logn)/2}.\hfill \end{array}$$

(11)

For a suitable choice of $\epsilon$ and by (8), we get

$$S_{j,n}-\mathrm{I}\mathrm{E}\left[S_{j,n}\right]=O_{a.co.}\left(\sqrt{\frac{{\tau }_{n}logn}{n^2}}\right).$$

□

Proof of Theorem 3. 

Once again, as in Lemma 2’s proof, we apply the exponential inequality of Fuck-Nagaev for another random variables as

$$T_i^j=\frac{1}{n{h}^j{\varphi }_x\left(h\right)}\left(d^j(x,X_i)Ker\left(h^{-1}d(x,X_i)\right)H\left(\ell -1(y-Y_i)\right)\right.$$

$$\left.-\mathrm{I}\mathrm{E}\left[d^j(x,X_i)Ker\left(h^{-1}d(x,X_i)\right)H\left(\ell -1(y-Y_i)\right)\right]\right).$$

Since

$$e_{j,n}=\sum _{i=1}^n{T}_i^j,$$

we get

$$\begin{array}{c}\hfill \sum _n\mathrm{I}\mathrm{P}\left[\left|e_{j,n}-\mathrm{I}\mathrm{E}\left[e_{j,n}\right]\right|>\epsilon \sqrt{n^{-2}{\tau }_{n}logn}\right]<\infty ,\end{array}$$

which allows to write

$$e_{j,n}-\mathrm{I}\mathrm{E}\left[e_{j,n}\right]=O_{a.co.}\left(\sqrt{\frac{{\tau }_{n}logn}{n^2}}\right).$$

This last achieves the proof of this lemma. □

4. Discussions and Comments

4.1. The kNN Method in Functional Statistics

Motivated by its flexibility in practice, the kNN-method is becoming a popular nonparametric data analysis. It was introduced in functional statistics by Burba et al. [7]. The implementation of this approach in functional data analysis is promising. Indeed, as with all nonparametric smoothing approaches, the kNN method has some drawbacks in multivariate analysis, such as its high sensitivity to feature vectors, the slowness of the execution-time when the data has large volume, or the excessive use of the memory. It is well known that all these drawbacks are due to the problem of the curse of dimensionality in vectorial statistics. This problem was handled by using the small probability function ${\varphi }_x$ to evaluate the asymptotic property of the estimator. Indeed as discussed in Ferraty et al. [7] the small probability function ${\varphi }_x\left(h\right)=\mathrm{I}\mathrm{P}(X\in Ba(x,h))$ quantifies the concentration property of the probability measure of the functional variable, and it has an inherent role in functional data analysis. In a sense that a less concentration of the probability measure of the functional variable implies a slower rate of convergence of the estimator. Then, the best way of resolving the above drawbacks is to increase the concentration of the functional variable in neighbor of the location point x. To do that, we use the fact that the small ball probability function depends crucially on the metric $d(.,.)$. Hence, from a statistical point of view, we can increase the concentration property by choosing the best metric d. Thus, we can say that the implantation of the kNN in functional data analysis is very beneficial in practice, and their drawbacks of the multivariate can be overcome by using the appropriate topological structure.

4.2. On the Impact of This Contribution

It is well known that the conditional distribution function has a pivotal role in nonparametric statistics modeling. Indeed, the nonparametric estimation of this model is an imperative step for various nonparametric models, such as conditional density, the conditional hazard, and the conditional quantile functions. The conditional cumulative distribution function in the prediction setting allows to construct many predictive intervals or, more generally, predictive regions. We mention, for instance, the conditional percentile interval, the shortest conditional modal interval (SCMI), and the maximum conditional density region (MCDR) (see Yao and Tong [18] for their definitions). Of course, the diversity of the applicability of the conditional distribution function highlights the importance of this conditional model, which has the advantage of completely characterizing the conditional law of the considered random variables. As mentioned in the bibliographical discussion of the introduction’s section, this model has been widely studied in nonparametric functional statistics. However, the novelty of the present contribution is mainly the estimation of this model by combining two important approaches: the local linear method and the k-Nearest Neighbors procedure. This combination allows to construct a new attractive estimator that inherits the advantages for both methods. Indeed, it is well known that the local linear method improves the bias property of the kernel method, while the weighting by the kNN algorithm offers a sophisticated procedure for the smoothing parameter selection. It is locally selected with respect to the vicinity at the conditioning point, which permits to construct an adaptive estimator to the data’s local structure. Such consideration is very important in (nonparametric functional data analysis, where the performance of estimators is closely linked to the local structure of the data through concentration properties of the probability measure (see Ferraty and Vieu [17]). Nevertheless, the establishment of the asymptotic properties of this estimator is more difficult than the classical case studied by Laksaci et al. [6] because, here, the bandwidth parameter is a random variable, unlike the standard case where the bandwidth parameter is a scalar. Considering the dependent case, which is a more general and more realistic situation, this difficulty becomes more complicated. We can say that the principal axes of this contribution are (1) the conditional distribution function as a pivotal model for various nonparametric conditional models, (2) the estimation method as a new proceeder even in the nonfunctional case (as far as we know, there is no work in the CCDF estimation by combining the LLE to kNN) and (3) the functional time series case as a generalization of the independent case. To emphasize the usefulness of the present contribution in the prediction issue, we discuss in the following section how we can predict future real characteristics of a continuous-time process given its past.

4.3. Some Particular Cases

One of the main features of the present work is treating the kNN-local linear estimation under the dependent case. The latter allows for regrouping several usual situations. To highlight the importance and the generality of the present contribution, we come back to particularize our study for these usual situations. In particular, we consider the independent case, the strong local dependency case, and the local constant method. Let us note that, for the sake of brevity, detailed proof of the corollaries is given in the Supplementary File.

• The independent case: The independent case is widely studied in the past for some alternative models. However, this case can be treated as a particular case for this contribution. It corresponds to the case of $\alpha \left(n\right)=0$. In this situation, the condition (2) is automatically stratified, and Theorem 1 leads straightforwardly to the following Corollary.

  Corollary 1.

  Under the conditions (6) and (7) and if the derivative of the function H exists and is an increasing function satisfies

  $${\displaystyle {\int |t|}^2{H}^{\prime }\left(t\right)dt<\infty ,}$$

  then,

  $$|\tilde{F}(y|x)-F(y|x)|=O{\left(\frac{l}{n}\right)}^{b_2}+O\left({\varphi }_x^{-1}{\left(\frac{k}{n}\right)}^{b_1}\right)+O_{a.co.}\left(\sqrt{\frac{logn}{k}}\right).$$

  We point out that this result is also new as the kNN-LLM estimator of the CCDF in the i.i.d. case.
• The strong local dependency: The second particular case is the case when the local dependency, measured by ${\mathsf{\Psi }}_x\left(r\right)$, is of order

  $${\mathsf{\Psi }}_x\left(r\right)\le C\left({\varphi }_x{\left(r\right)}^{(a+1)/a}\right).$$

  Then, in this situation, the Theorem 1 is reformulated as follows.

  Corollary 2.

  Under the conditions (6) and (7) and if the derivative of the function H exists and is an increasing function satisfies

  $${\displaystyle {\int |t|}^2{H}^{\prime }\left(t\right)dt<\infty ,}$$

  then,

  $$|\tilde{F}(y|x)-F(y|x)|=O{\left(\frac{l}{n}\right)}^{b_2}+O\left({\varphi }_x^{-1}{\left(\frac{k}{n}\right)}^{b_1}\right)+O_{a.co.}\left(\sqrt{\frac{logn}{k}}\right).$$

  Obviously, the convergence rate of this particular case is more speed than the general form given in Theorem 1.
• The local constant method: It is well known that the Nadaraya-Watson estimator can be viewed as a particular case of the local linear approach. It can be obtained by taking $b=0$. This case is so-called local constant approach and its kNN estimator is defined by

  $$F_n(y|x)=\frac{{\sum }_{i=1}^{n}Ker\left({\mathrm{I}\mathrm{H}}_k^{-1}d(x,X_i)\right)H\left({\ell }_l^{-1}(y-Y_i)\right)}{{\sum }_{i=1}^{n}Ker\left({\mathrm{I}\mathrm{H}}_k^{-1}d(x,X_i)\right)}.$$
  This estimator has been studied by Karra et al. [10]. They established its asymptomatic properties when the observations are independent identically distributed. While, here, we develop the dependent case. Once again, the kNN-LCM estimator’s consistency is also new in this context of nonparametric functional data analysis. It is given in the following corollary.

  Corollary 3.

  Using the approximation condition in (1) and under the conditions (6)–(8), if the derivative of the function H exists and is an increasing function satisfies

  $${\displaystyle {\int |t|}^2{H}^{\prime }\left(t\right)dt<\infty ,}$$

  then,

  $$|F_n(y|x)-F(y|x)|=O{\left(\frac{l}{n}\right)}^{b_2}+O\left({\varphi }_x^{-1}{\left(\frac{k}{n}\right)}^{b_1}\right)+O_{a.co.}\left(\sqrt{\frac{{\tau }_{n}logn}{n^2}}\right).$$

5. Real Data Applications

5.1. Application to Functional Time Series Prediction

One of the main feature of the CCDF is the possibility to construct several predictive regions $I_{\zeta }$. Of course, the efficiency of each prediction interval is assessed by the means of the length of the set $I_{\zeta }$ and the presence of the true value in $I_{\zeta }$. It is well documented that the width of the SCMI is the smallest compared to all predictive regions with the same coverage probability. It was introduced by Yao and Tong [18] and obtained by

$$[A_{1-\zeta },B_{1-\zeta }]=arg\underset{c,d}{min}\left\{Leb[c,d]|F^x\left(d\right)-F^x\left(c\right)\ge 1-\zeta \right\}.$$

The $Leb(·)$ refers to the Lebesgue measure. Using the CCFD estimators, we approximate the SCMI by

$$[A_{1-\zeta }\left(X_n\right),B_{1-\zeta }\left(X_n\right)]=arg\underset{c,d}{min}\left\{Leb[c,d]|\tilde{F}(d|X_n)-\tilde{F}(c|X_n)\ge 1-\zeta \right\}.$$

5.2. Example 1: Application to Climatological Time Series Data

In this first example, we show the applicability of the proposed estimators to climatological data. Indeed, we predict the monthly average temperature one year ahead in Debrecen’s station in Hungary. The link to the data is provided in the “Data Availability Statement” section. Let us note that the studied data is recently collected, and it contains only some missing values, which are replaced by the average of the values before and after the analyses. Now, from this observed data, we construct $n+1=100$ curves $\left(X_i\left(t\right)\right),i=1,\dots ,n+1$, where $X_i$ denotes the average temperature curve observed during the (1 year) 12 months of the i-th year. The observed data are plotted in Figure 1, representing the values of the monthly average of the temperature.

In Figure 2, we plot the ${\left(X_i\right)}_i$ that represents the yearly curves of the temperature.

The efficiency of the SCMI predictive interval is linked to the parameters’ choices in the estimator $\tilde{F}$. For this computational purpose, we compute $\tilde{F}$ with the quadratic kernel K. The metric d is determined according to the PCA-algorithm. In this application part, we shall compare the predictive interval (SCMI $\zeta =0.1$) using the kNN-LLM estimator instated to the CKM-estimator studied by Laksaci et al. [6]. In both estimators, we choose k and h by the same cross-validation method used by De Gooijer and Gannoun [19], which is based on the following criterion:

$$CV=\frac{1}{n}\sum _{j=1}^n\sum _{i=1}^n{(1{\mathrm{I}}_{\{Y_i\le Y_j\}}-{\widehat{F}}^{-j}(Y_i|x_j))}^2.$$

This criterion is optimized over the same subsets of k and h, proposed by Rachdi and Vieu [17].

Now, the SCMI predictive interval of the whole curve of the last year ($i=100$) of this sample knowing the functional covariates $X_{99}$, by commuting $\tilde{F}(·|X_{{99}^{*}})$, where $X_{{99}^{*}}$ is the nearest curve to $X_{99}$ using the learning sample ${(Y_i^j,X_i)}_{i=1\dots 99}$ with $Y_i^j=X_{i+1}\left(j\right)$, for each fixed month $j=1,\dots ,12$. Figure 3 displays the results where we plot three curves: The true curve in the dashed line and the extremities of the SCMI- interval are given in the continuous line.

The observed data is displayed in the dashed curve and the solid curves represent the estimated values for the two extremities of the SCMI predictive interval. It is clear that the kNN-LLM is significantly better than the CKM- estimator basis one. This gain is confirmed by the average of the SCMI-length(see

Table 1. The average of the length of the SCMI for kNN-LLM and CKM estimators.

	kNN-LLM Estimator	CKM Estimator
Average of the length of the SCMI	1.23	2.07

).

5.3. Example 2: Application to Air Quality Time Series Data

In addition to the first example that highlights the importance of the kNN approach over the classical kernel method, we emphasize in this second example the superiority of the local linear approach over the local constant approach. For this purpose, we consider a time series from air quality data. The importance of this kind of data is motivated by the fact that the air quality has a potential environmental impact on the quality of life of humans and the health of animals. In particular, it is well documented that exposures to ground ozone levels for a period of 1–8 h reduce various pulmonary functions and affect the respiratory tract’s tissues. Thus, the approximation of the excessive level of ozone concentration is a primordial subject in environmental sciences. In this example, we focus on the air quality in Westminster city in London. The time-series data was collected at the Marylebone road site by real-time monitoring. Let us point out that our study can be used to model the distribution of the ozone given the daily curves of the ocher polluting gases (carbon monoxide, carbon dioxide, Sulphur dioxide, Nitrogen dioxide, Nitric oxide, etc.). However, for the sake of shortness, we focus on two important indices of air quality that are the sulfur dioxide (SO₂) and the ozone concentration (O₃). It is well known that (SO₂) increases the stratospheric ozone concentrations when it reacts with ultraviolet rays. This example’s data is more complicated than the first example because the time series data is observed on a more fine grid. The link to this data is also provided in the “Data Availability Statement” section. Precisely, unlike the yearly curve of the first example, here, the ozone concentration is hourly observed, and the sulfur dioxide is observed by time-gride equal to 15 min. Thus, this kind of time series data is more adapted for our functional approach. It worth noting that the use of functional statistical models in this type of environmental studies has been widely studied by many authors FDA (see, for example, Quintela-del-Río and Francisco-Fernández (2011) to cite a few). In this example, we aim to analyze the relationship between the SO₂ and the O₃ in Westminster city using the SCMI predictive region. Specifically, we wish to predict the total ozone one day before using the daily curve of SO₂. Formally, we observe 364 days of air quality data $(X_i,Y_i)$ in Marylebone road station, where $X_i(.)$ is the daily curve of SO₂ in day i and $Y_i$ the total ozone of the day $i+1$. A sample of the functional regressors is shown in Figure 4. It concerns the daily curves of the sulphur dioxide observed by a time gride equal to 15 min, and the value of SO₂ in the vertical axis was measured in term of micrograms per cubic meter (μg/m³).

We highlight the importance of the local linear approach over the classical local constant one for this actual data set. In particular, the local linear approach reduces the bias term of the local constant. So, we shall quantify this gain in practice. To do this, we compare the SCMI of both estimators: the kNN-LLM and the kNN-CKM. In addition, we keep the same strategies as those used in the first example to select the parameters involved in the estimator. More precisely, we use the quadratic kernel on $(0,1)$, the PCA metric, and the criterion $CV$ to choose the number of neighborhood k.

For our comparison purpose, we put $\zeta =0.05$ and split the data sample randomly into two parts: the learning sample (260 observations) and the test sample (104 observations). Finally, we examine both estimators’ efficiency by Probability Coverage (PC), which is the main criterion to evaluate the predictive regions. In particular, we draw in the Figure 5, the PC of the testing sample obtained by the two estimation methods.

We see that the local linear estimation has better performance than that based on the local constant method. Of course, this conclusion is not surprising since it reflects the superiority of the local linear approach in the bias term. Undoubtedly, we can say that the kNN-LLM keeps its advantages over the local constant method in the functional time series case.

6. Conclusions and Perspectives

The present contribution investigates the problem of the local linear estimation of the distribution function of real random variable conditioning on a functional covariate. The main novelty of this paper is to construct an estimator using the double kernels kNN estimation procedure. The main feature of the built estimator is its smoothing property. The latter improves the estimators’ flexibility and broadens the scope of application of the conditional distribution estimation. From a theoretical point of view, the estimator’s asymptotic property is established under a more general domain called the functional time series case. Specifically, the dependence setting is modeled through the strong mixing condition. It is well known that this kind of dependency covers an extensive class of usual processes, including the AR process, ARMA process, Gaussian process, Markov process, Linear process, and m-dependent, among others. From a practical point of view, we have illustrated the constructed estimator’s feasibility using real data. The computational study shows that the proposed estimator has a good behavior as prediction models. It improves the prediction by the classical kernel method in both single predictions or by predictive region. This statement confirms the superiority of the kNN local linear estimation over the standard kernel one. Moreover, in addition to this considerable development of the nonparametric functional conditional models, the present contribution opens numerous future research tracks in nonparametric functional data analysis. For instance, it will be very interesting to establish the asymptotic normality of the proposed estimator, to consider the weak functional time series case or incomplete functional time series data case. On the other hand, the robustness of the predictors is a crucial issue in functional data analysis. At this stage, studying the consistency of the kNN-LLM estimator of the robust regression in functional time series is an important prospect of the present contribution. It permits the reduction of the sensitivity of the kNN approach to the noisy data, missing values, and the presence of outliers.