Analysis and Conditional Optimization of Projection Estimates for Distribution of Random Variable Using Legendre Polynomials

Abstract

Algorithms for jointly obtaining projection estimates of the density and distribution function of a random variable using Legendre polynomials are proposed. For these algorithms, a problem of conditional optimization is solved. Such optimization allows one to increase the approximation accuracy with minimum computational costs. The proposed algorithms are tested on examples with different degrees of smoothness of the density. A projection estimate of the density is compared to a histogram that is often used in applications to estimate distributions.

1. Introduction

Statistical projection estimates in the Monte Carlo method were first proposed by N.N. Chentsov [1]. He developed a general technique for optimizing such estimates; this technique, however, requires clarification in specific problems [2]. In the paper [3], projection estimates based on the Legendre polynomials for marginal probability densities of solutions to stochastic differential equations were proposed. The mean square error of the estimates was studied, and a comparison of both the obtained projection estimates and the histogram was carried out with examples. The analysis of the results showed that for the same sample size, the projection estimate approximates the density more accurately. In addition, such an estimate is specified analytically, and it is a smooth function. So, it is preferable in, e.g., filtering problems and the control of nonlinear systems [4,5,6,7].

This paper presents two statistical algorithms, based on algorithms using Legendre polynomials from [3], for jointly obtaining projection estimates of the density and distribution function of a random variable.

When solving different problems by statistical estimation algorithms, it is important to make an optimal (consistent) choice of their parameters for finding the mathematical expectation of a certain functional that depends on a random variable. Therefore, this paper analyzes the mean square error of the projection estimate from this point of view. Such parameters are the projection expansion length and sample size. We solve a conditional optimization problem considered by G.A. Mikhailov [8]. The objective of this problem is to minimize the algorithm complexity while achieving the required level of approximation accuracy. We study how to minimize the mean square error of the projection estimates of the density and distribution function by the equalization of its deterministic and stochastic components. The accuracy of the projection estimates also depends on the degree of smoothness of the density; therefore, in this paper, we consider the dependence of error not only on the projection expansion length but also on the degree of smoothness of the approximated function.

The obtained theoretical results are confirmed with examples using a two-parameter family of densities that allows one both to choose the degree of smoothness and to perform a simple calculation of the expansion coefficients with respect to Legendre polynomials.

The rest of this paper has the following structure: Section 2 contains the necessary information on the Fourier–Legendre series and the definition of projection estimates of the density and distribution function; in this section, relations for expansion coefficients of the density and distribution function with respect to Legendre polynomials are obtained. Section 3 presents algorithms for jointly obtaining randomized projection estimates of the density and distribution function. The analysis and conditional optimization of randomized projection estimates are carried out in Section 4. Section 5 proposes a two-parameter family of densities with different degrees of smoothness and related distribution functions; it presents an algorithm for modeling the corresponding random variables, gives the expansion coefficients of the considered densities and distribution functions, and studies the convergence rate of their expansions. Numerical experiments and their analysis are discussed in Section 6. In Section 7, a projection estimate of the density is compared with a histogram. Brief conclusions on the paper are given in Section 8.

2. Using Legendre Polynomials for Randomized Projection Estimates

2.1. Fourier–Legendre Series

The standardized Legendre polynomials ${\left\{P_i\right\}}_{i=0}^{\infty }$ are defined on the set $\Omega =[-1,1]$ as follows [9]:

$$P_i\left(x\right)={\displaystyle \frac{1}{2^{i}i!}}{\displaystyle \frac{d^i{(x^2-1)}^i}{d{x}^i}},i=0,1,2,\dots$$

They form a complete orthogonal system of functions in the space $L_2(\Omega )$ [10], where

$$L_2(\Omega )=\left\{u:{\int }_{\Omega }{\left|u\left(x\right)\right|}^{2}dx<\infty \right\},$$

and satisfy well-known recurrence relations:

$$\begin{array}{c}(i+1)P_{i+1}\left(x\right)=(2i+1)x{P}_i\left(x\right)-i{P}_{i-1}\left(x\right),i=1,2,\dots ,\end{array}$$

(1)

$$\begin{array}{c}(2i+1)P_i\left(x\right)=P_{i+1}^{\prime }\left(x\right)-P_{i-1}^{\prime }\left(x\right),i=1,2,\dots ,\end{array}$$

(2)

where $P_0\left(x\right)=1$ and $P_1\left(x\right)=x$. These relations can be formally applied for $i=0$ when $P_{-1}\left(x\right)=0$.

The explicit formula for the standardized Legendre polynomials is

$$P_i\left(x\right)={\displaystyle \frac{1}{2^i}}\sum _{k=0}^{\lfloor i/2\rfloor }{\displaystyle \frac{{(-1)}^k(2i-2k)!}{k!(i-k)!(i-2k)!}}{x}^{i-2k},i=0,1,2,\dots ,$$

(3)

where $\lfloor ·\rfloor$ is the floor function. The explicit formulae can be specified separately for even indices ($i=2j$), as

$$P_{2j}\left(x\right)={\displaystyle \frac{1}{2^{2j}}}\sum _{k=0}^j{\displaystyle \frac{{(-1)}^k(4j-2k)!}{k!(2j-k)!(2j-2k)!}}{x}^{2j-2k},$$

(4)

and for odd indices ($i=2j+1$), as

$$P_{2j+1}\left(x\right)={\displaystyle \frac{1}{2^{2j+1}}}\sum _{k=0}^j{\displaystyle \frac{{(-1)}^k(4j-2k+2)!}{k!(2j-k+1)!(2j-2k+1)!}}{x}^{2j-2k+1}.$$

(5)

Using the normalization, we obtain the Legendre polynomials ${\left\{{\widehat{P}}_i\right\}}_{i=0}^{\infty }$ that form a complete orthonormal system of functions in $L_2(\Omega )$:

$${\widehat{P}}_i\left(x\right)=\sqrt{{\displaystyle \frac{2i+1}{2}}}{P}_i\left(x\right),i=0,1,2,\dots$$

(6)

The expansion of an arbitrary function $u\in L_2(\Omega )$ into the Fourier–Legendre series has the form

$$u\left(x\right)=\sum _{i=0}^{\infty }{U}_i{\widehat{P}}_i\left(x\right),$$

(7)

where coefficients $U_i$ are defined as inner products, i.e.,

$$U_i={(u,{\widehat{P}}_i)}_{L_2(\Omega )}={\int }_{\Omega }u\left(x\right){\widehat{P}}_i\left(x\right)dx,i=0,1,2,\dots$$

This expansion is a basis for the approximation of u:

$$u\left(x\right)\approx u^{\langle n\rangle }\left(x\right)=\sum _{i=0}^n{U}_i{\widehat{P}}_i\left(x\right),$$

(8)

and the approximation accuracy is estimated depending on the chosen metric (or norm). For example, in $L_2(\Omega )$, we have [11]

$$\parallel u-u^{\langle n\rangle }{\parallel }_{L_2(\Omega )}⩽{\displaystyle \frac{C}{n^s}},$$

(9)

where $C>0$ does not depend on n, under the condition that the generalized derivatives of u up to order s belong to $L_2(\Omega )$. There is also the sharper estimate [12,13]

$$\parallel u-u^{\langle n\rangle }{\parallel }_{L_2(\Omega )}⩽\tilde{C}{\left({\displaystyle \frac{(n-s+1)!}{(n+s+1)!}}\right)}^{1/2},$$

(10)

for which it is assumed that $\tilde{C}>0$ does not depend on n, and the generalized derivatives of u up to order s belong to $L_2(\Omega )$ with weight $\rho \left(x\right)={(1-x^2)}^s$, $s⩽n+1$.

We also present the estimate corresponding to $C(\Omega )$, the space of all continuous functions on $\Omega$. If a function u is continuously differentiable r times on $\Omega$, i.e., $u\in C^r(\Omega )$, and its derivative $u^{\left(r\right)}$ satisfies the Lipschitz condition with parameter $\alpha \in (0,1]$, then

$$|u\left(x\right)-u^{\langle n\rangle }\left(x\right)|⩽{\displaystyle \frac{\widehat{C}}{n^{r+\alpha -1/2}}}\forall x\in \Omega ,$$

where $\widehat{C}>0$ does not depend on n, $r+\alpha >1/2$ [9].

Remark 1.

In inequality (9), the parameter s can be a real non-negative number [14]. In this case, we should use the condition $u\in W_2^s(\Omega )$ for $s⩾0$ instead of the condition that the generalized derivatives of u up to order s belong to $L_2(\Omega )$. If s is a natural number, then $W_2^s(\Omega )$, the Sobolev space, is defined in the usual way [10]:

$$W_2^s(\Omega )=\left\{u:u^{\left(k\right)}\in L_2(\Omega ),k=0,1,\dots ,s\right\}.$$

If s is not a natural number, then it is possible to define $W_2^s(\Omega )$, the Sobolev–Slobodetskij space, by the complex interpolation between $W_2^{\lfloor s\rfloor }(\Omega )$ and $W_2^{\lceil s\rceil }(\Omega )$ [10,15], where $\lceil s\rceil =\lfloor s\rfloor +1$ and $W_2^0(\Omega )=L_2(\Omega )$. However, we use another definition for $W_2^s(\Omega )$ from [15,16,17]:

$$W_2^s(\Omega )=\left\{u\in L_2(\Omega ):{\int }_{{\Omega }^2}{\displaystyle \frac{{|u\left(x\right)-u\left(y\right)|}^2}{{|x-y|}^{1+2s}}}dxdy<\infty \right\},s=\sigma \in (0,1),$$

(11)

and

$$W_2^s(\Omega )=\left\{u\in W_2^{\lfloor s\rfloor }(\Omega ),u^{(\lfloor s\rfloor )}\in W_2^{\sigma }(\Omega ),\sigma =s-\lfloor s\rfloor \right\},s⩾1.$$

The parameter s in inequalities (9) and (10) may be related to the condition that derivatives of fractional order s for u belong to $L_2(\Omega )$. For non-integer s, we should replace the ratio $(n-s+1)!/(n+s+1)!$ with the ratio $\Gamma (n-s+2)/\Gamma (n+s+2)$ in inequality (10), where Γ is the gamma function.

2.2. Projection Estimates of Density and Distribution Functions

Let $\xi$ be the random variable with range $\Omega =[-1,1]$, density g, and distribution function f. The function g equals to zero outside $\Omega$, and the function f is expressed through g [18]:

$$f\left(x\right)={\int }_{-\infty }^{x}g\left(\theta \right)d\theta .$$

(12)

Further, we consider projection estimates of $g,f\in L_2(\Omega )$ using their expansions into the Fourier–Legendre series (the restriction of the density and distribution function to $\Omega$ is implied). Note that the condition $f\in L_2(\Omega )$ follows from the condition $g\in L_2(\Omega )$, which is assumed to be satisfied.

As a function u in the above formulae, we can use the density g and distribution function f (below, we denote the corresponding expansion coefficients by $G_i$ and $F_i$). In this case, the sequence $f^{\langle n\rangle }$ converges to f faster than the sequence $g^{\langle n\rangle }$ converges to g, where

$$g^{\langle n\rangle }\left(x\right)=\sum _{i=0}^n{G}_i{\widehat{P}}_i\left(x\right),f^{\langle n\rangle }\left(x\right)=\sum _{i=0}^n{F}_i{\widehat{P}}_i\left(x\right),$$

(13)

since the degree of smoothness of f is greater by one than the degree of smoothness of g due to their relationship (12).

For example, let g be continuous on $\Omega$, differentiable in a generalized sense only, i.e., $s=1$, $r=0$, and satisfy the Lipschitz condition with parameter $\alpha =1$. Then,

$$\begin{array}{c}\parallel g-g^{\langle n\rangle }{\parallel }_{L_2(\Omega )}⩽{\displaystyle \frac{C^g}{n}}\mathrm{or}{\parallel g-g^{\langle n\rangle }\parallel }_{L_2(\Omega )}⩽{\displaystyle \frac{{\tilde{C}}^g}{\sqrt{(n+1)(n+2)}}},\\ |g\left(x\right)-g^{\langle n\rangle }\left(x\right)|⩽{\displaystyle \frac{{\widehat{C}}^g}{n^{1/2}}},\end{array}$$

and for f, the following estimates hold ($s=2$, $r=1$, $\alpha =1$):

$$\begin{array}{c}\parallel f-f^{\langle n\rangle }{\parallel }_{L_2(\Omega )}⩽{\displaystyle \frac{C^f}{n^2}}\mathrm{or}{\parallel f-f^{\langle n\rangle }\parallel }_{L_2(\Omega )}⩽{\displaystyle \frac{{\tilde{C}}^f}{\sqrt{n(n+1)(n+2)(n+3)}}},\\ |f\left(x\right)-f^{\langle n\rangle }\left(x\right)|⩽{\displaystyle \frac{{\widehat{C}}^f}{n^{3/2}}},\end{array}$$

where $C^g,{\tilde{C}}^g,{\widehat{C}}^g,C^f,{\tilde{C}}^f,{\widehat{C}}^f>0$ are some constants and $x\in \Omega$.

The projection estimates of the density and distribution function are given by expressions corresponding to their approximations (13):    

$${\overline{g}}^{\langle n\rangle }\left(x\right)=\sum _{i=0}^n{\overline{G}}_i{\widehat{P}}_i\left(x\right),{\overline{f}}^{\langle n\rangle }\left(x\right)=\sum _{i=0}^n{\overline{F}}_i{\widehat{P}}_i\left(x\right),$$

(14)

where ${\overline{G}}_i$ and ${\overline{F}}_i$ are estimates of expansion coefficients $G_i$ and $F_i$ based on observations of the random variable $\xi$ [19,20], $i=0,1,\dots ,n$.

2.3. Expansion Coefficients of Density and Distribution Function

The relation between expansion coefficients $G_i$ and the statistical characteristics of the random variable $\xi$ is established using a linear functional J defined by this density as

$$J\phi ={(g,\phi )}_{L_2(\Omega )}={\int }_{\Omega }g\left(x\right)\phi \left(x\right)dx=\mathrm{E}\phi \left(\xi \right),$$

(15)

where $\mathrm{E}$ means the mathematical expectation. In Formula (15), it is required to substitute the Legendre polynomials (6) instead of a function $\phi$, then

$$G_i=\mathrm{E}{\widehat{P}}_i\left(\xi \right),i=0,1,2,\dots$$

The series representation and the corresponding approximation by a partial sum of this series, similar to (7) and (8), can be obtained for an arbitrary orthonormal basis of $L_2(\Omega )$. But the Legendre polynomials have an important advantage; namely, expansion coefficients $G_i$ are easily expressed via initial moments of the random variable $\xi$:

$${\mathrm{M}}_k=\mathrm{E}{\xi }^k,k=1,2,\dots$$

Indeed, using Formulae (3) and (6), we have

$$\begin{array}{cc}\hfill \mathrm{E}{\widehat{P}}_i\left(\xi \right)& ={\displaystyle \frac{1}{2^i}}\sqrt{{\displaystyle \frac{2i+1}{2}}}\sum _{k=0}^{\lfloor i/2\rfloor }{\displaystyle \frac{{(-1)}^k(2i-2k)!}{k!(i-k)!(i-2k)!}}\mathrm{E}{\xi }^{i-2k}\hfill \\ \hfill & ={\displaystyle \frac{1}{2^i}}\sqrt{{\displaystyle \frac{2i+1}{2}}}\sum _{k=0}^{\lfloor i/2\rfloor }{\displaystyle \frac{{(-1)}^k(2i-2k)!}{k!(i-k)!(i-2k)!}}{\mathrm{M}}_{i-2k},\hfill \end{array}$$

(16)

where ${\mathrm{M}}_0=1$.

Another advantage of the chosen basis is that the antiderivative of any Legendre polynomial with index $i>0$ is represented as a linear combination of two Legendre polynomials with neighboring indices due to recurrence relation (2) (for $i=0$, one of the polynomials in a linear combination is also ${\widehat{P}}_0$). Thus, having a projection estimate of the density, we can find a projection estimate of the distribution function by the simplest operations. The expressions required for this are obtained below.

By integrating the left-hand and right-hand sides of relation (2) and taking into account the property of the Legendre polynomials [9], namely, the equality $P_{i+1}(-1)=P_{i-1}(-1)$, we have

$$(2i+1){\int }_{-1}^x{P}_i\left(\theta \right)d\theta =P_{i+1}\left(x\right)-P_{i-1}\left(x\right),i=1,2,\dots ,$$

(17)

or

$$\begin{array}{cc}\hfill & \sqrt{{\displaystyle \frac{2i+1}{2}}}{\int }_{-1}^x{P}_i\left(\theta \right)d\theta ={\displaystyle \frac{1}{\sqrt{2(2i+1)}}}{P}_{i+1}\left(x\right)-{\displaystyle \frac{1}{\sqrt{2(2i+1)}}}{P}_{i-1}\left(x\right)\hfill \\ \hfill & ={\displaystyle \frac{1}{\sqrt{(2i+1)(2i+3)}}}\sqrt{{\displaystyle \frac{2i+3}{2}}}{P}_{i+1}\left(x\right)-{\displaystyle \frac{1}{\sqrt{(2i-1)(2i+1)}}}\sqrt{{\displaystyle \frac{2i-1}{2}}}{P}_{i-1}\left(x\right),\hfill \end{array}$$

i.e.,

$${\int }_{-1}^x{\widehat{P}}_i\left(\theta \right)d\theta ={\displaystyle \frac{1}{\sqrt{(2i+1)(2i+3)}}}{\widehat{P}}_{i+1}\left(x\right)-{\displaystyle \frac{1}{\sqrt{(2i-1)(2i+1)}}}{\widehat{P}}_{i-1}\left(x\right),i=1,2,\dots$$

(18)

Given $i=0$, we obtain

$${\int }_{-1}^x{P}_0\left(\theta \right)d\theta ={\int }_{-1}^{x}d\theta =x+1=P_1\left(x\right)+P_0\left(x\right),$$

or

$${\displaystyle \frac{1}{\sqrt{2}}}{\int }_{-1}^x{P}_0\left(\theta \right)d\theta ={\displaystyle \frac{1}{\sqrt{2}}}{P}_1\left(x\right)+{\displaystyle \frac{1}{\sqrt{2}}}{P}_0\left(x\right)={\displaystyle \frac{1}{\sqrt{3}}}\sqrt{{\displaystyle \frac{3}{2}}}{P}_1\left(x\right)+{\displaystyle \frac{1}{\sqrt{2}}}{P}_0\left(x\right),$$

i.e.,

$${\int }_{-1}^x{\widehat{P}}_0\left(\theta \right)d\theta ={\displaystyle \frac{1}{\sqrt{3}}}{\widehat{P}}_1\left(x\right)+{\widehat{P}}_0\left(x\right).$$

(19)

Relationship (12) between functions g and f, as well as recurrent relations (18) and (19), implies that

$$F_0=G_0-{\displaystyle \frac{G_1}{\sqrt{3}}},F_i={\displaystyle \frac{G_{i-1}}{\sqrt{(2i-1)(2i+1)}}}-{\displaystyle \frac{G_{i+1}}{\sqrt{(2i+1)(2i+3)}}},i=1,2,\dots$$

(20)

Remark 2.

The condition $\Omega =[-1,1]$ does not reduce the generality of the above reasoning. Similar relations can be derived if the range of the random variable ξ is $\Omega =[a,b]$ and $g,f\in L_2(\Omega )$. In this case, the orthonormal Legendre polynomials ${\left\{{\widehat{P}}_i^{\ast }\right\}}_{i=0}^{\infty }$ are defined by the expression

$${\widehat{P}}_i^{\ast }\left(x\right)=\sqrt{{\displaystyle \frac{2i+1}{b-a}}}{P}_i\left({\displaystyle \frac{2x-(b+a)}{b-a}}\right).$$

3. Algorithms for Jointly Obtaining Randomized Projection Estimates of Density and Distribution Functions

The randomization of the projection estimate of the density g is obtained by the first equation of Formula (14) as a result of calculating linear functional (15) for the Legendre polynomials (6) by the Monte Carlo method with independent realizations of the random variable $\xi$:

$${\overline{G}}_i={\displaystyle \frac{1}{N}}\sum _{l=1}^N{\widehat{P}}_i\left({\xi }_l\right),i=0,1,2,\dots ,$$

(21)

where ${\xi }_l$ is the lth realization and N is the sample size (number of realizations).

Estimates ${\overline{G}}_i$ can also be obtained from expression (16) as

$${\overline{G}}_i={\displaystyle \frac{1}{2^i}}\sqrt{{\displaystyle \frac{2i+1}{2}}}\sum _{k=0}^{\lfloor i/2\rfloor }{\displaystyle \frac{{(-1)}^k(2i-2k)!}{k!(i-k)!(i-2k)!}}{\overline{\mathrm{M}}}_{i-2k},$$

(22)

where ${\overline{\mathrm{M}}}_k$ are statistical estimates of initial moments of the random variable $\xi$,

$${\overline{\mathrm{M}}}_k={\displaystyle \frac{1}{N}}\sum _{l=1}^N{\xi }_l^k,k=1,2,\dots ,$$

(23)

and ${\overline{\mathrm{M}}}_0=1$. Using Formulae (4) and (5), we can write the expressions for the estimates of the density expansion coefficients separately for even indices ($i=2j$), as

$${\overline{G}}_{2j}={\displaystyle \frac{1}{2^{2j}}}\sqrt{{\displaystyle \frac{4j+1}{2}}}\sum _{k=0}^j{\displaystyle \frac{{(-1)}^k(4j-2k)!}{k!(2j-k)!(2j-2k)!}}{\overline{\mathrm{M}}}_{2j-2k},$$

(24)

and for odd indices ($i=2j+1$), as

$${\overline{G}}_{2j+1}={\displaystyle \frac{1}{2^{2j+1}}}\sqrt{{\displaystyle \frac{4j+3}{2}}}\sum _{k=0}^j{\displaystyle \frac{{(-1)}^k(4j-2k+2)!}{k!(2j-k+1)!(2j-2k+1)!}}{\overline{\mathrm{M}}}_{2j-2k+1}.$$

(25)

The randomization of the projection estimate of the distribution function f is obtained by the second equation of Formula (14) and recurrence relations (20):

$${\overline{F}}_0={\overline{G}}_0-{\displaystyle \frac{{\overline{G}}_1}{\sqrt{3}}},{\overline{F}}_i={\displaystyle \frac{{\overline{G}}_{i-1}}{\sqrt{(2i-1)(2i+1)}}}-{\displaystyle \frac{{\overline{G}}_{i+1}}{\sqrt{(2i+1)(2i+3)}}},i=1,2,\dots$$

(26)

Here and below, the dependence of the estimates of both expansion coefficients and functions on the sample size N is not indicated for simplicity.

Remark 3.

To obtain a projection estimate of the distribution function f based on the first $n+1$ Legendre polynomials, it is necessary to find estimates of the expansion coefficients of the density g with respect to the first $n+2$ Legendre polynomials.

Further, we present Algorithm 1 for jointly obtaining randomized projection estimates of the density and distribution function of the random variable $\xi$.

Algorithm 1: Jointly obtaining projection estimates of the density and distribution function using explicit formulae for the Legendre polynomials.

0. Set the projection expansion length n and the sample size N. 1. Simulate N realizations ${\xi }_l$ of the random variable $\xi$, $l=1,2,\dots ,N$. 2. Find statistical estimates ${\overline{\mathrm{M}}}_k$ of initial moments of the random variable $\xi$ using Formula (23), $k=1,2,\dots ,n+1$. Set ${\overline{\mathrm{M}}}_0=1$.  3. Find estimates of expansion coefficients ${\overline{G}}_i$ of the density g using Formula (22) or Formulae (24) and (25), $i=0,1,\dots ,n+1$. 4. Find estimates of expansion coefficients ${\overline{F}}_i$ of the distribution function f using Formula (26), $i=0,1,\dots ,n$. 5. Find randomized projection estimates ${\overline{g}}^{\langle n\rangle }$ and ${\overline{f}}^{\langle n\rangle }$ of the density and distribution function, respectively, using Formula (14).

In step 3 in Algorithm 1, errors can occur due to the peculiarities of machine arithmetic when calculating expansion coefficients ${\overline{G}}_i$. To avoid this, it is recommended to use Formula (21) together with recurrence relation (1) and expression (6).

Next, we formulate Algorithm 2 for jointly obtaining randomized projection estimates of the density and distribution function.

Algorithm 2: Jointly obtaining projection estimates of the density and distribution function using the recurrence relation for the Legendre polynomials.

0. Set the projection expansion length n and the sample size N. 1. Simulate N realizations ${\xi }_l$ of the random variable $\xi$, $l=1,2,\dots ,N$. 2. Find estimates of expansion coefficients ${\overline{G}}_i$ of the density g using Formula (21) together with recurrence relation (1) and expression (6), $i=0,1,\dots ,n+1$. 3. Find estimates of expansion coefficients ${\overline{F}}_i$ of the distribution function f using Formula (26), $i=0,1,\dots ,n$. 4. Find randomized projection estimates ${\overline{g}}^{\langle n\rangle }$ and ${\overline{f}}^{\langle n\rangle }$ of the density and distribution function, respectively, using Formula (14).

4. Analysis and Conditional Optimization of Randomized Projection Estimates

In this section, we analyze the error of the projection estimate ${\overline{g}}^{\langle n\rangle }$ relative to the density g of the random variable $\xi$ in $L_2(\Omega )$. Let $B(g,{\overline{g}}^{\langle n\rangle })=\mathrm{E}{\parallel g-{\overline{g}}^{\langle n\rangle }\parallel }_{L_2(\Omega )}$; then,

$$B^2(g,{\overline{g}}^{\langle n\rangle })={\left(\mathrm{E}\parallel g-{\overline{g}}^{\langle n\rangle }{\parallel }_{L_2(\Omega )}\right)}^2⩽\mathrm{E}{\int }_{\Omega }{\left(g\left(x\right)-{\overline{g}}^{\langle n\rangle }\left(x\right)\right)}^{2}dx$$

due to Jensen’s inequality [21]. Further, we consider the following expression:

$${\int }_{\Omega }{\left(g\left(x\right)-{\overline{g}}^{\langle n\rangle }\left(x\right)\right)}^{2}dx=\parallel g-{\overline{g}}^{\langle n\rangle }{\parallel }_{L_2(\Omega )}^2={\parallel g-g^{\langle n\rangle }+g^{\langle n\rangle }-{\overline{g}}^{\langle n\rangle }\parallel }_{L_2(\Omega )}^2.$$

Functions $g-g^{\langle n\rangle }$ and $g^{\langle n\rangle }-{\overline{g}}^{\langle n\rangle }$ are orthogonal in $L_2(\Omega )$ since they belong to different subspaces. The first one is formed by the Legendre polynomials ${\widehat{P}}_i$ for $i=n+1,n+2,\dots$, and the second one is formed by the Legendre polynomials ${\widehat{P}}_i$ for $i=0,1,\dots ,n$. This is a consequence of the equalities

$$g\left(x\right)=\sum _{i=0}^{\infty }{G}_i{\widehat{P}}_i\left(x\right),g^{\langle n\rangle }\left(x\right)=\sum _{i=0}^n{G}_i{\widehat{P}}_i\left(x\right),{\overline{g}}^{\langle n\rangle }\left(x\right)=\sum _{i=0}^n{\overline{G}}_i{\widehat{P}}_i\left(x\right);$$

hence,

$$\parallel g-g^{\langle n\rangle }+g^{\langle n\rangle }-{\overline{g}}^{\langle n\rangle }{\parallel }_{L_2(\Omega )}^2=\parallel g-g^{\langle n\rangle }{\parallel }_{L_2(\Omega )}^2+{\parallel g^{\langle n\rangle }-{\overline{g}}^{\langle n\rangle }\parallel }_{L_2(\Omega )}^2.$$

According to Parseval’s identity [9], we have

$$\parallel g^{\langle n\rangle }-{\overline{g}}^{\langle n\rangle }{\parallel }_{L_2(\Omega )}^2=\parallel \sum _{i=0}^n(G_i-{\overline{G}}_i){\widehat{P}}_i\left(x\right){\parallel }_{L_2(\Omega )}^2=\sum _{i=0}^n{(G_i-{\overline{G}}_i)}^2$$

and

$${\int }_{\Omega }{\left(g\left(x\right)-{\overline{g}}^{\langle n\rangle }\left(x\right)\right)}^{2}dx=\sum _{i=0}^n{(G_i-{\overline{G}}_i)}^2+{\parallel g-g^{\langle n\rangle }\parallel }_{L_2(\Omega )}^2.$$

Therefore, taking into account that the unbiased estimate of mathematical expectation is used [18], namely, $\mathrm{E}{\overline{G}}_i=G_i$, we obtain

$$\begin{array}{cc}\hfill B^2(g,{\overline{g}}^{\langle n\rangle })& ⩽\mathrm{E}\left(\sum _{i=0}^n{(G_i-{\overline{G}}_i)}^2+{\parallel g-g^{\langle n\rangle }\parallel }_{L_2(\Omega )}^2\right)\hfill \\ \hfill & =\sum _{i=0}^n\mathrm{E}{(G_i-{\overline{G}}_i)}^2+{\parallel g-g^{\langle n\rangle }\parallel }_{L_2(\Omega )}^2\hfill \\ \hfill & =\sum _{i=0}^n\mathrm{D}{\overline{G}}_i+{\parallel g-g^{\langle n\rangle }\parallel }_{L_2(\Omega )}^2,\hfill \end{array}$$

(27)

where $\mathrm{D}$ denotes the variance.

The equality ${\overline{G}}_0=G_0$ is satisfied since ${\widehat{P}}_0\left(x\right)=\mathrm{const}$, i.e., $\mathrm{D}{\overline{G}}_0=0$. For $i>0$, the variance of estimates ${\overline{G}}_i$ and the sample size N are inversely proportional [22]; therefore, using additionally inequality (9) under the condition $g\in W_2^s(\Omega )$ (see Remark 1), we find

$$B^2(g,{\overline{g}}^{\langle n\rangle })⩽{\displaystyle \frac{C_1^{g}n}{N}}+{\displaystyle \frac{C_2^g}{n^{2s}}},$$

(28)

where $C_1^g,C_2^g>0$ are constants independent of n and N.

The error of the projection estimate ${\overline{f}}^{\langle n\rangle }$ relative to the distribution function f of the random variable $\xi$ in $L_2(\Omega )$ is analyzed similarly:

$$B^2(f,{\overline{f}}^{\langle n\rangle })⩽{\displaystyle \frac{C_1^{f}n}{N}}+{\displaystyle \frac{C_2^f}{n^{2s+2}}},$$

(29)

where $C_1^f,C_2^f>0$ are constants independent of n and N.

From the obtained estimates, it is clear how the mean square error depends on the projection expansion length n and the sample size N.

Further, we consider a problem of the optimal (consistent) choice of parameters for statistical algorithms to obtain projection estimates of the density and distribution function: the projection expansion length n and the sample size N. For jointly obtaining these estimates by Algorithms 1 and 2, it is sufficient to consider the optimal choice of parameters for density estimation only and use them for distribution function estimation. This is because the degree of smoothness of f is greater than the degree of smoothness of g, so the sequence ${\overline{f}}^{\langle n\rangle }$ converges to f faster than the sequence ${\overline{g}}^{\langle n\rangle }$ converges to g.

The main result is stated using the symbol “≍”. The expression $u\left(n\right)\asymp v\left(n\right)$ for suitable functions u and v means that $u\left(n\right)=O\left(v\right(n\left)\right)$ and $v\left(n\right)=O\left(u\right(n\left)\right)$ for $n\to \infty$; i.e., there exist constants $C_{\ast },C^{\ast }>0$ such that $C_{\ast }\left|v\left(n\right)\right|⩽\left|u\left(n\right)\right|⩽C^{\ast }\left|v\left(n\right)\right|$$\forall n>n^{\ast }$, where n and $n^{\ast }$ are natural numbers.

Theorem 1.

Let the density g of the random variable ξ belong to $W_2^s(\Omega )$, and let ${\overline{g}}^{\langle n\rangle }$ be the randomized projection estimate of the density obtained by Algorithms 1 or 2 with the projection expansion length n and the sample size N. Then, the minimum complexity of obtaining the estimate ${\overline{g}}^{\langle n\rangle }$ is achieved with the parameters $n=n_{\mathrm{opt}}$ and $N=N_{\mathrm{opt}}$ that satisfy relations

$$N_{\mathrm{opt}}\asymp n_{\mathrm{opt}}^{2s+1},N_{\mathrm{opt}}\asymp {\gamma }^{-(2s+1)/s},n_{\mathrm{opt}}\asymp {\gamma }^{-1/s},$$

where γ is the required approximation accuracy for the density g in $L_2(\Omega )$, i.e., $B(g,{\overline{g}}^{\langle n\rangle })⩽\gamma$.

Proof. 

To find the optimal parameters $n_{\mathrm{opt}}$ and $N_{\mathrm{opt}}$ for the estimate ${\overline{g}}^{\langle n\rangle }$, it is sufficient to equate terms [8] in Formula (28) and express the required approximation accuracy $\gamma$ from the relation

$${\displaystyle \frac{C_1^{g}n}{N}}+{\displaystyle \frac{C_2^g}{n^{2s}}}={\gamma }^2.$$

From the equality

$${\displaystyle \frac{C_1^{g}n}{N}}={\displaystyle \frac{C_2^g}{n^{2s}}},$$

we obtain the relationship for the optimal parameters, i.e., $N_{\mathrm{opt}}\asymp n_{\mathrm{opt}}^{2s+1}$, as well as expressions

$${\displaystyle \frac{C_1^{g}n}{N}}={\displaystyle \frac{{\gamma }^2}{2}}\mathrm{and}{\displaystyle \frac{C_2^g}{n^{2s}}}={\displaystyle \frac{{\gamma }^2}{2}},$$

and consequently, $N_{\mathrm{opt}}\asymp {\gamma }^{-(2s+1)/s}$ and $n_{\mathrm{opt}}\asymp {\gamma }^{-1/s}$.    □

Theorem 1 establishes the relationship between parameters in Algorithms 1 and 2, n and N, as well as the dependence of approximation accuracy on parameter s. By choosing parameters in this way, we have

$$B(g,{\overline{g}}^{\langle n\rangle })⩽c{N}^{-s/(2s+1)},$$

where $c>0$ is a constant independent of n and N.

Remark 4.

The error (28) of the randomized projection estimate ${\overline{g}}^{\langle n\rangle }$ relative to the density g is based on inequality (9), but inequality (10) can also be used. Then, taking into account Remark 1, we have

$$B^2(g,{\overline{g}}^{\langle n\rangle })⩽{\displaystyle \frac{C_1^{g}n}{N}}+C_2^g{\displaystyle \frac{\Gamma (n-s+2)}{\Gamma (n+s+2)}}.$$

It is possible to formulate an analogue of Theorem 1 and show that the following relationship between parameters for estimating the density g is conditionally optimal:

$$N_{\mathrm{opt}}\asymp n_{\mathrm{opt}}{\displaystyle \frac{\Gamma (n_{\mathrm{opt}}+s+2)}{\Gamma (n_{\mathrm{opt}}-s+2)}}.$$

Remark 5.

If the distribution function f of the random variable ξ belongs to $W_2^{s+1}(\Omega )$ (this condition holds if the density g of the random variable ξ belongs to $W_2^s(\Omega )$, i.e., if the conditions of Theorem 1 are satisfied), then the minimum complexity of obtaining randomized projection estimate ${\overline{f}}^{\langle n\rangle }$ by Algorithms 1 or 2 is achieved with parameters $n=n_{\mathrm{opt}}$ and $N=N_{\mathrm{opt}}$ that satisfy relations

$$N_{\mathrm{opt}}\asymp n_{\mathrm{opt}}^{2s+3},N_{\mathrm{opt}}\asymp {\gamma }^{-(2s+3)/(s+1)},n_{\mathrm{opt}}\asymp {\gamma }^{-1/(s+1)},$$

where γ is the required approximation accuracy for the function f in $L_2(\Omega )$, i.e., $B(f,{\overline{f}}^{\langle n\rangle })⩽\gamma$.

The proof is similar to the proof of Theorem 1. Another relationship between parameters can be found if we take inequality (10) instead of inequality (9) (see Remark 4).

5. Two-Parameter Family of Densities with Different Degrees of Smoothness

In this section, a special example is proposed to test statistical algorithms for obtaining projection estimates depending on the projection expansion length, sample size, and smoothness of the estimated function.

5.1. Densities with Different Degrees of Smoothness and Related Distribution Functions

Let $\xi$ be the random variable defined by the density $g_{{\nu }_1,{\nu }_2}$:

$$g_{{\nu }_1,{\nu }_2}\left(x\right)={\gamma }_{{\nu }_1,{\nu }_2}\left\{\begin{array}{cc}1-{(-x)}^{{\nu }_1}\hfill & \mathrm{for}x\in [-1,0),\hfill \\ 1-x^{{\nu }_2}\hfill & \mathrm{for}x\in [0,1],\hfill \\ 0\hfill & \mathrm{otherwise},\hfill \end{array}\right.$$

(30)

where ${\nu }_1$ and ${\nu }_2$ are parameters (natural numbers) and ${\gamma }_{{\nu }_1,{\nu }_2}$ is a normalizing constant, with

$${\displaystyle \frac{1}{{\gamma }_{{\nu }_1,{\nu }_2}}}={\int }_{-1}^0\left(1-{(-x)}^{{\nu }_1}\right)dx+{\int }_0^1(1-x^{{\nu }_2})dx={\displaystyle \frac{{\nu }_1}{{\nu }_1+1}}+{\displaystyle \frac{{\nu }_2}{{\nu }_2+1}},$$

because

$${\int }_{-1}^0\left(1-{(-x)}^{\nu }\right)dx={\int }_0^1(1-x^{\nu })dx=\left(x-{\displaystyle \frac{x^{\nu +1}}{\nu +1}}\right){|}_0^1=1-{\displaystyle \frac{1}{\nu +1}}={\displaystyle \frac{\nu }{\nu +1}}.$$

Further, we assume that ${\nu }_1\ne {\nu }_2$. The function $g_{{\nu }_1,{\nu }_2}$ has the following properties:

(a) $g_{{\nu }_1,{\nu }_2}$ is continuous on the set of real numbers;

(b) The support of $g_{{\nu }_1,{\nu }_2}$ is the set $\Omega =[-1,1]$;

(c) The normalization condition holds:

$${\int }_{-\infty }^{+\infty }{g}_{{\nu }_1,{\nu }_2}\left(x\right)dx={\int }_{-1}^1{g}_{{\nu }_1,{\nu }_2}\left(x\right)dx=1;$$

(d) $g_{{\nu }_1,{\nu }_2}$ is differentiable at $x=0$ only r times:

$$g_{{\nu }_1,{\nu }_2}\in C^r(\Omega ),r=min\{{\nu }_1,{\nu }_2\}-1;$$

(e) The derivative $g_{{\nu }_1,{\nu }_2}^{(r+1)}$ exists almost everywhere on $\Omega$; if the derivative is understood in a generalized sense, then

$$g_{{\nu }_1,{\nu }_2}^{\left(s\right)}\in L_2(\Omega ),g_{{\nu }_1,{\nu }_2}\in W_2^s(\Omega ),s=r+1=min\{{\nu }_1,{\nu }_2\}.$$

Next, we determine the distribution function $f_{{\nu }_1,{\nu }_2}$ for the random variable $\xi$ with density (30):

$$f_{{\nu }_1,{\nu }_2}\left(x\right)={\int }_{-\infty }^x{g}_{{\nu }_1,{\nu }_2}\left(\theta \right)d\theta .$$

(31)

It is easy to see that $f_{{\nu }_1,{\nu }_2}\left(x\right)=0$ for $x<-1$ and $f_{{\nu }_1,{\nu }_2}\left(x\right)=1$ for $x>1$. Moreover,

$$\begin{array}{c}{\int }_{-1}^x\left(1-{(-\theta )}^{\nu }\right)d\theta =\left(\theta +{(-1)}^{\nu +1}{\displaystyle \frac{{\theta }^{\nu +1}}{\nu +1}}\right){|}_{-1}^x={\displaystyle \frac{\nu }{\nu +1}}+x+{\displaystyle \frac{{(-x)}^{\nu +1}}{\nu +1}},x\in [-1,0],\\ {\int }_0^x(1-{\theta }^{\nu })d\theta =\left(\theta -{\displaystyle \frac{{\theta }^{\nu +1}}{\nu +1}}\right){|}_0^x=x-{\displaystyle \frac{x^{\nu +1}}{\nu +1}},x\in [0,1],\end{array}$$

and consequently,

$$f_{{\nu }_1,{\nu }_2}\left(x\right)=\left\{\begin{array}{cc}{\displaystyle 0\phantom{\frac{1}{2}}}\hfill & \mathrm{for}x<-1,\hfill \\ {\displaystyle {\gamma }_{{\nu }_1,{\nu }_2}\left(\frac{{\nu }_1}{{\nu }_1+1}+x+\frac{{(-x)}^{{\nu }_1+1}}{{\nu }_1+1}\right)}\hfill & \mathrm{for}x\in [-1,0),\hfill \\ {\displaystyle {\gamma }_{{\nu }_1,{\nu }_2}\left(\frac{{\nu }_1}{{\nu }_1+1}+x-\frac{x^{{\nu }_2+1}}{{\nu }_2+1}\right)}\hfill & \mathrm{for}x\in [0,1],\hfill \\ {\displaystyle 1\phantom{\frac{1}{2}}}\hfill & \mathrm{for}x>1.\hfill \end{array}\right.$$

(32)

The function $f_{{\nu }_1,{\nu }_2}$ is differentiable $r+1$ times at $x=0$ since $f_{{\nu }_1,{\nu }_2}^{(r+1)}=g_{{\nu }_1,{\nu }_2}^{\left(r\right)}$ due to relationship (31). Thus,

$$f_{{\nu }_1,{\nu }_2}\in C^{r+1}(\Omega ),f_{{\nu }_1,{\nu }_2}^{(s+1)}\in L_2(\Omega ),f_{{\nu }_1,{\nu }_2}\in W_2^{s+1}(\Omega ).$$

If we do not restrict ourselves to the space $W_2^s(\Omega )$ with natural s (see Remark 1), then we can show that $s<min\{{\nu }_1,{\nu }_2\}+1/2$ and $sups=min\{{\nu }_1,{\nu }_2\}+1/2$.

Consider the case ${\nu }_1>{\nu }_2$. Then, the generalized derivative $g_{{\nu }_1,{\nu }_2}^{\left({\nu }_2\right)}$ is represented as a linear combination of functions $\eta \left(x\right)$ and $x^{{\nu }_1-{\nu }_2}\eta (-x)$, where $\eta$ is the indicator of $(0,\infty )$. It suffices to find a condition for parameter $\sigma$ which ensures that $\eta \in W_2^{\sigma }(\Omega )$, where $W_2^{\sigma }(\Omega )$ is defined by Formula (11) and $\sigma \in (0,1)$.

If x and y have the same sign, then $\left|\eta \right(x)-\eta (y\left)\right|=0$, and if x and y have different signs, then $\left|\eta \right(x)-\eta (y\left)\right|=1$. Hence,

$${\int }_{{\Omega }^2}{\displaystyle \frac{{|\eta \left(x\right)-\eta \left(y\right)|}^2}{{|x-y|}^{1+2\sigma }}}dxdy={\int }_{-1}^0\left[{\int }_0^1{\displaystyle \frac{dy}{{|x-y|}^{1+2\sigma }}}\right]dx+{\int }_0^1\left[{\int }_{-1}^0{\displaystyle \frac{dy}{{|x-y|}^{1+2\sigma }}}\right]dx.$$

The integrals on the right-hand side of the latter equality coincide since the integrand does not change when the signs of x and y change simultaneously. The convergence condition for them is $\sigma <1/2$. Indeed, for $\sigma <1/2$, we have    

$$\begin{array}{cc}\hfill & {\int }_{-1}^0\left[{\int }_0^1{\displaystyle \frac{dy}{{|x-y|}^{1+2\sigma }}}\right]dx={\int }_{-1}^0\left[{\int }_0^1{\displaystyle \frac{dy}{{(y-x)}^{1+2\sigma }}}\right]dx=-{\displaystyle \frac{1}{2\sigma }}{\int }_{-1}^0{\displaystyle \frac{1}{{(y-x)}^{2\sigma }}}{|}_0^{1}dx\hfill \\ \hfill & =-{\displaystyle \frac{1}{2\sigma }}{\int }_{-1}^0\left[{\displaystyle \frac{1}{{(1-x)}^{2\sigma }}}-{\displaystyle \frac{1}{{(-x)}^{2\sigma }}}\right]dx={\displaystyle \frac{1}{2\sigma (2\sigma -1)}}\left[-{\displaystyle \frac{1}{{(1-x)}^{2\sigma -1}}}{|}_{-1}^0+{\displaystyle \frac{1}{{(-x)}^{2\sigma -1}}}{|}_{-1}^0\right]\hfill \\ \hfill & ={\displaystyle \frac{1}{2\sigma (2\sigma -1)}}\left[{\displaystyle \frac{1}{2^{2\sigma -1}}}-2\right]={\displaystyle \frac{2^{-2\sigma }-1}{\sigma (2\sigma -1)}}\hfill \end{array}$$

and

$${\int }_{{\Omega }^2}{\displaystyle \frac{{|\eta \left(x\right)-\eta \left(y\right)|}^2}{{|x-y|}^{1+2\sigma }}}dxdy={\displaystyle \frac{2(2^{-2\sigma }-1)}{\sigma (2\sigma -1)}}.$$

For $\sigma ⩾1/2$, these integrals obviously diverge. Therefore, $g_{{\nu }_1,{\nu }_2}^{\left({\nu }_2\right)}\in W_2^{\sigma }(\Omega )$ for $\sigma <1/2$, and $g_{{\nu }_1,{\nu }_2}\in W_2^s(\Omega )$ for $s<{\nu }_2+1/2$.

The case ${\nu }_1<{\nu }_2$ is similar to the considered case, so finally, $g_{{\nu }_1,{\nu }_2}\in W_2^s(\Omega )$ and $f_{{\nu }_1,{\nu }_2}\in W_2^{s+1}(\Omega )$, provided that $s<min\{{\nu }_1,{\nu }_2\}+1/2$ and $sups=min\{{\nu }_1,{\nu }_2\}+1/2$.

5.2. Modeling Random Variables with Given Test Distributions Using Monte Carlo Method

The modeling formula for the random variable $\xi$ with distribution function $f_{{\nu }_1,{\nu }_2}$ for parameters ${\nu }_1$ and ${\nu }_2$ can be derived using the inverse function method [22]: $\xi =f_{{\nu }_1,{\nu }_2}^{-1}\left(\alpha \right)$, where $f_{{\nu }_1,{\nu }_2}^{-1}$ is the inverse function to $f_{{\nu }_1,{\nu }_2}$ and $\alpha$ is the random variable having a uniform distribution on $(0,1)$.

Given the distribution function $f_{{\nu }_1,{\nu }_2}$, we can obtain the Algorithm 3 for modeling the random variable $\xi$.

Algorithm 3: Modeling the random variable with given test density and distribution function.

0. Set parameters ${\nu }_1$, ${\nu }_2$, calculate ${\gamma }_{{\nu }_1,{\nu }_2}$ and $f_{{\nu }_1,{\nu }_2}\left(0\right)$: $${\gamma }_{{\nu }_1,{\nu }_2}={\left({\displaystyle \frac{{\nu }_1}{{\nu }_1+1}}+{\displaystyle \frac{{\nu }_2}{{\nu }_2+1}}\right)}^{-1},f_{{\nu }_1,{\nu }_2}\left(0\right)={\gamma }_{{\nu }_1,{\nu }_2}{\displaystyle \frac{{\nu }_1}{{\nu }_1+1}}={\displaystyle \frac{{\nu }_1({\nu }_2+1)}{{\nu }_1+2{\nu }_1{\nu }_2+{\nu }_2}}.$$

1. Obtain a realization of the random variable $\alpha$ having a uniform distribution on $(0,1)$.

2. If $\alpha <f_{{\nu }_1,{\nu }_2}\left(0\right)$, then $\xi$ is a root of the algebraic equation $${\displaystyle \frac{{\nu }_1}{{\nu }_1+1}}+x+{\displaystyle \frac{{(-x)}^{{\nu }_1+1}}{{\nu }_1+1}}-{\displaystyle \frac{\alpha }{{\gamma }_{{\nu }_1,{\nu }_2}}}=0$$ (33)

from $(-1,0)$, otherwise $\xi$ is a root of the algebraic equation $${\displaystyle \frac{{\nu }_1}{{\nu }_1+1}}+x-{\displaystyle \frac{x^{{\nu }_2+1}}{{\nu }_2+1}}-{\displaystyle \frac{\alpha }{{\gamma }_{{\nu }_1,{\nu }_2}}}=0$$ (34)

from $[0,1)$.

For small ${\nu }_1$ and ${\nu }_2$, roots can be found analytically. Next, we obtain the modeling formulae for two cases used below.

Proposition 1.

For the random variable ξ with density

$$g\left(x\right)={\displaystyle \frac{6}{7}}\left\{\begin{array}{cc}1+x\hfill & \mathit{for}x\in [-1,0),\hfill \\ 1-x^2\hfill & \mathit{for}x\in [0,1],\hfill \\ 0\hfill & \mathit{otherwise},\hfill \end{array}\right.$$

the modeling formula is as follows:

$$\xi =\left\{\begin{array}{cc}{\displaystyle \sqrt{\frac{7\alpha }{3}}-1}\hfill & {\displaystyle \mathit{for}\alpha <\frac{3}{7},}\hfill \\ \\ {\displaystyle -\mathrm{Re}A+\sqrt{3}\mathrm{Im}A}\hfill & {\displaystyle \mathit{for}\alpha ⩾\frac{3}{7},}\hfill \end{array}\right.$$

(35)

where

$$A=\sqrt[3]{{\displaystyle \frac{3-7\alpha }{4}}+\sqrt{{\displaystyle \frac{{(7\alpha -3)}^2}{16}}-1}}.$$

Proof. 

The density g of the random variable $\xi$ is included in a two-parameter family (30) for ${\nu }_1=\nu$, ${\nu }_2=\nu +1$, and $\nu =1$: $g=g_{1,2}$. For given parameters, we have

$${\displaystyle \frac{{\nu }_1}{{\nu }_1+1}}={\displaystyle \frac{1}{2}},{\gamma }_{1,2}={\displaystyle \frac{6}{7}},f_{1,2}\left(0\right)={\displaystyle \frac{3}{7}}.$$

First, we consider the case $\alpha <f_{1,2}\left(0\right)=3/7$; i.e., we should solve Equation (33). This is the quadratic equation

$${\displaystyle \frac{1}{2}}+x+{\displaystyle \frac{x^2}{2}}-{\displaystyle \frac{7\alpha }{6}}=0,\mathrm{or}{x}^2+bx+c=0,$$

where $b=2$, $c=1-7\alpha /3>0$.

The function $v\left(x\right)=x^2+2x+1-7\alpha /3$ has a minimum at $x^{\ast }=-1$ since $v^{\prime }\left(x^{\ast }\right)=2x^{\ast }+2=0$, $v^{″}\left(x^{\ast }\right)=2>0$ and $v\left(x^{\ast }\right)=-7\alpha /3<0$. This means that the quadratic equation has two real roots (the discriminant is positive) and the largest of them determines $\xi$:

$$\xi =max\{x_1,x_2\}>x^{\ast },x_{1,2}=-{\displaystyle \frac{b}{2}}\pm \sqrt{{\displaystyle \frac{b^2}{4}}-c}=-1\pm \sqrt{{\displaystyle \frac{7\alpha }{3}}},\xi =\sqrt{{\displaystyle \frac{7\alpha }{3}}}-1\in (-1,0).$$

Second, we consider the case $\alpha ⩾f_{1,2}\left(0\right)=3/7$; i.e., we should solve Equation (34). This is the cubic equation

$${\displaystyle \frac{1}{2}}+x-{\displaystyle \frac{x^3}{3}}-{\displaystyle \frac{7\alpha }{6}}=0,\mathrm{or}{x}^3+px+q=0,$$

where $p=-3$, $q=(7\alpha -3)/2>0$.

The function $w\left(x\right)=x^3-3x+(7\alpha -3)/2$ has extrema at $x_{1,2}^{\ast }=\pm 1$: $w^{\prime }\left(x^{\ast }\right)=3{\left(x_{1,2}^{\ast }\right)}^2-3=0$. A minimum is reached at $x_1^{\ast }=1$: $w^{″}\left(x_1^{\ast }\right)=6x_1^{\ast }=6>0$ and $w\left(x_1^{\ast }\right)=7(\alpha -1)/2<0$. A maximum is reached at $x_2^{\ast }=-1$: $w^{″}\left(x_2^{\ast }\right)=6x_2^{\ast }=-6<0$ and $w\left(x_2^{\ast }\right)=(1+7\alpha )/2>0$. This means that the cubic equation has three real roots (the discriminant is positive) and $\xi$ is determined by the root that lies between the smallest and the largest roots. By using Cardano’s formulae for roots [23], we have

$$x_1=A+B,x_{2,3}=-{\displaystyle \frac{A+B}{2}}\pm {\displaystyle \frac{A-B}{2}}\sqrt{3}\mathrm{i},$$

where

$$A=\sqrt[3]{-{\displaystyle \frac{q}{2}}+\sqrt{Q}},B=\sqrt[3]{-{\displaystyle \frac{q}{2}}-\sqrt{Q}},Q={\left({\displaystyle \frac{p}{3}}\right)}^3+{\left({\displaystyle \frac{q}{2}}\right)}^2,$$

and $Q<0$, which corresponds to the positive discriminant; therefore, A and B are complex conjugate numbers:

$${\displaystyle \frac{A+B}{2}}=\mathrm{Re}A,{\displaystyle \frac{A-B}{2\mathrm{i}}}=\mathrm{Im}A\mathrm{and}{x}_1=2\mathrm{Re}A,x_{2,3}=-\mathrm{Re}A\mp \sqrt{3}\mathrm{Im}A.$$

Let $z=-q/2+\sqrt{Q}$. Since $\mathrm{Re}z=-q/2<0$ and $\mathrm{Im}z=\sqrt{-Q}>0$, we obtain $argz\in (\pi /2,\pi )$; therefore, $argA\in (\pi /6,\pi /3)$ and

$$\mathrm{Re}A,\mathrm{Im}A>0,tan{\displaystyle \frac{\pi }{6}}<{\displaystyle \frac{\mathrm{Im}A}{\mathrm{Re}A}}<tan{\displaystyle \frac{\pi }{3}},\mathrm{or}\mathrm{Re}A<\sqrt{3}\mathrm{Im}A<3\mathrm{Re}A.$$

Thus,

$$x_2<x_3<x_1,\xi =-{\displaystyle \frac{A+B}{2}}-{\displaystyle \frac{A-B}{2}}\sqrt{3}\mathrm{i}=-\mathrm{Re}A+\sqrt{3}\mathrm{Im}A\in (0,1),$$

i.e., Formula (35) is valid. □

Proposition 2.

For the random variable ξ with density

$$g\left(x\right)={\displaystyle \frac{12}{17}}\left\{\begin{array}{cc}1+x^3\hfill & \mathit{for}x\in [-1,0),\hfill \\ 1-x^2\hfill & \mathit{for}x\in [0,1],\hfill \\ 0\hfill & \mathit{otherwise},\hfill \end{array}\right.$$

the modeling formula is as follows:

$$\xi =\left\{\begin{array}{cc}{\displaystyle -\sqrt{\frac{Y}{2}}+\sqrt{-\frac{Y}{2}+\sqrt{\frac{2}{Y}}}}\hfill & {\displaystyle \mathrm{for}\alpha <\frac{9}{17},}\hfill \\ \\ {\displaystyle -\mathrm{Re}A+\sqrt{3}\mathrm{Im}A}\hfill & {\displaystyle \mathrm{for}\alpha ⩾\frac{9}{17},}\hfill \end{array}\right.$$

(36)

where

$$Y=\omega -{\displaystyle \frac{17\alpha -9}{9\omega }},\omega =\sqrt[3]{1+\sqrt{1+{\displaystyle \frac{{(17\alpha -9)}^3}{729}}}},A=\sqrt[3]{{\displaystyle \frac{9-17\alpha }{8}}+\sqrt{{\displaystyle \frac{{(17\alpha -9)}^2}{64}}-1}}.$$

Proof. 

The density g of the random variable $\xi$ is included in a two-parameter family (30) for ${\nu }_1=\nu +1$, ${\nu }_2=\nu$, and $\nu =2$: $g=g_{3,2}$. In this case,

$${\displaystyle \frac{{\nu }_1}{{\nu }_1+1}}={\displaystyle \frac{3}{4}},{\gamma }_{3,2}={\displaystyle \frac{12}{17}},f_{3,2}\left(0\right)={\displaystyle \frac{9}{17}}.$$

The proof is the same as for Proposition 1, so some details are omitted. We only note that for $\alpha <f_{3,2}\left(0\right)=9/17$, Equation (33) is the quartic equation

$${\displaystyle \frac{3}{4}}+x+{\displaystyle \frac{x^4}{4}}-{\displaystyle \frac{17\alpha }{12}}=0,\mathrm{or}{x}^4+bx+c=0,$$

where $b=4$, $c=3-17\alpha /3>0$, the polynomial $v\left(x\right)=x^4+4x+3-17\alpha /3$ has a minimum at $x^{\ast }=-1$, and $v\left(x^{\ast }\right)=-17\alpha /3<0$. The quartic equation has two real roots, and the largest of them determines $\xi$. It is convenient to find roots using Ferrari’s formulae [23].

For $\alpha ⩾f_{3,2}\left(0\right)=9/17$, Equation (34) is the cubic equation that is solved similarly to the cubic equation from the proof of Proposition 1. Such reasoning leads to Formula (36). □

5.3. Expansion Coefficients of Test Functions (Fourier–Legendre Series)

To exactly calculate the second term of the projection estimate error (27) in examples, we find expansion coefficients $G_{{\nu }_1,{\nu }_2,i}$ of the density $g_{{\nu }_1,{\nu }_2}$, as well as expansion coefficients $F_{{\nu }_1,{\nu }_2,i}$ of the distribution function $f_{{\nu }_1,{\nu }_2}$ with respect to Legendre polynomials (6):

$$g_{{\nu }_1,{\nu }_2}\left(x\right)=\sum _{i=0}^{\infty }{G}_{{\nu }_1,{\nu }_2,i}{\widehat{P}}_i\left(x\right),f_{{\nu }_1,{\nu }_2}\left(x\right)=\sum _{i=0}^{\infty }{F}_{{\nu }_1,{\nu }_2,i}{\widehat{P}}_i\left(x\right).$$

First, we obtain the following values:

$$Q_{\nu ,i}^{-}={\int }_{-1}^0{x}^{\nu }{P}_i\left(x\right)dx,Q_{\nu ,i}^{+}={\int }_0^1{x}^{\nu }{P}_i\left(x\right)dx.$$

(37)

We multiply the left-hand and right-hand sides of relation (1) by $x^{\nu -1}$ and integrate over the interval $[-1,0]$:

$$(i+1){\int }_{-1}^0{x}^{\nu -1}{P}_{i+1}\left(x\right)dx=(2i+1){\int }_{-1}^0{x}^{\nu }{P}_i\left(x\right)dx-i{\int }_{-1}^0{x}^{\nu -1}{P}_{i-1}\left(x\right)dx,$$

or

$$(i+1)Q_{\nu -1,i+1}^{-}=(2i+1)Q_{\nu ,i}^{-}-i{Q}_{\nu -1,i-1}^{-},Q_{\nu ,i}^{-}={\displaystyle \frac{(i+1)Q_{\nu -1,i+1}^{-}+i{Q}_{\nu -1,i-1}^{-}}{2i+1}}.$$

(38)

Similarly, by multiplying the left-hand and right-hand sides of relation (1) by $x^{\nu -1}$ and integrating over the interval $[0,1]$, we obtain

$$Q_{\nu ,i}^{+}={\displaystyle \frac{(i+1)Q_{\nu -1,i+1}^{+}+i{Q}_{\nu -1,i-1}^{+}}{2i+1}}.$$

These relations can be formally applied for $i=0$ when $Q_{\nu ,-1}^{-}=Q_{\nu ,-1}^{+}=0$ but not for $\nu =0$. Therefore, we should consider the case $\nu =0$ separately:

$$Q_{0,i}^{-}={\int }_{-1}^0{P}_i\left(x\right)dx,Q_{0,i}^{+}={\int }_0^1{P}_i\left(x\right)dx.$$

For $i=0$ and $i=1$, we have

$$Q_{0,0}^{-}={\int }_{-1}^0{P}_0\left(x\right)dx={\int }_{-1}^{0}dx=1,Q_{0,1}^{-}={\int }_{-1}^0{P}_1\left(x\right)dx={\int }_{-1}^{0}xdx=-{\displaystyle \frac{1}{2}},$$

and then we use relation (17):

$$Q_{0,i}^{-}={\int }_{-1}^0{P}_i\left(x\right)dx={\displaystyle \frac{P_{i+1}\left(0\right)-P_{i-1}\left(0\right)}{2i+1}},i=1,2,\dots$$

If $i\ne 0$ is even, then $P_{i+1}\left(0\right)=P_{i-1}\left(0\right)=0$ according to Formula (5), and $Q_{0,i}^{-}=0$. If i is odd, then we can apply the explicit Formula (4) or obtain an additional recurrence relation. We choose the latter way and take into account relation (1) for $x=0$:

$$(i+1)P_{i+1}\left(0\right)=-i{P}_{i-1}\left(0\right).$$

Then,

$$\begin{array}{cc}\hfill {\displaystyle \frac{Q_{0,i}^{-}}{Q_{0,i-2}^{-}}}& ={\displaystyle \frac{P_{i+1}\left(0\right)-P_{i-1}\left(0\right)}{2i+1}}{\displaystyle \frac{2i-3}{P_{i-1}\left(0\right)-P_{i-3}\left(0\right)}}\hfill \\ \hfill & ={\displaystyle \frac{-\frac{i}{i+1}{P}_{i-1}\left(0\right)-P_{i-1}\left(0\right)}{2i+1}}{\displaystyle \frac{2i-3}{P_{i-1}\left(0\right)+\frac{i-1}{i-2}{P}_{i-1}\left(0\right)}}\hfill \\ \hfill & ={\displaystyle \frac{-\frac{i}{i+1}-1}{2i+1}}{\displaystyle \frac{2i-3}{1+\frac{i-1}{i-2}}},\hfill \end{array}$$

i.e.,

$$Q_{0,i}^{-}={\displaystyle \frac{2-i}{i+1}}{Q}_{0,i-2}^{-},i=3,5,7,\dots$$

The same reasoning leads to the following results:

$$Q_{0,0}^{+}={\int }_0^1{P}_0\left(x\right)dx={\int }_0^{1}dx=1,Q_{0,1}^{+}={\int }_0^1{P}_1\left(x\right)dx={\int }_0^{1}xdx={\displaystyle \frac{1}{2}},$$

and

$$Q_{0,i}^{+}={\displaystyle \frac{P_{i-1}\left(0\right)-P_{i+1}\left(0\right)}{2i+1}},i=1,2,\dots ,$$

where $Q_{0,i}^{+}=0$ for even $i\ne 0$. For odd i, we have

$$Q_{0,i}^{+}={\displaystyle \frac{2-i}{i+1}}{Q}_{0,i-2}^{+},i=3,5,7,\dots$$

Thus, we obtain the general expression for calculating $Q_{\nu ,i}^{-}$ and $Q_{\nu ,i}^{+}$ with arbitrary non-negative integers $\nu$ and i:

$$Q_{\nu ,i}^{\pm }=\left\{\begin{array}{cc}{\displaystyle 1\phantom{\frac{1}{2}}}\hfill & \mathrm{for}\nu =i=0,\hfill \\ {\displaystyle \pm \frac{1}{2}}\hfill & \mathrm{for}\nu =0\mathrm{and}i=1,\hfill \\ {\displaystyle 0\phantom{\frac{1}{2}}}\hfill & \mathrm{for}\nu =0\mathrm{and}i=2,4,6,\dots ,\hfill \\ {\displaystyle \frac{2-i}{i+1}{Q}_{0,i-2}^{\pm }}\hfill & \mathrm{for}\nu =0\mathrm{and}i=3,5,7,\dots ,\hfill \\ {\displaystyle \frac{(i+1)Q_{\nu -1,i+1}^{\pm }+i{Q}_{\nu -1,i-1}^{\pm }}{2i+1}}\hfill & \mathrm{otherwise},\hfill \end{array}\right.$$

so that the expansion coefficients of the density $g_{{\nu }_1,{\nu }_2}$ with respect to Legendre polynomials (6) are expressed as follows (relation (37) is also used here):

$$\begin{array}{cc}\hfill G_{{\nu }_1,{\nu }_2,i}& ={\int }_{-1}^1{g}_{{\nu }_1,{\nu }_2}\left(x\right){\widehat{P}}_i\left(x\right)dx\hfill \\ \hfill & ={\gamma }_{{\nu }_1,{\nu }_2}\sqrt{{\displaystyle \frac{2i+1}{2}}}\left({\int }_{-1}^0\left(1-{(-x)}^{{\nu }_1}\right)P_i\left(x\right)dx+{\int }_0^1(1-x^{{\nu }_2})P_i\left(x\right)dx\right)\hfill \\ \hfill & ={\gamma }_{{\nu }_1,{\nu }_2}\sqrt{{\displaystyle \frac{2i+1}{2}}}\left(Q_{0,i}^{-}-{(-1)}^{{\nu }_1}{Q}_{{\nu }_1,i}^{-}+Q_{0,i}^{+}-Q_{{\nu }_2,i}^{+}\right).\hfill \end{array}$$

Expressions for the expansion coefficients of the distribution function $f_{{\nu }_1,{\nu }_2}$ are similarly obtained:

$$\begin{array}{cc}\hfill F_{{\nu }_1,{\nu }_2,i}& ={\int }_{-1}^1{f}_{{\nu }_1,{\nu }_2}\left(x\right){\widehat{P}}_i\left(x\right)dx\hfill \\ \hfill & ={\gamma }_{{\nu }_1,{\nu }_2}\sqrt{{\displaystyle \frac{2i+1}{2}}}({\int }_{-1}^0\left[{\displaystyle \frac{{\nu }_1}{{\nu }_1+1}}+x+{\displaystyle \frac{{(-x)}^{{\nu }_1+1}}{{\nu }_1+1}}\right]P_i\left(x\right)dx\hfill \\ \hfill & +{\int }_0^1\left[{\displaystyle \frac{{\nu }_1}{{\nu }_1+1}}+x-{\displaystyle \frac{x^{{\nu }_2+1}}{{\nu }_2+1}}\right]P_i\left(x\right)dx)\hfill \\ \hfill & ={\gamma }_{{\nu }_1,{\nu }_2}\sqrt{{\displaystyle \frac{2i+1}{2}}}({\displaystyle \frac{{\nu }_1}{{\nu }_1+1}}{Q}_{0,i}^{-}+Q_{1,i}^{-}+{\displaystyle \frac{{(-1)}^{{\nu }_1+1}}{{\nu }_1+1}}{Q}_{{\nu }_1+1,i}^{-}\hfill \\ \hfill & +{\displaystyle \frac{{\nu }_1}{{\nu }_1+1}}{Q}_{0,i}^{+}+Q_{1,i}^{+}-{\displaystyle \frac{1}{{\nu }_2+1}}{Q}_{{\nu }_2+1,i}^{+}).\hfill \end{array}$$

These expressions for the expansion coefficients of the density $g_{{\nu }_1,{\nu }_2}$ and distribution function $f_{{\nu }_1,{\nu }_2}$ are used for their approximation as

$$g_{{\nu }_1,{\nu }_2}\left(x\right)\approx g_{{\nu }_1,{\nu }_2}^{\langle n\rangle }\left(x\right)=\sum _{i=0}^n{G}_{{\nu }_1,{\nu }_2,i}{\widehat{P}}_i\left(x\right),f_{{\nu }_1,{\nu }_2}\left(x\right)\approx f_{{\nu }_1,{\nu }_2}^{\langle n\rangle }\left(x\right)=\sum _{i=0}^n{F}_{{\nu }_1,{\nu }_2,i}{\widehat{P}}_i\left(x\right),$$

and for the approximation error:

$$\begin{array}{cc}\hfill {\epsilon }_g^{\langle n\rangle }& =\parallel g_{{\nu }_1,{\nu }_2}-g_{{\nu }_1,{\nu }_2}^{\langle n\rangle }{\parallel }_{L_2(\Omega )}={\left({\int }_{-1}^1{g}_{{\nu }_1,{\nu }_2}^2\left(x\right)dx-\sum _{i=0}^n{G}_{{\nu }_1,{\nu }_2,i}^2\right)}^{1/2},\hfill \\ \hfill {\epsilon }_f^{\langle n\rangle }& =\parallel f_{{\nu }_1,{\nu }_2}-f_{{\nu }_1,{\nu }_2}^{\langle n\rangle }{\parallel }_{L_2(\Omega )}={\left({\int }_{-1}^1{f}_{{\nu }_1,{\nu }_2}^2\left(x\right)dx-\sum _{i=0}^n{F}_{{\nu }_1,{\nu }_2,i}^2\right)}^{1/2}.\hfill \end{array}$$

(39)

5.4. Analysis of Convergence Rate for Expansions of Test Functions

Consider the function

$$y_{\nu }^{-}\left(x\right)=\left\{\begin{array}{cc}{x}^{\nu }\hfill & \mathrm{for}x\in [-1,0),\hfill \\ 0\hfill & \mathrm{for}x\in [0,1],\hfill \end{array}\right.$$

where $\nu$ is a parameter (natural number). Its expansion coefficients $Y_{\nu ,i}^{-}$ with respect to Legendre polynomials (6) are expressed through the previously found values $Q_{\nu ,i}^{-}$:

$$y_{\nu }^{-}\left(x\right)=\sum _{i=0}^{\infty }{Y}_{\nu ,i}^{-}{\widehat{P}}_i\left(x\right),Y_{\nu ,i}^{-}=\sqrt{{\displaystyle \frac{2i+1}{2}}}{Q}_{\nu ,i}^{-}.$$

Further, we derive the recurrence relation for $Q_{\nu ,i}^{-}$, different from relation (38). We multiply the left-hand and right-hand sides of relation (1) by $x^{\nu }$ and integrate over the interval $[-1,0]$:

$$(i+1){\int }_{-1}^0{x}^{\nu }{P}_{i+1}\left(x\right)dx=(2i+1){\int }_{-1}^0{x}^{\nu +1}{P}_i\left(x\right)dx-i{\int }_{-1}^0{x}^{\nu }{P}_{i-1}\left(x\right)dx.$$

Next, we use the rule of integration by parts:

$${\int }_{-1}^0{x}^{\nu +1}{P}_i\left(x\right)dx=x^{\nu +1}{\Theta }_i\left(x\right){|}_{-1}^0-(\nu +1){\int }_{-1}^0{x}^{\nu }{\Theta }_i\left(x\right)dx,{\Theta }_i\left(x\right)={\int }_{-1}^x{P}_i\left(\theta \right)d\theta ,$$

and consequently, taking into account equality (17), we obtain

$${\int }_{-1}^0{x}^{\nu +1}{P}_i\left(x\right)dx=-{\displaystyle \frac{\nu +1}{2i+1}}{\int }_{-1}^0{x}^{\nu }\left(P_{i+1}\left(x\right)-P_{i-1}\left(x\right)\right)dx;$$

therefore,

$$(i+1){\int }_{-1}^0{x}^{\nu }{P}_{i+1}\left(x\right)dx=-(\nu +1){\int }_{-1}^0{x}^{\nu }\left(P_{i+1}\left(x\right)-P_{i-1}\left(x\right)\right)dx-i{\int }_{-1}^0{x}^{\nu }{P}_{i-1}\left(x\right)dx,$$

or

$$(i+1)Q_{\nu ,i+1}^{-}=-(\nu +1)(Q_{\nu ,i+1}^{-}-Q_{\nu ,i-1}^{-})-i{Q}_{\nu ,i-1}^{-},$$

i.e.,

$$Q_{\nu ,i+1}^{-}={\displaystyle \frac{\nu -i+1}{\nu +i+2}}{Q}_{\nu ,i-1}^{-}\mathrm{and}{Y}_{\nu ,i+1}^{-}=\sqrt{{\displaystyle \frac{2i+3}{2i-1}}}{\displaystyle \frac{\nu -i+1}{\nu +i+2}}{Y}_{\nu ,i-1}^{-}.$$

The series formed by the squared expansion coefficients $Y_{\nu ,i}^{-}$ converges since $y_{\nu }^{-}\in L_2(\Omega )$. It can be represented as a sum of two series:

$$\sum _{i=0}^{\infty }{\left(Y_{\nu ,i}^{-}\right)}^2=\sum _{j=0}^{\infty }{\left(Y_{\nu ,2j}^{-}\right)}^2+\sum _{j=0}^{\infty }{\left(Y_{\nu ,2j+1}^{-}\right)}^2.$$

The Raabe–Duhamel test [24] implies that the first series (similarly for the second one) converges in the same way as the series

$$\sum _{j=1}^{\infty }{\displaystyle \frac{1}{j^{2\nu +2}}},$$

since the sequence

$$R_j=j\left({\displaystyle \frac{{\left(Y_{\nu ,2j}^{-}\right)}^2}{{\left(Y_{\nu ,2j+2}^{-}\right)}^2}}-1\right)=j\left({\displaystyle \frac{4j-1}{4j+3}}{\left[{\displaystyle \frac{\nu +2j+2}{\nu -2j+1}}\right]}^2-1\right)$$

has the limit $\underset{j\to \infty }{lim}{R}_j=2\nu +2$, but the convergence of this series is equivalent to the convergence of the integral

$${\int }_1^{+\infty }{\displaystyle \frac{dt}{t^{2\nu +2}}}=-{\displaystyle \frac{1}{(2\nu +1)t^{2\nu +1}}}{|}_1^{+\infty }=-{\displaystyle \frac{1}{2\nu +1}}\left(\underset{t\to \infty }{lim}{\displaystyle \frac{1}{t^{2\nu +1}}}-1\right)$$

which takes place under the condition $2\nu +1>0$, or $\nu >-1/2$.

As a result, using Parseval’s identity, we find

$$\parallel y_{\nu }^{-}-y_{\nu }^{-\langle n\rangle }{\parallel }_{L_2(\Omega )}^2=\sum _{i=n+1}^{\infty }{\left(Y_{\nu ,i}^{-}\right)}^2⩽{\displaystyle \frac{C^2}{n^{2\nu +1}}}\mathrm{and}{\parallel y_{\nu }^{-}-y_{\nu }^{-\langle n\rangle }\parallel }_{L_2(\Omega )}⩽{\displaystyle \frac{C}{n^{\nu +1/2}}},C>0,$$

where

$$y_{\nu }^{-\langle n\rangle }\left(x\right)=\sum _{i=0}^n{Y}_{\nu ,i}^{-}{\widehat{P}}_i\left(x\right),$$

and this corresponds to estimate (9) with the limit value $s=\nu +1/2$ (see Remark 1).

The obtained result can be transferred to the function

$$y_{\nu }^{+}\left(x\right)=\left\{\begin{array}{cc}0\hfill & \mathrm{for}x\in [-1,0),\hfill \\ x^{\nu }\hfill & \mathrm{for}x\in [0,1],\hfill \end{array}\right.$$

and its expansion coefficients $Y_{\nu ,i}^{+}$ with respect to Legendre polynomials (6). The easiest way to prove this is the use of the equality $|Y_{\nu ,i}^{-}|=|Y_{\nu ,i}^{+}|$, which follows from the relation between expansion coefficients of an arbitrary function from $L_2(\Omega )$ and its reflection [25]. Therefore, the same result holds for the function $g_{{\nu }_1,{\nu }_2}$. This result can be extended to the function $f_{{\nu }_1,{\nu }_2}$ due to its degree of smoothness is greater by one than the degree of smoothness of $g_{{\nu }_1,{\nu }_2}$.

Thus,

$${\epsilon }_g^{\langle n\rangle }\approx {\displaystyle \frac{C^g}{n^{min\{{\nu }_1,{\nu }_2\}+1/2}}},{\epsilon }_f^{\langle n\rangle }\approx {\displaystyle \frac{C^f}{n^{min\{{\nu }_1,{\nu }_2\}+3/2}}},$$

(40)

where $C^g,C^f>0$ are some constants. Moreover, we can assume that ${\nu }_1$ and ${\nu }_2$ are real positive numbers, and if we consider $g_{{\nu }_1,{\nu }_2}$ not to be a density but some function not bound by the probability theoretical framework, then the condition ${\nu }_1,{\nu }_2>-1/2$ is admissible. Such a convergence reflects that $g_{{\nu }_1,{\nu }_2}^{\left(s\right)}\in W_2^s(\Omega )$ and $f_{{\nu }_1,{\nu }_2}^{\left(s\right)}\in W_2^{s+1}(\Omega )$ subject to $s<min\{{\nu }_1,{\nu }_2\}+1/2$ and $sups=min\{{\nu }_1,{\nu }_2\}+1/2$. It corresponds to estimate (9).

6. Numerical Experiments

In this section, we present the results of the joint estimation of the density and distribution function for two examples that use a two-parameter family (30) of densities with different degrees of smoothness. In these examples, the results are presented in the tables that contain the errors of the projection estimates of the density and distribution function in $L_2(\Omega )$. We study the dependence of error on the projection expansion length (for the maximum degree n of the Legendre polynomials, values 4, 8, 16, 32, and 64 are used, i.e., $n=2^{k+2}$ for $k=0,1,\dots ,4$), on the sample size $N=2^{m+9}$ for $m=0,1,\dots ,18$, and on the degree of smoothness of the approximated density (see Examples 1 and 2 below). Algorithm 2 is applied for estimation.

These examples show the approximation errors ${\epsilon }_g^{\langle n\rangle }$ and ${\epsilon }_f^{\langle n\rangle }$, which are calculated using Formula (39), i.e., deterministic components of projection estimate errors. In the tables, they are in rows marked by the symbol “∗”. The remaining rows contain errors that include deterministic and stochastic components. The formulae for these errors follow from Parseval’s identity:

$$\begin{array}{cc}\hfill {\overline{\epsilon }}_g^{\langle n\rangle }& ={\left({\int }_{-1}^1{g}_{{\nu }_1,{\nu }_2}^2\left(x\right)dx-\sum _{i=0}^n{G}_{{\nu }_1,{\nu }_2,i}^2+\sum _{i=0}^n{(G_{{\nu }_1,{\nu }_2,i}-{\overline{G}}_{{\nu }_1,{\nu }_2,i})}^2\right)}^{1/2},\hfill \\ \hfill {\overline{\epsilon }}_f^{\langle n\rangle }& ={\left({\int }_{-1}^1{f}_{{\nu }_1,{\nu }_2}^2\left(x\right)dx-\sum _{i=0}^n{F}_{{\nu }_1,{\nu }_2,i}^2+\sum _{i=0}^n{(F_{{\nu }_1,{\nu }_2,i}-{\overline{F}}_{{\nu }_1,{\nu }_2,i})}^2\right)}^{1/2},\hfill \end{array}$$

where ${\overline{G}}_{{\nu }_1,{\nu }_2,i}$ and ${\overline{F}}_{{\nu }_1,{\nu }_2,i}$ are estimates of expansion coefficients $G_{{\nu }_1,{\nu }_2,i}$ and $F_{{\nu }_1,{\nu }_2,i}$, respectively. For an arbitrary density g, a similar formula was used to obtain estimate (27).

Example 1.

Let $g_{{\nu }_1,{\nu }_2}$ be the density from a two-parameter family (30), and let $f_{{\nu }_1,{\nu }_2}$ be the related distribution function described by Formula (32), with the following parameters: ${\nu }_1=\nu$, ${\nu }_2=\nu +1$ for $\nu =1$. The modeling formula is given in Proposition 1.

The function $g_{1,2}$ is continuous, non-differentiable at $x=0$, but its first-order derivative exists almost everywhere on Ω: $g_{1,2}\in C(\Omega )$, $g_{1,2}^{\left(1\right)}\in L_2(\Omega )$, i.e., $g_{1,2}\in W_2^1(\Omega )$. However, if we do not restrict ourselves to the space $W_2^s(\Omega )$ with natural s (see Remark 1), then $g_{1,2}\in W_2^s(\Omega )$ for $s<min\{{\nu }_1,{\nu }_2\}+1/2=3/2$ ($sups=3/2$). The degree of smoothness of $f_{1,2}$ is greater, and $f_{1,2}\in W_2^{s+1}(\Omega )$. This corresponds to Formulae (40), which take the form

$${\epsilon }_g^{\langle n\rangle }\approx {\displaystyle \frac{C^g}{n^{3/2}}},{\epsilon }_f^{\langle n\rangle }\approx {\displaystyle \frac{C^f}{n^{5/2}}}$$

for given parameters ${\nu }_1$ and ${\nu }_2$.

According to Theorem 1, for the limit value $s=3/2$, to achieve the required approximation accuracy $B(g_{1,2},{\overline{g}}_{1,2}^{\langle n\rangle })=\gamma ⩽c{N}^{-3/8}$, $c>0$, the conditionally optimal parameters should be as follows: $N_{\mathrm{opt}}\asymp n_{\mathrm{opt}}^4$; this is confirmed by the statistical modeling results (see

Table 1. The approximation errors ${\epsilon }_g^{\langle n\rangle }$ (in the row “∗”) and ${\overline{\epsilon }}_g^{\langle n\rangle }$ (in the remaining rows) for sample sizes $N=2^{m+9}$ (Example 1).

m	$\mathit{n}=4$	$\mathit{n}=8$	$\mathit{n}=16$	$\mathit{n}=32$	$\mathit{n}=64$
∗	0.024614	0.009800	0.003838	0.001442	0.000527
0	0.065300	0.068625	0.119784	0.168044	0.209996
1	0.029999	0.020867	0.064303	0.107275	0.144097
2	0.026360	0.031022	0.046008	0.061732	0.093907
3	0.035117	0.028862	0.036685	0.045302	0.064636
4	0.028603	0.020236	0.026896	0.035032	0.050834
5	0.025879	0.015305	0.018405	0.025655	0.034847
6	0.025867	0.014401	0.014068	0.017283	0.023956
7	0.026084	0.015807	0.014762	0.017245	0.021088
8	0.024814	0.011465	0.008550	0.010850	0.015997
9	0.024640	0.010035	0.005534	0.006907	0.010124
10	0.024638	0.009976	0.004925	0.005121	0.007505
11	0.024622	0.009888	0.004352	0.003211	0.005116
12	0.024616	0.009815	0.004112	0.002461	0.003081
13	0.024616	0.009814	0.003937	0.002064	0.002367
14	0.024616	0.009807	0.003881	0.001813	0.001689
15	0.024615	0.009802	0.003852	0.001623	0.001308
16	0.024615	0.009803	0.003851	0.001553	0.001093
17	0.024615	0.009802	0.003852	0.001503	0.000830
18	0.024615	0.009801	0.003845	0.001470	0.000647

and

Table 2. The approximation errors ${\epsilon }_f^{\langle n\rangle }$ (in the row “∗”) and ${\overline{\epsilon }}_f^{\langle n\rangle }$ (in the remaining rows) for sample sizes $N=2^{m+9}$ (Example 1).

m	$\mathit{n}=4$	$\mathit{n}=8$	$\mathit{n}=16$	$\mathit{n}=32$	$\mathit{n}=64$
∗	0.005772	0.001069	0.000198	0.000036	$6.48·{10}^{-6}$
0	0.027351	0.026935	0.027769	0.028143	0.028244
1	0.010817	0.009287	0.010189	0.010621	0.010828
2	0.006060	0.004198	0.004744	0.005001	0.005172
3	0.008635	0.007038	0.007179	0.007255	0.007313
4	0.006667	0.004018	0.004166	0.004234	0.004284
5	0.006149	0.002746	0.002680	0.002754	0.002788
6	0.006159	0.002666	0.002519	0.002544	0.002561
7	0.006186	0.002722	0.002571	0.002594	0.002604
8	0.005845	0.001519	0.001160	0.001182	0.001205
9	0.005784	0.001152	0.000535	0.000539	0.000558
10	0.005801	0.001230	0.000661	0.000647	0.000656
11	0.005783	0.001140	0.000449	0.000411	0.000420
12	0.005773	0.001081	0.000263	0.000183	0.000186
13	0.005775	0.001087	0.000282	0.000210	0.000209
14	0.005773	0.001080	0.000250	0.000161	0.000158
15	0.005772	0.001070	0.000202	0.000060	0.000051
16	0.005772	0.001072	0.000212	0.000086	0.000079
17	0.005772	0.001071	0.000208	0.000075	0.000067
18	0.005772	0.001070	0.000200	0.000046	0.000030

). In the row “∗” of Table 1, the deterministic component of error decreases by approximately $2^{3/2}\approx 2.8$ times (in Table 2, by approximately $2^{5/2}\approx 5.6$ times) when the projection expansion length n is doubled. In the rest of this table, errors corresponding to optimal parameters $n_{\mathrm{opt}}$ and $N_{\mathrm{opt}}$ are highlighted in bold, and they are consistent with the relationship $N_{\mathrm{opt}}\asymp n_{\mathrm{opt}}^4$. Table 2 demonstrates higher accuracy of distribution function estimation.

In our calculations with the formulae for errors, the following values are used (squared norms of the functions $g_{1,2}$ and $f_{1,2}$):

$${\int }_{-1}^1{g}_{1,2}^2\left(x\right)dx={\displaystyle \frac{156}{245}},{\int }_{-1}^1{f}_{1,2}^2\left(x\right)dx={\displaystyle \frac{235}{343}}.$$

For clarity, Figure 1 contains the approximation errors in graphical form. One axis shows $k=0,1,\dots ,4$, which determines the projection expansion length, $n=2^{k+2}$, and another axis shows $m=0,1,\dots ,18$, which determines the sample size, $N=2^{m+9}$. The vertical axis corresponds to the density approximation errors.

The example under consideration corresponds to the left part of Figure 1, which presents two surfaces. The first one (red) corresponds to the obtained computational error, it is formed on data from Table 1. The second one (blue, with marked nodes) corresponds to the theoretical error according to Formula (28) with $s=3/2$:

$${ϵ}_g^{\langle n\rangle }\approx \sqrt{{\displaystyle \frac{C_1^{g}n}{N}}+{\displaystyle \frac{C_2^g}{n^3}}}.$$

Constants $C_1^g=0.885$ and $C_2^g=0.276$ are approximately determined based on the condition of a minimum sum of squared deviations between the theoretical error and computational error.

Example 2.

Let $g_{{\nu }_1,{\nu }_2}$ be the density from a two-parameter family (30), and let $f_{{\nu }_1,{\nu }_2}$ be the related distribution function described by Formula (32), with the following parameters: ${\nu }_1=\nu +1$, ${\nu }_2=\nu$ for $\nu =2$. The modeling formula is given in Proposition 2.

The function $g_{3,2}$ is continuous, differentiable everywhere on Ω, and its second-order derivative exists almost everywhere on Ω: $g_{3,2}\in C^1(\Omega )$, $g_{3,2}^{\left(2\right)}\in L_2(\Omega )$; hence, $g_{3,2}\in W_2^2(\Omega )$. Considering the space $W_2^s(\Omega )$ with real non-negative s (see Remark 1), we affirm that $g_{3,2}\in W_2^s(\Omega )$ for $s<min\{{\nu }_1,{\nu }_2\}+1/2=5/2$ ($sups=5/2$). The degree of smoothness of $f_{3,2}$ is greater, and $f_{3,2}\in W_2^{s+1}(\Omega )$. This corresponds to Formula (40) when substituting given parameters ${\nu }_1$ and ${\nu }_2$:

$${\epsilon }_g^{\langle n\rangle }\approx {\displaystyle \frac{C^g}{n^{5/2}}},{\epsilon }_f^{\langle n\rangle }\approx {\displaystyle \frac{C^f}{n^{7/2}}}.$$

Theorem 1 implies that in order to achieve the required approximation accuracy $B(g_{3,2},{\overline{g}}_{3,2}^{\langle n\rangle })=\gamma ⩽c{N}^{-5/12}$, $c>0$, with the limit value $s=5/2$, the conditionally optimal parameters should satisfy the relationship $N_{\mathrm{opt}}\asymp n_{\mathrm{opt}}^6$, and this is illustrated by the statistical modeling results from

Table 3. The approximation errors ${\epsilon }_g^{\langle n\rangle }$ (in the row “∗”) and ${\overline{\epsilon }}_g^{\langle n\rangle }$ (in the remaining rows) for sample sizes $N=2^{m+9}$ (Example 2).

m	$\mathit{n}=4$	$\mathit{n}=8$	$\mathit{n}=16$	$\mathit{n}=32$	$\mathit{n}=64$
∗	0.009757	0.001782	0.000327	0.000059	0.000011
0	0.095213	0.096279	0.110003	0.163112	0.202505
1	0.069075	0.072595	0.077403	0.101639	0.156190
2	0.037189	0.042272	0.055580	0.075076	0.109453
3	0.019912	0.020148	0.028046	0.044567	0.077312
4	0.015445	0.015796	0.018872	0.035310	0.052812
5	0.010799	0.007478	0.014730	0.021591	0.037072
6	0.011292	0.007615	0.010409	0.013876	0.026574
7	0.012527	0.009047	0.011426	0.013697	0.021402
8	0.010221	0.004889	0.006611	0.008514	0.014083
9	0.009919	0.002619	0.003359	0.004879	0.008398
10	0.009851	0.002347	0.003053	0.004197	0.005689
11	0.009779	0.002255	0.001818	0.002358	0.003913
12	0.009783	0.002329	0.001788	0.001999	0.003051
13	0.009766	0.001968	0.001211	0.001580	0.002321
14	0.009769	0.001911	0.000960	0.001176	0.001615
15	0.009761	0.001839	0.000798	0.000778	0.001129
16	0.009757	0.001793	0.000472	0.000455	0.000766
17	0.009757	0.001789	0.000409	0.000425	0.000553
18	0.009758	0.001789	0.000382	0.000298	0.000388

and

Table 4. The approximation errors ${\epsilon }_f^{\langle n\rangle }$ (in the row “∗”) and ${\overline{\epsilon }}_f^{\langle n\rangle }$ (in the remaining rows) for sample sizes $N=2^{m+9}$ (Example 2).

m	$\mathit{n}=4$	$\mathit{n}=8$	$\mathit{n}=16$	$\mathit{n}=32$	$\mathit{n}=64$
∗	0.002779	0.000139	0.000014	$1.40·{10}^{-6}$	$1.35·{10}^{-7}$
0	0.050042	0.050077	0.050178	0.050338	0.050393
1	0.032134	0.032449	0.032474	0.032568	0.032637
2	0.020694	0.020649	0.020779	0.020854	0.020905
3	0.010729	0.010445	0.010513	0.010577	0.010651
4	0.007291	0.006854	0.006890	0.006980	0.007017
5	0.003914	0.002828	0.002946	0.003002	0.003052
6	0.003984	0.002933	0.002982	0.002999	0.003031
7	0.004923	0.004130	0.004160	0.004169	0.004182
8	0.003304	0.001842	0.001872	0.001879	0.001893
9	0.002904	0.000866	0.000874	0.000883	0.000893
10	0.002803	0.000414	0.000439	0.000454	0.000460
11	0.002790	0.000322	0.000301	0.000306	0.000311
12	0.002792	0.000341	0.000320	0.000322	0.000325
13	0.002782	0.000208	0.000167	0.000171	0.000174
14	0.002783	0.000215	0.000169	0.000170	0.000171
15	0.002780	0.000171	0.000108	0.000107	0.000108
16	0.002779	0.000147	0.000053	0.000052	0.000053
17	0.002779	0.000144	0.000043	0.000043	0.000043
18	0.002780	0.000158	0.000076	0.000075	0.000075

. In the row “∗” of Table 3, if the projection expansion length n is doubled, then the deterministic component of error decreases by approximately $2^{5/2}\approx 5.6$ times (in Table 4, by approximately $2^{7/2}\approx 11.2$ times). In the rest of this table, errors corresponding to optimal parameters $n_{\mathrm{opt}}$ and $N_{\mathrm{opt}}$ are shown in bold, and they are consistent with the relationship $N_{\mathrm{opt}}\asymp n_{\mathrm{opt}}^6$. Table 4 shows higher accuracy of distribution function estimation.

In our calculations with the formulae for errors, the following values are used (squared norms of the functions $g_{3,2}$ and $f_{3,2}$):

$${\int }_{-1}^1{g}_{3,2}^2\left(x\right)dx={\displaystyle \frac{5928}{10115}},{\int }_{-1}^1{f}_{3,2}^2\left(x\right)dx={\displaystyle \frac{7801}{10115}}.$$

Figure 1 contains a graphical representation of the numerical experiment. The meaning of different axes is described in Example 1. This example corresponds to the right part of Figure 1 with two surfaces. The first one (red) is constructed from the obtained computational error given in Table 3, and the second one (blue, with marked nodes) corresponds to the theoretical error according to Formula (28) with $s=5/2$:

$${ϵ}_g^{\langle n\rangle }\approx \sqrt{{\displaystyle \frac{C_1^{g}n}{N}}+{\displaystyle \frac{C_2^g}{n^5}}},$$

where constants $C_1^g=0.890$ and $C_2^g=0.545$ are chosen from the same condition as in Example 1.

7. Comparison of Projection Density Estimate and Histogram

The classical method of estimating the density of a random variable is associated with a histogram [18], which is very often used in applied problems. We consider a histogram a projection estimate since the main results of this paper are related to projection estimates.

We can define block pulse functions on the set $\Omega$ as

$$\begin{array}{c}{\widehat{\Pi }}_i\left(x\right)={\displaystyle \frac{1}{\sqrt{h}}}\left\{\begin{array}{cc}1\hfill & \mathrm{for}x\in [ih-1,(i+1)h-1),\hfill \\ 0\hfill & \mathrm{for}x\notin [ih-1,(i+1)h-1),\hfill \end{array}\right.\\ i=0,1,\dots ,L-1,\end{array}$$

(41)

where L, a natural number, is the number of block pulse functions and $h=2/L$. It is advisable to redefine the function ${\widehat{\Pi }}_{L-1}$ in such a way that it becomes continuous on the left at $x=1$.

Block pulse functions (41) form an orthonormal system of functions in $L_2(\Omega )$. This system is not complete, but it can be used to approximate an arbitrary function $u\in L_2(\Omega )$:

$$u\left(x\right)\approx u^{\langle L\rangle \ast }\left(x\right)=\sum _{i=0}^{L-1}{U}_i^{\ast }{\widehat{\Pi }}_i\left(x\right),$$

where

$$U_i^{\ast }={(u,{\widehat{\Pi }}_i)}_{L_2(\Omega )}={\int }_{\Omega }u\left(x\right){\widehat{\Pi }}_i\left(x\right)dx,i=0,1,\dots ,L-1.$$

For $L=2^m$ with natural m, the first L Walsh or Haar functions on $\Omega$ are exactly expressed through block pulse functions (41). Therefore, the results of this section can easily be adapted to the projection estimates of the distribution of a random variable using Walsh or Haar functions that form complete orthonormal systems of functions [26].

The approximation accuracy in $L_2(\Omega )$ is usually estimated as follows [27]:

$$\parallel u-u^{\langle L\rangle \ast }{\parallel }_{L_2(\Omega )}⩽{\displaystyle \frac{C^{\ast }}{L}},$$

(42)

where $C^{\ast }>0$ does not depend on L, under the condition $u\in W_2^1(\Omega )$.

As a function u in the given formulae, we can use the density g and the distribution function f. We restrict ourselves to the density g only (the corresponding expansion coefficients are further denoted by $G_i^{\ast }$):

$$g^{\langle L\rangle \ast }\left(x\right)=\sum _{i=0}^{L-1}{G}_i^{\ast }{\widehat{\Pi }}_i\left(x\right),$$

(43)

where

$$\begin{array}{cc}\hfill G_i^{\ast }& ={\int }_{\Omega }g\left(x\right){\widehat{\Pi }}_i\left(x\right)dx={\displaystyle \frac{1}{\sqrt{h}}}{\int }_{ih-1}^{(i+1)h-1}g\left(x\right)dx\hfill \\ \hfill & ={\displaystyle \frac{f\left(\right(i+1)h-1)-f(ih-1)}{\sqrt{h}}},i=0,1,\dots ,L-1.\hfill \end{array}$$

Thus, the calculation of expansion coefficients $G_{{\nu }_1,{\nu }_2,i}^{\ast }$ of the density $g_{{\nu }_1,{\nu }_2}$ from a two-parameter family (30) is reduced to the calculation of values of the corresponding distribution function $f_{{\nu }_1,{\nu }_2}$ described by Formula (32), i.e.,

$$G_{{\nu }_1,{\nu }_2,i}^{\ast }={\displaystyle \frac{f_{{\nu }_1,{\nu }_2}((i+1)h-1)-f_{{\nu }_1,{\nu }_2}(ih-1)}{\sqrt{h}}},i=0,1,\dots ,L-1.$$

To calculate the approximation error for the density $g_{{\nu }_1,{\nu }_2}$, we can use the formula similar to the first equation of Formula (39):

$${\epsilon }_g^{\langle L\rangle \ast }={\parallel g_{{\nu }_1,{\nu }_2}-g_{{\nu }_1,{\nu }_2}^{\langle L\rangle \ast }\parallel }_{L_2(\Omega )}={\left({\int }_{-1}^1{g}_{{\nu }_1,{\nu }_2}^2\left(x\right)dx-\sum _{i=0}^{L-1}{\left(G_{{\nu }_1,{\nu }_2,i}^{\ast }\right)}^2\right)}^{1/2}.$$

(44)

The histogram can be defined by the expression based on approximation (43):

$${\overline{g}}^{\langle L\rangle \ast }\left(x\right)=\sum _{i=0}^{L-1}{\overline{G}}_i^{\ast }{\widehat{\Pi }}_i\left(x\right),$$

where ${\overline{G}}_i^{\ast }$ are estimates of expansion coefficients $G_i$ based on observations of the random variable $\xi$, $i=0,1,\dots ,L-1$. For example,

$${\overline{G}}_i^{\ast }={\displaystyle \frac{1}{N}}\sum _{l=1}^N{\widehat{\Pi }}_i\left({\xi }_l\right),i=0,1,\dots ,L-1,$$

where ${\xi }_l$ is the lth realization and N is the sample size (number of realizations). The value h depending on L specifies the histogram step.

The error of the histogram ${\overline{g}}^{\langle n\rangle \ast }$ relative to the density g includes deterministic and stochastic components, and it is estimated from below by the value ${\epsilon }_g^{\langle L\rangle \ast }$:

$${\overline{\epsilon }}_g^{\langle L\rangle \ast }={\parallel g_{{\nu }_1,{\nu }_2}-{\overline{g}}_{{\nu }_1,{\nu }_2}^{\langle L\rangle \ast }\parallel }_{L_2(\Omega )}⩾{\epsilon }_g^{\langle L\rangle \ast }.$$

The results of calculations using Formula (44) for both densities from Examples 1 and 2 are given in

Table 5. The approximation error ${\epsilon }_g^{\langle L\rangle \ast }$ (Example 1).

$\mathit{n}=4$	$\mathit{n}=8$	$\mathit{n}=16$	$\mathit{n}=32$	$\mathit{n}=64$
0.186263	0.094153	0.047203	0.023618	0.011811

and

Table 6. The approximation error ${\epsilon }_g^{\langle L\rangle \ast }$ (Example 2).

$\mathit{n}=4$	$\mathit{n}=8$	$\mathit{n}=16$	$\mathit{n}=32$	$\mathit{n}=64$
0.171305	0.089038	0.044945	0.022526	0.011270

, respectively. They should be compared with the rows “∗” in Table 1 and Table 3. Such a comparison shows the undoubted advantage of projection density estimates using Legendre polynomials. The approximation accuracy when using block pulse functions for $L=64$ corresponds to the approximation accuracy when using Legendre polynomials for $n=8$ in Example 1 and for $n=4$ in Example 2. If the number of block pulse functions L is doubled, then the deterministic component of error decreases by approximately 2 times, and this conclusion does not depend on the degree of smoothness of the estimated density.

The problem of the conditional optimization of the algorithm for obtaining the histogram was solved in [1]. In this problem, optimal parameters satisfy the relationship $N_{\mathrm{opt}}\asymp L_{\mathrm{opt}}^3$ which corresponds to inequality (42). A generalization of this result in the context of stochastic differential equations can be found in [3]. Choosing parameters in this way, the computational error ${\overline{\epsilon }}_g^{\langle L\rangle \ast }$ will be approximately twice as large as its deterministic component ${\epsilon }_g^{\langle L\rangle \ast }$, since the conditional optimization of the algorithm for obtaining the histogram assumes the equalization of deterministic and stochastic components.

For densities from Examples 1 and 2, to reduce the approximation error using the histogram by 2 times, it is necessary to increase the number of block pulse functions L by 2 times and the sample size N by $2^3=8$ times. The projection estimates of densities using Legendre polynomials are more effective in these examples. To reduce the approximation error by 2 times in Example 1, it suffices to increase the projection expansion length n by $2^{2/3}\approx 1.587$ times and the sample size N by ${\left(2^{2/3}\right)}^4=2^{8/3}\approx 6.350$ times. In Example 2, it suffices to increase the projection expansion length n by $2^{2/5}\approx 1.320$ times and the sample size N by ${\left(2^{2/5}\right)}^6=2^{12/5}\approx 5.278$ times. In this case, increasing n and N implies their subsequent rounding-up.

8. Conclusions

A distinctive feature of the algorithms proposed in this paper is the jointly obtaining of the projection estimates of the density and distribution function of a random variable. In addition, the problem of the conditional optimization of the algorithms is solved under the assumption that the estimated functions belong to the Sobolev–Slobodetskij space. This assumption allows one to use a more precise estimate of the convergence rate for projection expansions compared with the case where the ordinary Sobolev space or the space of continuously differentiable functions are used. Numerical experiments confirm the obtained theoretical results.

The proposed algorithms and conditionally optimal relationships between parameters ensure the minimum complexity when obtaining projection estimates to achieve the required approximation accuracy. These algorithms are important for studying the properties of solutions to stochastic differential equations [28,29], including stochastic differential equations with a Poisson component [6,30] or with the switching right-hand side [31,32].