# Semi-Parametric Estimation Using Bernstein Polynomial and a Finite Gaussian Mixture Model

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Background

#### The Gaussian Mixture Model and Em Algorithm

**(i)**- E-step: The conditional expectation of the complete-data log-likelyhood given the observed data, using the current fit ${\theta}^{\left(l\right)}$, is defined by$$\begin{array}{c}\hfill \phi \left(\right)open="("\; close=")">\theta |{\theta}^{\left(l\right)}\\ =& {\mathbb{E}}_{{\theta}^{\left(l\right)}}\left(\right)open="("\; close=")">L({X}_{1},\dots ,{X}_{n},{Z}_{1},\dots ,{Z}_{n},\theta )|{X}_{1},\dots ,{X}_{n}.\hfill \end{array}$$The posterior probability that ${X}_{i}$ belongs to the $j$ component of the mixture at the $l$ iteration, is expressed as$$\begin{array}{c}\hfill {\tau}_{ij}^{\left(l\right)}={\mathbb{E}}_{{\theta}^{\left(l\right)}}\left(\right)open="("\; close=")">{Z}_{ij}|{X}_{1},\dots ,{X}_{n}=\frac{{\pi}_{j}^{\left(l\right)}\mathcal{N}\left(\right)open="("\; close=")">{\mu}_{j}^{\left(l\right)},{\left({\sigma}^{2}\right)}_{j}^{\left(l\right)}}{\left({X}_{i}\right)}{\sum}_{h=1}^{K}{\pi}_{h}^{\left(l\right)}\mathcal{N}{\left(\right)}^{{\mu}_{h}^{\left(l\right)}}\left(l\right)\left({X}_{i}\right)\\ .\end{array}$$Finally, we obtain$$\begin{array}{c}\hfill \phi \left(\right)open="("\; close=")">\theta |{\theta}^{\left(l\right)}\\ =& \sum _{i=1}^{n}\sum _{j=1}^{K}{\tau}_{ij}^{\left(l\right)}\left(\right)open="["\; close="]">log\left({\pi}_{j}\right)+log\left(\right)open="("\; close=")">\mathcal{N}({\mu}_{j},{\sigma}_{j})\left({X}_{i}\right)\hfill & .\end{array}$$
**(ii)**- M-step: It consists of a global maximization of $\phi \left(\right)open="("\; close=")">\theta |{\theta}^{\left(l\right)}$ with respect to $\theta $.$$\begin{array}{ccc}\hfill {\theta}^{(l+1)}& =& arg\underset{\theta}{max}\phi \left(\right)open="("\; close=")">\theta |{\theta}^{\left(l\right)}.\hfill \end{array}$$The updated estimates are stated by$$\begin{array}{c}\hfill {\pi}_{j}^{(l+1)}=\frac{1}{n}\sum _{i=1}^{n}{\tau}_{ij}^{\left(l\right)},\end{array}$$$$\begin{array}{ccc}\hfill {\mu}_{j}^{(l+1)}& =& \frac{{\sum}_{i=1}^{n}{\tau}_{ij}^{\left(l\right)}{X}_{i}}{{\sum}_{i=1}^{n}{\tau}_{ij}^{\left(l\right)}},\hfill \end{array}$$$$\begin{array}{ccc}\hfill {\left({\sigma}_{j}^{2}\right)}^{(l+1)}& =& \frac{{\sum}_{i=1}^{n}{\tau}_{ij}^{\left(l\right)}{\left(\right)}^{{X}_{i}}2}{}{\sum}_{i=1}^{n}{\tau}_{ij}^{\left(l\right)}.\hfill \end{array}$$

## 3. Proposed Approach

- Step 1
- We consider the Bernstein estimator of the density function f, which is defined as$$\begin{array}{ccc}\hfill {\tilde{f}}_{1,n,m}\left(x\right)& =& m\sum _{i=0}^{m-1}\left(\right)open="["\; close="]">{F}_{n}\left(\right)open="("\; close=")">\frac{i+1}{m}-{F}_{n}\left(\right)open="("\; close=")">\frac{i}{m}\hfill \\ {b}_{i}(m-1,x)\end{array}$$
- Step 2
- In view of (13), we consider the Gaussian mixture density as an estimator of the density function f, given by$$\begin{array}{c}\hfill {\tilde{f}}_{2,n}\left(x\right)=\sum _{k=1}^{K}{\widehat{\pi}}_{k}\mathcal{N}({\widehat{\mu}}_{k},{\widehat{\sigma}}_{k})\left(x\right),\end{array}$$
- Step 3
- We consider the shrinkage density estimator ${\widehat{f}}_{n,m}$ form defined by$${\widehat{f}}_{n,m}\left(x\right)=\lambda {\tilde{f}}_{1,n,m}\left(x\right)+(1-\lambda ){\tilde{f}}_{2,n}\left(x\right),$$

**1.**- E-step: The conditional expectation of the complete-data log-likelihood given the observed data, using the current ${\lambda}^{\left(t\right)}$, is provided by$$\begin{array}{ccc}\hfill Q\left(\lambda \right|{\lambda}^{\left(t\right)})& =& \sum _{i=1}^{n}{\mathbb{E}}_{{\lambda}^{\left(t\right)}}\left(\right)open="("\; close=")">{W}_{i1}\mid {X}_{i}log{\tilde{f}}_{1,n,m}\left({X}_{i}\right)+{\mathbb{E}}_{{\lambda}^{\left(t\right)}}\left(\right)open="("\; close=")">{W}_{i2}\mid {X}_{i}\hfill & log{\tilde{f}}_{2,n}\left({X}_{i}\right),\end{array}$$$$\begin{array}{ccc}\hfill {\overline{\tau}}_{i1}^{\left(t\right)}& =& \frac{{\tilde{f}}_{1,n,m}\left({X}_{i}\right){\lambda}^{\left(t\right)}}{{\lambda}^{\left(t\right)}{\tilde{f}}_{1,n,m}\left({X}_{i}\right)+(1-{\lambda}^{\left(t\right)}){\tilde{f}}_{2,n}\left({X}_{i}\right)},\hfill \end{array}$$$$\begin{array}{ccc}\hfill {\overline{\tau}}_{i2}^{\left(t\right)}& =& \frac{{\tilde{f}}_{2,n}\left({X}_{i}\right){\lambda}^{\left(t\right)}}{{\lambda}^{\left(t\right)}{\tilde{f}}_{1,n,m}\left({X}_{i}\right)+(1-{\lambda}^{\left(t\right)}){\tilde{f}}_{2,n}\left({X}_{i}\right)}=1-{\overline{\tau}}_{i1}^{\left(t\right)}.\hfill \end{array}$$
**2.**- M-step: It consists of a global maximization of $Q\left(\lambda \right|{\lambda}^{\left(t\right)})$ with respect to $\lambda $.$${\lambda}^{(t+1)}=arg\underset{\lambda}{max}Q(\lambda \mid {\lambda}^{\left(t\right)}).$$The updated estimate of $\lambda $ is indicated by$${\lambda}^{(t+1)}=\frac{1}{n}\sum _{i=1}^{n}{\overline{\tau}}_{i1}^{\left(t\right)}.$$

## 4. Convergence

**Proposition**

**1**

**.**If $m=o\left(\right)open="("\; close=")">n/log\left(n\right)$, then, for $x\in [0,1]$, we have

**Lemma**

**1.**

**Proof of Proposition**

**1.**

**(A1)**- For almost $x\in [0,1]$ and for all $i,j,h=1\dots ,\nu $, the partial derivatives $\partial g/\partial {\xi}_{i}$, ${\partial}^{2}g/\partial {\xi}_{i}\partial {\xi}_{j}$ and ${\partial}^{3}g/\partial {\xi}_{i}\partial {\xi}_{j}\partial {\xi}_{h}$ of the density g exist and satisfy that $\left(\right)open="|"\; close="|">\frac{\partial g\left(x\right|\theta )}{\partial {\xi}_{i}},$$\left(\right)open="|"\; close="|">\frac{{\partial}^{2}g\left(x\right|\theta )}{\partial {\xi}_{i}{\xi}_{j}}$ and $\left(\right)open="|"\; close="|">\frac{{\partial}^{3}g\left(x\right|\theta )}{\partial {\xi}_{i}{\xi}_{j}{\xi}_{h}}$ are bounded, respectively, by ${J}_{i}$, ${J}_{ij}$ and ${J}_{ijh}$, where ${J}_{i}$ and ${J}_{ij}$ are integrable and ${J}_{ijh}$, satisfies$${\int}_{0}^{1}{J}_{ijh}\left(x\right)g\left(x\right|\widehat{\theta})dx<\infty .$$
**(A2)**- The Fisher information matrix $I\left(\theta \right)$ is positively defined at $\widehat{\theta}$.

**Proposition**

**2**

**.**Under the regularity conditions

**(A1)**–

**(A2)**, if $f\left(x\right)>0$ for all $x\in [0,1]$, $2\le m\le (n/logn)$ and $\underset{n,m\to \infty}{lim}{n}^{2/3}/m=0$, then, we obtain

**Proof of Proposition**

**2.**

**Corollary**

**1.**

## 5. Numerical Studies

#### 5.1. Comparison Study

**1.**- Let us suppose that X is concentrated on a finite support $[a,b]$; then, we work with the sample values ${Y}_{1},\dots ,{Y}_{n}$, where ${Y}_{i}=({X}_{i}-a)/(b-a)$.
**2.**- For the density functions concentrated on $\mathbb{R}$, we can use the transformed sample ${Y}_{i}=1/2+{\pi}^{-1}arctan\left({X}_{i}\right)$, which transforms the range to the interval $(0,1)$.
**3.**- For the support ${\mathbb{R}}_{+}$, we can use the transformed sample ${Y}_{i}={X}_{i}/(1+{X}_{i})$, which transforms the range to the interval $(0,1)$.

**(a)**- The beta mixture density $0.5\mathcal{B}(3,9)+0.5\mathcal{B}(9,3)$;
**(b)**- The beta mixture density $0.5\mathcal{B}(3,1)+0.5\mathcal{B}(10,10)$;
**(c)**- The normal mixture density $1/4\mathcal{N}(2,1)+3/4\mathcal{N}(-3,1)$;
**(d)**- The chi-squared ${\chi}_{n}\left(2\right)$ density.
**(e)**- The gamma mixture density $0.5\mathcal{G}(1,6)+0.5\mathcal{G}(6,1)$;
**(f)**- The gamma mixture density $0.5\mathcal{G}(1,2)+0.5\mathcal{G}(4,2)$.

**-**- We first generated a random sample ${\left({X}_{i}\right)}_{1\le i\le n}$ of size n from the models’ density $\left(a\right)-\left(f\right)$.
**-**- We then split the generated data into a training set of a size of $2/3$ of the considered sample and a test set of a size of $1/3$ of the considered sample.
**-**- We applied the proposed estimator, using the observed data ${X}_{i}$ only from the training set, in order to estimate the density function.
**-**- The test set was then used to compute the estimation errors $\widehat{ISE}$, $\widehat{IAE}$ and $\widehat{KL}$.

**-****-**- Using the proposed estimator, we obtained better results than those given by the other estimators in a large part of the cases.
**-**- The Figure 2 and 3 give a better sense of where the error is located.
**-**- For the case (e) of the gamma mixture, the average $\widehat{ISE}$ and $\widehat{IAE}$ of Guan’s estimator (1.3) were smaller than those obtained by the proposed density estimator (3.4) and the Bernstein estimator (1.2). However, in all the other cases, using an appropriate choice of the degree m, the average $\widehat{ISE}$ and $\widehat{IAE}$ of the proposed density estimator (3.4) were smaller than what achieved by the kernel estimator (1.1), the Bernstein estimator (1.2) and Guan’s estimator (1.3), even when the sample size was large for same cases.
**-**- When we changed the parameters of the gamma mixture density in the sense that we had a smaller bias, our estimator was more competitive than the other approaches and we obtained better results.
**-****-**- In the considered distribution $0.5\mathcal{B}(3,9)+0.5\mathcal{B}(9,3)$, by choosing the appropriate m, the curve of the proposed distribution estimator (3.4) was closer to the true distribution than that of Guan’s estimator (1.3), even when the sample size was very large.

**-**- None of the estimators for the gamma mixture density $0.5\mathcal{G}(6,1)+0.5\mathcal{G}(1,6)$ had good approximations near $x=0$. However, the $\widehat{ISE}$ of the proposed estimator was closer to zero than that of the Bernstein estimator and the kernel estimator, especially near the edge $x=1$.
**-**- Guan’s estimator and the kernel estimator for the normal mixture density $0.25\mathcal{N}(2,1)+0.75\mathcal{N}(-3,1)$ had good approximations near $x=0$. However, the $\widehat{ISE}$ of the proposed estimator was closer to zero than that of the other estimators, especially near the two edges.

#### 5.2. Real Dataset

#### 5.2.1. COVID-19 Data

#### 5.2.2. Tuna Data

## 6. Conclusions

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Conflicts of Interest

## References

- Rosenblatt, M. Remarks on Some Nonparametric Estimates of a Density Function. Ann. Math. Stat.
**1956**, 27, 832–837. [Google Scholar] [CrossRef] - Parzen, E. On Estimation of a Probability Density Function and Mode. Ann. Math. Stat.
**1962**, 33, 1065–1076. [Google Scholar] [CrossRef] - Härdle, W. Smoothing Techniques with Implementation in S; S. Springer Science and Business Media: Berlin, Germany, 1991. [Google Scholar]
- Schuster, E.F. Incorporating support constraints into nonparametric estimators of densities. Comm. Stat. Theory Methods
**1985**, 14, 1123–1136. [Google Scholar] [CrossRef] - Müller, H.-G. Smooth optimum kernel estimators near endpoints. Biometrika
**1991**, 78, 521–530. [Google Scholar] [CrossRef] - Müller, H.-G. On the boundary kernel method for nonparametric curve estimation near endpoints. Scand. J. Statist.
**1993**, 20, 313–328. [Google Scholar] - Müller, H.-G.; Wang, J.-L. Hazard rate estimation under random censoring with varying kernels and bandwidths. Biometrika
**1994**, 50, 61–76. [Google Scholar] [CrossRef] - Lejeune, M.; Sarda, P. Smooth estimators of distribution and density functions. Comput. Stat. Data Anal.
**1992**, 14, 457–471. [Google Scholar] [CrossRef] - Jones, M.C. Simple boundary correction for density estimation kernel. Stat. Comput.
**1993**, 13, 135–146. [Google Scholar] [CrossRef] - Chen, S.X. Beta kernel estimators for density functions. Comput. Stat. Data Anal.
**1999**, 31, 131–145. [Google Scholar] [CrossRef] - Chen, S.X. Probability density function estimation using gamma kernels. Ann. Inst. Stat. Math.
**2000**, 52, 471–480. [Google Scholar] [CrossRef] - Leblanc, A. A bias-reduced approach to density estimation using Bernstein polynomials. J. Nonparametr. Stat.
**2010**, 22, 459–475. [Google Scholar] [CrossRef] - Slaoui, Y. Bias reduction in kernel density estimation. J. Nonparametr. Stat.
**2018**, 30, 505–522. [Google Scholar] [CrossRef] - Vitale, R.A. A bernstein polynomial approach to density function estimation. Stat. Inference Relat. Topics
**1975**, 2, 87–99. [Google Scholar] - Ghosal, S. Convergence rates for density estimation with Bernstein polynomials. Ann. Stat.
**2000**, 29, 1264–1280. [Google Scholar] [CrossRef] - Babu, G.J.; Canty, A.J.; Chaubey, Y.P. Application of Bernstein polynomials for smooth estimation of a distribution and density function. J. Stat. Plan. Inference
**2002**, 105, 377–392. [Google Scholar] [CrossRef] - Kakizawa, Y. Bernstein polynomial probability density estimation. J. Nonparametr. Stat.
**2004**, 16, 709–729. [Google Scholar] [CrossRef] - Rao, B.L.S.P. Estimation of distribution and density functions by generalized Bernstein polynomials. Indian J. Pure Appl. Math.
**2005**, 36, 63–88. [Google Scholar] - Igarashi, G.; Kakizawa, Y. On improving convergence rate of Bernstein polynomial density estimator. J. Nonparametr. Stat.
**2014**, 26, 61–84. [Google Scholar] [CrossRef] - Slaoui, Y.; Jmaei, A. Recursive density estimators based on Robbins-Monro’s scheme and using Bernstein polynomials. Stat. Interface
**2019**, 12, 439–455. [Google Scholar] [CrossRef] [Green Version] - Li, J.Q.; Barron, A.R. Mixture density estimation. Adv. Neural Inf. Process. Syst.
**2000**, 12, 279–285. [Google Scholar] - Pearson, K. Contributions to the mathematical theory of evolution. Philos. Trans. R. Soc. Lond. Ser. A Math. Phys. Eng. Sci.
**1984**, 185, 71–110. [Google Scholar] - McLachlan, G.; Peel, D. Finite Mixture Models; John Wiley and Sons: New York, NY, USA, 2004. [Google Scholar]
- Roeder, K.; Wasserman, L. Practical Bayesian density estimation using mixtures of normals. J. Am. Stat. Assoc.
**1997**, 92, 894–902. [Google Scholar] [CrossRef] - Schwarz, G. Estimating the dimension of a model. Ann. Stat.
**1978**, 6, 461–464. [Google Scholar] [CrossRef] - Leroux, B. Consistent estimation of a mixing distribution. Ann. Stat.
**1992**, 20, 1350–1360. [Google Scholar] [CrossRef] - Guan, Z. Efficient and robust density estimation using Bernstein type polynomials. J. Nonparametr. Stat.
**2016**, 28, 250–271. [Google Scholar] [CrossRef] [Green Version] - James, W.; Stein, C. Estimation with quadratic loss. In Breakthroughs in Statistics; Springer: New York, NY, USA, 1992; pp. 443–460. [Google Scholar]
- Dempster, A.P.; Laird, N.M.; Rubin, D.B. Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. Ser. B Stat. Methodol.
**1997**, 39, 1–22. [Google Scholar] - Wu, C.J. On the convergence properties of the EM algorithm. Ann. Stat.
**1983**, 11, 95–103. [Google Scholar] [CrossRef] - Stein, C. Estimation of the mean of a multivariate normal distribution. Ann. Stat.
**1981**, 9, 1135–1151. [Google Scholar] [CrossRef] - Oman, S.D. Contracting towards subspaces when estimating the mean of a multivariate normal distribution. J. Multivar. Anal.
**1982**, 12, 270–290. [Google Scholar] [CrossRef] [Green Version] - Oman, S.D. Shrinking towards subspaces in multiple linear regression. Technometrics
**1982**, 24, 307–311. [Google Scholar] [CrossRef] - Lehmann, E.L.; Casella, G. Theory of Point Estimation; Springer Science and Business Media: Berlin/Heidelberg, Germany, 2006. [Google Scholar]
- Zitouni, M.; Zribi, M.; Masmoudi, A. Asymptotic properties of the estimator for a finite mixture of exponential dispersion models. Filomat
**2018**, 32, 6575–6598. [Google Scholar] [CrossRef] [Green Version] - Redner, R.A.; Walker, H.F. Mixture densities, maximum likelihood and the EM algorithm. SIAM Rev.
**1984**, 26, 195–239. [Google Scholar] [CrossRef] - Tibshirani, R.; Walther, G.; Hastie, T. Estimating the number of clusters in a data set via the gap statistic. J. R. Stat. Soc. Ser.
**2001**, 63, 411–423. [Google Scholar] [CrossRef] - Chen, S.X. Empirical likelihood confidence intervals for nonparametric density estimation. Biometrika
**1996**, 83, 329–341. [Google Scholar] [CrossRef]

**Figure 1.**Quantitative comparison between the proposed estimator and Guan’s estimator of $0.5\mathcal{B}(3,9)+0.5\mathcal{B}(9,3)$ for $n=50$ (

**left**) and $n=100$ (

**right**).

**Figure 2.**Quantitative comparison among the mean squared error of the kernel estimator, the Bernstein estimator, the Guan’s estimator and the proposed estimator of $0.5\mathcal{G}(1,6)+0.5\mathcal{G}(6,1)$ for $n=200$.

**Figure 3.**Quantitative comparison among the mean squared error of the kernel estimator, the Bernstein estimator, the Guan’s estimator and the proposed estimator of $0.25\mathcal{N}(2,1)+0.75\mathcal{N}(-3,1)$ for $n=200$.

**Table 1.**Average $\widehat{ISE}$ for $N=500$ trials of Bernstein estimator, standard Gaussian kernel estimator and the proposed estimator ${\widehat{f}}_{n,m}$, for $n=50$, $n=100$ and $n=200$. The bold values indicate the smallest values of $ISE$.

Density | n | Proposed | Bernstein | Kernel | Guan’s |
---|---|---|---|---|---|

Estimator | Estimator | Estimator | Estimator | ||

50 | $\mathbf{0.092242}$ | $0.096684$ | $0.197497$ | $0.140323$ | |

$\left(a\right)$ | 100 | $\mathbf{0.091364}$ | $0.092242$ | $0.174251$ | $0.089129$ |

200 | $\mathbf{0.075532}$ | $0.079299$ | $0.148143$ | $0.086305$ | |

50 | $1.157827$ | $1.215347$ | $0.530446$ | $\mathbf{0.816906}$ | |

$\left(b\right)$ | 100 | $\mathbf{0.235402}$ | $0.306704$ | $0.482152$ | $0.276573$ |

200 | $\mathbf{0.199786}$ | $0.289870$ | $0.474805$ | $0.255716$ | |

50 | $\mathbf{0.001423}$ | $1.808252$ | $2.222369$ | $1.035522$ | |

$\left(c\right)$ | 100 | $\mathbf{0.000384}$ | $1.410606$ | $1.641689$ | $1.014602$ |

200 | $\mathbf{0.000227}$ | $1.348292$ | $1.077352$ | $0.994346$ | |

50 | $0.525192$ | $2.812448$ | $4.936701$ | $0.589465$ | |

$\left(d\right)$ | 100 | $\mathbf{0.492752}$ | $2.483141$ | $2.331765$ | $0.579595$ |

200 | $\mathbf{0.162917}$ | $0.898103$ | $1.154646$ | $0.507362$ | |

50 | $2.180849$ | $2.231986$ | $2.424656$ | $\mathbf{1.084340}$ | |

$\left(e\right)$ | 100 | $2.050098$ | $2.133496$ | $2.295932$ | $\mathbf{0.835670}$ |

200 | $2.042379$ | $2.086204$ | $2.053453$ | $\mathbf{0.717715}$ | |

50 | $\mathbf{0.313388}$ | $0.896995$ | $1.397111$ | $0.663889$ | |

$\left(f\right)$ | 100 | $\mathbf{0.253988}$ | $0.656400$ | $0.762742$ | $0.516530$ |

200 | $\mathbf{0.186290}$ | $0.577408$ | $0.417980$ | $0.472094$ |

**Table 2.**Average $\widehat{IAE}$ for $N=500$ trials of Bernstein estimator, standard Gaussian kernel estimator and the proposed estimator ${\widehat{f}}_{n,m}$, for $n=50$, $n=100$ and $n=200$. The bold values indicate the smallest values of $IAE$.

Density | n | Proposed | Bernstein | Kernel | Guan’s |
---|---|---|---|---|---|

Estimator | Estimator | Estimator | Estimator | ||

50 | $\mathbf{0.250241}$ | $\mathbf{0.250241}$ | $0.391072$ | $0.251641$ | |

$\left(a\right)$ | 100 | $\mathbf{0.196423}$ | $0.207109$ | $0.367361$ | $0.232399$ |

200 | $\mathbf{0.180536}$ | $0.191673$ | $0.348499$ | $0.214117$ | |

50 | $0.855008$ | $0.823416$ | $\mathbf{0.621137}$ | $0.747562$ | |

$\left(b\right)$ | 100 | $\mathbf{0.417735}$ | $0.457722$ | $0.669027$ | $0.438088$ |

200 | $\mathbf{0.386280}$ | $0.455983$ | $0.583057$ | $0.423595$ | |

50 | $\mathbf{0.035161}$ | $0.971720$ | $1.238839$ | $0.948451$ | |

$\left(c\right)$ | 100 | $\mathbf{0.019331}$ | $0.960044$ | $1.157838$ | $0.944000$ |

200 | $\mathbf{0.013233}$ | $0.923259$ | $0.953675$ | $0.931293$ | |

50 | $0.669269$ | $0.807667$ | $1.942776$ | $\mathbf{0.648617}$ | |

$\left(d\right)$ | 100 | $0.657815$ | $1.486974$ | $1.415036$ | $\mathbf{0.642986}$ |

200 | $\mathbf{0.351205}$ | $1.573886$ | $0.881866$ | $0.637942$ | |

50 | $0.830317$ | $0.834203$ | $1.499828$ | $\mathbf{0.678362}$ | |

$\left(e\right)$ | 100 | $0.706770$ | $0.713225$ | $1.374609$ | $\mathbf{0.645112}$ |

200 | $0.681767$ | $0.692357$ | $1.225502$ | $\mathbf{0.620315}$ | |

50 | $\mathbf{0.453948}$ | $0.640596$ | $1.068483$ | $0.608421$ | |

$\left(f\right)$ | 100 | $\mathbf{0.414676}$ | $0.714844$ | $0.782939$ | $0.584572$ |

200 | $\mathbf{0.383248}$ | $0.883320$ | $0.516527$ | $0.584320$ |

**Table 3.**Average $\widehat{KL}$ for $N=500$ trials of Bernstein estimator, standard Gaussian kernel estimator and the proposed estimator ${\widehat{f}}_{n,m}$, for $n=50$, $n=100$ and $n=200$. The bold values indicate the smallest values of $KL$.

Density | n | Proposed | Bernstein | Kernel | Guan’s |
---|---|---|---|---|---|

Estimator | Estimator | Estimator | Estimator | ||

50 | $\mathbf{0.025048}$ | $0.025048$ | $0.289818$ | $0.066817$ | |

$\left(a\right)$ | 100 | $\mathbf{0.012086}$ | $0.015023$ | $0.081830$ | $0.058541$ |

200 | $0.003256$ | $\mathbf{0.001788}$ | $0.060468$ | $0.029419$ | |

50 | $1.088053$ | $1.120246$ | $\mathbf{0.575298}$ | $0.667079$ | |

$\left(b\right)$ | 100 | $\mathbf{0.284795}$ | $0.325933$ | $0.381406$ | $0.659105$ |

200 | $0.255697$ | $0.310759$ | $\mathbf{0.150096}$ | $0.654338$ | |

50 | $\mathbf{2.689781}$ | $3.871172$ | $4.732324$ | $3.360316$ | |

$\left(c\right)$ | 100 | $\mathbf{0.011844}$ | $3.591156$ | $4.196295$ | $3.356093$ |

200 | $\mathbf{0.009426}$ | $3.505050$ | $3.169852$ | $3.318666$ | |

50 | $0.976450$ | $0.870222$ | $4.702359$ | $\mathbf{0.878995}$ | |

$\left(d\right)$ | 100 | $0.960633$ | $1.572251$ | $3.031783$ | $\mathbf{0.763044}$ |

200 | $\mathbf{0.281862}$ | $1.584022$ | $1.537355$ | $0.742696$ | |

50 | $\mathbf{1.549035}$ | $1.560172$ | $5.538142$ | $1.799133$ | |

$\left(e\right)$ | 100 | $\mathbf{1.207084}$ | $1.217043$ | $3.498273$ | $1.557894$ |

200 | $\mathbf{1.153420}$ | $1.169645$ | $1.322299$ | $1.387843$ | |

50 | $\mathbf{0.528337}$ | $1.052789$ | $1.952294$ | $0.651422$ | |

$\left(f\right)$ | 100 | $\mathbf{0.292017}$ | $0.805962$ | $1.191070$ | $0.537625$ |

200 | $\mathbf{0.062893}$ | $0.589790$ | $0.679154$ | $0.339442$ |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Helali, S.; Masmoudi, A.; Slaoui, Y.
Semi-Parametric Estimation Using Bernstein Polynomial and a Finite Gaussian Mixture Model. *Entropy* **2022**, *24*, 315.
https://doi.org/10.3390/e24030315

**AMA Style**

Helali S, Masmoudi A, Slaoui Y.
Semi-Parametric Estimation Using Bernstein Polynomial and a Finite Gaussian Mixture Model. *Entropy*. 2022; 24(3):315.
https://doi.org/10.3390/e24030315

**Chicago/Turabian Style**

Helali, Salima, Afif Masmoudi, and Yousri Slaoui.
2022. "Semi-Parametric Estimation Using Bernstein Polynomial and a Finite Gaussian Mixture Model" *Entropy* 24, no. 3: 315.
https://doi.org/10.3390/e24030315