# Replacing Histogram with Smooth Empirical Probability Density Function Estimated by K-Moments

## Abstract

**:**

## 1. Introduction

## 2. K-Moments and Their Relevance

#### 2.1. Definition and Interpretation

#### 2.2. Estimation of K-Moments

#### 2.3. Estimation of the Distribution Function at K-Moment Values

## 3. Results

#### 3.1. Estimation of Probability Density

#### 3.2. Uncertainty Assessment

#### 3.3. Entropy Estimation

## 4. Discussion

## 5. Conclusions

- The ability to reliably estimate from a sample, moments of high order, up to the sample size.
- The ability to assign values of the distribution function to each estimated value of K-moment.
- The smoothness of the estimated values, which are linear combinations of a number of observations, rather than based on a single observation as in other approaches.

- The faithful representation of the true density, both in the body and the tails of the distribution.
- The dense and smooth shape, owing to the ability to estimate values of the density at very many points (even for any arbitrary point) within the range of the available observations.
- The low uncertainty of estimates.
- The ability to provide both point and interval estimates (confidence limits), with the latter becoming possible by Monte Carlo simulation.
- The simplicity of the calculations, which can be made in a typical spreadsheet environment.

- We sort the observed sample in ascending order.
- We calculate the estimates ${\widehat{\underset{\_}{K}}}_{p}^{\prime},{\widehat{\overline{K}}}_{p}^{\prime},i=1,\dots ,n$ from Equations (17) and (18).
- We estimate the coefficient ${\Lambda}_{1}$ from Equation (24) and calculate ${\overline{\Lambda}}_{1}$ from Equation (30).
- We calculate the estimates $\widehat{F}\left({\widehat{K}}_{p}^{\prime}\right)$ and $\widehat{F}\left({\widehat{\overline{K}}}_{p}^{\prime}\right)$ from Equations (28) and (32) for all ${\widehat{\underset{\_}{K}}}_{p}^{\prime},{\widehat{\overline{K}}}_{p}^{\prime}$ derived in step 2. (By plotting the tails $\widehat{\overline{F}}\left({\widehat{K}}_{p}^{\prime}\right)$ and $\widehat{F}\left({\widehat{\overline{K}}}_{p}^{\prime}\right)$ in double logarithmic graphs, we check whether the empirically estimated tail indices agree with those assumed in Step 4 and, if not, we repeat steps 4 and 5 with new estimates).
- We calculate the estimates of $f\left(x\right)$ from Equation (33).

## Supplementary Materials

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## Appendix A. Illustration of Alternative Techniques

**Figure A1.**Illustration of the estimate of the distribution function $F\left(x\right)$ using Equation (1) (

**upper**, with inset to better depict the staircase form) and that of the probability density $f\left(x\right)$ (

**lower**) as the numerical derivative of $F\left(x\right)$ with the staircase form replaced by a broken line form. The data series of Figure 1 was used ($n=100$ values, generated from a lognormal distribution with parameters $\varsigma =\lambda =1$; see Table 2). The abscissae of the points of estimates are the midpoints of the intervals $\left({x}_{\left(i:n\right)},{x}_{\left(i+1:n\right)}\right)$. As the estimates of $f\left(x\right)$ vary by orders of magnitude, logarithmic axes are used. In both panels the estimates by the proposed method (from Figure 1) are also shown for comparison.

**Figure A2.**Illustration of the probability density estimate using (a) the histogram (Equation (4) with 10 bins for the range [0, 20]), (b) the uniform kernel (Equation (A2) with $h=1$) and (c) the normal kernel (Equation (A3) with $h=1$), plotted in Cartesian (

**upper**) and logarithmic (

**lower**) axes. The data series of Figure 1 was used ($n=100$ values, generated from a lognormal distribution with parameters $\varsigma =\lambda =1$; see Table 2).

## Appendix B. Proof of Equation (48)

## References

- Kolmogorov, A.N. Grundbegriffe der Wahrscheinlichkeitsrechnung; Ergebnisse der Math: Berlin, Germany, 1933. [Google Scholar]
- Kolmogorov, A.N. Foundations of the Theory of Probability, 2nd ed.; Chelsea Publishing Company: New York, NY, USA, 1956; p. 84. [Google Scholar]
- Kolmogorov, A.N. Sulla determinazione empirica di una legge di distribuzione. Inst. Ital. Attuari Giorn
**1933**, 4, 83–91. [Google Scholar] - Papoulis, A. Probability and Statistics; Prentice-Hall: Hoboken, NJ, USA, USA, 1990. [Google Scholar]
- Weisstein, E.W. Plotting Position. From MathWorld—A Wolfram Web Resource. Available online: https://mathworld.wolfram.com/PlottingPosition.html (accessed on 6 November 2022).
- Koutsoyiannis, D. Stochastics of Hydroclimatic Extremes—A Cool Look at Risk, 2nd ed.; Kallipos Open Academic Editions: Athens, Greece, 2022; p. 346. ISBN 978-618-85370-0-2. [Google Scholar] [CrossRef]
- Parzen, E. On estimation of a probability density function and mode. Ann. Math. Stat.
**1962**, 33, 1065–1076. [Google Scholar] [CrossRef] - Sturges, H.A. The choice of a class interval. J. Am. Stat. Assoc.
**1926**, 21, 65–66. [Google Scholar] [CrossRef] - Scott, D.W. Sturges’ rule. Wiley Interdiscip. Rev. Comput. Stat.
**2009**, 1, 303–306. [Google Scholar] [CrossRef] - Rosenblatt, M. Remarks on some nonparametric estimates of a density function. Ann. Math. Stat.
**1956**, 27, 832–837. [Google Scholar] [CrossRef] - Scott, D.W. On optimal and data-based histograms. Biometrika
**1979**, 66, 605–610. [Google Scholar] [CrossRef] - Terrell, G.R.; Scott, D.W. Variable kernel Density estimation. Ann. Stat.
**1992**, 20, 1236–1265. [Google Scholar] [CrossRef] - Chen, Y.C. A tutorial on kernel density estimation and recent advances. Biostat. Epidemiol.
**2017**, 1, 161–187. [Google Scholar] [CrossRef] - Węglarczyk, S. Kernel density estimation and its application. In ITM Web of Conferences; EDP Sciences: Les Ulis, France, 2018; Volume 23, p. 00037. [Google Scholar] [CrossRef]
- Scott, D.W. Kernel density estimation. Wiley StatsRef Stat. Ref. Online
**2018**, 1–7. [Google Scholar] [CrossRef] - Eremenko, S.Y. Atomic Machine Learning. J. Neurocomput.
**2018**, 3, 13–28. [Google Scholar] - Lombardo, F.; Volpi, E.; Koutsoyiannis, D.; Papalexiou, S.M. Just two moments! A cautionary note against use of high-order moments in multifractal models in hydrology. Hydrol. Earth Syst. Sci.
**2014**, 18, 243–255. [Google Scholar] [CrossRef] [Green Version] - Koutsoyiannis, D. Knowable moments for high-order stochastic characterization and modelling of hydrological processes. Hydrol. Sci. J.
**2019**, 64, 19–33. [Google Scholar] [CrossRef] [Green Version] - Koutsoyiannis, D.; Sargentis, G.-F. Entropy and Wealth. Entropy
**2021**, 23, 1356. [Google Scholar] [CrossRef] [PubMed]

**Figure 1.**Illustration of the probability density estimate using the proposed method, plotted in Cartesian (

**upper**) and logarithmic (

**lower**) axes. A data series of $n=100$ values was used, generated from a lognormal distribution (see Table 2) with parameters $\varsigma =\lambda =1$. The density of the generating distribution is marked as “true”. The points marked as “estimate” are calculated by the proposed method (Equation (33)) and their abscissae are the midpoints of the intervals $\left({\widehat{K}}_{p}^{\prime},{\widehat{K}}_{p+1}^{\prime}\right)$ and $\left({\widehat{\overline{K}}}_{p+1}^{\prime},{\widehat{\overline{K}}}_{p}^{\prime}\right)$. For comparison, the histogram of 10 bins, calculated as in Equation (4) for the range [0, 20] (width $w=2$) is also shown.

**Figure 2.**Illustration of the median estimate and uncertainty (in terms of prediction limits) of the probability density using the proposed method, plotted in Cartesian (

**upper**) and logarithmic (

**middle**) axes. The original results from the proposed method are interpolated at the points that are plotted in the graphs. For comparison, results for the classical histogram with 10 bins are also shown (

**lower**), plotted with abscissae equal to the midpoints of the bins. The true distribution is exponential with parameters as in Table 4, from which 100 data series of $n=100$ values each were generated and processed to produce the uncertainty band.

**Figure 7.**Comparison of the Monte Carlo simulation results for prediction limits of the lognormal distribution (with parameters as in Table 4) with the minimal version of the method (

**upper**; copy of the middle panel of Figure 4) and for the confidence limits of the empirical probability density of Figure 1 (

**lower**). The plotted “point estimates” are precisely those shown in Figure 1.

**Table 1.**Special cases of K-moment estimator coefficients (adapted from [6], p. 194).

Case | ${\mathit{b}}_{\mathit{i}\mathit{n}\mathit{p}}$ | Case | ${\mathit{b}}_{\mathit{i}\mathit{n}\mathit{p}}$ |
---|---|---|---|

$p=1$ | ${b}_{in1}=\frac{1}{n}$ | $p=n-1$ | ${b}_{n-1,n,n-1}=\frac{1}{n},{b}_{n,n,n-1}=1-\frac{1}{n}$ |

$p=2$ | ${b}_{in2}=\frac{2}{n}\frac{i-1}{n-1}$ | $p=n$ | ${b}_{nnn}=1$ |

$p=3$ | ${b}_{in2}=\frac{3}{n}\frac{i-1}{n-1}\frac{i-2}{n-2}$ | $i=n$ | ${b}_{nnp}=\frac{p}{n}$ |

$p=4$ | ${b}_{in4}=\frac{4}{n}\frac{i-1}{n-1}\frac{i-2}{n-2}\frac{i-3}{n-3}$ | $i=p$ | ${b}_{pnp}=p\mathrm{B}\left(p,n-p+1\right)$ * symmetry: ${b}_{pnp}={b}_{n-p,n,n-p}$ (minimum at $p=n/2$) |

Name, Parameters *, Domain | $\mathbf{Probability}\mathbf{Density}\mathbf{Function},\mathbf{f}\left(\mathit{x}\right)$ | $\mathbf{Mean},{\mathit{\mu}}_{1}^{\prime}$ | $\mathbf{Variance},{\mathit{\mu}}_{2}$ | $\mathbf{Entropy}\mathit{\Phi}$ |
---|---|---|---|---|

Exponential $\mu >0,x\ge 0$ | ${\mathrm{e}}^{\u2013x/\mu}/\mu $ | $\mu $ | ${\mu}^{2}$ | $\mathrm{ln}\mathrm{e}\mu $ |

Normal $\mu \in \mathbb{R},\sigma 0,x\in \mathbb{R}$ | $\frac{\mathrm{exp}\left(-\frac{{\left(x-\mu \right)}^{2}}{2{\sigma}^{2}}\right)}{\sqrt{2\mathsf{\pi}}\sigma}$ | $\mu $ | ${\sigma}^{2}$ | $\mathrm{ln}\left(\sqrt{2e\mathsf{\pi}}\sigma \right)$ |

Lognormal $\varsigma >0,\lambda 0,x\ge 0$ | $\frac{\mathrm{exp}\left(-\frac{1}{2{\varsigma}^{2}}{\left(\mathrm{ln}\left(\frac{x}{\lambda}\right)\right)}^{2}\right)}{\sqrt{2\mathsf{\pi}}\varsigma x}$ | ${\mathrm{e}}^{\frac{{\varsigma}^{2}}{2}}\lambda $ | ${\mathrm{e}}^{{\varsigma}^{2}}\left({\mathrm{e}}^{{\varsigma}^{2}}-1\right){\lambda}^{2}$ | $\mathrm{ln}\left(\sqrt{2e\mathsf{\pi}}\lambda \varsigma \right)$ |

Pareto $\xi >0,\lambda 0,x\ge 0$ | $\frac{1}{\lambda}{\left(1+\xi \frac{x}{\lambda}\right)}^{-1-\frac{1}{\xi}}$ | $\frac{\lambda}{1-\xi}$ | $\frac{{\lambda}^{2}}{{\left(1-\xi \right)}^{2}\left(1-2\xi \right)}$ | $\xi +\mathrm{ln}\mathrm{e}\lambda $ |

Distribution | ${\mathit{\Lambda}}_{1}$ | ${\mathit{\Lambda}}_{\mathit{\infty}}$ | ${\overline{\mathit{\Lambda}}}_{1}$ | ${\overline{\mathit{\Lambda}}}_{\mathit{\infty}}$ |
---|---|---|---|---|

Exponential | $\mathrm{e}=2.718$ | ${\mathrm{e}}^{\mathsf{\gamma}}=1.781$ * | $\frac{\mathrm{e}}{\mathrm{e}-1}=1.582$ | 1 |

Normal | 2 | ${\mathrm{e}}^{\mathsf{\gamma}}=1.781$ | $2$ | ${\mathrm{e}}^{\mathsf{\gamma}}=1.781$ |

Lognormal | $\frac{2}{\mathrm{erfc}\left(\varsigma /{2}^{3/2}\right)}$ | ${\mathrm{e}}^{\mathsf{\gamma}}=1.781$ | $\frac{2}{2-\mathrm{erfc}\left(\varsigma /{2}^{3/2}\right)}$ | 1 ^{†} |

Pareto | ${\left(1-\xi \right)}^{-1/\xi}$ | $\mathsf{\Gamma}{\left(1-\xi \right)}^{1/\xi}$ | $\frac{1}{1-{\left(1-\xi \right)}^{1/\xi}}$ | 1 |

^{†}The theoretically consistent value is ${\mathrm{e}}^{\mathsf{\gamma}}$, but the convergence to the limit is very slow and thus the value 1 (like in the exponential and Pareto distribution) provides more accurate numerical results for typical sample sizes.

**Table 4.**Estimated means, standard deviations and entropies: averages from the 100 Monte Carlo simulations performed in the study with sample size 100.

Distribution, Parameters | Mean | Standard Deviation | Entropy | |||||
---|---|---|---|---|---|---|---|---|

True | Est. 1 * | Est. 2 * | True | Est. 1 | Est. 2 | True | Est. 2 | |

Exponential, $\mu =1$ | 1 | 0.99 | 1.00 | 1 | 0.98 | 0.98 | 1 | 0.96 |

Normal, $\mu =0,\sigma =1$ | 0 | 0.01 | 0.01 | 1 | 0.99 | 0.99 | 1.42 | 1.37 |

Lognormal, $\varsigma =\lambda =1$ | 1.65 | 1.69 | 1.80 | 2.16 | 2.26 | 2.31 | 1.42 | 1.42 |

Pareto, $\xi =0.2,\lambda =1$ | 1.25 | 1.26 | 1.27 | 1.61 | 1.56 | 1.57 | 1.20 | 1.18 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2022 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Koutsoyiannis, D.
Replacing Histogram with Smooth Empirical Probability Density Function Estimated by K-Moments. *Sci* **2022**, *4*, 50.
https://doi.org/10.3390/sci4040050

**AMA Style**

Koutsoyiannis D.
Replacing Histogram with Smooth Empirical Probability Density Function Estimated by K-Moments. *Sci*. 2022; 4(4):50.
https://doi.org/10.3390/sci4040050

**Chicago/Turabian Style**

Koutsoyiannis, Demetris.
2022. "Replacing Histogram with Smooth Empirical Probability Density Function Estimated by K-Moments" *Sci* 4, no. 4: 50.
https://doi.org/10.3390/sci4040050