Abstract
Entropies are useful measures of variation. However, explicit expressions for entropies available in the literature are limited. In this paper, we provide a comprehensive collection of explicit expressions for four of the most common entropies for over sixty continuous univariate distributions. Most of the derived expressions are new. The explicit expressions involve known special functions.
1. Introduction
Let X denote a continuous random variable with probability density and cumulative distribution functions specified by and , respectively. Four of the most popular entropies are the geometric mean [1,2], Shannon entropy ([3], pp. 379–423; [3], pp. 623–656), Rényi entropy [4] and the cumulative residual entropy [5], defined by
and
respectively, for and .
There have been several papers giving explicit expressions for entropies. Ref. [6] derived expressions for for twenty univariate distributions. Ref. [7] derived expressions for for five multivariate distributions. Ref. [8] derived expressions for and mutual information for eight multivariate distributions. Ref. [9] derived expressions for and for fifteen bivariate distributions. Ref. [10] derived expressions for and for fifteen multivariate distributions. Ref. [11] derived expressions for , and the q-entropy for the Dagum distribution. Ref. [12] derived expressions for for certain binomial type distributions. Ref. [13] derived expressions for and for three Lindley type distributions.
All of these and other papers are restrictive in terms of the entropies considered and the number of distributions considered. In this paper, we derive expressions for (1)–(4) for more than sixty continuous univariate distributions, see Section 3. Most of the derived expressions are new. Some technicalities used in the derivations are given in Section 2. The derivations themselves are not given and can be obtained from the corresponding author. Some conclusions and future work are noted in Section 4.
The calculations of this paper involve several special functions, including the the exponential integral defined by
the gamma function defined by
the lower incomplete gamma function defined by
the upper incomplete gamma function defined by
the digamma function defined by
the standard normal distribution function defined by
the error function defined by
the complementary error function defined by
the beta function defined by
the incomplete beta function defined by
the incomplete beta function ratio defined by
the modified Bessel function of the first kind of order defined by
the modified Bessel function of the second kind defined by
the confluent hypergeometric function defined by
where denotes the ascending factorial; the Kummer function defined by
the Gauss hypergeometric function defined by
the degenerate hypergeometric series of two variables defined by
the degenerate hypergeometric function of two variables defined by
The properties of these special functions can be found in [14,15].
2. Technical Lemmas
The derivations in Section 3 use the following two lemmas.
Lemma 1.
Proof.
Note that
Hence, the result. □
Lemma 2.
The cumulative residual entropy defined by (4) can be calculated using
Proof.
Using the Taylor series expansion for , we can write
Hence, the result. □
3. The Tabulation
In this section, we give expressions for (the probability density function), (the cumulative distribution function), (the geometric mean), (Shannon entropy), (Rényi entropy), and (the cumulative residual entropy) for over sixty continuous univariate distributions.
1. Gauss hypergeometric beta distribution [16]: for this distribution,
and
for , , , and , where and .
2. q Weibull distribution [17]: for this distribution,
and
for , , if and if .
3. q exponential distribution [17]: for this distribution,
and
for , if and if .
4. Weighted exponential distribution: for this distribution,
and
for , and .
5. Teissier distribution [18]: for this distribution,
and
for and .
6. Maxwell distribution [19,20]: for this distribution,
and
for and .
7. Inverse Maxwell distribution: for this distribution,
and
for and .
8. Power Maxwell distribution [21]: for this distribution,
and
for , and .
9. Inverse power Maxwell distribution [22]: for this distribution,
and
for , and .
10. Omega distribution [23]: for this distribution,
and
for , and .
11. Colak et al.’s distribution [24]: for this distribution,
and
for , and .
12. Bimodal beta distribution [25]: for this distribution,
and
for , , , and , where , , , and .
13. Confluent hypergeometric beta distribution [26]: for this distribution,
and
for , , and .
14. Libby and Novick’s beta distribution [27]: for this distribution,
and
for , , , and .
15. Generalized beta distribution [28]: for this distribution,
and
for , , , and .
16. Log-logistic distribution: for this distribution,
and
for , and .
17. Inverse Gaussian distribution [29]: for this distribution,
and
for , and .
18. Gompertz distribution [30]: for this distribution,
and
for , and .
19. Exponential distribution: for this distribution,
and
for and .
20. Inverse exponential distribution: for this distribution,
and
for and .
21. Exponentiated exponential distribution [31]: for this distribution,
and
for and .
22. Gamma distribution: for this distribution,
and
for , and .
23. Chisquare distribution: for this distribution,
and
for and .
24. Chi distribution: for this distribution,
and
for and .
25. Inverse gamma distribution: for this distribution,
and
for , and .
26. Inverse chisquare distribution: for this distribution,
and
for and .
27. Inverse chi distribution: for this distribution,
and
for and .
28. Rayleigh distribution: for this distribution,
and
for and .
29. Weibull distribution [32]: for this distribution,
and
for , and .
30. Inverse Rayleigh distribution: for this distribution,
and
for and .
31. Inverse Weibull distribution: for this distribution,
and
for , and .
32. Gumbel distribution [33]: for this distribution,
and
for , and .
33. Generalized extreme value distribution [34]: for this distribution,
and
for if , if , and .
34. Generalized gamma distribution [35]: for this distribution,
and
for , , and .
35. Pareto distribution of type I [36]: for this distribution,
and
for , and .
36. Pareto distribution of type II [37]: for this distribution,
and
for , and .
37. Generalized Pareto distribution [38]: for this distribution,
and
for if and if .
38. Uniform distribution: for this distribution,
and
for and .
39. Power function distribution of type I: for this distribution,
and
for and .
40. Power function distribution of type II: for this distribution,
and
for and .
41. Arcsine distribution: for this distribution,
and
for .
42. Beta distribution: for this distribution,
and
for , and .
43. Inverted beta distribution: for this distribution,
and
for , and .
44. Kumaraswamy distribution [39]: for this distribution,
and
for , and .
45. Inverted Kumaraswamy distribution [40]: for this distribution,
and
for , and .
46. Normal distribution: for this distribution,
and
for , and .
47. Lognormal distribution: for this distribution,
and
for , and .
48. Half normal distribution: for this distribution,
and
for and .
49. Student’s t distribution [41]: for this distribution,
and
for and .
50. Cauchy distribution: for this distribution,
and
for .
51. Laplace distribution [42]: for this distribution,
and
for , and .
52. Logistic distribution of type I: for this distribution,
and
for , and .
53. Logistic distribution of type II: for this distribution,
and
for , and .
54. Logistic distribution of type III: for this distribution,
and
for , and .
55. Logistic distribution of type IV [43]: for this distribution,
and
for , , and .
56. Burr distribution [44]: for this distribution,
and
for , and .
57. Dagum distribution [45]: for this distribution,
and
for , and .
58. J shaped distribution [46]: for this distribution,
and
for and .
59. Nadarajah–Haghighi distribution [47]: for this distribution,
and
for , and .
60. Two-sided power distribution [48]: for this distribution,
and
for .
61. Power Lindley distribution [49]: for this distribution,
and
for , and .
62. Modified slash Lindley–Weibull distribution [50]: for this distribution,
and
for , and .
63. Reciprocal distribution: for this distribution,
and
for .
4. Conclusions
We have derived the most comprehensive collection of explicit expressions for the geometric mean, Shannon entropy, Rényi entropy and the cumulative residual entropy for the following continuous univariate distributions: 1. Gauss hypergeometric beta distribution, 2. q Weibull distribution, 3. q exponential distribution, 4. Weighted exponential distribution, 5. Teissier distribution, 6. Maxwell distribution, 7. Inverse Maxwell distribution, 8. Power Maxwell distribution, 9. Inverse power Maxwell distribution, 10. Omega distribution, 11. Colak et al.’s distribution, 12. Bimodal beta distribution, 13. Confluent hypergeometric beta distribution, 14. Libby and Novick’s beta distribution, 15. Generalized beta distribution, 16. Log-logistic distribution, 17. Inverse Gaussian distribution, 18. Gompertz distribution, 19. Exponential distribution, 20. Inverse exponential distribution, 21. Exponentiated exponential distribution, 22. Gamma distribution, 23. Chisquare distribution, 24. Chi distribution, 25. Inverse gamma distribution, 26. Inverse chisquare distribution, 27. Inverse chi distribution, 28. Rayleigh distribution, 29. Weibull distribution, 30. Inverse Rayleigh distribution, 31. Inverse Weibull distribution, 32. Gumbel distribution, 33. Generalized extreme value distribution, 34. Generalized gamma distribution, 35. Pareto distribution of type I, 36. Pareto distribution of type II, 37. Generalized Pareto distribution, 38. Uniform distribution, 39. Power function distribution of type I, 40. Power function distribution of type II, 41. Arcsine distribution, 42. Beta distribution, 43. Inverted beta distribution, 44. Kumaraswamy distribution, 45. Inverted Kumaraswamy distribution, 46. Normal distribution, 47. Lognormal distribution, 48. Half normal distribution, 49. Student’s t distribution, 50. Cauchy distribution, 51. Laplace distribution, 52. Logistic distribution of type I, 53. Logistic distribution of type II, 54. Logistic distribution of type III, 55. Logistic distribution of type IV, 56. Burr distribution, 57. Dagum distribution, 58. J shaped distribution, 59. Nadarajah–Haghighi distribution, 60. Two-sided power distribution, 61. Power Lindley distribution, 62. Modified slash Lindley–Weibull distribution, 63. Reciprocal distribution. This collection could be a useful reference for both theoreticians and practitioners of entropies. Future work will be to derive similar collections of explicit expressions for entropies of discrete univariate distributions, continuous bivariate distributions, discrete bivariate distributions, continuous multivariate distributions, discrete multivariate distributions, continuous matrix variate distributions, discrete matrix variate distributions, continuous complex variate distributions, and discrete complex variate distributions.
Author Contributions
Conceptualization, S.N. and M.K.; methodology, S.N. and M.K.; writing—original draft preparation, S.N.; writing—review and editing, S.N. All authors have read and agreed to the published version of the manuscript.
Funding
This research received no external funding.
Institutional Review Board Statement
Not applicable.
Data Availability Statement
Not applicable.
Acknowledgments
The authors would like to thank the Editor and the two referees for careful reading and comments which improved the paper.
Conflicts of Interest
Authors declare no conflicts of interest.
References
- Feng, C.; Wang, H.; Tu, X.M. Geometric mean of nonnegative random variable. Commun. Stat. Theory Methods 2013, 42, 2714–2717. [Google Scholar] [CrossRef]
- Vogel, R.M. The geometric mean? Commun. Stat. Theory Methods 2022, 51, 82–94. [Google Scholar] [CrossRef]
- Shannon, C.E. A mathematical theory of communication. Bell Syst. Tech. J. 1948, 27, 379–423. [Google Scholar] [CrossRef]
- Rényi, A. On measures of information and entropy. In Proceedings of the 4th Berkeley Symposium on Mathematics, Statistics and Probability, Berkeley, CA, USA, 20 June–30 July 1960; Neyman, J., Ed.; University of California Press: Berkeley, CA, USA, 1960; Volume 1, pp. 547–561. [Google Scholar]
- Rao, M.; Chen, Y.; Vemuri, B.C.; Wang, F. Cumulative residual entropy: A new measure of information. IEEE Trans. Inf. Theory 2004, 50, 1220–1228. [Google Scholar] [CrossRef]
- Lazo, A.V.; Rathie, P. On the entropy of continuous probability distributions. IEEE Trans. Inf. Theory 1978, 24, 120–122. [Google Scholar] [CrossRef]
- Ahmed, N.A.; Gokhale, D.V. Entropy expressions and their estimators for multivariate distributions. IEEE Trans. Inf. Theory 1989, 35, 688–692. [Google Scholar] [CrossRef]
- Darbellay, G.A.; Vajda, I. Entropy expressions for multivariate continuous distributions. IEEE Trans. Inf. Theory 2000, 46, 709–712. [Google Scholar] [CrossRef]
- Nadarajah, S.; Zografos, K. Expressions for Rényi and Shannon entropies for bivariate distributions. Inf. Sci. 2005, 170, 173–189. [Google Scholar] [CrossRef]
- Zografos, K.; Nadarajah, S. Expressions for Rényi and Shannon entropies for multivariate distributions. Stat. Probab. Lett. 2005, 71, 71–84. [Google Scholar] [CrossRef]
- Khan, M.Z.; Khan, M.A. Explicit expressions for three entropies of Dagum distribution. In Proceedings of the 13th International Conference on Statistical Sciences, Peshawar, Pakistan, 16–18 March 2015; Volume 28, pp. 193–196. [Google Scholar]
- Cheraghchi, M. Expressions for the entropy of binomial-type distributions. In Proceedings of the 2018 IEEE International Symposium on Information Theory, Vali, CO, USA, 17–22 June 2018; pp. 2520–2524. [Google Scholar]
- Giuclea, M.; Popescu, C.-C. On geometric mean and cumulative residual entropy for two random variables with Lindley type distribution. Mathematics 2022, 10, 1499. [Google Scholar] [CrossRef]
- Prudnikov, A.P.; Brychkov, Y.A.; Marichev, O.I. Integrals and Series; Gordon and Breach Science Publishers: Amsterdam, The Netherlands, 1986; Volume 1–3. [Google Scholar]
- Gradshteyn, I.S.; Ryzhik, I.M. Table of Integrals, Series, and Products, 6th ed.; Academic Press: San Diego, CA, USA, 2000. [Google Scholar]
- Armero, C.; Bayarri, M.J. Prior assessments for prediction in queues. Statistician 1994, 43, 139–153. [Google Scholar] [CrossRef]
- Picoli, S., Jr.; Mendes, R.S.; Malacarne, L.C. q-exponential, Weibull, and q-Weibull distributions: An empirical analysis. Phys. Stat. Mech. Its Appl. 2003, 324, 678–688. [Google Scholar] [CrossRef]
- Teissier, G. Recherches sur le vieillissement et sur les lois de mortalite. Ann. Physiol. Phys. Chim. Biol. 1934, 10, 237–284. [Google Scholar]
- Maxwell, J.C. Illustrations of the dynamical theory of gases. Part I. On the motions and collisions of perfectly elastic spheres. Lond. Edinb. Dublin Philos. Mag. J. Sci. 1860, 19, 19–32. [Google Scholar] [CrossRef]
- Maxwell, J.C. Illustrations of the dynamical theory of gases. Part II. On the process of diffusion of two or more kinds of moving particles among one another. Lond. Edinb. Dublin Philos. Mag. J. Sci. 1860, 20, 21–37. [Google Scholar] [CrossRef]
- Yadav, A.S.; Bakouch, H.S.; Singh, S.K.; Singh, U. Power Maxwell distribution: Statistical properties, estimation and application. arXiv 2022, arXiv:1807.01200. [Google Scholar]
- Al-Kzzaz, H.S.; Abd El-Monsef, M.M.E. Inverse power Maxwell distribution: Statistical properties, estimation and application. J. Appl. Stat. 2022, 49, 2287–2306. [Google Scholar] [CrossRef]
- Dombi, J.; Jonas, T.; Toth, Z.E.; Arva, G. The omega probability distribution and its applications in reliability theory. Qual. Reliab. Eng. Int. 2019, 35, 600–626. [Google Scholar] [CrossRef]
- Colak, A.B.; Sindhu, T.N.; Lone, S.A.; Akhtar, M.T.; Shafiq, A. A comparative analysis of maximum likelihood estimation and artificial neural network modeling to assess electrical component reliability. Qual. Reliab. Eng. Int. 2022. accepted. [Google Scholar] [CrossRef]
- Vila, R.; Alfaia, L.; Meneze, A.F.B.; Cankaya, M.N.; Bourguignon, M. A model for bimodal rates and proportions. J. Appl. Stat. 2022. accepted. [Google Scholar] [CrossRef]
- Gordy, M.B. Computationally convenient distributional assumptions for common-value auctions. Comput. Econ. 1988, 12, 61–78. [Google Scholar] [CrossRef]
- Libby, D.L.; Novick, M.R. Multivariate generalized beta-distributions with applications to utility assessment. J. Educ. Stat. 1982, 7, 271–294. [Google Scholar] [CrossRef]
- McDonald, J.B.; Xu, Y.J. A generalization of the beta distribution with applications. J. Econom. 1995, 66, 133–152. [Google Scholar] [CrossRef]
- Wald, A. On cumulative sums of random variables. Ann. Math. Stat. 1944, 15, 283–296. [Google Scholar] [CrossRef]
- Gompertz, B. On the nature of the function expressive of the law of human mortality, and on a new mode of determining the value of life contingencies. Philos. Trans. R. Soc. Lond. 1825, 115, 513–583. [Google Scholar]
- Gupta, R.D.; Kundu, D. Exponentiated exponential family: An alternative to gamma and Weibull distributions. Biom. J. 2001, 43, 117–130. [Google Scholar] [CrossRef]
- Weibull, W. A statistical distribution function of wide applicability. J. Appl. Mech. 1951, 18, 293–297. [Google Scholar] [CrossRef]
- Gumbel, E.J. Les valeurs extremes des distributions statistiques. Ann. L’Institut Henri Poincare 1935, 5, 115–158. [Google Scholar]
- Jenkinson, A.F. The frequency distribution of the annual maximum (or minimum) values of meteorological elements. Q. J. R. Meteorol. Soc. 1955, 81, 158–171. [Google Scholar] [CrossRef]
- Stacy, E.W. A generalization of the gamma distribution. Ann. Math. Stat. 1962, 33, 1187–1192. [Google Scholar] [CrossRef]
- Pareto, V. La legge della domanda. G. Degli Econ. 1895, 10, 59–68. [Google Scholar]
- Lomax, K.S. Business failures; Another example of the analysis of failure data. J. Am. Stat. Assoc. 1954, 49, 847–852. [Google Scholar] [CrossRef]
- Pickands, J. Statistical inference using extreme order statistics. Ann. Stat. 1975, 3, 119–131. [Google Scholar]
- Kumaraswamy, P. A generalized probability density function for double bounded random processes. J. Hydrol. 1980, 46, 79–88. [Google Scholar] [CrossRef]
- Abd Al-Fattah, A.M.; El-Helbawy, A.A.; Al-Dayian, G.R. Inverted Kumaraswamy distribution: Properties and estimation. Pak. J. Stat. 2017, 33, 37–61. [Google Scholar]
- Gosset, W.S. The probable error of a mean. Biometrika 1908, 6, 1–25. [Google Scholar]
- Laplace, P.S. Memoire sur la probabilite des causes par les evenements. Mem. L’Academie R. Des Sci. Present. Par Divers. Savan 1774, 6, 621–656. [Google Scholar]
- Prentice, R.L. Discrimination among some parametric models. Biometrika 1975, 62, 607–614. [Google Scholar] [CrossRef]
- Burr, I.W. Cumulative frequency functions. Ann. Math. Stat. 1942, 13, 215–232. [Google Scholar] [CrossRef]
- Dagum, C. A model of income distribution and the conditions of existence of moments of finite order. Bull. Int. Stat. Inst. 1975, 46, 199–205. [Google Scholar]
- Topp, C.W.; Leone, F.C. A family of J-shaped frequency functions. J. Am. Stat. Assoc. 1955, 50, 209–219. [Google Scholar] [CrossRef]
- Nadarajah, S.; Haghighi, F. An extension of the exponential distribution. Statistics 2011, 45, 543–558. [Google Scholar] [CrossRef]
- van Dorp, J.R.; Kotz, S. The standard two-sided power distribution and its properties: With applications in financial engineering. Am. Stat. 2002, 56, 90–99. [Google Scholar] [CrossRef]
- Ghitany, M.E.; Al-Mutairi, D.K.; Balakrishnan, N.; Al-Enezi, L.J. Power Lindley distribution and associated inference. Comput. Stat. Data Anal. 2013, 64, 20–33. [Google Scholar] [CrossRef]
- Reyes, J.; Arrue, J.; Venegas, O.; Gomez, H.W. The modified slash Lindley-Weibull distribution with applications to nutrition data. J. Appl. Stat. 2022, 49, 4206–4224. [Google Scholar] [CrossRef] [PubMed]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).