# The Expected Missing Mass under an Entropy Constraint

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

- Every local maximum $\mathbf{p}$ of $\mathbb{E}{U}_{t}$ is of the form$${p}_{1}={p}_{2}=\dots ={p}_{n-1}\le {p}_{n},$$
- There exists a threshold $\tau =\tau \left(n\right)>n$ such that:
- (a)
- For $t\le \tau $, there is a unique global maximum:$${p}_{1}={p}_{2}=\dots ={p}_{n-1}={p}_{n}=\frac{1}{n}.$$
- (b)
- For $t>\tau $, there is a unique global maximum, and it has the form:$${p}_{1}={p}_{2}=\dots ={p}_{n-1}<{p}_{n}.$$

## 2. Main Results

**Proposition**

**1.**

**Corollary**

**1.**

**Theorem**

**1.**

**Theorem**

**2.**

- (i)
- For each $h>0$, the function $\mathbb{E}{U}_{t}$ attains its maximum.
- (ii)
- If $\mathbf{p}$ is a global maximum point of $\mathbb{E}{U}_{t}$, then $\mathbf{p}$ has a finite support.

**Theorem**

**3.**

**Theorem**

**4.**

**Proposition**

**2.**

**Proposition**

**3.**

**Proposition**

**4.**

**Remark**

**1.**

## 3. Proofs

**Proof**

**of**

**Proposition**

**1.**

**Proof**

**of**

**Theorem**

**1.**

- ${p}_{i}\ge 0$ for each i and $\sum _{i=1}^{\infty}{p}_{i}=1.$
- $H\left(\mathbf{p}\right)\le h.$

**Lemma**

**1.**

**Proof**

**of**

**Lemma**

**1.**

**Example**

**1.**

**Lemma**

**2.**

**Proof**

**of**

**Lemma**

**2.**

**Proof**

**of**

**Theorem**

**2.**

- (i)
- Follows from Lemma 1 and Lemma 2.
- (ii)
- Suppose that $\mathbf{p}=({p}_{1},{p}_{2},\dots )$ does not have a finite support. Then, we can find an ${n}_{0}$ such that the first ${n}_{0}$ entries ${p}_{1},{p}_{2},\dots ,{p}_{{n}_{0}}$ of $\mathbf{p}$ assume more than four different values. Put $\tilde{\mathbf{p}}=({p}_{1},{p}_{2},\dots ,{p}_{{n}_{0}})$ and let $c={p}_{1}+{p}_{2}+\dots +{p}_{{n}_{0}}$ and $\tilde{h}=H\left(\tilde{\mathbf{p}}\right)$. Consider the optimization problem$$\begin{array}{cc}\hfill \underset{\mathbf{p}}{\mathrm{max}}& \sum _{i=1}^{{n}_{0}}{p}_{i}{(1-{p}_{i})}^{t}\hfill \end{array}$$$$\begin{array}{}\mathrm{(10)}& \sum _{i=1}^{{n}_{0}}{p}_{i}\mathrm{ln}\frac{1}{{p}_{i}}\le \tilde{h},\hfill \mathrm{(11)}& \sum _{i=1}^{{n}_{0}}{p}_{i}=c.\hfill \end{array}$$

**Proof**

**of**

**Theorem**

**3.**

**Proof**

**of**

**Theorem**

**4.**

**Proof**

**of**

**Proposition**

**2.**

**Remark**

**2.**

**Proof**

**of**

**Proposition**

**3.**

**Proof**

**of**

**Proposition**

**4.**

## Acknowledgments

## Author Contributions

## Conflicts of Interest

## References

- Good, I.J. The Population Frequencies of Species and the Estimation of Population Parameters. Biometrika
**1953**, 40, 237–264. [Google Scholar] [CrossRef] - McAllester, D.; McAllester, R.E. On The Convergence Rate of Good-Turing Estimators. In Proceedings of the Thirteenth Annual Conference on Computational Learning Theory, Stanford, CA, USA, 28 June–1 July 2000; pp. 1–6. [Google Scholar]
- Haslinger, R.; Pipa, G.; Lewis, L.D.; Nikolić, D.; Williams, Z.; Brown, E. Encoding through Patterns: Regression Tree-Based Neuronal Population Models. Neural Comput.
**2013**, 25, 1953–1993. [Google Scholar] [CrossRef] [PubMed] - Berend, D.; Kontorovich, A. The Missing Mass Problem. Stat. Probab. Lett.
**2012**, 82, 1102–1110. [Google Scholar] [CrossRef] - Berend, D.; Kontorovich, A. On The Concentration of the Missing Mass. Electron. Commun. Probab.
**2013**, 18, 1–7. [Google Scholar] [CrossRef] - Kontorovich, A.; Hendler, D.; Menahem, E. Metric Anomaly Detection via Asymmetric Risk Minimization. In SIMBAD 2011. Lecture Notes in Computer Science; Springer: Berlin/Heidelberg, Germany, 2011; Volume 7005, pp. 17–30. [Google Scholar]
- Luo, H.P.; Schapire, R. Towards Minimax Online Learning with Unknown Time Horizon. In Proceedings of the 31st International Conference on International Conference on Machine Learning, Beijing, China, 21–26 June 2014. [Google Scholar]
- Sadeqi, M.; Holte, R.C.; Zilles, S. Detecting Mutex Pairs in State Spaces by Sampling. In Australasian Conference on Artificial Intelligence (2013); Lecture Notes in Computer Science; Springer: Cham, Switzerland; Volume 8272, pp. 490–501.
- Ben-Hamou, A.; Boucheron, S.; Ohannessian, M.I. Concentration Inequalities in the Infinite Urn Scheme for Occupancy Counts and the Missing Mass, with Applications. Bernoulli
**2017**, 23, 249–287. [Google Scholar] [CrossRef] - Han, Y.J.; Jiao, J.T.; Weissman, T. Minimax Estimation of Discrete Distributions under ℓ
_{1}Loss. IEEE Trans. Inf. Theory**2015**, 61, 6343–6354. [Google Scholar] [CrossRef] - Hussain, S.; Pečarić, J. An Improvement of Jensen’s Inequality with Some Applications. Asian Eur. J. Math.
**2009**, 2, 85–94. [Google Scholar] [CrossRef]

© 2017 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Berend, D.; Kontorovich, A.; Zagdanski, G.
The Expected Missing Mass under an Entropy Constraint. *Entropy* **2017**, *19*, 315.
https://doi.org/10.3390/e19070315

**AMA Style**

Berend D, Kontorovich A, Zagdanski G.
The Expected Missing Mass under an Entropy Constraint. *Entropy*. 2017; 19(7):315.
https://doi.org/10.3390/e19070315

**Chicago/Turabian Style**

Berend, Daniel, Aryeh Kontorovich, and Gil Zagdanski.
2017. "The Expected Missing Mass under an Entropy Constraint" *Entropy* 19, no. 7: 315.
https://doi.org/10.3390/e19070315