# Maximizing Diversity in Biology and Beyond

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

## 1. Introduction

- Which distribution(s) $\mathbf{p}$ maximize the diversity ${}^{q}\phantom{\rule{-0.166667em}{0ex}}{D}^{\mathbf{Z}}\left(\mathbf{p}\right)$ of order q?
- What is the value of the maximum diversity ${sup}_{\mathbf{p}}{}^{q}\phantom{\rule{-0.166667em}{0ex}}{D}^{\mathbf{Z}}\left(\mathbf{p}\right)$?

**Theorem 1**

#### Conventions

## 2. A Spectrum of Viewpoints on Diversity

## 3. Distributions on a Set with Similarities

**Example 1.**The simplest similarity matrix $\mathbf{Z}$ is the identity matrix $\mathbf{I}$. This is called the naive model in [1], since it embodies the assumption that distinct species have nothing in common. Crude though this assumption is, it is implicit in the diversity measures most popular in the ecological literature (Table 1 of [1] ).

**Example 2.**With the rapid fall in the cost of DNA sequencing, it is increasingly common to measure similarity genetically (in any of several ways). Thus, the coefficients ${Z}_{ij}$ may be chosen to represent percentage genetic similarities between species. This is an effective strategy even when the taxonomic classification is unclear or incomplete [1], as is often the case for microbial communities [7].

**Example 3.**Given a suitable phylogenetic tree, we may define the similarity between two present-day species as the proportion of evolutionary time before the point at which the species diverged.

**Example 4.**In the absence of more refined data, we can measure species similarity according to a taxonomic tree. For instance, we might define

**Example 5.**In purely mathematical terms, an important case is where the similarity matrix arises from a metric d on the set $\{1,\dots ,n\}$ via the formula ${Z}_{ij}={e}^{-d(i,j)}$. Thus, the community is modelled as a probability distribution on a finite metric space. (The naive model corresponds to the metric defined by $d(i,j)=\infty $ for all $i\ne j$.) The diversity measures that we will shortly define can be understood as (the exponentials of) Rényi-like entropies for such distributions.

## 4. The Diversity Measures

**Proposition 1.**

**Lemma 1.**

- i.
- ${}^{q}\phantom{\rule{-0.166667em}{0ex}}{D}^{\mathbf{Z}}\left(\mathbf{p}\right)$ is continuous in $q\in [0,\infty ]$ for each distribution $\mathbf{p}$;
- ii.
- ${}^{q}\phantom{\rule{-0.166667em}{0ex}}{D}^{\mathbf{Z}}\left(\mathbf{p}\right)$ is continuous in $\mathbf{p}$ for each $q\in (0,\infty )$.

**Proof.**

**Lemma 2**

**Proof.**

## 5. Preparatory Lemmas

**Definition 1.**

**Lemma 3.**

- i.
- $\mathbf{p}$ is invariant;
- ii.
- ${\left(\mathbf{Z}\mathbf{p}\right)}_{i}={\left(\mathbf{Z}\mathbf{p}\right)}_{j}$ for all $i,j\in supp\left(\mathbf{p}\right)$;
- iii.
- $\mathbf{p}=\mathbf{p}\left(\mathbf{w}\right)$ for some nonnegative weighting $\mathbf{w}$ on ${\mathbf{Z}}_{B}$ and some nonempty subset $B\subseteq \{1,\dots ,n\}$.

**Proof.**

**Lemma 4.**

**Proof.**

**Definition 2.**

**Lemma 5.**

**Proof.**Let $0\le {q}^{\prime}\le q\le \infty $ and let $\mathbf{p}$ be an invariant distribution that maximizes ${}^{{q}^{\prime}}\phantom{\rule{-0.166667em}{0ex}}{D}^{\mathbf{Z}}$. Then for all $\mathbf{r}\in {\Delta}_{n}$,

## 6. The Main Theorem

**Theorem 1**

**Proof.**

**Corollary 1.**

**Definition 3.**

**Corollary 2.**

**Proof.**

## 7. The Computation Theorem

**Theorem 2**

- i.
- For all $q\in [0,\infty ]$,$$\underset{\mathbf{p}\in {\Delta}_{n}}{sup}{}^{q}\phantom{\rule{-0.166667em}{0ex}}{D}^{\mathbf{Z}}\left(\mathbf{p}\right)=\underset{B}{max}\left(\right)open="|"\; close="|">{\mathbf{Z}}_{B}$$
- ii.
- The maximizing distributions are precisely those of the form $\mathbf{p}\left(\mathbf{w}\right)$ where $\mathbf{w}$ is a nonnegative weighting on ${\mathbf{Z}}_{B}$ for some B attaining the maximum in Equation (10).

**Proof.**

**Remark 1.**The computation theorem provides a finite-time algorithm for finding all the maximizing distributions and computing ${D}_{\mathrm{max}}\left(\mathbf{Z}\right)$, as follows. For each of the ${2}^{n}$ subsets B of $\{1,\dots ,n\}$, perform some simple linear algebra to find the space of nonnegative weightings on ${\mathbf{Z}}_{B}$; if this space is nonempty, call B feasible and record the magnitude $\left(\right)open="|"\; close="|">{\mathbf{Z}}_{B}$. Then ${D}_{\mathrm{max}}\left(\mathbf{Z}\right)$ is the maximum of all the recorded magnitudes. For each feasible B such that $\left(\right)open="|"\; close="|">{\mathbf{Z}}_{B}={D}_{\mathrm{max}}\left(\mathbf{Z}\right)$, and each nonnegative weighting $\mathbf{w}$ on ${\mathbf{Z}}_{B}$, the distribution $\mathbf{p}\left(\mathbf{w}\right)$ is maximizing. This generates all of the maximizing distributions.

## 8. Simple Examples

**Example 6.**First consider the naive model $\mathbf{Z}=\mathbf{I}$, in which different species are deemed to be entirely dissimilar. As noted in Section 4, ${}^{q}\phantom{\rule{-0.166667em}{0ex}}{D}^{\mathbf{I}}\left(\mathbf{p}\right)$ is the exponential of the Rényi entropy of order q. It is well-known that Rényi entropy of any order $q>0$ is maximized uniquely by the uniform distribution. This result also follows trivially from Corollary 2: for clearly ${}^{\infty}\phantom{\rule{-0.166667em}{0ex}}{D}^{\mathbf{I}}\left(\mathbf{p}\right)=1/{max}_{i}{p}_{i}$ is uniquely maximized by the uniform distribution, and the corollary implies that the same is true for all values of $q>0$. Moreover, ${D}_{\mathrm{max}}\left(\mathbf{I}\right)=\left|\mathbf{I}\right|=n$.

**Example 7.**For a general matrix $\mathbf{Z}$ satisfying conditions (1), a two-species system is always maximized by the uniform distribution ${p}_{1}={p}_{2}=1/2$. When $n=3$, however, nontrivial examples arise. For instance, take the system shown in Figure 3, consisting of one species of newt and two species of frog. Let us first consider intuitively what we expect the maximizing distribution to be, then compare this with the answer given by Theorem 2.

**Example 8.**Let $\mathbf{Z}=\left(\begin{array}{cc}1& 1/2\\ 0& 1\end{array}\right)$, which satisfies all of our standing hypotheses except symmetry. Consider a distribution $\mathbf{p}=({p}_{1},{p}_{2})\in {\Delta}_{2}$. If $\mathbf{p}$ is $(1,0)$ or $(0,1)$ then ${}^{q}\phantom{\rule{-0.166667em}{0ex}}{D}^{\mathbf{Z}}\left(\mathbf{p}\right)=1$ for all q. Otherwise,

## 9. Maximum Diversity on Graphs

**Proposition 2.**

**Proof.**

**Remark 2.**The first part of the proof (together with Corollary 2) shows that a maximizing distribution can be constructed by taking the uniform distribution on some independent set of largest cardinality, then extending by zero to the whole vertex-set. Except in the trivial case $\mathbf{Z}=\mathbf{I}$, this maximizing distribution never has full support. We return to this point in Section 11.

**Example 9.**The reflexive graph $G=\u2022\phantom{\rule{-0.166667em}{0ex}}\phantom{\rule{-0.166667em}{0ex}}-\phantom{\rule{-0.166667em}{0ex}}\phantom{\rule{-0.166667em}{0ex}}\u2022\phantom{\rule{-0.166667em}{0ex}}\phantom{\rule{-0.166667em}{0ex}}-\phantom{\rule{-0.166667em}{0ex}}\phantom{\rule{-0.166667em}{0ex}}\u2022$ (loops not shown) has adjacency matrix $\mathbf{Z}=\left(\begin{array}{ccc}1& 1& 0\\ 1& 1& 1\\ 0& 1& 1\end{array}\right)$. The independence number of G is 2; this, then, is the maximum diversity of $\mathbf{Z}$. There is a unique independent set of cardinality 2, and a unique maximizing distribution, $(1/2,0,1/2)$.

**Example 10.**The reflexive graph $\u2022\phantom{\rule{-0.166667em}{0ex}}\phantom{\rule{-0.166667em}{0ex}}-\phantom{\rule{-0.166667em}{0ex}}\phantom{\rule{-0.166667em}{0ex}}\u2022\phantom{\rule{-0.166667em}{0ex}}\phantom{\rule{-0.166667em}{0ex}}-\phantom{\rule{-0.166667em}{0ex}}\phantom{\rule{-0.166667em}{0ex}}\u2022\phantom{\rule{-0.166667em}{0ex}}\phantom{\rule{-0.166667em}{0ex}}-\phantom{\rule{-0.166667em}{0ex}}\phantom{\rule{-0.166667em}{0ex}}\u2022$ again has independence number 2. There are three independent sets of maximal cardinality, so by Remark 2, there are at least three maximizing distributions,

**Example 11.**Let d be a metric on $\{1,\cdots ,n\}$. For a given $\epsilon >0$, the covering number $N(d,\epsilon )$ is the minimum cardinality of a subset $A\subseteq \{1,\cdots ,n\}$ such that

**Corollary 3.**

**Proof.**

**Remark 3.**Proposition 2 implies that computationally, finding the maximum diversity of an arbitrary $\mathbf{Z}$ is at least as hard as finding the independence number of a reflexive graph. This is a very well-studied problem, usually presented in its dual form (find the clique number of an irreflexive graph) and called the maximum clique problem [37]. It is $\mathbf{NP}$-hard, so on the assumption that $\mathbf{P}\ne \mathbf{NP}$, there is no polynomial-time algorithm for computing maximum diversity, nor even for computing the support of a maximizing distribution.

## 10. Positive Definite Similarity Matrices

**Lemma 6.**

**Proof.**

**Lemma 7.**

**Proof.**

**Proposition 3.**

**Proof.**

**Corollary 4.**

**Example 12.**Call $\mathbf{Z}$ ultrametric if ${Z}_{ik}\ge min\{{Z}_{ij},{Z}_{jk}\}$ for all $i,j,k$ and ${Z}_{ii}>{max}_{j\ne k}{Z}_{jk}$ for all i. (Under the assumptions (1) on $\mathbf{Z}$, the latter condition just states that distinct species are not completely similar.) If $\mathbf{Z}$ is ultrametric then $\mathbf{Z}$ is positive definite with positive weighting, by Proposition 2.4.18 of [13].

**Example 13.**Let $\mathbf{r}\in {\Delta}_{n}$ be a probability distribution of full support, and write $\mathbf{Z}$ for the diagonal matrix with entries $1/{r}_{1},\dots ,1/{r}_{n}$. Then for $0<q<\infty $,

**Example 14.**The identity matrix $\mathbf{Z}=\mathbf{I}$ is certainly positive definite with positive weighting. By topological arguments, there is a neighbourhood U of $\mathbf{I}$ in the space of symmetric matrices such that every matrix in U also has these properties. (See the proofs of Propositions 2.2.6 and 2.4.6 of [13].) Quantitative versions of this result are also available. For instance, in Proposition 2.4.17 of [13] it was shown that $\mathbf{Z}$ is positive definite with positive weighting if ${Z}_{ii}=1$ for all i and ${Z}_{ij}<1/(n-1)$ for all $i\ne j$. In fact, this result can be improved:

**Proposition 4.**

**Proof.**

## 11. Preservation of Species

**Lemma 8.**

**Proof.**

**Proposition 5.**

- i.
- there exists a maximizing distribution for $\mathbf{Z}$ of full support;
- ii.
- $\mathbf{Z}$ is positive semidefinite and admits a positive weighting.

**Proof.**

**Proposition 6.**

- i.
- every maximizing distribution for $\mathbf{Z}$ has full support;
- ii.
- $\mathbf{Z}$ has exactly one maximizing distribution, which has full support;
- iii.
- $\mathbf{Z}$ is positive definite with positive weighting;
- iv.
- ${D}_{\mathrm{max}}\left(\mathbf{Z}\right)>{D}_{\mathrm{max}}\left({\mathbf{Z}}_{B}\right)$ for every nonempty proper subset B of $\{1,\dots ,n\}$.

**Proof.**

## 12. Open Questions

## Acknowledgments

## Author Contributions

## Conflicts of Interest

## References

- Leinster, T.; Cobbold, C.A. Measuring diversity: The importance of species similarity. Ecology
**2012**, 93, 477–489. [Google Scholar] [CrossRef] [PubMed] - Rao, C.R. Diversity and dissimilarity coefficients: A unified approach. Theor. Popul. Biol.
**1982**, 21, 24–43. [Google Scholar] [CrossRef] - Rényi, A. On Measures of Entropy and Information. In Proceedings of the 4th Berkeley Symposium on Mathematical Statistics and Probability, Berkeley, CA, USA, 20 June–30 July 1960; University of California Press: Oakland, CA, USA, 1961; Volume 1, pp. 547–561. [Google Scholar]
- Tsallis, C. Possible generalization of Boltzmann–Gibbs statistics. J. Stat. Phys.
**1988**, 52, 479–487. [Google Scholar] [CrossRef] - Patil, G.P.; Taillie, C. Diversity as a concept and its measurement. J. Am. Stat. Assoc.
**1982**, 77, 548–561. [Google Scholar] [CrossRef] - Havrda, J.; Charvát, F. Quantification method of classification processes: concept of structural α-entropy. Kybernetika
**1967**, 3, 30–35. [Google Scholar] - Veresoglou, S.D.; Powell, J.R.; Davison, J.; Lekberg, Y.; Rillig, M.C. The Leinster and Cobbold indices improve inferences about microbial diversity. Fungal Ecol.
**2014**, 11, 1–7. [Google Scholar] [CrossRef] - Bakker, M.G.; Chaparro, J.M.; Manter, D.K.; Vivanco, J.M. Impacts of bulk soil microbial community structure on rhizosphere microbiomes of Zea mays. Plant Soil
**2015**, 392, 115–126. [Google Scholar] [CrossRef] - Jeziorski, A.; Tanentzap, A.J.; Yan, N.D.; Paterson, A.M.; Palmer, M.E.; Korosi, J.B.; Rusak, J.A.; Arts, M.T.; Keller, W.; Ingram, R.; et al. The jellification of north temperate lakes. Proc. R. Soc. B
**2015**, 282. [Google Scholar] [CrossRef] [PubMed] - Chalmandrier, L.; Münkemüller, T.; Lavergne, S.; Thuiller, W. Effects of species’ similarity and dominance on the functional and phylogenetic structure of a plant meta-community. Ecology
**2015**, 96, 143–153. [Google Scholar] [CrossRef] [PubMed] - Bromaghin, J.F.; Rode, K.D.; Budge, S.M.; Thiemann, G.W. Distance measures and optimization spaces in quantitative fatty acid signature analysis. Ecol. Evol.
**2015**, 5, 1249–1262. [Google Scholar] [CrossRef] [PubMed] - Wang, L.; Zhang, M.; Jajodia, S.; Singhal, A.; Albanese, M. Modeling Network Diversity for Evaluating the Robustness of Networks against Zero-Day Attacks. In Proceedings of the 19th European Symposium on Research in Computer Security (ESORICS 2014), Wroclaw, Poland, 7–11 September 2014; pp. 494–511.
- Leinster, T. The magnitude of metric spaces. Doc. Math.
**2013**, 18, 857–905. [Google Scholar] - Leinster, T. The Euler characteristic of a category. Doc. Math.
**2008**, 13, 21–49. [Google Scholar] - Barceló, J.A.; Carbery, A. On the magnitudes of compact sets in Euclidean spaces. 2015; arXiv:1507.02502. [Google Scholar]
- Meckes, M.W. Magnitude, diversity, capacities, and dimensions of metric spaces. Potential Anal.
**2015**, 42, 549–572. [Google Scholar] [CrossRef] - Willerton, S. On the magnitude of spheres, surfaces and other homogeneous spaces. Geom. Dedicata
**2014**, 168, 291–310. [Google Scholar] [CrossRef] - Leinster, T. The magnitude of a graph. 2014; arXiv:1401.4623. [Google Scholar]
- Hepworth, R.; Willerton, S. Categorifying the magnitude of a graph. 2015; arXiv:1505.04125. [Google Scholar]
- Chuang, J.; King, A.; Leinster, T. On the magnitude of a finite dimensional algebra. Theory Appl. Categories
**2016**, 31, 63–72. [Google Scholar] - Leinster, T. A maximum entropy theorem with applications to the measurement of biodiversity. 2009; arXiv:0910.0906. [Google Scholar]
- Fremlin, D.H.; Talagrand, M. Subgraphs of random graphs. Trans. Am. Math. Soc.
**1985**, 291, 551–582. [Google Scholar] [CrossRef] - Simpson, E.H. Measurement of diversity. Nature
**1949**, 163, 688. [Google Scholar] [CrossRef] - Whittaker, R.H. Vegetation of the Siskiyou mountains, Oregon and California. Ecol. Monogr.
**1960**, 30, 279–338. [Google Scholar] [CrossRef] - Magurran, A.E. Measuring Biological Diversity; Wiley-Blackwell: Hoboken, NJ, USA, 2003. [Google Scholar]
- Hurlbert, S.H. The nonconcept of species diversity: A critique and alternative parameters. Ecology
**1971**, 52, 577–586. [Google Scholar] [CrossRef] - Kimura, M.; Crow, J.F. The number of alleles that can be maintained in a finite population. Genetics
**1964**, 49, 725–738. [Google Scholar] [PubMed] - Hannah, L.; Kay, J.A. Concentration in the Modern Industry: Theory, Measurement, and the U.K. Experience; MacMillan: London, UK, 1977. [Google Scholar]
- McBratney, A.; Minasny, B. On measuring pedodiversity. Geoderma
**2007**, 141, 149–154. [Google Scholar] [CrossRef] - Hardy, G.H.; Littlewood, J.E.; Pólya, G. Inequalities, 2nd ed.; Cambridge University Press: Cambridge, UK, 1952. [Google Scholar]
- Chao, A.; Chiu, C.H.; Jost, L. Phylogenetic diversity measures based on Hill numbers. Philos. Trans. R. Soc. B
**2010**, 365, 3599–3609. [Google Scholar] [CrossRef] [PubMed] - Lawvere, F.W. Metric spaces, generalized logic and closed categories. Rendiconti del Seminario Matematico e Fisico di Milano
**1973**, 43, 135–166, reprinted in Repr. Theory Appl. Categories**2002**, 1, 1–37. [Google Scholar] [CrossRef] - Gromov, M. Metric Structures for Riemannian and Non-Riemannian Spaces; Birkhäuser: Boston, MA, USA, 2001. [Google Scholar]
- Pavoine, S.; Bonsall, M.B. Biological diversity: distinct distributions can lead to the maximization of Rao’s quadratic entropy. Theor. Popul. Biol.
**2009**, 75, 153–163. [Google Scholar] [CrossRef] [PubMed] - Kolmogorov, A.N. On certain asymptotic characteristics of completely bounded metric spaces. Doklady Akademii Nauk SSSR
**1956**, 108, 385–388. [Google Scholar] - Berarducci, A.; Majer, P.; Novaga, M. Infinite paths and cliques in random graphs. Fundam. Math.
**2012**, 216, 163–191. [Google Scholar] [CrossRef] - Karp, R.M. Reducibility among Combinatorial Problems. In Complexity of Computer Computations; Miller, R.E., Thatcher, J.W., Eds.; Plenum Press: New York, NY, USA, 1972; pp. 85–103. [Google Scholar]
- Meckes, M.W. Positive definite metric spaces. Positivity
**2013**, 17, 733–757. [Google Scholar] [CrossRef] - Horn, R.A.; Johnson, C.R. Matrix Analysis, 2nd ed.; Cambridge University Press: Cambridge, UK, 2012. [Google Scholar]
- Broom, M.; Rychtář, J. Game-Theoretical Models in Biology; Chapman & Hall/CRC Press: Boca Raton, FL, USA, 2013. [Google Scholar]
- Haigh, J. Game theory and evolution. Adv. Appl. Probab.
**1975**, 7, 8–11. [Google Scholar] [CrossRef] - Bishop, D.T.; Cannings, C. Models of animal conflict. Adv. Appl. Probab.
**1976**, 8, 616–621. [Google Scholar] [CrossRef] - Broom, M.; Cannings, C.; Vickers, G.T. On the number of local maxima of a constrained quadratic form. Proc. R. Soc. A
**1993**, 443, 573–584. [Google Scholar] [CrossRef] - Zhang, Y.J.; Harte, J. Population dynamics and competitive outcome derive from resource allocation statistics: The governing influence of the distinguishability of individuals. Theor. Popul. Biol.
**2015**, 105, 53–63. [Google Scholar] [CrossRef] [PubMed] - Leinster, T.; Willerton, S. On the asymptotic magnitude of subsets of Euclidean space. Geom. Dedicata
**2013**, 164, 287–310. [Google Scholar] [CrossRef] - Cover, T.M.; Thomas, J.A. Elements of Information Theory; Wiley: New York, NY, USA, 1991. [Google Scholar]

**Figure 1.**Two bird communities. Heights of stacks indicate species abundances. In (

**a**), there are four species, with the first dominant and the others relatively rare; in (

**b**), the fourth species is absent but the community is otherwise evenly balanced.

**Figure 2.**Visualizations of the main theorem: (

**a**) in terms of how different values of q rank the set of distributions; and (

**b**) in terms of diversity profiles.

**Figure 3.**Hypothetical three-species system. Distances between species indicate degrees of dissimilarity between them (not to scale).

**Figure 4.**Hypothetical community consisting of one species of oak (▪) and ten species of pine (•), to which one further species of pine is then added (◦). Distances between species indicate degrees of dissimilarity (not to scale).

© 2016 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons by Attribution (CC-BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Leinster, T.; Meckes, M.W.
Maximizing Diversity in Biology and Beyond. *Entropy* **2016**, *18*, 88.
https://doi.org/10.3390/e18030088

**AMA Style**

Leinster T, Meckes MW.
Maximizing Diversity in Biology and Beyond. *Entropy*. 2016; 18(3):88.
https://doi.org/10.3390/e18030088

**Chicago/Turabian Style**

Leinster, Tom, and Mark W. Meckes.
2016. "Maximizing Diversity in Biology and Beyond" *Entropy* 18, no. 3: 88.
https://doi.org/10.3390/e18030088