# Symmetry Properties of Bi-Normal and Bi-Gamma Receiver Operating Characteristic Curves are Described by Kullback-Leibler Divergences

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

_{1}(x) (for cases) and f

_{2}(x) (for controls). The corresponding cumulative distribution functions (cdfs) are F

_{1}(x) and F

_{2}(x), respectively.

_{1}(x)] against FPP [=1 − F

_{2}(x)], with pairs of TPP and FPP values obtained by allowing a single threshold risk score to vary over the range of the indicator variable. Thus, points along the curve represent potential thresholds on the scale of the indicator variable, from each of which a binary test may be characterized. An ROC curve can therefore provide a useful summary of the characteristics of an indicator variable used as the basis for a binary test. Depending on the choice of model for risk scores for cases and controls, it may be possible to write down an analytical equation for the ROC curve, but this is immaterial in the present context. ROC curves that are monotone increasing above the main diagonal of the plot over the whole domain are sometimes referred to as “proper” ROC curves (see, e.g., Section 4.6 in [1]). Some continuous parametric ROC curves are proper, some are not; for example, it is well-known that the bi-Normal ROC curve is not in general proper, while the bi-gamma ROC curve is proper [8].

## 2. Analytical Background

#### 2.1. Geometric Symmetry of ROC Curves

**Figure 1.**Graphical description of symmetric and asymmetric ROC curves. The dotted lines show, for reference, TPP = 1 − FPP (the negative diagonal) and the lines FPP = a (vertical) and TPP = 1 − a (horizontal). The FPP coordinate of point A = a, and the FPP coordinate of point C = a*, such that a < a*. The solid line is a symmetric ROC curve passing through the points A (a, b) and B (a

_{1}, b

_{1}) (such that a

_{1}= 1 − b, b

_{1}= 1 − a). Point C (a*, 1 − a*) also lies on the symmetric ROC curve. Asymmetries are defined by reference to the symmetric curve passing through point A, as follows. The dashed line is a TPP-asymmetric ROC curve passing through the points A (a, b) and D (a

_{2}, b

_{2}) (such that a

_{2}> 1 − b, b

_{2}= 1 − a). The dot-dashed line is a TNP-asymmetric ROC curve passing through the points A (a, b) and E (a

_{3}, b

_{3}) (such that a

_{3}< 1 − b, b

_{3}= 1 − a).

#### 2.2. Kullback-Leibler Divergences

_{1}(x) (for cases) and f

_{2}(x) (for controls). Then the Kullback-Leibler divergences (KLDs) [18] are I(f

_{1},f

_{2}) (with cases as the comparison distribution and controls as the reference distribution):

_{2},f

_{1}) (with controls as the comparison distribution and cases as the reference distribution):

_{1}and f

_{2}.

_{1},f

_{2}) and I(f

_{2},f

_{1}) ≥ 0, with equality only if f

_{1}(x) and f

_{2}(x) are identical. Typically, I(f

_{1},f

_{2}) ≠ I(f

_{2},f

_{1}) [10] although for an ROC curve based on f

_{1}(x) (for cases) and f

_{2}(x) (for controls) that is symmetric about the negative diagonal, I(f

_{1},f

_{2}) = I(f

_{2},f

_{1}) [17]. A KLD can be interpreted as a kind of distance between probability distributions [19], although the asymmetry in its arguments (apart from some special cases) clearly indicates it is not a distance in the Euclidian sense. We will work in natural logarithms, so the KLDs are denominated in nits [20]. For a discussion of measures of distance between distributions as used in summarizing ROC curves, see Section 4.3.4 in [1].

#### 2.3. The Pareto Distribution

_{1},f

_{2}) and I(f

_{2},f

_{1}) ≥ 0, with equality only if f

_{1}(x) and f

_{2}(x) are identical (i.e., if ${\lambda}_{1}={\lambda}_{2}$), as required. Figure 2A shows the graphical plots of the two Pareto KLDs $I\left({f}_{1},{f}_{2}\right)={g}_{1}\left(z\right)=\frac{1}{z}-1-\mathrm{ln}\left(\frac{1}{z}\right)$ and $I\left({f}_{2},{f}_{1}\right)={g}_{2}\left(z\right)=z-1-\mathrm{ln}\left(z\right)$, from which it appears that (for z > 0):

- $I\left({f}_{1},{f}_{2}\right)>I\left({f}_{2},{f}_{1}\right)$ when z < 1,
- $I\left({f}_{1},{f}_{2}\right)=I\left({f}_{2},{f}_{1}\right)$ when z = 1,
- $I\left({f}_{1},{f}_{2}\right)<I\left({f}_{2},{f}_{1}\right)$ when z > 1.

_{1},f

_{2}) and I(f

_{2},f

_{1}) shown in Figure 2. We will use these results on the Pareto distribution in the following sections.

**Figure 2.**(

**A**). The figure shows graphical plots of Kullback-Leibler divergences for two Pareto densities: ${g}_{1}\left(z\right)=I\left({f}_{1},{f}_{2}\right)=\left(1/z\right)-1-\mathrm{ln}\left(1/z\right)$ (the solid line), and ${g}_{2}\left(z\right)=I\left({f}_{2},{f}_{1}\right)=z-1-\mathrm{ln}\left(z\right)$ (the dashed line), with $z={\lambda}_{2}/{\lambda}_{1}$. (

**B**). The derivatives ${g}_{1}^{/}\left(z\right)=\left(z-1\right)/{z}^{2}$ (the solid line) and ${g}_{2}^{/}\left(z\right)=\left(z-1\right)/z$ (the dashed line).

## 3. The Bi-Normal ROC Curve

_{1}(x); and ${\mu}_{2}\hspace{0.17em}\text{and}\hspace{0.17em}{\sigma}_{2}^{2}$ denote the mean and variance, respectively, of f

_{2}(x). The indicator variable is calibrated so that μ

_{1}> μ

_{2}.

_{1}(x) is Normal with mean μ

_{1}= 3.4 and standard deviation σ

_{1}= 1 and the distribution of risk scores for controls f

_{2}(x) is Normal with mean μ

_{2}= 2 and standard deviation σ

_{2}= 1. The resulting ROC curve is geometrically symmetric [14] and I(f

_{1},f

_{2}) = I(f

_{2},f

_{1}) = 0.980 nits.

_{1}and μ

_{2}from Killeen and Taylor (Figure 1 in [14]). We note also from Figure 3 that the point where the two curves intersect characterizes the symmetric ROC curve with I(f

_{1},f

_{2}) = I(f

_{2},f

_{1}) = 0.980 nits.

**Figure 3.**Analysis of a bi-Normal ROC curve. The graph shows the Kullback-Leibler divergences I(f

_{1},f

_{2}) (the solid line) and I(f

_{2},f

_{1}) (the dashed line) for two Normal densities; f

_{1}(x) for cases has μ

_{1}= 3.4 and σ

_{1}is varied over a range that includes σ

_{1}= 1, and f

_{2}(x) for controls has μ

_{2}= 2.0 and σ

_{2}= 1. When σ

_{2}/σ

_{1}= 1, I(f

_{1},f

_{2}) = I(f

_{2},f

_{1}) and the corresponding ROC curve is symmetric about the negative diagonal. When σ

_{2}/σ

_{1}< 1, I(f

_{1},f

_{2}) > I(f

_{2},f

_{1}) and the corresponding ROC curve is TPP-asymmetric; when σ

_{2}/σ

_{1}> 1, I(f

_{2},f

_{1}) > I(f

_{1},f

_{2}) and the corresponding ROC curve is TNP-asymmetric.

## 4. The Bi-Exponential ROC Curve

_{1}(x) against 1−F

_{2}(x) then provides the ROC curve. Such ROC curves are TPP-asymmetric (as described in Figure 1) (see, e.g., Figure 1 in [25]).

## 5. The Bi-Gamma ROC Curve

_{1}(x) and f

_{2}(x), we have X~gamma(x, r

_{1}, λ

_{1}) and X~gamma(x, r

_{2}, λ

_{2}) and the corresponding KLDs (e.g., [27]) are:

#### 5.1. The Constant-Shape Bi-Gamma ROC Curve

_{1}(x) and f

_{2}(x) respectively, X~gamma(x, r, λ

_{1}) and X~gamma(x, r, λ

_{2}). The indicator variable is calibrated so that the mean of the case distribution is larger than the mean of the control distribution, which requires $r{\lambda}_{1}>r{\lambda}_{2}$. A graphical plot of 1−F

_{1}(x) against 1−F

_{2}(x) then provides the ROC curve. Such curves are TPP-asymmetric (as described in Figure 1). For example, see Dorfman et al. [8]. If r = 1, f

_{1}(x) and f

_{2}(x) are the same as for the bi-exponential ROC curve (above), and the symmetry properties then follow. Otherwise, the general gamma KLDs above simplify to:

#### 5.2. The Constant-Scale Bi-Gamma ROC Curve

_{1}(x) and f

_{2}(x) respectively, X~gamma(x, r

_{1}, λ) and X~gamma(x, r

_{2}, λ). The indicator variable is calibrated so that the mean of the case distribution is larger than the mean of the control distribution, which requires ${r}_{1}\lambda >{r}_{2}\lambda $. A graphical plot of 1−F

_{1}(x) against 1−F

_{2}(x) then provides the ROC curve. Such curves are TNP-asymmetric (as described in Figure 1). The general gamma KLDs above simplify to:

_{2}= 1 and r

_{1}= ζ, ζ > 0. Figure 4A shows the graphical plots of:

- $I\left({f}_{1},{f}_{2}\right)={g}_{1}\left(\zeta \right)=-\mathrm{ln}\left(\Gamma \left(\zeta \right)\right)+(\zeta -1)\cdot \mathsf{\Psi}\left(\zeta \right)$
- $I\left({f}_{2},{f}_{1}\right)={g}_{2}\left(\zeta \right)=\mathrm{ln}\left(\Gamma \left(\zeta \right)\right)+\left(1-\zeta \right)\cdot \mathsf{\Psi}\left(1\right)$

- $I\left({f}_{1},{f}_{2}\right)>I\left({f}_{2},{f}_{1}\right)$ when ζ < 1,
- $I\left({f}_{1},{f}_{2}\right)=I\left({f}_{2},{f}_{1}\right)$ when ζ = 1,
- $I\left({f}_{1},{f}_{2}\right)<I\left({f}_{2},{f}_{1}\right)$ when ζ > 1.

- when ζ < 1, ${g}_{3}^{/}\left(\zeta \right)$ is negative and ${g}_{3}\left(\zeta \right)={g}_{2}^{/}\left(\zeta \right)-{g}_{1}^{/}\left(\zeta \right)$ is decreasing,
- when ζ = 1, ${g}_{3}^{/}\left(1\right)$ is zero and ${g}_{3}\left(\zeta \right)$ is stationary,
- when ζ > 1, ${g}_{3}^{/}\left(\zeta \right)$ is positive and ${g}_{3}\left(\zeta \right)$ is increasing,

_{1},f

_{2}) and I(f

_{2},f

_{1}) shown in Figure 4.

**Figure 4.**Analysis of a constant scale bi-gamma ROC curve. (

**A**). Graphical plots of Kullback-Leibler divergences: ${g}_{1}\left(\zeta \right)=I\left({f}_{1},{f}_{2}\right)=-\mathrm{ln}\left(\Gamma \left(\zeta \right)\right)+(\zeta -1)\cdot \mathsf{\Psi}\left(\zeta \right)$ (the solid line), and ${g}_{2}\left(\zeta \right)=I\left({f}_{2},{f}_{1}\right)=\mathrm{ln}\left(\Gamma \left(\zeta \right)\right)+\left(1-\zeta \right)\cdot \mathsf{\Psi}\left(1\right)$ (the dashed line), with r

_{2}= 1 and r

_{1}= ζ, ζ > 0. (

**B**). The derivatives ${g}_{1}^{/}\left(\zeta \right)=\left(\zeta -1\right)\cdot {\mathsf{\Psi}}^{\left(1\right)}\left(\zeta \right)$ (the solid line) and ${g}_{2}^{/}\left(\zeta \right)=\mathsf{\Psi}\left(\zeta \right)+\gamma $ (the dashed line).

## 6. Discussion

_{1}(x) (for cases) and f

_{2}(x) (for controls) that is symmetric about the negative diagonal, I(f

_{1},f

_{2}) = I(f

_{2},f

_{1}) [17]. Second, although the lack of symmetry of the KLD has been referred to as a nuisance in applications [36], in this particular study we find that the asymmetry of the KLD usefully characterizes the asymmetry of bi-Normal and bi-gamma ROC curves.

## Acknowledgments

## References

- Pepe, M.S. The Statistical Evaluation of Medical Tests for Classification and Prediction; Oxford University Press: Oxford, UK, 2003. [Google Scholar]
- Thrusfield, M. Veterinary Epidemiology, 3rd ed.; Blackwell Science Ltd.: Oxford, UK, 2007. [Google Scholar]
- Madden, L.V.; Hughes, G.; van den Bosch, F. The Study of Plant Disease Epidemics; APS Press: St Paul, MN, USA, 2007. [Google Scholar]
- Egan, J.P. Signal Detection Theory and ROC Analysis; Academic Press Inc.: New York, NY, USA, 1975. [Google Scholar]
- Krzanowski, W.J.; Hand, D.J. ROC Curves for Continuous Data; Chapman & Hall/CRC: Boca Raton, FL, USA, 2009. [Google Scholar]
- Swets, J.A.; Dawes, R.M.; Monahan, J. Better decisions through science. Sci. Am.
**2000**, 283, 70–75. [Google Scholar] [CrossRef] - Zweig, M.H.; Campbell, G. Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine. Clin. Chem.
**1993**, 39, 561–577. [Google Scholar] [PubMed] - Dorfman, D.D.; Berbaum, K.S.; Metz, C.E.; Lenth, R.V.; Hanley, J.A.; Dagga, H.A. Proper receiver operating characteristic analysis: the bigamma model. Acad. Radiol.
**1996**, 4, 138–149. [Google Scholar] [CrossRef] - Hanley, J.A.; McNeil, B.J. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology
**1982**, 143, 29–36. [Google Scholar] [CrossRef] [PubMed] - Broemeling, L.D. Bayesian methods for medical test accuracy. Diagnostics
**2011**, 1, 1–35. [Google Scholar] [CrossRef] - Obuchowski, N.A. ROC analysis. Am. J. Roentgenol.
**2005**, 184, 364–372. [Google Scholar] [CrossRef] [PubMed] - Olatinwo, R.O.; Paz, J.O.; Brown, S.L.; Kemerait, R.C., Jr.; Culbreath, A.K.; Hoogenboom, G. Impact of early spring weather factors on the risk of tomato spotted wilt in peanut. Plant. Dis.
**2009**, 93, 783–788. [Google Scholar] [CrossRef] - Green, D.M.; Swets, J.A. Signal Detection Theory and Psychophysics; John Wiley and Sons Inc.: New York, NY, USA, 1966. [Google Scholar]
- Killeen, P.R.; Taylor, T.J. Symmetric receiver operating characteristics. J. Math. Psychol.
**2004**, 48, 432–434. [Google Scholar] [CrossRef] - Hughes, G. Applications of Information Theory to Epidemiology; APS Press: St Paul, MN, USA, 2012. [Google Scholar]
- Taylor, M.M. Detectability theory and the interpretation of vigilance data. Acta Psychol.
**1967**, 27, 390–399. [Google Scholar] [CrossRef] - Bhattacharya, B.; Hughes, G. Symmetry of receiver operating characteristic curves and Kullback-Leibler divergences between the signal and noise populations. J. Math. Psychol.
**2011**, 5, 365–367. [Google Scholar] [CrossRef] - Kullback, S. Information Theory and Statistics, 2nd ed.; Dover Publications: New York, NY, USA, 1968. [Google Scholar]
- Cover, T.M.; Thomas, J.A. Elements of Information Theory, 2nd ed.; Wiley-Interscience: Hoboken, NJ, USA, 2006. [Google Scholar]
- MacDonald, D.K.C. Information theory and its application to taxonomy. J. Appl. Phys.
**1952**, 23, 529–531. [Google Scholar] [CrossRef] - Ullah, A. Uses of entropy and divergence measures for evaluating econometric approximations and inference. J. Econometrics
**2002**, 107, 313–326. [Google Scholar] [CrossRef] - Applebaum, D. Probability and Information—an Integrated Approach; Cambridge University Press: Cambridge, UK, 1996. [Google Scholar]
- Belov, D.I.; Armstrong, R.D. Distributions of the Kullback-Leibler divergence with applications. Brit. J. Math. Stat. Psy.
**2011**, 64, 291–309. [Google Scholar] [CrossRef] [PubMed] - Marzban, C. The ROC curve and the area under it as performance measures. Weather Forecast.
**2004**, 19, 1106–1114. [Google Scholar] [CrossRef] - Betinec, M. Testing the difference of the ROC curves in biexponential model. Tatra Mt. Math. Publ.
**2008**, 39, 215–223. [Google Scholar] - Asadi, M.; Ebrahimi, N.; Hamedani, G.G.; Soofi, E.S. Information measures for Pareto distributions and order statistics. In Statistics for Industry and Technology: Advances in Distribution Theory, Order Statistics, and Inference; Balakrishnan, N., Sarabia, J.M., Castillo, E., Eds.; Birkhäuser: Boston, MA, USA, 2006; pp. 207–223. [Google Scholar]
- Mathiassen, J.R.; Skavhaug, A.; Bø, K. Texture similarity measure using Kullback-Leibler divergence between gamma distributions. Lect. Notes Comput. Sc.
**2002**, 2352, 133–147. [Google Scholar] - Faraggi, D.; Reiser, B. Estimation of the area under the ROC curve. Stat. Med.
**2002**, 21, 3093–3106. [Google Scholar] [CrossRef] [PubMed] - Faraggi, D.; Reiser, B.; Schisterman, E.F. ROC curve analysis for biomarkers based on pooled assessments. Stat. Med.
**2003**, 22, 2515–2527. [Google Scholar] [CrossRef] [PubMed] - Hussain, E. The bi-gamma ROC curve in a straightforward manner. J. Basic Appl. Sci.
**2012**, 8, 309–314. [Google Scholar] [CrossRef] - Oldham, K.B.; Myland, J.; Spanier, J. An Atlas of Functions: with Equator, the Atlas Function Calculator, 2nd ed.; Springer: New York, NY, USA, 2009. [Google Scholar]
- Hanley, J.A. The robustness of the “binormal” assumptions used in fitting ROC curves. Med. Decis. Making
**1988**, 8, 197–203. [Google Scholar] [CrossRef] [PubMed] - Tang, L.; Du, P.; Wu, C. Compare diagnostic tests using transformation-invariant smoothed ROC curves. J. Stat. Plan. Infer.
**2010**, 140, 3540–3551. [Google Scholar] [CrossRef] [PubMed] - Hand, D.J.; Anagnostopoulos, C. When is the area under the receiver operating characteristic curve an appropriate measure of classifier performance? Pattern Recogn. Lett.
**2013**, 34, 492–495. [Google Scholar] [CrossRef] - Johnson, D.H.; Gruner, C.M.; Baggerly, K.; Seshagiri, C. Information-theoretic analysis of neural coding. J. Comput. Neurosci.
**2001**, 10, 47–69. [Google Scholar] [CrossRef] [PubMed] - Sinanović, S.; Johnson, D.H. Toward a theory of information processing. Signal Process.
**2007**, 87, 1326–1344. [Google Scholar] [CrossRef]

© 2013 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/).

## Share and Cite

**MDPI and ACS Style**

Hughes, G.; Bhattacharya, B.
Symmetry Properties of Bi-Normal and Bi-Gamma Receiver Operating Characteristic Curves are Described by Kullback-Leibler Divergences. *Entropy* **2013**, *15*, 1342-1356.
https://doi.org/10.3390/e15041342

**AMA Style**

Hughes G, Bhattacharya B.
Symmetry Properties of Bi-Normal and Bi-Gamma Receiver Operating Characteristic Curves are Described by Kullback-Leibler Divergences. *Entropy*. 2013; 15(4):1342-1356.
https://doi.org/10.3390/e15041342

**Chicago/Turabian Style**

Hughes, Gareth, and Bhaskar Bhattacharya.
2013. "Symmetry Properties of Bi-Normal and Bi-Gamma Receiver Operating Characteristic Curves are Described by Kullback-Leibler Divergences" *Entropy* 15, no. 4: 1342-1356.
https://doi.org/10.3390/e15041342