Next Article in Journal
Novel Criteria for Deterministic Remote State Preparation via the Entangled Six-Qubit State
Next Article in Special Issue
Indicators of Evidence for Bioequivalence
Previous Article in Journal
Noise Suppression in 94 GHz Radar-Detected Speech Based on Perceptual Wavelet Packet
Previous Article in Special Issue
The Logical Consistency of Simultaneous Agnostic Hypothesis Tests
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

The Structure of the Class of Maximum Tsallis–Havrda–Chavát Entropy Copulas

by
Jesús E. García
1,
Verónica A. González-López
1,* and
Roger B. Nelsen
2
1
Department of Statistics, University of Campinas, Rua Sérgio Buarque de Holanda, 651, Campinas, São Paulo 13083-859, Brazil
2
Department of Mathematical Sciences, Lewis & Clark College, 0615 SW Palatine Hill Rd., Portland, OR 97219, USA
*
Author to whom correspondence should be addressed.
Entropy 2016, 18(7), 264; https://doi.org/10.3390/e18070264
Submission received: 21 May 2016 / Revised: 1 July 2016 / Accepted: 14 July 2016 / Published: 19 July 2016
(This article belongs to the Special Issue Statistical Significance and the Logic of Hypothesis Testing)

Abstract

:
A maximum entropy copula is the copula associated with the joint distribution, with prescribed marginal distributions on [ 0 , 1 ] , which maximizes the Tsallis–Havrda–Chavát entropy with q = 2 . We find necessary and sufficient conditions for each maximum entropy copula to be a copula in the class introduced in Rodríguez-Lallena and Úbeda-Flores (2004), and we also show that each copula in that class is a maximum entropy copula.

1. Introduction

In thermodynamics, entropy is a measure of randomness or disorder, and in statistics, entropy is a measure of uncertainty in a probability distribution. During the past one and a half centuries, a number of such measures have been proposed. In terms of a continuous bivariate distribution with density f ( x , y ) , three well known entropies are the following:
B ( f ) = f ( x , y ) ln ( f ( x , y ) ) d x d y [ Boltzmann - Gibbs - Shannon ( BGS ) ] ; R q ( f ) = 1 1 q log 2 f ( x , y ) q d x d y , q 0 , q 1 [ Rényi ] ; T q ( f ) = 1 q 1 1 f ( x , y ) q d x d y , q 0 , q 1 [ Tsallis - Havrda - Chavát ( THC ) ] .
These measures are related to one another. For example, the limit as q approaches 1 of the THC entropy (see [1]) is the BGS entropy; and the THC and Rényi entropies for a common value of q are monotone functions of each other.
One objective in the study of entropy is to identify the joint distributions that maximize entropy subject to given margins, and to identify the copulas associated with those distributions. In [2] Pougaza and Mohammad-Djafari use Lagrange multipliers to show that when the support of f ( x , y ) is I 2 , I = [ 0 , 1 ] , the joint probability density function that maximizes BGS entropy is f ( x , y ) = f 1 ( x ) f 2 ( y ) , the product of the two marginal probability density functions. Thus, in the case of BGS entropy, every joint distribution maximizing BGS entropy has the same copula, namely the independence copula.
Pougaza and Mohammad-Djafari [2] also identified the joint distributions that maximize THC entropy in the case q = 2 (a case of interest since it coincides with Simpson’s diversity index [3]). In [4] the authors construct several families of copulas by considering the THC entropy with index q = 2 and particular expressions for the marginals. Our goal in this paper is to study the copulas associated with those joint distributions. We show that the family of maximum entropy copulas (in the THC q = 2 sense) coincides with the family of copulas studied previously by Rodríguez-Lallena and Úbeda-Flores [5]. In practical terms, this result guarantees that researchers using maximum entropy copulas (in the THC q = 2 sense) can find in [5] all the relevant properties of the copulas in [2].
We proceed as follows. In the next section we review the maximization of THC entropy in the case q = 2 . Since not every solution to the Lagrange multiplier optimization in [2] yields a proper bivariate distribution, in Section 3 we establish necessary and sufficient conditions for the solutions to be proper bivariate distributions and identify the associated copulas. We illustrate the results with several examples.

2. Preliminaries

Consider the bivariate probability density function f ( x , y ) with ( x , y ) I 2 , I= [ 0 , 1 ] , which maximizes the THC entropy T 2 ( f ) , given by
T 2 ( f ) = 1 0 1 0 1 f 2 ( x , y ) d x d y ,
with the following constraints
0 1 f ( x , y ) d y = f 1 ( x ) , x I ,
0 1 f ( x , y ) d x = f 2 ( y ) , y I ,
0 1 0 1 f ( x , y ) d x d y = 1 .
We use the notations T 2 ( f ) and T 2 ( F ) to refer to the THC ([1]) entropy of the density f and the entropy of the cumulative distribution F , respectively.
We note that distributions with the same copula but different marginals (hence different joint densities) may have different entropies (1). For example, consider the following cases, all with the product copula and different margins: F 1 ( x , y ) = x y , x , y I , T 2 ( F 1 ) = 0 ;   F 2 ( x , y ) = x 2 y 2 , x , y I , T 2 ( F 2 ) = 7 9 ; and F 3 ( x , y ) = x y , x , y I , T 2 ( F 3 ) = .
According to [2] for absolutely continuous distribution functions F 1 ( x ) and F 2 ( y ) , the joint density and distribution functions f ( x , y ) and F ( x , y ) on I 2 with maximum entropy given by Equation (1), under the constraints (2)–(4) are f ( x , y ) = f 1 ( x ) + f 2 ( y ) 1 and
F ( x , y ) = y F 1 ( x ) + x F 2 ( y ) x y , x , y I
for appropriate functions F 1 and F 2 , and with a maximum entropy (ME) copula
C ( u , v ) = u F 2 1 ( v ) + v F 1 1 ( u ) F 2 1 ( v ) F 1 1 ( u ) , u , v I .
Not every choice of cumulative distribution functions F 1 and F 2 in (5) yields a bivariate distribution function F . For example, let F 1 ( x ) = x 3 and F 2 ( y ) = y 3 . Then f ( x , y ) = 3 x 2 + 3 y 2 1 , which equals 1 3 when x = y = 1 3 .
Since the only maximum entropy copula for BGS entropy is the product copula, the entropy (1) identifies models of greater flexibility in terms of dependence structure than models derived from BGS entropy.
Given real functions f , g : I R , define
α = inf { f ( u ) : u A } , β = sup { f ( u ) : u A } , where A = { u I : f ( u ) exists } ,
γ = inf { g ( v ) : v B } , δ = sup { g ( v ) : v B } , where B = { v I : g ( v ) exists } ,
and consider the function C given by
C ( u , v ) = u v + f ( u ) g ( v ) , for all u , v I .
According to Theorem 2.3 in [5], C in (9) is a copula, if and only if (i) f ( 0 ) = f ( 1 ) = g ( 0 ) = g ( 1 ) = 0 ; (ii) f and g are absolutely continuous and (iii) min { α δ , β γ } 1 , where α , β , γ and δ are introduced in Equations (7) and (8). Furthermore, in such a case, C is absolutely continuous. We call a copula given by Equation (9) a RU (Rodríguez-Lallena and Úbeda-Flores) copula.

3. Results

We initiate this section with a result, showing that every ME copula (Equation (6)) has the form of a RU copula (Equation (9)). Since each inverse cumulative distribution function F i 1 is absolutely continuous and nondecreasing on its domain I, the inverse F i of F i 1 is an absolutely continuous distribution function on I. We find necessary and sufficient conditions on the functions F 1 and F 2 (in terms of the derivatives of their inverses) for the ME copula in Equation (6) to be a RU copula. These conditions also insure that the functions F 1 and F 2 yield a proper joint distribution function in (5).
Theorem 1. 
i
Every ME copula has the functional form of a RU copula.
ii
Let F 1 1 ( u ) and F 2 1 ( v ) be inverse distribution functions on I, and set A = { u I : d d u F 1 1 ( u ) e x i s t s } , B = { v I : d d v F 2 1 ( v ) e x i s t s } , m 1 = inf { d d u F 1 1 ( u ) : u A } , M 1 = sup { d d u F 1 1 ( u ) : u A } , m 2 = inf { d d v F 2 1 ( v ) : v B } , M 2 = sup { d d v F 2 1 ( v ) : v B } . Then the function C ( u , v ) in Equation (6) is a copula-in fact a RU copula-if and only if max { ( M 1 1 ) ( M 2 1 ) , ( m 1 1 ) ( m 2 1 ) } 1 .
Proof. 
i
Equation (6) is equivalent to
C ( u , v ) = u v + [ u F 1 1 ( u ) ] [ F 2 1 ( v ) v ] ,
which has the form of a RU copula with f ( u ) = u F 1 1 ( u ) and g ( v ) = F 2 1 ( v ) v .
ii
Since f ( u ) = 1 d d u F 1 1 ( u ) and g ( v ) = d d v F 2 1 ( v ) 1 , α = 1 M 1 = inf { f ( u ) : u A } , β = 1 m 1 = sup { f ( u ) : u A } , γ = m 2 1 = inf { g ( v ) : v B } , and δ = M 2 1 = sup { g ( v ) : v B } . The condition min { α δ , β γ } 1 given in Theorem 2.3 [5] is equivalent to max { ( M 1 1 ) ( M 2 1 ) , ( m 1 1 ) ( m 2 1 ) } 1 , and the conclusion follows.
 ☐
Example 1. 
Consider the function C ( u , v ) = v u a + 1 + u v b + 1 u a + 1 v b + 1 for a , b 0 , which has the form of Equation (6) with F 1 1 ( u ) = u a + 1 and F 2 1 ( v ) = v b + 1 . It follows from differentiation that m 1 , m 2 = 0 or 1 , M 1 = a + 1 and M 2 = b + 1 . Hence C is a copula if and only if a b 1 . C also has the form u v [ 1 ( 1 u a ) ( 1 v b ) ] , a generalized Farlie–Gumbel–Morgenstern copula.
Example 2. 
Let F 1 1 ( u ) = 3 u 2 2 u 3 and F 2 1 ( v ) = ( 1 + θ ) v θ ( 3 v 2 2 v 3 ) on I , with θ [ 1 , 2 ] . Here m 1 = 0 , M 1 = 3 2 . In addition m 2 = 0 , M 2 = 3 , since g ( v ) = θ f ( v ) (see Proof of Theorem 1) and for 1 θ 2 the supremum and the infimum of g ( v ) will occur when θ = 1 or θ = 2 . So m 2 = 1 + min { f ( 0 ) , f ( 1 ) , 2 f ( 1 2 ) } = 1 + ( 1 ) = 0 and M 2 = 1 + max { f ( 1 2 ) , 2 f ( 0 ) , 2 f ( 1 ) } = 1 + 2 = 3 . Hence max { ( M 1 1 ) ( M 2 1 ) , ( m 1 1 ) ( m 2 1 ) } = 1 . In this case the functions C in Equation (10) are C ( u , v ) = u v + θ u v ( 1 u ) ( 1 v ) ( 1 2 u ) ( 1 2 v ) , copulas with cubic sections in Example 3.15 in [6].
We now state and prove a converse of Theorem 1 by showing that every RU copula is a ME copula.
Theorem 2. 
Let C ( u , v ) = u v + f ( u ) g ( v ) be a RU copula. Then there exists a constant k > 0 such that setting F 1 1 ( u ) = u k f ( u ) and F 2 1 = 1 k g ( v ) + v yields the ME copula C ( u , v ) = u F 2 1 ( v ) + v F 1 1 ( u ) F 2 1 ( v ) F 1 1 ( u ) .
Proof. 
Let C ( u , v ) = u v + f ( u ) g ( v ) be a RU copula. We need to show that there exists a k > 0 such that G 1 ( u ) = u k f ( u ) and G 2 ( v ) = 1 k g ( v ) + v are inverse distribution functions on I. Clearly, from Theorem 2.3 in [5], G 1 ( 0 ) = G 2 ( 0 ) = 0 and G 1 ( 1 ) = G 2 ( 1 ) = 1 , so it suffices to show that G 1 ( u ) = 1 k f ( u ) 0 and G 2 ( v ) = 1 k g ( v ) + 1 0 for some k > 0 . Since min { α δ , β γ } 1 from Theorem 2.3 in [5], both α δ 1 and β γ 1 . But β > 0 and γ < 0 imply that 0 < γ 1 β , hence there exists a k > 0 such that γ k 1 β . Thus f ( u ) β implies that k f ( u ) k β 1 , and hence G 1 ( u ) 0 . Similarly g ( v ) γ implies that 1 k g ( v ) 1 k γ 1 , and hence G 2 ( v ) 0 . Thus G 1 ( u ) and G 2 ( v ) are inverse distribution functions on I. A similar result follows from α δ 1 setting G 1 ( u ) = k f ( u ) + u and G 2 ( v ) = v 1 k g ( v ) .  ☐
Example 3. 
The function C ( u , v ) = u v u v ( 1 u ) ( 1 v ) ( 1 + u ) ( 1 + v ) is a RU copula with f ( u ) = u ( 1 u ) 1 + u and g ( v ) = v ( 1 v ) 1 + v , with α = 1 , β = 1 2 , γ = 1 2 , and δ = 1 . Setting k = 1 yields F 1 1 ( u ) = u f ( u ) = 2 u 1 + u and F 2 1 ( v ) = v + g ( v ) = 2 v 1 + v . Thus F 1 ( x ) = x 2 x , F 2 ( y ) = y 2 y , and the maximum entropy bivariate cumulative distribution function F ( x , y ) with margins F 1 ( x ) and F 2 ( y ) , is F ( x , y ) = x y ( x + y x y ) ( 2 x ) ( 2 y ) (see Equation (5)) with T 2 ( F ) = 1 3 . As a point of comparison, in the independence case we have T 2 ( F 1 F 2 ) = 13 36 < 1 3 . The functions F 1 ( x ) and F 2 ( y ) are not unique. If we set F 1 1 ( u ) = u + f ( u ) = 2 u 2 1 + u and F 2 1 ( v ) = v g ( v ) = 2 v 2 1 + v we obtain F 1 ( x ) = 1 4 ( x + x 2 + 8 x ) , F 2 ( y ) = 1 4 ( y + y 2 + 8 y ) , and a different F ( x , y ) with T 2 ( F ) = .
Example 4. 
Let F 1 1 ( u ) = u + a 2 u ( 1 u ) and F 2 1 ( v ) = v a v ( 1 v ) on I with a [ 1 , 1 ] . Then the copula C in Equation (6) with maximum entropy is C ( u , v ) = u v + a 3 u v ( 1 u ) ( 1 v ) , a Farlie–Gumbel–Morgenstern copula with parameter θ = a 3 . Note that m 1 = 1 a 2 , M 1 = 1 + a 2 , m 2 = min { 1 a , 1 + a } , and M 2 = max { 1 a , 1 + a } . The joint distribution F given by Equation (5) has entropy T 2 ( F ) = 2 1 2 a 2 ln 1 + a 2 ( 1 + a ) 1 a ( 1 a ) 1 + a for a 0 , and T 2 ( F ) = 0 for a = 0 , where F 1 ( x ) = 1 + a 2 1 + 2 a 2 + a 4 4 a 2 x 2 a 2 , f 1 ( x ) = 1 1 + 2 a 2 + a 4 4 a 2 x and F 2 ( y ) = a 1 + ( a 1 ) 2 + 4 a y 2 a , f 2 ( y ) = 1 1 2 a + a 2 + 4 a y .
The next example is an application to real data. We use a Farlie–Gumbel–Morgenstern copula to model the dependence between two indicators, under this model we test the independence assumption between such indicators, since it is the solution indicated by the BGS entropy. The probability density function in the Farlie–Gumbel–Morgenstern copula case is c ( u , v ) = 1 + θ ( 1 2 u ) ( 1 2 v ) , u , v I with θ [ 1 , 1 ] . The Farlie–Gumbel–Morgenstern copulas are solutions that maximize THC entropy in certain cases (see Example 4). Because we use a data set with moderate sample size, to test the independence, we apply the Full Bayesian Significance Test (FBST) [7], a Bayesian approach especially designed to test precise hypothesis. The aim of this application is to use the family of ME (or RU) copulas derived from THC entropy in Example 4 to test the hypothesis of the independence assumption.
Example 5. 
The database is composed by paired observations { ( x i , y i ) } i = 1 n coming from the bivariate vector ( X , Y ) . The observations are taken from each one of the 50 American states and the District of Columbia. The observations of X are “Gonorrhea Rates per 100,000 Population” collected for each of the n=51 units by the Centers for Disease Control and Prevention in 2011. The data can be obtained from Table 14 in [8]. See also [9]. The observations of Y are records of poverty “400% FPL and over” records the upper section of the distribution of the total population by Federal Poverty Level (FPL) for the 51 units, in 2011. The data can be obtained from the Kaiser Family Foundation [10]. The hypothesis of independence in the case of Farlie–Gumbel–Morgenstern copula is translated to test θ = 0 , which leads naturally to apply the procedure FBST as follows: H 0 : θ = 0 versus H 1 : θ 0 , compute E v = 1 T g ( θ | D ) d θ , where T = { θ : 1 θ 1 and g ( θ | D ) > g ( 0 | D ) } and D : = { ( u i , v i ) } i = 1 n , u i = r a n k ( x i ) n , v i = r a n k ( y i ) n , i = 1 , , n , with posterior density on θ given by g ( θ | D ) , θ [ 1 , 1 ] . In FBST, H 0 is rejected when E v is small, for details about this methodology see [11]. In order to explore some possibilities we compare the results using a prior density on θ proportional to ( 1 + θ 2 ) a 1 ( 1 θ 2 ) b 1 with a and b hyperparameters of the distribution. We fixed the hyperparameters to consider five settings (i) uniform distribution on [ 1 , 1 ] , (ii) prior density with a mode on θ = 0 , (iii) prior density with a mode on θ = 0 . 5 , (iv) prior density with a mode on θ = 0 . 5 , (v) prior density with a mode near to θ = 1 . This last setting is included in concordance with the Spearman’s coefficient observed in the data, ρ s = 0 . 32113 , that indicates a θ value near to −0.96.
According to our results (Table 1), we have evidence to reject the independence assumption under the Farlie–Gumbel–Morgenstern copula model, except in a setting as (iv), in which the prior distribution on θ and the likelihood of the data expose some conflict of information, allowing us to disregard this case.

4. Conclusions

In [2], Pougaza and Mohammad-Djafari studied bivariate distributions with maximum entropy (in the Tsallis–Havrda–Chavát sense with q = 2 ) for the purpose of generating new families of copulas, which we call ME copulas. In this paper we show that every ME copula coincides with a member of a family studied by Rodríguez-Lallena and Úbeda-Flores in [5], copulas we call RU copulas. Furthermore, we show that every RU copula is a ME copula, and provide illustrative examples.

Acknowledgments

The authors gratefully acknowledge the financial support provided by FAEPEX-PRP of University of Campinas. The authors also wish to express their gratitude to three referees and an associate editor for their helpful comments on an earlier draft of this paper.

Author Contributions

The authors of this paper jointly conceived the idea for this paper, discussed the agenda for the research, performed the theoretical and numerical calculations, and prepared each draft of the paper. All authors have read and approved the final manuscript.

Conflicts of Interest

The authors declare no conflict of interest.

Abbreviations

The following abbreviations are used in this manuscript:
BGS
Boltzmann–Gibbs–Shannon
THC
Tsallis–Havrda–Chavát
ME
maximum entropy
RU
Rodríguez-Lallena and Úbeda-Flores
FBST
Full Bayesian Significance Test

References

  1. Tsallis, C. Possible generalization of Boltzmann–Gibbs statistics. J. Stat. Phys. 1988, 52, 479–487. [Google Scholar] [CrossRef]
  2. Pougaza, D.B.; Mohammad-Djafari, A. Maximum entropies copulas. In Proceedings of the 30th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering, Chamoni, France, 4–9 July 2011; Volume 1305, pp. 329–336.
  3. Ellerman, D. Counting distinctions: On the conceptual foundations of Shannon’s information theory. Synthese 2009, 168, 119–149. [Google Scholar] [CrossRef]
  4. Pougaza, D.B.; Mohammad-Djafari, A. New copulas obtained by maximizing Tsallis or Rényi entropies. In Proceedings of AIP Conference, Garching, Germany, 15–20 July 2012; p. 238.
  5. Rodríguez-Lallena, J.A.; Úbeda-Flores, M. A new class of bivariate copulas. Stat. Probab. Lett. 2004, 3, 315–325. [Google Scholar] [CrossRef]
  6. Nelsen, R.B. An Introduction to Copulas; Springer: Berlin/Heidelberg, Germany, 2006. [Google Scholar]
  7. De Bragança Pereira, C.A.; Stern, J.M. Evidence and credibility: Full Bayesian significance test for precise hypotheses. Entropy 1999, 1, 99–110. [Google Scholar] [CrossRef]
  8. 2011 Sexually Transmitted Disease Surveillance. Available online: http://www.cdc.gov/std/stats11/ (accessed on 15 July 2016).
  9. Table 14. Gonorrhea—Reported Cases and Rates by State/Area and Region in Alphabetical Order, United States and Outlying Areas, 2007–2011. Available online: http://www.cdc.gov/std/stats11/tables/14.htm (accessed on 15 July 2016).
  10. Distribution of the Total Population by Federal Poverty Level (above and below 400% FPL). Available online: http://kff.org/other/state-indicator/population-up-to-400-fpl/ (accessed on 15 July 2016).
  11. Madruga, M.R.; Esteves, L.G.; Wechsler, S. On the bayesianity of Pereira-Stern tests. Test 2001, 10, 291–299. [Google Scholar] [CrossRef]
Table 1. Evidence against H 0 : θ = 0 , prior density on θ ( 1 + θ 2 ) a 1 ( 1 θ 2 ) b 1 .
Table 1. Evidence against H 0 : θ = 0 , prior density on θ ( 1 + θ 2 ) a 1 ( 1 θ 2 ) b 1 .
SettingabEv
(i)110.02686
(ii)220.05580
(iii)240.01702
(iv)420.28670
(v)2100.00050

Share and Cite

MDPI and ACS Style

García, J.E.; González-López, V.A.; Nelsen, R.B. The Structure of the Class of Maximum Tsallis–Havrda–Chavát Entropy Copulas. Entropy 2016, 18, 264. https://doi.org/10.3390/e18070264

AMA Style

García JE, González-López VA, Nelsen RB. The Structure of the Class of Maximum Tsallis–Havrda–Chavát Entropy Copulas. Entropy. 2016; 18(7):264. https://doi.org/10.3390/e18070264

Chicago/Turabian Style

García, Jesús E., Verónica A. González-López, and Roger B. Nelsen. 2016. "The Structure of the Class of Maximum Tsallis–Havrda–Chavát Entropy Copulas" Entropy 18, no. 7: 264. https://doi.org/10.3390/e18070264

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop