Essential Conditions for the Full Synergy of Probability of Occurrence Distributions

In this contribution, we specify the conditions for assuring the validity of the synergy of the distribution of probabilities of occurrence. We also study the subsequent restriction on the maximal extension of the strict concavity region on the parameter space of Sharma–Mittal entropy measures, which has been derived in a previous paper in this journal. The present paper is then a necessary complement to that publication. Some applications of the techniques introduced here are applied to protein domain families (Pfam databases, versions 27.0 and 35.0). The results will show evidence of their usefulness for testing the classification work performed with methods of alignment that are used by expert biologists.


Introduction
We have been working since 2015 on the problem of testing the alignment of protein domain families which are proposed by expert biologists and bioinformaticians. We have found that the use of selected entropy measures is very proficient for testing the results published by those professionals and they favour a rigorous ANOVA statistical analysis [1]. In order to reduce the search space for admissible values of entropy measures, we have emphasized the need for work in the region related to strict concavity of these entropies. This study has been undertaken in a previous work, and we present in Section 2 a summary of those developments. In the present work, we aim to complement the results of a previous publication [2], and a subsequent restriction on the parameter space has to be performed in order to guarantee the synergy of the probability distributions to be tested. Non-synergetic distributions are not worthwhile for working because they will not preserve the fundamental property of getting more information of amino acids into t-sets of columns than to sum up the information obtained from individual columns. In Section 3, a brief digression is then made for introducing the Sharma-Mittal class of entropy measures. Section 4 emphasizes the aspects of synergy of the distributions and their consequences for the reduction of the parameter space of Sharma-Mittal entropies. In Section 5, we treat the analysis of the maximal extension of the parameter space, and we repeat the reduction process imposed by the requirement of fully synergetic distributions of Section 4. We conclude the paper in Section 6 by studying the relation of Hölder and generalized Khinchin-Shannon (GKS) inequalities.

The Construction of the Probabilistic Space
Let us consider a set of m f domains (m f rows) from a chosen family of protein domains. In order to associate a rectangular array with this family, to be taken as its representative in the probabilistic space we are constructing, we specify its number of columns as n f = n. This means that among m f rows, we disregard all rows such that the number of their amino acids satisfies n f < n and preserve m f rows whose number of amino acids satisfies n f ≤ n, p j 1 ...j t (a 1 , . . . , a t ) = n j 1 ...j t (a 1 , . . . , a t ) m , where n j 1 ...j t (a 1 , . . . , a t ) stands for the number of occurrences of the set a 1 , . . . , a t in the t columns of the subarray m × t of the representative array m × n (1 ≤ t ≤ n). The symbols a 1 , . . . , a t will be running over the letters of the one-letter code for the twenty amino acids: We also introduce the conditional probabilities of occurrence, which are given implicitly by where p j 1 ...j t (a 1 , . . . , a t−1 |a t ) is the probability of occurrence of the amino acids in the columns j 1 , . . . , j t−1 , if the distribution of amino acids in the j t -th column is known a priori. The Bayes' law for probabilities of occurrence [2,3] can be written as = p j t ...j 3 j 1 j 2 (a t , . . . , a 3 |a 1 , a 2 )p j 1 j 2 (a 1 , a 2 ) = p j t ...j 2 j 1 (a t , . . . , a 2 |a 1 )p j 1 (a 1 ) = p j t ...j 1 (a t , . . . , a 1 ) .
The equality of the three first right-side members, as well as the equality of the three last ones, does correspond to the application of Bayes' law [2,3]. The symmetries for the joint probability distribution p j 1 ...j t (a 1 , . . . , a t ) are due to the ordering of the columns for the distributions of amino acids. From the ordering j 1 < j 2 < . . . < j t , the values assumed by the variables j 1 , . . . , j t are respectively given by We then have ( n t ) = n! t!(n−t)! geometric objects p j 1 ...j t (a 1 , . . . , a t ) of t columns and (20) t components each.

The Sharma-Mittal Class of Entropy Measures
As emphasized in Ref. [2], the introduction of random variable functions such as entropy measures associated with the probabilities of occurrence, is suitable to provide an analysis of the evolution of these probabilities through the regions of the parameter space of entropies. The class of Sharma-Mittal entropy measures seems to be particularly adapted to this task when related to the occurrence of amino acids in the objects p j 1 ...j t (a 1 , . . . , a t ). The thermodynamic interpretation of the notion of entropy greatly helps to classify the distribution of its values associated with protein domain databases and to interpret its evolution through the Fokker-Planck equations to be treated in forthcoming articles in this line of research.
The two-parameter Sharma-Mittal class of entropy measures is usually given by where α (s) The parameters r, s must bound a region corresponding to a strict concavity in the parameter space. A necessary requirement to be satisfied [3] is wherep j 1 ...j t (a 1 , . . . , a t ) stands for the escort probability associated with the joint probability p j 1 ...j t (a 1 , . . . , a t ), or, p j 1 ...j t (a 1 , . . . , a t ) = p j 1 ...j t (a 1 , . . . , a t ) Equation (8) leads to r ≥ s > 0 .
Some special cases of one-parameter entropies are commonplace in the scientific literature [3][4][5][6][7][8][9]: The r = s region is the domain of the Havrda-Charvat [6] entropy measure H (s) The r = 2 − s, 0 ≤ s ≤ 1, region will stand for the domain of the Landsberg-Vedral [7] entropy measure, L (s) The Renyi R After using the definition of α (s) j 1 ...j t , Equation (7), and lim s→1 α (s) (1) and (2), we get: where S j 1 ...j t is the Gibbs-Shannon entropy measure The Gibbs-Shannon entropy measure, Equation (15), is also obtained by taking the convenient limits of the special cases of Sharma-Mittal entropies, Equations (11)-(14): We shall analyse in the next section the structure of the two-parameter space of Sharma-Mittal entropy by taking into consideration these special cases.
We are now reminded that for the limit of Gibbs-Shannon entropy, a conditional entropy measure is defined [3] by We then have analogously for the conditional Sharma-Mittal entropy measure [3] (SM) j 1 ..
It is easy to show by trivial calculation that, analogously to Equation (16), we will have From Equations (6), (7) and (18) and the application of the Bayes' law, Equation (4), we can write Entropy 2022, 24, 993 5 of 16

Aspects of Synergy and the Reduction of the Parameter Space for Fully Synergetic Distributions
For the Gibbs-Shannon entropy measure, the inequality written by A. Y. Khinchin [3,10] is This inequality would be described by Khinchin as: "On the average, the knowledge a priori of the distribution on the column j t can only decrease the uncertainty of the distribution on the j 1 , . . . , j t−1 columns". We can write an analogous inequality for the Sharma-Mittal class of entropies We then get from Equations (20) and (22) After iteration of this equation, The inequalities in (21)-(24) are associated with what are called "synergetic conditions". In this section, we also derive the fully synergetic conditions as GKS inequalities. After using Equations (7) and (9) in Equation (23), we get and after iteration and use of Equation (24) The hatched region of strict concavity in the parameter space of Sharma-Mittal entropies, C = {(s, r) | r ≥ s > 0}, is depicted in Figure 1. The special cases corresponding to Havrda-Charvat's (r = s), Landsberg-Vedral's (r = 2 − s), Renyi's (r = 1), and "nonextensive" Gaussian's (s = 1) entropies are also represented.
We can identify three subregions in Figure 1. They will correspond to where the ordering of α-symbols has been obtained from Equation (26). The subregions R I and R III are what we call fully synergetic subregions, and the corresponding inequalities are the GKS inequalities [2]. The subregions R I , R II , and R III are depicted in Figure 2a-c, respectively. The union of subregions R I and R III is the fully synergetic Khinchin-Shannon restriction to be imposed on the strict concavity region of Figure 1 and it is depicted in Figure 2d below.

The Maximal Extension of the Parameter Space and Its Reduction for Fully Synergetic Distribution
In Figures 1 and 2d, we have depicted the structure of the strict concavity region for Sharma-Mittal entropy measures and its reduction to a subregion by the application of the requirement of fully synergetic distributions, respectively. Our analysis has used a coarse-grained approach to concavity given by Equations (8) and (10). We now introduce some necessary refinements for characterizing the probability of occurrence in subarrays of m rows and t columns, m × t. For t columns, there are (20) t possibilities of occurrence of amino acids, which could be a large number, but we could count not individual amino acids, but groups of t-sets of amino acids (µ-groups) which appear on the m rows of the m × t array. We characterize these µ-groups by µ = 1, . . . , m, from all equal µ-groups (µ = 1) to m different µ-groups (µ = m). We also call q µ , the number of equal t-sets of a given µ-group.
In Equation (2), the sum is over all the amino acids that make up the geometric object defined in Equation (1), the probability of occurrence. We can now perform the sum over µ-groups and write where a q µ 1 , . . . , a q µ t are the t-sets of a µ-group. We also have from Equation (7) ∑ a The principal minors are given by We then have det H q µ q ν (µ, ν = 1, . . . , k) < 0 , k odd; > 0 , k even.
This does correspond to the criterion of negative definiteness of the Hessian matrix for strict concavity of multivariate functions [11]. Each k-value is associated with the k-epigraph region, which is the k-extension of the strict concavity region presented in Figure 1. These regions are given by The greatest lower bound of the sequence of k-curves is given by σ m (s) = 1. We then have We can then write for the maximal extended region of strict concavity The region corresponding to Equation (41) is depicted in Figure 3 below.
We can then write for the maximal extended region of strict concavity: The region corresponding to Equations (41) is depicted in Figure 3 below. We are now ready to undertake the application of restrictions for fully synergetic distributions (validity of GKS inequalities) to the maximal strict concavity region of Figure 3.
We start by identifying two regions included in Figure 3. They will be given by These regions are depicted in Figures 4a and 4b, respectively. In order to find the reduced region corresponding to Figure 3, analogously to what has been done for Figure 1, we also need the subregions R I , R III , Equations (27), (29): the resulting subregion of fully synergetic distributions is given by R IV ∪ R I ∪ R III and is depicted in Figure 5.

Hölder Inequalities and GKS Inequalities. A Possible Conjecture
In this section, we study the relation between GKS inequalities [2] and Hölder inequalities by using examples of distributions obtained from databases of protein domain families. In order to start, some definitions and properties of the probabilistic space are now in order. We are now ready to undertake the application of restrictions for fully synergetic distributions (validity of GKS inequalities) to the maximal strict concavity region of Figure 3.
We start by identifying two regions included in Figure 3. They will be given by These regions are depicted in Figure 4a,b, respectively.    In order to find the reduced region corresponding to Figure 3, analogously to what has been done for Figure 1, we also need the subregions R I , R III , Equations (27) and (29): the resulting subregion of fully synergetic distributions is given by R IV ∪ R I ∪ R III and is depicted in Figure 5. Let us firstly introduce the definition of the conditional probability of occurrence of the escort probability of occurrence [13]. This is a simple application to escort probabilities of Equation (3):p From the definitions of escort probabilities, Equation (9), we can write, p j 1 ...j t (a 1 , . . . , a t ) = p j 1 ...j t (a 1 , . . . , a t ) and,p In Equations (44)-(46), the symbols a 1 , . . . , a t ; b 1 , . . . , b t assume the representative letters

Hölder Inequalities and GKS Inequalities: A Possible Conjecture
In this section, we study the relation between GKS inequalities [2] and Hölder inequalities by using examples of distributions obtained from databases of protein domain families. In order to start, some definitions and properties of the probabilistic space are now in order.
Let us first introduce the definition of the conditional probability of occurrence of the escort probability of occurrence [12]. This is a simple application to escort probabilities of Equation (3):p From the definitions of escort probabilities, Equation (9), we can writê p j 1 ...j t (a 1 , . . . , a t ) = p j 1 ...j t (a 1 , . . . , a t ) In Equations (44)-(46), the symbols a 1 , . . . , a t ; b 1 , . . . , b t assume the representative letters of the one-letter code for the 20 amino acids, and from Equation (46) p j 1 ...j t (a 1 , . . . , a t−1 |a t ) = p j 1 ...j t (a 1 , . . . , a t−1 |a t ) We also write the definition of escort probability of occurrence of the conditional probability of occurrence [12] p j 1 ...j t (a 1 , . . . , a t−1 |a t ) = p j 1 ...j t (a 1 , . . . , a t−1 |a t ) We can check the definitions of Equations (48) and (49) from the equality of the two escort probabilities with the original conditional probability, for s = 1 s = 1 ⇒p j 1 ...j t (a 1 , . . . , a t−1 |a t ) = p j 1 ...j t (a 1 , . . . , a t−1 |a t ) = p j 1 ...j t (a 1 , . . . , a t−1 |a t ) . (50) We should note that the denominators of the right-hand sides of Equations (48) and (49), or, and X(a t ) ≡ ∑ a 1 ,...,a t−1 will be equal if all amino acids in the j t column are equal. If we have, for instance, the j t column given by: The unit vectors of probabilitiesp j t and p j t will also be equal and given by This means that for this special case of an event of rare occurrence, we also have the equality of the conditional of the escort probability and the escort probability of the conditional probability, or the left-hand sides of Equations (48) and (49), respectively. For a j t -column with a generic distribution of amino acids, the denominators Z and X(a t ) on the right-hand sides of Equations (48) and (49) will no longer be equal. An ordering of these denominators should be decided from the probabilities of amino acid occurrence in a chosen protein domain family.
This study is undertaken with the help of the functions Z and X(a t ) of Equations (51) and (52) and with the functions J and U, defined below: Our method will then be the comparison of pairs of functions in order to proceed with the search for the effect of fully synergetic distributions of amino acids.
There are six comparisons to study:

Concluding Remarks
The first comment we want to make to the present work is about the possibility of working in a region of the parameter space that preserves the strict concavity and the fully synergetic structure of the Sharma-Mittal class of entropy measure distributions to be visited by solutions of a new successful statistical mechanics approach. The usual work with Havrda-Charvat distributions describes the evolution along the boundary (r = s) of the region (r ≥ s > 0) that was correctly considered to correspond to strict concavity, but it is also known to be non-synergetic for s > 1. We now have the opportunity to develop this statistical mechanics approach along an extended boundary, preserving the strict concavity and providing the study of the evolution of fully synergetic entropy distributions. A first sketch of these developments will be presented in a forthcoming publication.
With respect to Figures 6 and 7, we could hypothesize that if the ordering of B and H could not be obtained, this would be due to the poor alignment of some protein domain families we have been using, but we are not confident enough that we could do this, because we would need much more information "in silico" to be obtained from many other protein domain families. In other words, we expect that a good alignment of a protein domain family will result in the ordering of B and H , but we need to verify this in a large number of families from different Pfam versions before we proceed with a proposal of a method to improve the Pfam database. This looks promising for good scientific work in the line of research we have been aiming to introduce in Ref. [2] and in this contribution.

Conflicts of Interest:
The authors declare no conflict of interest.