A Non-Negative Measure of Information for Continuous Probability Distributions

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

See the attached.

Comments for author File: Comments.pdf

Author Response

 

Response to the first referee
----------------------------------------------------------------------

• The referee requested:
• Put a full stop at the end of equations 1, 3, 4 and 5.
• Line 150: Place the full stop at the right position. (modifications between lines 150 and 156)
• Line 185: Place the full stop at the right position.
• Put full stop at the end of line 194.
• Line 195, change “Which” to “which”
• Place full stop at the end of line 219 and 251.
• Place comma at the end of lines, 221, 238 and 254 and everywhere necessary.

Our response: all these points above have been addressed. See modifications in the marked-up version and in the list of major changes.

• The referee questioned
• Section 5; is that “Power law” or “Pareto”?

Our response: It is “power law”. To stress that wee used Pareto’s law as an example, the following statement was added in the line 361: “A typical example of continuous power law is …”.

• The referee wrote: The manuscript mainly motivates varentropy through its ability to remain positive for continuous distributions. However, positivity alone may not be enough to establish a new entropy framework. The authors are encouraged to further discuss the theoretical and practical advantages of varentropy in comparison with existing entropy measures such as Rényi entropy and Tsallis entropy, which have been extensively studied for continuous distributions.

Our response: Thanks for the reminding. Indeed, the advantages of varentropy and its difference from other entropies were not sufficiently addressed. The introduction is modified between the lines 67 and 115, as well as in the section 2 between the lines 164 and 178 in order to highlight the fundamental problems of negative continuous entropies, the difference between varentropy and other continuous entropies. For this purpose, the concluding discussion is also modified in lines 383-406 and 436-437.

Response to the first referee
----------------------------------------------------------------------

• The referee requested:
• Put a full stop at the end of equations 1, 3, 4 and 5.
• Line 150: Place the full stop at the right position. (modifications between lines 150 and 156)
• Line 185: Place the full stop at the right position.
• Put full stop at the end of line 194.
• Line 195, change “Which” to “which”
• Place full stop at the end of line 219 and 251.
• Place comma at the end of lines, 221, 238 and 254 and everywhere necessary.

Our response: all these points above have been addressed. See modifications in the marked-up version and in the list of major changes.

• The referee questioned
• Section 5; is that “Power law” or “Pareto”?

Our response: It is “power law”. To stress that wee used Pareto’s law as an example, the following statement was added in the line 361: “A typical example of continuous power law is …”.

• The referee wrote: The manuscript mainly motivates varentropy through its ability to remain positive for continuous distributions. However, positivity alone may not be enough to establish a new entropy framework. The authors are encouraged to further discuss the theoretical and practical advantages of varentropy in comparison with existing entropy measures such as Rényi entropy and Tsallis entropy, which have been extensively studied for continuous distributions.

Our response: Thanks for the reminding. Indeed, the advantages of varentropy and its difference from other entropies were not sufficiently addressed. The introduction is modified between the lines 67 and 115, as well as in the section 2 between the lines 164 and 178 in order to highlight the fundamental problems of negative continuous entropies, the difference between varentropy and other continuous entropies. For this purpose, the concluding discussion is also modified in lines 383-406 and 436-437.

Response to the first referee
----------------------------------------------------------------------

• The referee requested:
• Put a full stop at the end of equations 1, 3, 4 and 5.
• Line 150: Place the full stop at the right position. (modifications between lines 150 and 156)
• Line 185: Place the full stop at the right position.
• Put full stop at the end of line 194.
• Line 195, change “Which” to “which”
• Place full stop at the end of line 219 and 251.
• Place comma at the end of lines, 221, 238 and 254 and everywhere necessary.

Our response: all these points above have been addressed. See modifications in the marked-up version and in the list of major changes.

• The referee questioned
• Section 5; is that “Power law” or “Pareto”?

Our response: It is “power law”. To stress that wee used Pareto’s law as an example, the following statement was added in the line 361: “A typical example of continuous power law is …”.

• The referee wrote: The manuscript mainly motivates varentropy through its ability to remain positive for continuous distributions. However, positivity alone may not be enough to establish a new entropy framework. The authors are encouraged to further discuss the theoretical and practical advantages of varentropy in comparison with existing entropy measures such as Rényi entropy and Tsallis entropy, which have been extensively studied for continuous distributions.

Our response: Thanks for the reminding. Indeed, the advantages of varentropy and its difference from other entropies were not sufficiently addressed. The introduction is modified between the lines 67 and 115, as well as in the section 2 between the lines 164 and 178 in order to highlight the fundamental problems of negative continuous entropies, the difference between varentropy and other continuous entropies. For this purpose, the concluding discussion is also modified in lines 383-406 and 436-437.

Response to the first referee
----------------------------------------------------------------------

• The referee requested:
• Put a full stop at the end of equations 1, 3, 4 and 5.
• Line 150: Place the full stop at the right position. (modifications between lines 150 and 156)
• Line 185: Place the full stop at the right position.
• Put full stop at the end of line 194.
• Line 195, change “Which” to “which”
• Place full stop at the end of line 219 and 251.
• Place comma at the end of lines, 221, 238 and 254 and everywhere necessary.

Our response: all these points above have been addressed. See modifications in the marked-up version and in the list of major changes.

• The referee questioned
• Section 5; is that “Power law” or “Pareto”?

Our response: It is “power law”. To stress that wee used Pareto’s law as an example, the following statement was added in the line 361: “A typical example of continuous power law is …”.

• The referee wrote: The manuscript mainly motivates varentropy through its ability to remain positive for continuous distributions. However, positivity alone may not be enough to establish a new entropy framework. The authors are encouraged to further discuss the theoretical and practical advantages of varentropy in comparison with existing entropy measures such as Rényi entropy and Tsallis entropy, which have been extensively studied for continuous distributions.

Our response: Thanks for the reminding. Indeed, the advantages of varentropy and its difference from other entropies were not sufficiently addressed. The introduction is modified between the lines 67 and 115, as well as in the section 2 between the lines 164 and 178 in order to highlight the fundamental problems of negative continuous entropies, the difference between varentropy and other continuous entropies. For this purpose, the concluding discussion is also modified in lines 383-406 and 436-437.

Response to the first referee
----------------------------------------------------------------------

• The referee requested:
• Put a full stop at the end of equations 1, 3, 4 and 5.
• Line 150: Place the full stop at the right position. (modifications between lines 150 and 156)
• Line 185: Place the full stop at the right position.
• Put full stop at the end of line 194.
• Line 195, change “Which” to “which”
• Place full stop at the end of line 219 and 251.
• Place comma at the end of lines, 221, 238 and 254 and everywhere necessary.

Our response: all these points above have been addressed. See modifications in the marked-up version and in the list of major changes.

• The referee questioned
• Section 5; is that “Power law” or “Pareto”?

Our response: It is “power law”. To stress that wee used Pareto’s law as an example, the following statement was added in the line 361: “A typical example of continuous power law is …”.

• The referee wrote: The manuscript mainly motivates varentropy through its ability to remain positive for continuous distributions. However, positivity alone may not be enough to establish a new entropy framework. The authors are encouraged to further discuss the theoretical and practical advantages of varentropy in comparison with existing entropy measures such as Rényi entropy and Tsallis entropy, which have been extensively studied for continuous distributions.

Our response: Thanks for the reminding. Indeed, the advantages of varentropy and its difference from other entropies were not sufficiently addressed. The introduction is modified between the lines 67 and 115, as well as in the section 2 between the lines 164 and 178 in order to highlight the fundamental problems of negative continuous entropies, the difference between varentropy and other continuous entropies. For this purpose, the concluding discussion is also modified in lines 383-406 and 436-437.

Reviewer 2 Report

Comments and Suggestions for Authors

The comments are available in the attached file.

Comments for author File: Comments.pdf

Author Response

Response to the second referee 
---------------------------------------------------------------------- 
 

• The referee wrote: The authors treat negative differential entropy as a fundamental inconsistency without adequately discussing the modern interpretation of continuous entropy. The text would be more robust if it explicitly acknowledged that negativity can be accepted in information theory, even if this may be undesirable in certain contexts. Varentropy should be presented as a complementary alternative to differential entropy. 

Response:  We accept the referee’s suggestion and have added some statements in lines 67-115, 127-129 and lines 383-386, to detail more the features of continuous entropies and the open questions, and for the reader to see the long history of continuous entropies accepted by many physicists and some of their applications. Ref [21] to [26] are added for this purpose.

• The referee wrote: The manuscript implicitly assumes that any random variable can play the same role as energy in the variational formulation of entropy. However, in thermodynamics, energy possesses specific physical properties: it is associated with conservation laws, has a clear mechanical interpretation, and is directly related to thermodynamic equilibrium.

1) By replacing ?? with an arbitrary variable ??, the authors should clearly define: I) Which properties of energy remain valid after this substitution? II) Does the new quantity still possess a thermodynamic interpretation? Does this analogy have physical meaning? The manuscript does not sufficiently discuss these limitations.

Response:  Some discussions have been added to clarify these points in the lines 188-201. If the random variable is not energy, varentropy has nothing to do with thermodynamics.

• The referee wrote:

The definition of varentropy appears to derive from a particular choice of variational form. I) Could other equally valid generalizations exist? II) Could other functionals be derived from the first law of thermodynamics? III) Are there other possible extensions for continuous variables? IV) Why should this particular choice be preferred? V) Which criteria justify its adoption?

Response:  The definition is not a particular choice among many possible. The form  is a necessity if we follow the first law of equilibrium thermodynamics and write it in statistical form, as shown in lines 179-188. We have added a sentence in lines 188-190 to highlight this important point: “From above calculation, we see that this variational expression of thermodynamic entropy as a function of  multiplied by the random variable energy  is just a statistical form of the first law.”

• The referee wrote: A central property of Shannon entropy is concavity. Concavity guarantees stability, a consistent physical interpretation, and increased uncertainty under mixing. Another important property is additivity. Does varentropy satisfy concavity? Is it additive?

Response:  As varentropy is defined by a variational form, its exact functional form is different for different distributions (we have added a sentence in lines 198-200 to underline this characteristic of varentropy:  measures the uncertainty in , and its functional form and property (extensivity, additivity, concavity etc.) are entirely determined by  and .). Hence, this kind of properties can be discussed only when varentropy takes definitive form for given probability distribution as has been done in the references [10,11,12,14]. We have added some statements in lines 212-218 to remind this point: “For example,  takes the form of the Boltzmann-Shannon entropy Eq.(1) when  is exponential distribution,  is the nonadditive Tsallis or additive Renyi entropy if  is q-exponential distribution, and can take different forms for other distributions (Power law, stretched exponential, Cauchy, Gauss etc.),  (nonadditive) for power law for example.”

• The referee wrote : Differential entropy depends on scale and changes under variable transformations. The authors implicitly criticize this behavior, but they do not clearly demonstrate whether varentropy resolves this issue. I) Is it invariant under coordinate transformations? II) Does it depend on parametrization? III) Does it preserve geometric information?

Response:   It will be interesting to study these properties for each given formula of varentropy for given distribution. In general, different forms of  should have different invariant and geometric properties. We didn’t check these aspects in this work. Reference [10] presented a study of scale invariance of some distributions and their varentropies.

In order to highlight the scale invariant property of varentropy, we have added some remarks in lines 275-280 and 286-288 for  of uniform distribution, and lines 306-307 for exponential distribution. The other formulas of  for other distributions have the same scaling property. 

• The referee wrote: The point requiring the greatest attention from the authors in this section is a clearer definition of the role of the constant ?. The authors explicitly state that ? must be chosen in such a way as to guarantee positivity. Given this, is positivity truly an intrinsic property of varentropy, or is it merely imposed artificially through the choice of ?? In later examples, ? = 1 or ? = −1 appears to be selected according to necessity. This raises the question of whether the sign of the entropy emerges naturally from the theory or whether it is simply an arbitrary adjustment. The manuscript should clarify the role of this constant in the proposed generalization.

Response:   Indeed, the different choice of A as well as C for different varentropies gives the impression that the positivity is something artificial and ad hoc. But if we consider the absence of such parameters in the different continuous entropies to make them positive, A and C are rather an advantage of varentropy. It is worth noticing that once A and C are chosen for a given varentropy with a given formula with a given distribution, they do not change anymore. On the other hand, it is impossible to do this for other continuous entropies to make them positive. For example, for the continuous entropy Eq.(2), It is impossible to write a given A times the formula to make it always positive because the formula is sometimes positive sometime negative. It is impossible to choose a given C to add to the formula to make it always positive because the formula can go to minus infinity. To highlight this point, we have added a discussion of two examples of continuous entropy in lines 96-108, with a remark in lines 417-430 in concluding discussion.

Indeed, if one wanted to make the continuous entropy of uniform distribution positive, he would have to change A and C constantly for a given formula. From this point of view, a given A and a given C for a given formula are acceptable.

On the other hand, A also plays the role of a Lagrange multiplier in the maximization of varentropy to generate probability distribution. In general, different distributions can have different multipliers, depending on the nature of the distributions. Some discussion has been added in lines 243-246 in this sense.

 

• The referee wrote: The authors claim that ?? can be maximized for any distribution, unlike Shannon entropy. This is an extremely strong statement. In my opinion, the manuscript does not provide sufficient conceptual foundation to support such a claim.

Response:  We agree that the statement “maximizable for any distribution” is a little bit too strong. We have changed it to “maximizable to generate probability distribution” (line 165) and added some precisions about these words in lines 165-178, and 223-235.

 

 

• The referee wrote: Many of the issues attributed by the authors to differential entropy are already addressed through relative entropies or through the introduction of reference measures. The authors should discuss why varentropy would be preferable. What formal advantages does it possess, and in which contexts does it outperform existing measures?

At several points, the manuscript implicitly suggests that varentropy would constitute a “better” or more “universal” measure. Again, in my opinion, the authors overstate their conclusions without sufficient general evidence. A more appropriate tone would be to describe varentropy as an “alternative framework” or a “possible extension.”

At present, varentropy may be regarded more as a promising exploratory proposal than as a replacement for traditional continuous entropy.

However, the section also clearly exposes the main weaknesses of the proposal, such as the seemingly arbitrary dependence of the constants A and C, the absence of a clear physical interpretation, and the lack of demonstrated general properties.

Furthermore, it is worth mentioning the increasing complexity of the expressions and the absence of evidence of superiority beyond positivity.

Varentropy seems viable, but it does not yet convincingly establish its necessity or universality as a fundamental measure of continuous information.

The discussion adequately summarizes the objectives and results of the article and suggests potentially interesting applications in statistical physics and quantum mechanics. However, it also highlights the main conceptual limitations of the proposal.

The defense of varentropy continues to rely almost exclusively on the positivity of the measure, without convincingly demonstrating that the negativity of differential entropy constitutes, in fact, a fundamental problem in modern continuous information theory.

Thus, the discussion reinforces the idea that varentropy may be an interesting mathematical alternative, but it still does not establish its necessity as a fundamental replacement for traditional continuous entropy.

Response:  The relative entropy or Kulback-Leibler divergence is a solution to problem of the negative entropy; we have added a statement in lines 113-115 to highlight this interesting approach, although the conceptual problems of entropy itself is always there.

We accept the suggestion of the referee and have made modification in lines 128-129 to stress the objective of the work is to present an alternative uncertainty measure which can help to avoid some undesirable features of continuous entropy.

The concluding discussion of section 4 is rewritten with additional statements in lines 383-406 and 436-437, including a list of reasons for which varentropy may be a good candidate for the measure of uncertainty of continuous distributions, a candidate to be confirmed by further investigation.

 

• In addition, I would like to mention the issue of the similarity of this text to another text published by the same authors. iThenticate identified a 26% match. Qiuping Alexandre Wang, Qiong Ye. Derivation of the maximum entropy principle from the virtual work principle. The European Physical Journal Plus, 2025, 140, pp. 1234. Doi:10.1140/epjp/s13360-025-07174-3. Only one of the authors of the published text coincides with the authors of this article (under review). A textual revision is necessary so that the new versions present a more original text without plagiarism.

Response:   The mentioned publication is the reference [13] (updated in this revision) of the present manuscript. As the title indicates, that work focuses on the derivation of the principle Maxent of Jaynes from a principle of virtual work in mechanics. The conclusion is maxent should be considered as a law of physics while using thermodynamic entropy, instead of a fundamental principle on its own, because it can be derived from a well established fundamental principle. There may be some common words, but the aim, the starting point, the mathematical reasoning and the results are 100% different from the present manuscript where the result of that work is used as a support to an intrinsic feature varentropy : each derived formulas of  can be maximized to generate its own distribution, as discussed in lines 229-237.

Round 2

Reviewer 2 Report

Comments and Suggestions for Authors

The authors provided satisfactory answers. The modifications and updates made the manuscript suitable for publication in a journal such as Mathematics.