Tsallis Entropy Theory for Modeling in Water Engineering: A Review

Abstract

Water engineering is an amalgam of engineering (e.g., hydraulics, hydrology, irrigation, ecosystems, environment, water resources) and non-engineering (e.g., social, economic, political) aspects that are needed for planning, designing and managing water systems. These aspects and the associated issues have been dealt with in the literature using different techniques that are based on different concepts and assumptions. A fundamental question that still remains is: Can we develop a unifying theory for addressing these? The second law of thermodynamics permits us to develop a theory that helps address these in a unified manner. This theory can be referred to as the entropy theory. The thermodynamic entropy theory is analogous to the Shannon entropy or the information theory. Perhaps, the most popular generalization of the Shannon entropy is the Tsallis entropy. The Tsallis entropy has been applied to a wide spectrum of problems in water engineering. This paper provides an overview of Tsallis entropy theory in water engineering. After some basic description of entropy and Tsallis entropy, a review of its applications in water engineering is presented, based on three types of problems: (1) problems requiring entropy maximization; (2) problems requiring coupling Tsallis entropy theory with another theory; and (3) problems involving physical relations.

1. Introduction

Water resources systems serve a multitude of human needs. They are needed for water supply, water transfer, water diversion, irrigation, land reclamation, drainage, flood control, hydropower generation, river training, navigation, coastal protection, pollution abatement, transportation and recreation, among others. Many of the systems (e.g., channels, culverts, impoundments) have been with us since the birth of human civilization. Some (e.g., spillways, small dams, levees) are several centuries old, while some others (e.g., large dams, long-distance water transfer structures) are of more recent origin. In the beginning, systems were designed more or less empirically. Then, engineering and economics constituted the sole foundation of design. About fifty years ago, planning and design of hydraulic structures went through a dramatic metamorphosis. These days, they are based on both engineering and non-engineering aspects. Engineering aspects encompass planning, development, design, operation and management, while non-engineering aspects include environmental impact assessment, socio-economic analysis, policy making and impact on society.

For designing water engineering systems (e.g., channels, levees, bridge piers, drainage structures, dams, reservoirs, spillways), some key questions that need to be addressed relate to the following, among others: peak discharge; velocity distribution; sediment yield, concentration and discharge; pollutant load, concentration and transport; river bed profile, meandering and braiding; downstream and at-a-point hydraulic geometry; flow depth, discharge and velocity routing; and seepage through a dam. For water supply systems, the key questions are concerned with, among others: reliability; loss of energy in the distribution system; and pipe sizes. In addition, pollutant concentration and transport as well as pollution abatement are also now considered as essential components of water resources system design. In a similar manner, because of increased public awareness, primarily triggered by environmental movement, non-engineering aspects of hydraulic design, dubbed under “socio-economic analyses,” play a critical role. Besides engineering feasibility and economic viability, issues related to public health, political support, legal and judicial restrictions and social acceptability determine, to a large extent, if the water resources project will go off the ground.

A survey of the water engineering literature shows that there are myriad techniques for addressing questions pertaining to the design of water resources systems. The techniques range from empirical to semi-empirical to physically-based ones. Empirical techniques are data-based; examples are regression, time series analysis and other statistical methods. Semi-empirical methods, also sometimes referred to as conceptual or systems-based techniques, employ mass conservation and some empirical relationship or hypothesis; a good example is the unit hydrograph theory. Physically-based methods employ the laws of conservation of mass, momentum and energy; an example is the use of St. Venant’s equations for flow routing or Richards’ equation for computing infiltration. Strictly speaking, even physically-based methods also employ empirical parameters and, thus, are not entirely “physically-based.” Indeed, all these three types of techniques employ some physics through data or hypotheses or laws. Extensive details of these methods are already well-documented in the literature [1,2,3,4,5,6]. A more recent and comprehensive account of these methods and applications is also presented in [7].

Because of the large diversity of these techniques, based on different hypotheses and assumptions, it is difficult to present the developments in any subject or field of interest in a unified and coherent manner. This becomes particularly challenging when undertaking water resources system engineering design. There are, of course, some theories that do apply to a wide variety of problems, such as kinematic wave theory and diffusion wave theory [8,9]. These theories can be applied to solve a wide variety of problems where the movement of water, sediment and/or pollutant is involved. However, many problems in water engineering design require a statistical treatment. For addressing such problems, entropy theory can serve as a unifying theory. During the past three decades, entropy theory has been applied to address a wide spectrum of problems in water engineering, including rainfall-runoff [1,3,8], infiltration [2], soil moisture [2], network design [10], velocity distributions [11,12,13,14], sediment concentration and discharge [15,16,17], hydraulic geometry [18,19,20,21,22] and reliability [23], among others. For recent comprehensive accounts of entropy theory applications in water and environmental engineering, see [24,25,26].

The origin of entropy theory is in the second law of thermodynamics. Koutsoyiannis [27] has presented a nice account of the historical background of entropy. The most commonly used measure of entropy is the Boltzmann-Gibbs-Shannon (BGS) entropy [28], which is often referred simply as the Shannon entropy [29]. Tsallis [30] introduced a more general entropy function for complex systems, which is now referred to as the Tsallis entropy. Tsallis entropy specializes in the Shannon entropy. During the last two decades or so, Tsallis entropy theory has found many applications in water and environmental engineering and there is certainly a great potential to extend the applications to a much wider spectrum of water systems and associated problems.

This paper aims to provide a review of the applications of Tsallis entropy theory in water engineering. It revisits the Tsallis entropy theory, presents a general methodology for application of the theory, shows how entropy theory couples statistical information with physical laws and how it can be employed to derive useful physical constructs in time and/or space and provides a review of physical applications of the Tsallis entropy theory in water engineering.

The rest of the paper is organized as follows. Section 2 reviews the Tsallis entropy theory. Section 3, Section 4, Section 5 and Section 6 review the applications of the Tsallis entropy theory in water engineering. Section 3 presents an overview of three types of problems in water engineering. Section 4 reviews problems requiring entropy maximization. Section 5 reviews coupling entropy theory with another theory. Section 6 reviews physical relations. Section 7 draws some conclusions.

2. Tsallis Entropy Theory

2.1. Definition of Entropy

The concept of entropy is closely linked with the concept of uncertainty, information, chaos, disorder, surprise or complexity. Indeed, there are often different interpretations of entropy in different fields [24,25,26,31,32,33]. For instance, in statistics, entropy is regarded as a measure of randomness, objectivity or unbiasedness, dependence, or departure from the uniform distribution. In ecology, it is a measure of diversity of species or lack of concentration. In water engineering, it is a measure of information of uncertainty. In industrial engineering, it is a measure of complexity. In manufacturing, it is a measure of interdependence. In management, it is a measure of similarity. In social sciences, it is measure of equality. This may be illustrated as follows.

Consider a random variable X that takes on values $x_i,i=1,2,\dots ,N,$ that occur with probabilities $p_i,i=1,2,\dots ,N.$ If p_i = 1 (the event is certain to occur), p_j = 0 (the event is certain not to occur), $\mathit{i}\ne \mathit{j},$ then it can be said that there is no surprise about the occurrence or non-occurrence of event X = x_i and the occurrence or non-occurrence of this event provides no information. The system that produces such an event has no complexity and is not chaotic or disorderly. On the other hand, if an event x_i occurs with very small probability p_i, say 0.01, then our anticipation of event x_i is highly uncertain and if x_i does indeed occur, then there will be a great deal of surprise about its occurrence. The occurrence of this event provides a great deal of information and the system producing such an event is complex, chaotic and disorderly. Intuitively, the information content of event x_i or the anticipatory uncertainty of x_i prior to the observation decreases as the value of probability p(x_i) increases [24,26]. It may be noted that information is gained only if the variable takes on different values. The value that occurs with a higher probability conveys less information and vice versa.

It is logical to deduce that if a system has more uncertainty, then more information will be needed to characterize it and vice versa. That is, information reduces uncertainty, meaning that, for a system, more information means less uncertainty. Uncertainty increases the need for more information; that is, more uncertainty means more information is needed. Shannon [29] formulated entropy as the expected value of the probabilities of values that a variable or event may take on. The information gained is indirectly measured as the amount of reduction of uncertainty. Thus, entropy is defined as a measure of disorder, chaos, uncertainty, surprise, or information.

From an informational perspective, the information gain from the occurrence of any event x_i, $\Delta H(x_i),$ can be expressed as:

$$\Delta H(x_i)=-\log p_i$$

(1)

Equation (1) says that the information gained is minus the logarithm of the inverse of the probability of occurrence. For N events, the average or expected information gain, H_s, is the weighted average of Equation (1):

$$H_s={\displaystyle \sum _{i=1}^{N}H(x_i)}=-{\displaystyle \sum _{i=1}^N{p}_i\log p_i}$$

(2)

Equation (2) is the Shannon entropy [29], also called informational entropy. Equation (1) shows that the information gain is directly a function of probability and is, hence, called gain function. Equation (1) can be generalized by expressing it as a power function, given by:

$$\Delta I(x_i)=\frac{1}{m-1}(1-p_i{}^{m-1}),{\displaystyle \sum _{i=1}^N{p}_i=1}$$

(3)

where ΔI(x_i) is the gain in information from an event i which occurs with probability p_i and m is any real number. The gain function computed from Equation (3) for m = −1, 0, 1 and 2, as shown in Figure 1, decreases with an increase in the probability value regardless of the value of m. For increasing value of m, the gain function diminishes for the same probability value. The gain function has a much longer tail, showing very low values of gain as the probability increases.

Analogous to Shannon entropy, the average or expected gain function for N events, H_m, is the weighted average of Equation (3), given by:

$$H_m=E[\Delta I_i]={\displaystyle \sum _{i=1}^N{p}_i[\frac{1}{m-1}(1-p_i{}^{m-1})}]=\frac{1}{m-1}{\displaystyle \sum _{i=1}^N{p}_i[1-}{p}_i{}^{m-1}]$$

(4)

where H_m is designated as the Tsallis entropy or m-entropy [30], which is often referred to as the non-extensive statistic, m-statistic, or Tsallis statistic. It can be shown that as $m\to 1,$ Tsallis entropy converges to Shannon entropy. The quantity m is often referred to as the non-extensivity index, Tsallis entropy index, or simply entropy index and its value can be positive or negative. Entropy index m reflects the microscopic dynamics and the degree of nonlinearity of the system. Since almost all real systems (e.g., water systems) are inherently nonlinear in nature, the Tsallis entropy has a clear advantage when compared to the Shannon entropy. Tsallis [34] noted that super-extensivity, extensivity and sub-extensivity occur when m < 1, m = 1 and m > 1, respectively. Interestingly, if $m\ge 0,m<1$ corresponds to the rare events $(0\le m<1)$ and m > 1 corresponds to frequent events [35,36], pointing to the stretching or compressing of the entropy curve to lower or higher maximum entropy positions.

If random variable X is non-negative continuous with a probability density function (PDF), f(x), then the Shannon entropy can be written as:

$$H_s(X)=H_s(f)=-{\displaystyle \underset{0}{\overset{\infty }{\int }}f(x)\log f(x)dx}$$

(5)

Likewise, Tsallis entropy can be expressed as [37,38,39]:

$$H_m(X)=H_m(f)=\frac{1}{m-1}{\displaystyle \underset{0}{\overset{\infty }{\int }}\{f(x)-{[f(x)]}^m\}dx=}\frac{1}{m-1}\{1-{\displaystyle \underset{0}{\overset{\infty }{\int }}{[f(x)]}^m\}dx}$$

(6)

From now onwards, subscript m will be deleted and H_m will be simply denoted by H.

A plot of H versus p for m = −1, −0.5, 0, 0.5, 1 and 2 is given in Figure 2. For m < 0, the Tsallis entropy is concave; and for m > 0, it becomes convex. For m = 0, H = (N − 1) for all p_i’s. For m = 1, it converges to Shannon entropy. For all cases, the Tsallis entropy decreases as m increases.

2.2. Properties of Tsallis Entropy

As mentioned earlier, the concept of entropy is closely linked with the concept of uncertainty, information, chaos, disorder, surprise or complexity. Tsallis entropy has some interesting properties [30,40] that are briefly summarized here.

(1) m-entropy: The m-surprise or m-unexpectedness is defined as log_m(1/p_i), where the logarithm is the base m. Hence, the m-entropy can be defined as:

$$H=E[{\log }_m\frac{1}{p_i}]$$

(7)

which coincides with the Tsallis entropy:

$$H=E[\frac{1-p_i{}^{m-1}}{m-1}]$$

(8)

in which E is the expectation.

(2) Maximum value: Equation (4) attains an extreme value for all values of m when all p_i’s are equal, i.e., p_i = 1/N. For m > 0, it attains a maximum value and for m < 0 it attains a minimum value. The extremum of H becomes

$$H=\frac{N^{m-1}-1}{1-m}$$

(9)

(3) Concavity: For two probability distributions $P=\{p_i,i=1,2,\dots ,N\}$ and $Q=\{q_i,i=1,2,\dots ,N\}$ corresponding to a unique set of N possibilities, an intermediate probability distribution $G=\{g_i,i=1,2,\dots ,N\}$ can be defined for a real a, such that 0 < a < 1, as:

$$g_i=a{p}_i+(1-a)q_i$$

(10)

for all i’s. It can be shown that for m > 0,

$$H[G]\ge aH[P]+(1-a)H[Q]$$

(11)

and for m < 0,

$$H[G]\le aH[P]+(1-a)H[Q]$$

(12)

Functional $H(G)\ge 0$ if m > 0 and is, hence, concave; H(G) = 0 if m = 0; and $H(G)\le 0$ if m < 0 and is, therefore, convex.

(4) Additivity: For two independent systems A and B with ensembles of configurational possibilities $E^A=\{1,2,\dots ,N\}$ with probability distribution $P^A=\{p_i{}^A,i=1,2,\dots ,N\}$ and configurational possibilities $E^B=\{1,2,\dots ,M\}$ with probability distribution $P^B=\{p_j{}^B,j=1,2,\dots ,M\},$ one can express the union of two systems $A\cup B$ and their corresponding ensembles of possibilities $E^{A\cup B}=\{(1,1),(1,2),\dots ,(i,j),\dots ,(N,M)\}.$ If $p_{ij}{}^{A\cup B}$ represents the corresponding probabilities, then, by virtue of independence, the joint probability will be equal to the product of individual probabilities, i.e., $p_{ij}{}^{A\cup B}=p_i{}^A{p}_j{}^B$ or $p_{ij}(A+B)=p_i(A)p_j(B)$, for all i and j. One can write the entropy of the union of two systems A and B as:

$$H(A+B)=H(A)+H(B)+(1-m)H(A)H(B)$$

(13)

Equation (13) describes the additivity property, which can be extended to any number of systems. In all cases, $H\ge 0$ (non-negativity property). If systems A and B are correlated, then

$$p_{ij}{}^{A\cup B}\ne [{\displaystyle \sum _{i=1}^N{p}_{ij}{}^{A\cup B}][{\displaystyle \sum _{j=1}^M{p}_{ij}{}^{A\cup B}]}}$$

(14)

for all (i, j). One may define mutual information of the two systems or transinformation S as:

$$T[(p_{ij}{}^{A\cup B})]=H^{A\cup B}[(p_{ij}{}^{A\cup B})]-H^A[({\displaystyle \sum _{i=1}^N{p}_{ij})]-H^B[({\displaystyle \sum _{j=1}^M{p}_{ij})]}}$$

(15)

Considering Equation (15), T(p_ij) = 0 for all m, if A and B are independent and Equation (15) will reduce to Equation (13). For correlated A and B, T(p_ij) < 0 for m = 1 and $T(p_{ij})=0$ for m = 0. For arbitrary values of m, it will be sensitive to p_ij; it can take on negative or positive values for both m < 1 and m > 1 with no particular regularity and can exhibit more than one extremum.

(5) Composability: The entropy H(A + B) of a system comprised of two sub-systems A and B can be computed from the entropies of sub-systems, H(A) and H(B) and the entropy index m.

(6) Interacting sub-systems: Consider a set of N possibilities arbitrarily separated into two sub-systems with W₁ and W₂ possibilities, where W = W₁ + W₂. Defining $P_{W_1}={\displaystyle \sum _{i=1}^{W_1}{p}_i}$ and $P_{W_2}={\displaystyle \sum _{j=1}^{W_2}{p}_j}$, $P_{W_1}+P_{W_2}=1={\displaystyle \sum _{k=1}^W{p}_k}$, it can be shown that

$$H(P_W)=H(P_{W_1},P_{W_2})+P_{W_1}{}^{m}H(p_i|P_{W_1})+P_{N_2}{}^{m}H(p_j|P_{W_2})$$

(16)

where $p_i|P_{W_1}$ and $p_j|P_{W_2}$ are the conditional probabilities. Note that $p_i{}^m>p_i$ for m < 1 and $p_i{}^m<p_i$ for m > 1. Hence, m < 1 corresponds to rare events and m > 1 to frequent events [41]. This property can be extended to any number R of interacting sub-systems: $W={\displaystyle \sum _{j=1}^R{W}_j}.$ Then, defining ${\mathit{w}}_j={\displaystyle \sum _{i=1}^{N_j}{\mathit{p}}_i},\mathit{j}=1,2,\dots ,{\mathit{N}}_j$, ${\displaystyle \sum _{j=1}^N{\mathit{w}}_j=1,}$ Equation (16) can be generalized as:

$$H(\{p_i\})=H(\{w_j\})+{\displaystyle \sum _{j=1}^R{w}_j{}^{m}H(\{p_i|w_j\})}$$

(17)

Here, $p_j=w_j.$

2.3. Principle of Maximum Entropy

Quite often, some information may be known about the random variable X. Then, it seems logical to choose a PDF of X that is consistent with the known information. Since there can be more than one PDF that may satisfy this condition, Jaynes [42,43] formulated the principle of maximum entropy (POME), which states that one should choose the distribution that has the highest entropy, subject to the given information. This distribution will be the least-biased distribution. Furthermore, it is equivalent to the minimum relative (cross) entropy condition when no prior is given, which is also called as the Kullback-Leibler principle [44]. The implication here is that POME takes into account all of the given information and, at the same time, avoids consideration of any information that is not given. This is consistent with Laplace’s principle of insufficient reason (or principle of indifference), according to which all outcomes of an experiment should be considered equally likely unless there is information to the contrary. Therefore, POME enables entropy theory to achieve the probability distribution of a given random variable.

The procedures for applications of POME entails the following steps [24,26]: (1) definition of Tsallis entropy; (2) specification of constraints; (3) maximization of entropy, in accordance with POME, using of the method of Lagrange multipliers; (4) derivation of the probability distribution in terms of constraints; (5) determination of Lagrange multipliers; and (6) determination of the maximum entropy. Since the definition of entropy has already been reviewed earlier, it will not be repeated here. The remaining steps, (2)–(6), are briefly discussed here.

2.3.1. Specification of Constraints

For deriving the PDF of a random variable X using POME, appropriate constraints need to be defined. Papalexiou and Koutsoyiannis [28] suggested three considerations for defining constraints: (1) Simplicity and physical meaningfulness; (2) little variability in the future; and (3) definition in terms of laws of physics—mass, momentum and energy conservation and constitutive laws—as far as possible. For simplicity, constraints are often defined in terms of statistical moments and, fortunately, the moments are related to the laws of physics.

Let C_i denote the i-th constraint, i = 0, 1, 2, …, n, where n is the number of constraints. Let g_r(x) be an arbitrary function of X. Then, constraints C_r, r = 0, 1, 2, …, n, can be defined as:

$$C_0={\displaystyle \underset{a}{\overset{b}{\int }}f(x)dx=1}$$

(18)

$$C_r={\displaystyle \underset{a}{\overset{b}{\int }}{g}_r(x)f(x)dx=}E[g_r(x)]=\overline{g_r(x)},r=1,2,\dots ,n$$

(19)

where g₀(x) = 0; g_r(x), r = 1, 2, …, n, represents some function of x and $\overline{g_r(x)}$ is the expectation of g_r(x). Equation (18) states the total probability theorem that the PDF must satisfy. If r = 1 (first moment) and g₁(x) = x, then Equation (19) represents the mean $\overline{x}$; likewise, for r = 2 (second moment) and $g_2(x)={(x-\overline{x})}^2$, it denotes the variance (σ²) of X. The first moment corresponds to the conservation of mass and the second moment to the conservation of momentum. Similarly, r = 3 (third moment), measuring skewness, corresponds to the conservation of energy. It is noticed that the order of moments higher than 3 may be unreliable [45]. In water engineering, two or three constraints usually suffice.

2.3.2. Entropy Maximizing Using Lagrange Multipliers

Tsallis entropy, given by Equation (6), can be maximized, subject to constraints defined by Equations (18) and (19), using the method of Lagrange multipliers [42,43]. Therefore, the Lagrangian function L can be expressed as:

$$L={\displaystyle \underset{0}{\overset{\infty }{\int }}f(x)}\frac{1-{[f(x)]}^{m-1}}{m-1}dx+({\lambda }_0-\frac{1}{m-1})[{\displaystyle \underset{0}{\overset{\infty }{\int }}f(x)dx-1]+{\displaystyle \sum _{i=1}^M{\lambda }_i[{\displaystyle \underset{0}{\overset{\infty }{\int }}{g}_i(x)f(x)dx-\overline{g_i(x)}}]}}$$

(20)

where λ_i, i = 0, 1, 2, …, M, are the Lagrange multipliers. Note that −1/(m − 1) is added to the zeroth Lagrange multiplier for algebraic simplification. Using the Euler-Lagrange calculus of variation, differentiating Equation (20) with respect to f(x) and equating the derivative to zero, we obtain:

$$\frac{dL}{df(x)}=0=\frac{1}{m-1}-{[f(x)]}^{m-1}+{\lambda }_0-\frac{1}{m-1}+{\displaystyle \sum _{i=1}^M{\lambda }_i{g}_i(x)}$$

(21)

2.3.3. Determination of Probability Distribution

Equation (21) yields the least-biased probability distribution of X:

$$f(x)=[{\lambda }_0+{\displaystyle \sum _{i=1}^M{\lambda }_i{g}_i(x){]}^{\frac{1}{m-1}}}$$

(22)

Integrating Equation (22), the cumulative distribution function F(x) is obtained as:

$$F(x)={\displaystyle \underset{0}{\overset{x}{\int }}[}{\lambda }_0+{\displaystyle \sum _{i=1}^M{\lambda }_i{g}_i(x){]}^{\frac{1}{m-1}}}dx$$

(23)

The properties of the probability distribution will depend on the value of m, Lagrange multipliers, functions g_i(x) and M.

2.3.4. Determination of the Lagrange Multipliers

Equation (22) contains M unknown Lagrange multipliers that can be determined with the use of Equations (18) and (19). Substituting Equation (22) in Equations (18) and (19), the result is, respectively:

$${\displaystyle \underset{0}{\overset{\infty }{\int }}[}{\lambda }_0+{\displaystyle \sum _{i=1}^M{\lambda }_i{g}_i(x){]}^{\frac{1}{m-1}}}dx=1$$

(24)

$${\displaystyle \underset{0}{\overset{\infty }{\int }}{g}_i(x)[}{\lambda }_0+{\displaystyle \sum _{i=1}^M{\lambda }_i{g}_i(x){]}^{\frac{1}{m-1}}}dx=\overline{g_i(x)},i=1,2,\dots ,M$$

(25)

Solution of Equations (24) and (25) yields the unknown Lagrange multipliers ${\lambda }_i,i=1,2,\dots ,M.$ It may be noted from Equation (24) that λ₀ can be expressed as a function of other Lagrange multipliers and, therefore, the unknown multipliers are λ_i, i = 1, 2, …, M. Except for simple cases, Equations (24) and (25) do not have an analytical solution but can be solved numerically.

2.3.5. Determination of Maximum Entropy

Substitution of Equation (24) in Equation (6) leads to the maximum Tsallis entropy:

$$H=\frac{1}{m-1}\{1-{\displaystyle \underset{0}{\overset{\infty }{\int }}[{\lambda }_0+}{\displaystyle \sum _{i=1}^M{\lambda }_i{g}_i(x){]}^{\frac{m}{m-1}}dx\}}$$

(26)

Equation (26) shows that the entropy of the probability distribution of X depends only on the specified constraints, since the Lagrange multipliers themselves can be expressed in terms of the specified constraints.

3. Applications in Water Engineering: Overview

The problems that can be addressed using the Tsallis entropy theory can be classified into three groups. The first group consists of problems that only require the maximization of the Tsallis entropy, which can be accomplished using POME. Examples of such problems include frequency analysis, parameter estimation, network evaluation and design, spatial and inverse spatial analysis, geomorphologic analysis, grain size distribution analysis, flow forecasting, complexity analysis and clustering [46,47,48,49,50,51,52].

The types of problems included in the second group require coupling with another theory, such as the theory of stream power or theory of minimum energy dissipation rate. Examples of problems in this group include hydraulic geometry [19,20] and evaporation [53]. Also included in this group are problems wherein first relations between entropy and design variables are derived and then relations between design variables and system characteristics are established. Examples include geomorphologic relations for elevation, slope and fall; and evaluation of water distribution systems [54,55].

The third group includes problems that require deriving a physical relation either in time or in space. This means, the domain of analysis requires a shift from the probability domain to a real domain (space or time), which is accomplished by invoking a relation between the cumulative probability distribution function and the design variable. Examples of such problems are infiltration capacity, soil moisture movement in vadose zone, groundwater head distribution, velocity distribution, rainfall-runoff relation, channel geometry, rating curve, flow-duration curve, erosion and sediment yield, sediment concentration and discharge, debris flow, longitudinal river profile, hydraulic geometry, channel cross-section shape and rating curve [19,20,21,22,56,57,58,59,60,61]. The objective in this class of problems is to derive a relation of the design variable as a function of time or space.

These three kinds of problems are further detailed in the following sections, with examples from water engineering.

4. Problems Requiring Entropy Maximization

Fundamental to solving problems that only require entropy maximization is the derivation of probability distribution. A multitude of problems in water engineering involve essentially data analysis for deriving either a probability distribution or computing entropy.

4.1. Frequency Distributions

The procedure for deriving a maximum entropy-based frequency distribution has been discussed above. The procedure is illustrated here using only two constraints: mean (first moment) and second moment. For any probability density function, f(x), of a random variable X, the total probability must equal one, i.e.,

$${\displaystyle \underset{0}{\overset{\infty }{\int }}f(x)dx=1}$$

(27)

The first and second moments can be defined, respectively, as:

$${\displaystyle \underset{0}{\overset{\infty }{\int }}xf(x)dx}=E(x)={\mu }_1$$

(28)

and

$${\displaystyle \underset{0}{\overset{\infty }{\int }}{x}^{2}f(x)dx}=E(x^2)={\mu }_2$$

(29)

In order to obtain the least-biased f(x), subject to Equations (27)–(29), Equation (6) can be maximized for m > 0 using the method of Lagrange multipliers. The Lagrange function L can be written as:

$$L={\displaystyle \underset{0}{\overset{\infty }{\int }}f(x)\frac{1-{[f(x)]}^{m-1}}{m-1}}dx-{\lambda }_0[{\displaystyle \underset{0}{\overset{\infty }{\int }}f(x)dx-1]-{\lambda }_1[{\displaystyle \underset{0}{\overset{\infty }{\int }}xf(x)dx-{\mu }_1]}}-{\lambda }_2[{\displaystyle \underset{0}{\overset{\infty }{\int }}{x}^{2}f(x)dx-{\mu }_2]}$$

(30)

Differentiating L with respect to f(x) and equating the derivative to zero, one obtains:

$$f(x)={\{\frac{1}{m}+\frac{(1-m)}{m}[{\lambda }_0+{\lambda }_{1}x+{\lambda }_2{x}^2]\}}^{1/(m-1)}$$

(31)

Defining k = (1–m)/m [37] and ${\alpha }_i=m{\lambda }_i,i=0,1,2,$ Equation (31) can be written as:

$$f(x)={(1+k)}^{-1-1/k}{[1+k({\alpha }_0+{\alpha }_{1}x+{\alpha }_2{x}^2)]}^{-1-1/k}$$

(32)

Equation (32) is the entropy-based probability density function of power type.

The Lagrange multipliers, λ_i, i = 0, 1, 2 and, consequently, ${\alpha }_i=m{\lambda }_i,i=0,1,2$, can be estimated using Equations (27) to (29). For simplification, let λ₂ be assumed as 0. Then, Equation (32) becomes:

$$f(x)={(1+k)}^{-1-1/k}{[1+k({\alpha }_0+{\alpha }_{1}x)]}^{-1-1/k}$$

(33)

whose parameters ${\alpha }_i=m{\lambda }_i,i=0,1,$ are estimated using Equations (27) and (28). Substituting Equation (33) in Equation (27), one obtains:

$${\alpha }_1={(1+k)}^{-1-1/k}{(1+k{\alpha }_0)}^{-1/k}$$

(34)

or

$${\alpha }_0=-1+\frac{1}{k}{[{\alpha }_1{(1+k)}^{1+1/k}]}^k$$

(35)

The other Lagrange multiplier can be determined by inserting Equation (33) in Equation (28) and is given as:

$$-\frac{{(1+k)}^{-1-1/k}}{k(1-\frac{1}{k})}{(1+k{\alpha }_0)}^{1-1/k}={\alpha }_1^2\overline{x}$$

(36)

By solving Equations (34) or (35) with (36), Lagrange multipliers α₁ and α₀ can be determined. Inserting Equation (34) in Equation (33), one obtains

$$f(x)={(1+k)}^{-1-1/k}{(1+k{\alpha }_0)}^{-1-1/k}[1+k{(1+k)}^{-1-1/k}k{(1+k{\alpha }_0)}^{-1-1/k}x]$$

(37)

Let

$$\beta ={[(1+k)(1+k{\alpha }_0)]}^{-1-1/k}$$

(38)

Using Equation (38), Equation (37) becomes

$$f(x)=\frac{1}{\beta }{[1+\frac{kx}{\beta }]}^{-1-1/k}$$

(39)

Equation (39) is a two-parameter generalized Pareto distribution.

Let

$$y=x+\frac{\beta }{k}\Rightarrow x=y-\frac{\beta }{k}$$

(40)

Equation (39) then becomes

$$f(y)=\frac{1}{\beta }{[\frac{ky}{\beta }]}^{-1-1/k}$$

(41)

Equation (41) is a two-parameter Pareto distribution. If k → 0, then Equation (41) leads to an exponential distribution:

$$f(y)=\frac{1}{\beta }\exp [\frac{y}{\beta }]$$

(42)

Koutsoyiannis [37,38] proposed a generalization, following which Equation (32) can be expressed as:

$$f(x)={(1+k)}^{-1-1/k}{[1+k({\alpha }_0+{\alpha }_{1}x+{\alpha }_2{x}^{c_1})]}^{-1-1/k}{x}^{c_2-1}$$

(43)

where c₁ and c₂ are shape parameters. Equation (43) has four parameters: scale parameter α₁ and shape parameters k and c₁ and c₂. Note that α₀ is not a parameter, because it is a constant based on the satisfaction of Equation (27). It is, however, not clear as to what led to the generalized form of Equation (43). Koutsoyiannis [37] suggested that random variable $X^{c_2}$ would have Beta Prime (also referred to as Beta of the second kind) distribution [62]. Then, the distribution of X would be referred to as the power-transformed Beta Prime (PBP). Koutsoyiannis [37] showed that Equation (43) can specialize into several exponential and power-type probability distributions, such as PBP-L1 $(k\to 0)$, gamma $(k\to 0,c_1\to 1)$, Weibull $(k\to 0,c_2=c_1)$, Pareto $(c_2=c_1=1)$, Beta Prime $(c_1=1)$, PBP-L2 $(k\to \infty ,k{\alpha }_0\to k_0,k{\alpha }_1\to {\alpha }_1)$ and others.

4.2. Network Evaluation and Design

Hydrometric data are required for an efficient planning, design, development, operation and management of water engineering systems. Many studies have applied the Shannon entropy theory to assess and optimize data collection networks (e.g., water quality, rainfall, streamflow, hydrometric, elevation, landscape, etc.) [63,64] but not the Tsallis entropy theory. The basic idea for developing a methodology for data collection network design is that it must take into account the information of each gaging station or potential gaging station in the network. A station with a higher information content is given a higher priority over other stations that have lower information content but the information content of a station must be tempered with the degree of use. That is, a station that is used by one user might be given a lower priority than a station that has diverse uses.

A framework for network design or evaluation considers a range of factors, such as: (a) objectives of sampling; (b) variables to be sampled; (c) locations of measurement stations; (d) frequency of sampling; (e) duration of sampling; (f) uses and users of data; and (g) socio-economic considerations. A network design has two modes: (1) number of gages and their locations (space evaluation); and (2) time interval for measurement (time evaluation).

Let there be two stations A and B with ensembles of configurational possibilities $E^A=\{1,2,\dots ,N\}$ with probability distribution $P^A=\{p_j{}^A,i=1,2,\dots ,N\}$ and configurational possibilities $E^B=\{1,2,\dots ,M\}$ with probability distribution $P^B=\{p_j{}^B,j=1,2,\dots ,M\}.$ Then, the union of the two stations $A\cup B$ and their corresponding ensembles of possibilities are $E^{A\cup B}=\{(1,1),(1,2),\dots ,(i,j),\dots ,(N,M)\}.$ If $p_{ij}{}^{A\cup B}$ represents the corresponding probabilities, then the mutual information or transinformation S can be expressed as:

$$T[(p_{ij}{}^{A\cup B})]=H^{A\cup B}[(p_{ij}{}^{A\cup B})]-H^A[({\displaystyle \sum _{i=1}^N{p}_{ij})]-H^B[({\displaystyle \sum _{j=1}^M{p}_{ij})]}}$$

(44)

From Equation (44), T(p_ij) = 0 for all m, if X and Y are independent. For correlated X and Y, T(p_ij) < 0 for m = 1 and $T(p_{ij})=0$ for m = 0. For arbitrary values of m, it will be sensitive to p_ij; it can take on negative or positive values for both m < 1 and m > 1 with no particular regularity and can exhibit more than one extremum.

4.3. Directional Information Transfer Index

By dividing by the marginal entropy, the mutual information T can be normalized [65] as:

$$\frac{T}{H}=DIT=\frac{(H-H_{Lost})}{H}=1-\frac{H_{Lost}}{H}$$

(45)

where H_Lost is the amount of information lost. The ratio of T by H is called the Directional Information Transfer (DIT) index. Mogheir and Singh [66] called it as the Information Transfer Index (ITI). The DIT varies from zero to unity and denotes the fraction of information transferred from one station to another. A zero value of DIT corresponds to the case where sites are independent and, therefore, no information is transmitted. A value of unity for DIT corresponds to the case where sites are fully dependent and no information is lost. Since DIT_XY = T/H(X) is not the same as DIT_YX = T/H(Y), DIT is not symmetrical. The term DIT_XY describes the fractional information inferred by station X about station Y, whereas DIT_YX describes the fractional information inferred by station Y about station X. Between two stations, the station with the higher DIT should be given higher priority because of its greater capability in inferring (predicting) the information at other sites. The DIT can be applied for regionalization of the network or watersheds.

4.4. Reliability of Water Distribution Networks

A water distribution system can be designed by minimizing head losses, costs, risks and departures from specified values of water quantity, pressure and quality and also maximizing reliability [67]. Thus, it becomes a multi-objective optimization problem. However, it is not uncommon to formulate the design problem as a single-objective optimization problem, where the system capital and operational costs are minimized and, at the same time, the laws of hydraulics are satisfied and the targets of water quantity and pressure at demand nodes are met. Fundamental to either type of optimization is reliability [68,69,70].

To develop a Tsallis entropy-based redundancy measure of the network with N nodes, where the nodes may be considered to constitute sub-systems, the Tsallis entropy of a node j can now be expressed in terms of W_ij as:

$$S_j=\frac{1}{m-1}[{\displaystyle \sum _{i=1}^{n(j)}{W}_{ij}}-W_{ij}{}^m)]=\frac{1}{m-1}\{{\displaystyle \sum _{i=1}^{n(j)}[\frac{q_{ij}}{Q_j}}-{(\frac{q_{ij}}{Q_j})}^m]\}$$

(46)

where m is the entropy index and is a real number and S_j is an entropic measure of redundancy at node j and is local redundancy. Maximizing S_j would maximize redundancy of node j and is equivalent to maximizing entropy at node j. The maximum value of S_j is achieved when all W_ij’s or q_ij/Q_j’s are equal. This occurs when all q_ij’s are equal. For the entire water distribution network, redundancy is a function of redundancies S_j’s of individual nodes in the network.

The overall network redundancy can be assessed in two ways. First, the network redundancy can be assessed by the relative importance of a link to its node and its importance recognized by q_ij/Q_j. In this case, the redundancy is maximized at each node. It may, however, be noted that the network redundancy is not a sum of nodal redundancies. Second, the network redundancy can be assessed by the relative importance of a link to the total flow and its importance recognized by q_ij/Q₀. Here, the proposition is that the importance of a link relative to the local flow is not as important as it is to the total flow. In this case too, the network redundancy is not a sum of nodal redundancies. In order to acknowledge the relative importance of a link to the entire network, the nodal redundancy S_j* can be expressed as:

$$S_{j*}=\frac{1}{m-1}{\displaystyle \sum _{i=1}^{n(j)}[\frac{q_{ij}}{Q_0}}-{(\frac{q_{ij}}{Q_0})}^m]=\frac{1}{m-1}[\frac{Q_j}{Q_0}-{\displaystyle \sum _{i=1}^{n(j)}{(\frac{q_{ij}}{Q_0})}^m]}$$

(47)

It may be noted that S_j*, given by Equation (47), is similar to the S_j given by Equation (46). In this case too, the maximum value of S_j* will occur when the q_ij values are equal at the j-th node. It can also be shown that the maximum network redundancy will be achieved when all the q_ij values are equal. It may, however, be noted that

$$S_{j*}=\frac{1}{m-1}{\displaystyle \sum _{i=1}^{n(j)}[\frac{q_{ij}}{Q_0}}-{(\frac{q_{ij}}{Q_0})}^m]\ne \frac{1}{m-1}[1-{\displaystyle \sum _{i=1}^{n(j)}{(\frac{q_{ij}}{Q_0})}^{m-1}]}$$

(48)

This is because,

$${\displaystyle \sum _{i=1}^{n(j)}{q}_{ij}=Q_j\ne Q_0}\mathrm{and}{\displaystyle \sum _{i=1}^{n(j)}\frac{q_{ij}}{Q_0}\ne 1}$$

(49)

Therefore, in the second approach, Equation (47) can be used in the spirit of Tsallis entropy or considering it via partial Tsallis entropy [36].

The network redundancy for N nodes is a function of redundancies of individual nodes, S_j’s, in the network. However, it will not be a simple summation of these nodal redundancies, because of the non-extensive property of the Tsallis entropy. For the first approach, it can be shown that the network redundancy (with N nodes) can be expressed as:

$$\begin{array}{l}{S}_{1\cup 2\cup \dots \cup N}={\displaystyle \sum _{1\le j\le N}{S}_j}+(1-m){\displaystyle \sum _{1\le j1<j2\le N}{S}_{j1}}{S}_{j2}+{(1-m)}^2{\displaystyle \sum _{1\le j1<j2<j3\le N}{S}_{j1}}{S}_{j2}{S}_{j3}+\dots \\ +{(1-m)}^{N-1}{\displaystyle \sum _{1\le j1<j2\dots <jN\le N}{S}_{j1}}{S}_{j2}\dots S_{jN}\end{array}$$

(50)

In order to develop an appreciation for Equation (47), it will be instructive to expand Equation (47) in terms of flow quantities. Equation (47) is just the sum of nodal redundancies.

For the second approach, the network redundancy (with N nodes: 1, 2, …, N) can be expressed as:

$$\begin{array}{ll}{S}_{1\cup 2\cup 3\cup 4\dots \cup N*}& ={(m-1)}^{N-1}\underset{j=1}{\overset{N}{\Pi }}{S}_{j*}-\frac{{(m-1)}^{N-2}}{Q_0}[Q_1{S}_{2*}\dots S_{N*}+Q_2{S}_{1*}{S}_{3*}\dots S_{N*}+\dots Q_N{S}_{1*}{S}_{2*}\dots S_{N-1*}]\\ & +\dots +\frac{(m-1)}{Q_0{}^{N-1}}[Q_1{Q}_2\dots Q_{N-1}{S}_{N*}+Q_1{Q}_2\dots Q_{N-2}{S}_{N-1*}+\dots +Q_2{Q}_2\dots Q_N{Q}_1]\end{array}$$

(51)

It can be seen that Equation (51) for network redundancy with the second approach is significantly different from Equation (50) with the first approach.

5. Problems Requiring Coupling with another Theory

There are some problems where the entropy theory can only be part of the solution methodology and needs to be coupled with another theory. Consider, for example, hydraulic geometry of a river or channel, which is defined by the relations between discharge and each of hydraulic variables (e.g., flow width, depth, velocity under bankfull conditions) and each of geometric variables (e.g., bed roughness, slope). Hydraulic geometry is of two types: (1) downstream; and (2) at-a-point. For either type, the values of hydraulic and geometric variables are average annual values corresponding to the equilibrium or stable condition of the river. At this condition, the river will try to spend the least amount of energy for transporting water and sediment. In response to the influx of water and sediment coming from the watershed, the river will adjust these variables or characteristics in order to attain the stable condition. This means, the river will spread the adjustment as equitably as the environment will allow and will follow the theory of minimum energy dissipation rate. The equal rate of adjustment can be described by the principle of maximum entropy (POME) that the river will follow. In this manner, the theory of minimum energy dissipation rate and entropy theory are combined for determining the hydraulic geometry of a river or designing a stable canal. The advantage of the entropy theory is that it explicitly allows to incorporate the constraints imposed by the watershed and the design. For example, if a river is leveed, then it cannot adjust its width and the adjustment will be shared by depth and velocity. Likewise, if a canal is lined, it cannot adjust its width, slope and roughness and will have to adjust its depth and velocity. In this manner, a whole hierarchy of hydraulic geometry relations can be obtained, depending on the constraints imposed. The other existing theories do not allow this flexibility. In what follows, some specific examples of problems requiring coupling entropy theory with other theories are presented.

5.1. Hydraulic Geometry

Hydraulic geometry is defined by relations between discharge (Q) and each of channel width (B), flow depth (d), flow velocity (V), slope (S) and roughness factor (say Manning’s n). Hydraulic geometry is either downstream hydraulic geometry or at-a-station hydraulic geometry [71]. Langbein [72] and Yang et al. [73] reasoned that hydraulic geometry relations correspond to the equilibrium state of the channel. In order to attain this state, the channel adjusts its hydraulic variables and the adjustment is shared equally among these variables. In practice, the channel is seldom in equilibrium, meaning that the adjustment among hydraulic variables will be unequal. This, then, suggests that there will be a family of hydraulic geometry relations, depending on the adjustment of hydraulic variables and the adjustment should explain the variability in the parameters of hydraulic geometry relations. For downstream hydraulic geometry, Singh et al. [19,20] hypothesized that, for a given influx of discharge from the watershed, the channel would minimize its stream power by adjusting three controlling variables: depth, width and friction.

Coupling the theory of minimum energy dissipation rate with the principle of maximum entropy, three possibilities can occur corresponding to the spatial rate of adjustment of friction, the spatial rate of adjustment of width and the spatial rate of adjustment of flow depth. These possibilities then lead to the formulation of, respectively, the proportion of the adjustment of stream power (SP) by friction, the proportion of the adjustment of SP by channel width and the proportion of the adjustment of SP by flow depth and, hence, to four sets of hydraulic geometry relations, as follows: (1) the spatial change in SP is accomplished by an equal spatial adjustment between flow width B and resistance expressed by Manning’s n; (2) the spatial variation in SP is accomplished by an equal spatial adjustment between flow depth and flow width; (3) the spatial variation in SP is accomplished by an equal spatial adjustment between flow depth and resistance; and (4) the spatial variation in SP is accomplished by an equal spatial adjustment between flow depth, flow width and resistance. These four possibilities can occur in the same river in different reaches or in the same reach at different times, or in different rivers at the same time or at different times.

The hydraulic geometry relations are expressed as:

$$B=a{Q}^b,d=c{Q}^f,V=k{Q}^m,n=N{Q}^p,S=s{Q}^y$$

(52)

where a, c, k, N and s are parameters; and b, f, m, p and y are exponents. Values of these exponents and parameters depend upon the possibility under consideration and the entropy theory permits explicit expressions for the exponents and parameters. Singh et al. [19,20] showed that most of the downstream hydraulic geometry relations reported in the literature can be derived as special cases of the entropy-based equations.

For at-a-station hydraulic geometry, when discharge changes, a river cross-section can adjust its width, depth, velocity, roughness and slope or a combination thereof. Singh and Zhang [21,22] reasoned that the channel cross-section will adjust or minimize its SP by adjusting these four variables: (1) P_B can be interpreted as the proportion of the temporal change of SP due to the temporal rate of adjustment of width; (2) P_h as the proportion of the temporal change of SP due to the temporal rate of adjustment of depth; (3) P_α as the proportion of the temporal change of SP due to the temporal rate of adjustment of friction; and (4) P_S as the proportion of the temporal change of SP due to the temporal rate of adjustment of slope. These cases involve probabilities of four variables, meaning that any adjustment in hydraulic variables in combinations of two, three or four may occur. These give rise to different configurations of adjustment that do indeed occur in nature [74]. Thus, the equality among four probabilities yields 11 possibilities and, hence, leads to 11 sets of equations: (1) P_B = P_h; (2) P_B = P_α; (3) P_B = P_S; (4) P_h = P_α; (5) P_h = P_S; (6) P_α = P_S; (7) P_B = P_h = P_α; (8) P_B = P_α = P_S; (9) P_B = P_h = P_S; (10) P_h = P_α = P_S; and (11) P_B = P_h = P_α = P_S. It should be noted that all eleven possibilities can occur in the same river cross-section at different times, or in different river cross-sections at the same time or at different times. Williams [75] explored 11 cases, which are similar to the above 11 possibilities. The resulting hydraulic geometry relations are of the same form as Equation (52) but expression for exponents and parameters therein are different.

5.2. Evaporation

Evaporation is a process by which liquid water is converted into water vapor. The process of evaporation entails four elements [76]: (1) supply of energy; (2) supply of water; (3) tendency of liquid water molecules to escape; and (4) turbulent transport. The source of energy that is needed for evaporation to occur from land surfaces is solar radiation, which can be defined by radiative flux. The supply of water can be from precipitation or irrigation, determining soil wetness, which is characterized by soil moisture. Fugacity refers to the tendency of water molecules to escape and is expressed by the saturated vapor pressure at the liquid-vapor interface. The turbulent transport of water vapor and heat is determined by wind speed and thermal instability of the surface layer and is defined by turbulent sensible heat flux into the atmosphere. Wang et al. [76] argued that, under thermodynamic equilibrium, thermal and hydrologic states of the land surface resulting from the interaction between land and atmospheric processes tend to maximize evaporation.

For a given radiative energy flux, the rate of evaporation depends on the combination of surface soil moisture, surface soil temperature and sensible heat flux, as well as the dynamic feedbacks among them at the land surface. There can be many combinations of ground and sensible heat fluxes, evaporation rate and net radiation that can satisfy the energy balance. Wang et al. [76] hypothesized that the preferred combination is the one that maximizes evaporation. Denoting the rate of evaporation by E, surface soil moisture by w, surface soil temperature by T, sensible heat flux into the atmosphere by H, ground heat flux by G and net radiative by R_n at the surface, maximizing evaporation, subject to the energy balance, the result is:

$$E=\max [E(w,T,H;R)|E+H+G=R_n]$$

(53)

for all combinations of independent variables w, T and H. Wang et al. [76] investigated three cases: (1) $R=R_n-G,$ representing the turbulent energy budget, as described by the Bowen ratio; (2) $R=R_n,$ corresponding to the partitioning of the net radiation into latent, sensible and ground heat fluxes; and (3) $R=R_i,$ representing the budget of all surface energy fluxes. These cases express land-atmosphere interactions.

6. Problems Involving Physical Relations

In water engineering, we often need to determine physical relationships, such as infiltration rate as a function of time, runoff or discharge as a function of time, soil moisture as a function of depth from the soil surface, velocity as a function of flow depth measured from the bottom, sediment concentration as a function of flow depth and sediment discharge as a function of time. For deriving physical relationships, the probability domain and the physical domain need to be concatenated and this can be done by hypothesizing the cumulative distribution function (CDF) of a design (dependent) variable (e.g., flux, say discharge) in terms of independent (concentration) variable (e.g., stage of flow).

6.1. Hypotheses on Cumulative Probability Distribution Function

Different types of hypotheses have been formulated when applying entropy theory to derive relationships for design variables. Examples of a linear hypothesis include velocity distribution as a function of flow depth, wind velocity as a function of height, sediment concentration profile along the flow depth, rating curve and groundwater discharge along the horizontal direction of flow. It is noted that the CDF should have a one-to-one relation with the design variable (i.e., random variable) of interest and its value should only be between 0 and 1; 0 for the minimum value of the random variable and 1 for the maximum value. For deriving a two-dimensional velocity (u) distribution, Cui and Singh [77] hypothesized a general form of the CDF as:

$$F(u)={[1-{(\frac{x}{B})}^2]}^b{(\frac{y}{D})}^a\mathrm{for}\mathrm{all}\left(x,y\right)\mathrm{on}I(u)$$

(54)

in which y is the vertical dimension, x is the transverse direction, a and b are shape parameters and B and D act as scale parameters or normalizing quantities. The CDF given by Equation (54) has a one-to-one relationship with the velocity value u; in other words, the CDF is unique on each isovel I(u) and has a value of 0 at I(0) and 1 at I(u_max). Also, CDF is 0 at x = B or y = 0 and is 1 at x = 0 and y = D (Figure 3). It is continuous and differentiable in both x and y.

The two-dimensional (2-D) velocity distribution can also be expressed by transforming the Cartesian coordinates [vertical (y) and transverse (x)] into curvilinear s-r coordinate system in which r has a unique, one-to-one relation with a value of velocity and s (coordinate) curves are their orthogonal trajectories [56]. Then, the CDF of velocity in a channel cross section can be expressed as:

$$F(u)=\frac{r-r_0}{r_{\max }-r_0}$$

(55)

where r is a coordinate between r₀ and r_max and corresponds to y.

For the one-dimensional (1-D) form of Equation (54) with b = 0, it is seen from Equation (54) that the CDF will remain between 0 and 1 when 0 ≤ x ≤ B and 0 ≤ y ≤ D. Thus, the CDF for 1-D velocity distribution can be simplified as:

$$F(u)={(\frac{y}{D})}^a\mathrm{for}\mathrm{all}\left(y\right)\mathrm{on}I(u)$$

(56)

This equation implies that the shape parameter a is critical. If a = 1, then

$$F(u)=(\frac{y}{D})\mathrm{for}\mathrm{all}\left(y\right)\mathrm{on}I(u)$$

(57)

In cases where the design variable is maximum at y = 0 and minimum at maximum y (i.e., y = D), as is the case for suspended sediment concentration, it is distributed from a maximum value at the channel bed and decreases towards the water surface. Hence, the CDF should be assumed in a way so that it is 1 at the channel bed (y = 0) and 0 at the water surface (y = D). The CDF of sediment concentration can then be expressed as:

$$F(c)=1-{(\frac{y}{D})}^a$$

(58)

It is seen from Equation (58) that F(c) will be between 0 and 1 if 0 ≤ y ≤ D.

6.2. One-Dimensional Velocity Distribution

Velocity distribution is needed for determining flow discharge, scour around bridge piers, erosion and sediment transport, pollutant transport, energy and momentum distribution coefficients, hydraulic geometry, watershed runoff and river behavior. Velocity distributions have been derived using experimental, hydrodynamic, or entropy methods. Shannon entropy has been applied to derive one- and two-dimensional velocity distributions [26]. Singh and Luo [11,78] and Luo and Singh [12] employed Tsallis entropy to derive the 1-D velocity distribution. The Tsallis entropy-based velocity distribution has been shown to have an advantage over the Shannon entropy-based distribution. However, in these entropy-based velocity distributions, the CDF has been assumed to be linear, meaning the velocity is equally likely along the vertical from the channel bed to the water surface. This assumption is fundamental to the derivation of velocity distributions but has not been adequately scrutinized. Further, this assumption is weak and may partly explain the reason why these velocity distributions do not accurately describe the velocity near the channel bed.

For deriving 1-D velocity distribution, the constraint equation is derived using the continuity equation such that g₁(u) = u and limits of integration are 0 and u_D (maximum velocity at the surface or flow depth D). The entropy theory then yields the probability density function of velocity u:

$$f(u)={\left[\frac{m-1}{m}\left({\lambda }_{*}+{\lambda }_{1}u\right)\right]}^{\frac{1}{m-1}}$$

(59)

where ${\lambda }_{*}=\frac{1}{m-1}+{\lambda }_0$.

Following Cui and Singh [77], the CDF of u can be hypothesized as Equation (56). Then, a general velocity distribution based on the Tsallis entropy theory [14] is obtained as:

$$u=-\frac{{\lambda }_{*}}{{\lambda }_1}+\frac{1}{{\lambda }_1}\frac{m}{m-1}{\left[{\lambda }_1{\left(\frac{y}{D}\right)}^a+{\left(\frac{m-1}{m}{\lambda }_{*}\right)}^{\frac{m}{m-1}}\right]}^{\frac{m-1}{m}}$$

(60)

Equation (60) can be simplified by defining a dimensionless parameter G as:

$$G=\frac{{\lambda }_1{u}_{\max }}{{\lambda }_1{u}_{\max }+{\lambda }_{*}}$$

(61)

Parameter G can be regarded as an index of the velocity distribution uniformity. Equation (60) can be cast as:

$$\frac{u}{u_{\max }}=1-\frac{1}{G}\left(1-{({\left(1-G\right)}^{\frac{m}{m-1}}+\left(1-{\left(1-G\right)}^{\frac{m}{m-1}}\right){\left(\frac{y}{D}\right)}^a)}^{\frac{m-1}{m}}\right)$$

(62)

Equation (62) shows that for a given m value, the velocity distribution can be obtained with only two parameters: a and G. A bigger G value tends to slow the growth of the velocity from the channel bed to the water surface, while the parameter a has an opposite effect. A lower value of G tends to linearize the velocity distribution and a higher value non-linearize. The opposite is the case for exponent a. The velocity distribution is more sensitive to a than to G.

6.3. Two-Dimensional Velocity Distribution

For two-dimensional (2-D) velocity distribution, Chiu [56] proposed a transformation that converts the Cartesian coordinates [vertical (y) and transverse (z)] into curvilinear s-r coordinate system, in which r has a unique, one-to-one relation with a value of velocity and s (coordinate) curves are orthogonal trajectories to r. Following the coordinate system of Chiu [56], Luo and Singh [12] employed the Tsallis entropy. Then, expressing the CDF of velocity in a channel cross section by Equation (57), the velocity distribution becomes

$$u=(\frac{m}{m-1})\frac{1}{{\lambda }_1}{\left\{\left({\lambda }_1\frac{r-r_0}{r_{\max }-r_0}\right)+{\left[(\frac{m-1}{m}){\lambda }_{*}\right]}^{\frac{m}{m-1}}\right\}}^{\frac{m-1}{m}}-\frac{{\lambda }_{*}}{{\lambda }_1}$$

(63)

Defining entropy parameter $M={\lambda }_1{u}_{\max }^2$, the dimensionless velocity distribution can be expressed as:

$$\frac{u}{u_{\max }}=\frac{2}{M}{\left[M(\frac{r-r_0}{r_{\max }-r_0})+\frac{{\left(4-M\right)}^2}{16}\right]}^{\frac{1}{2}}-\frac{4-M}{2M}$$

(64)

The Tsallis entropy-based approach of Luo and Singh [12] was either superior or comparable to Chiu’s distribution for the data sets used therein for testing. However, due to the complexity of the coordinate system and a large number of parameters used, the application of these methods may be limited. Later, using normal x-y coordinate system, Cui and Singh [13] obtained the 2-D velocity distribution equation as:

$$u=-\frac{{\lambda }_{*}}{{\lambda }_1}+\frac{1}{{\lambda }_1}\frac{m}{m-1}{\left[{\lambda }_{1}F(u)+{(\frac{m-1}{m}{\lambda }_{*})}^{\frac{m}{m-1}}\right]}^{\frac{m-1}{m}}$$

(65)

where the CDF, F(u), is defined as Equation (54). As defined in the case of one-dimensional velocity distribution, the dimensionless entropy parameter G is also used here as:

$$G=\frac{{\lambda }_1{u}_{\max }}{{\lambda }_1{u}_{\max }+{\lambda }_{*}}$$

(66)

Parameter G is found to be related to the ratio of mean and maximum velocity and a quadratic relation is obtained by computing from observed mean and maximum velocity values as [13]:

$$\frac{\overline{u}}{u_{\max }}=\mathsf{\Psi }(G)=0.554G^2-0.077G+0.568$$

(67)

Now, with the use of Equation (67), the general velocity distribution Equation (65) can be cast as:

$$\frac{u}{u_{\max }}=1-\frac{1}{G}\left(1-{\left[{\left(1-G\right)}^{\frac{m}{m-1}}+\left(1-{\left(1-G\right)}^{\frac{m}{m-1}}\right)F(u)\right]}^{\frac{m-1}{m}}\right)$$

(68)

Equation (68) is the general 2-D velocity distribution equation in terms of parameter G and maximum velocity.

6.4. Suspended Sediment Concentration

The sediment concentration distribution can also be derived using Tsallis entropy. For deriving the one-dimensional sediment concentration distribution, the constraint equation is derived using the mean concentration, such that g₁(c) = c and limits of integration are a = c₀ (maximum concentration at the bed surface or flow depth D = 0) and b = 0 at the water surface, as [16]:

$$f(c)={\left[\frac{m-1}{m}({\lambda }_{*}+{\lambda }_{1}c)\right]}^{\frac{1}{m-1}}$$

(69)

where ${\lambda }_{*}=\frac{1}{m-1}+{\lambda }_0$. Equation (69) is the least-biased entropy-based probability density function of sediment concentration, which is fundamental to determining the sediment concentration distribution. The dimensionless sediment concentration can then be obtained as:

$$\frac{c}{c_0}=1-\frac{1}{N}\left(1-{\left\{{\left(1-N\right)}^{\frac{m}{m-1}}+\left[1-{\left(1-N\right)}^{\frac{m}{m-1}}\right]F(c)\right\}}^{\frac{m-1}{m}}\right)$$

(70)

where N is a dimensionless parameter as a function of maximum concentration and the Lagrange multipliers expressed as:

$$N=\frac{{\lambda }_1{c}_0}{{\lambda }_1{c}_0+{\lambda }_{*}}$$

(71)

By plotting the empirical observations of the mean over the maximum concentration values and corresponding N values, the implicit function can be regressed as a quadratic polynomial as:

$$\frac{\overline{c}}{c_0}=0.176+0.5083N-0.1561N^2$$

(72)

with a coefficient of determination as 0.99. Thus, N can be used for deriving sediment concentration distribution instead of solving nonlinear equations for λ₁ and ${\lambda }_{*}$.

6.5. Sediment Discharge

The suspended sediment discharge can be computed simultaneously by integrating sediment concentration and velocity over the cross-section where the velocity distribution and sediment concentration can be obtained empirically as well as using the Tsallis entropy theory. That is, suspended sediment discharge can be derived using different combinations of entropy-based and empirical methods for velocity and sediment concentration distributions. Thus, the suspended sediment discharge can be computed with the use of entropy in three ways: (1) velocity distribution by a standard formula and concentration distribution by an entropy-based equation; (2) velocity distribution by an entropy-based equation and concentration distribution by a standard formula; and (3) velocity distribution and concentration distribution both by entropy-based equations. In the discussion that follows, only the combination where both are derived using Tsallis entropy is considered.

The Tsallis entropy-based velocity distribution [13], given by Equation (68) and sediment concentration [16], given by Equation (70), are integrated from bottom of the channel to the water surface to obtain the suspended sediment discharge, as:

$$\begin{array}{l}{q}_s=u_{\max }{c}_0{\displaystyle \underset{0}{\overset{D}{\int }}\left[1-\frac{1}{N}\left(1-{\left(\left(1+0.5\ln N\right)F(c)-0.5\ln N\right)}^{2/3}\right)\right]}\\ \left[1-\frac{1}{G}{\left(1-\left((1+0.5\ln G\right)F(u)-0.5\ln G\right)}^{2/3}\right]dy\end{array}$$

(73)

To get an explicit solution of Equation (73) is difficult; however, it can be simplified with the use of mean values. The first term in the integration can be replaced by mean sediment concentration and the second term can be replaced by mean velocity, such that Equation (73) reduces to

$$q_s=D{u}_{\max }{c}_0(0.554G^2-0.777G+0.568)(0.554N^2-0.777N+0.568)$$

(74)

which provides a simple method to compute sediment discharge. Entropy parameters G and N are fixed for each channel cross-section [17]. Thus, once the entropy parameters have been obtained for some known cross-section, with observed maximum velocity and sediment concentration, the sediment discharge can be obtained with ease.

6.6. Flow-Duration Curve

The flow-duration curve (FDC) is used for predicting the distribution of future flows, forecasting of future recurrence frequencies, determining low-flow thresholds for defining droughts, generating hydropower, constructing load-duration curves, determining power-duration curves and comparing watersheds. For deriving an FDC, it is assumed that temporally-averaged discharge Q is a random variable, varying from a minimum value Q_min to a maximum value Q_max, with a probability density function (PDF) denoted as f(Q). The time interval for which the discharge is averaged depends on the purpose of constructing an FDC but it is often taken as one day. Considering g₁(x) = mean Q = Q_mean, the PDF of Q can be derived as [79]:

$$f(Q)={\left[\frac{m-1}{m}\left(\frac{1}{m-1}-{\lambda }_0-{\lambda }_{1}Q\right)\right]}^{\frac{1}{m-1}}$$

(75)

It is interesting to note that at Q = 0, f(Q) becomes ${\{[(m-1)/m][(1/(m-1))-{\lambda }_0]\}}^{1/(m-1)}.$ Similar to velocity distribution, a dimensionless parameter M can be defined as:

$$M=\frac{{\lambda }_1{Q}_{\max }}{{\lambda }_1{Q}_{\max }-{\lambda }_{*}}$$

(76)

It was found [79] that M is linearly related to the ratio between the mean flow and the maximum flow, which, using regression, can be written as:

$$M=2.246-4.891\frac{\overline{Q}}{Q_{\max }}$$

(77)

with the squared correlation coefficient of 0.9972. The FDC can be expressed as:

$$\begin{gathered} \frac{Q}{Q_{\max }}=1-\frac{1}{M}-(1-\frac{1}{M}){\left\{-{(\frac{1}{{\lambda }_{*}})}^{\frac{m}{m-1}}{\lambda }_1{(\frac{m}{m-1})}^{\frac{1}{m-1}}F(Q)+1\right\}}^{\frac{m-1}{m}} \\ =1-\frac{1}{M}\left(1-{\left\{{\left(\frac{m}{m-1}\right)}^{\frac{m}{m-1}}\frac{M}{Q_{\max }}\left[F(Q)-1\right]+1\right\}}^{\frac{m-1}{m}}\right) \end{gathered}$$

(78)

7. Conclusions

A survey of the literature shows that the Tsallis entropy theory has great potential to address a wide range of problems in water engineering and in many other fields, as reviewed in this paper; see also [80,81,82,83,84,85] for some more recent studies. As generalization of the Shannon entropy, the Tsallis entropy can be applied in generalized equilibrium and statics in physics. The advantage of using the theory is that it can combine statistical information with physical laws, permits deriving physical relations as functions of time or space and derives probability distributions in terms of the specified constraints. However, analysis using the Tsallis entropy theory becomes complicated when one or two constraints are involved and it is more difficult to obtain the analytical expressions. Besides, the m-statistic must be carefully chosen when applying the Tsallis entropy, since it sometimes involves complex operations. Although entropy is a thermodynamic quantity, development of the thermodynamic basis of entropy-based relations has not been accomplished yet. Until now, the Tsallis entropy has been applied to determine frequency distributions, network evaluation, hydraulic geometry, evaporation, velocity distribution, sediment concentration distribution, flow duration curves and several other problems, as reviewed above. The outcomes are certainly encouraging. Until now, as we have reviewed in this paper, the Tsallis entropy theory has been applied in water engineering particularly with the principle of maximum entropy (POME). There is, however, also potential to combine the Tsallis entropy with the minimum relative (cross) entropy, for the condition that prior assumptions can be made. There is no question that the Tsallis entropy theory has a much greater potential to study a wide spectrum of problems in water engineering. It is hoped that this review will stimulate further interest in this fascinating field.