Maximum entropy production (MEP) may be an organizational principle applicable to physical and biological systems. Nonequilibrium thermodynamics and MEP can be used to derive overall evolutionary trends of Earth’s past; for example, they can identify the role that life plays in driving thermodynamic states far from equilibrium [3,6–13,27–35]. Various derivations of MEP by using the maximum information entropy (MIE) procedure by Jaynes [36] exist in the literature [2]. In these derivations the information entropy (IE) is not defined by a probability measure on phase space, but on path space for the stationary nonequilibrium systems [26].

Implementation of the MEP principle also includes the steepest entropy ascent formalism based on quantum thermodynamics approach [37–39]. Beretta [37] has examined the steepest entropy ascent formalism for transition between states and adopted a geometrical view of the dynamics in probability space based on identifying a unique path of locally maximal entropy increase.

Consider M sites with a variable n_i(t) (i = 1,2,…,M) at each site with t = 0,1,…,τ. The flux (time asymmetric) occurring randomly at every time step, J_ij = − J_ji from i to j depends on a parameter c_ij = c_ji, such that J_ij(t) = ± c_ij with stochastic sign. A microscopic path a is a set of values ± c_ij so that n_i,a (t+1) −n_i,a(t)= −∑_j J_ij,a(t). The path dependent time average flux is J̄_ij,a = (1/τ)∑_t J_ij(t) and n_i(0) does not depend on the complete path. With the microscopic path dependent probability p_a, the path ensemble average is 〈J̄_ij〉 = ∑_ap_aJ̄_ij,a. By using Jaynes’ information theory and maximizing the path information entropy [2,24,25]:

with the constraints [2,25]:

the most likely probability on the path space is estimated as:

where N_ij is the numerical value of the time and path ensemble average of the flux J_ij, A_a is the path action A_a = ∑_i λ_i n_ia(0) + ∑_ij n_ij J̄_ij,a in which λ_i and n_ij = − n_ji are the Lagrange multipliers of constraints (8) and (9), respectively, and Z is the partition function [2,24,25].

However, a trajectory of a quantity possessed by a system may fluctuate wildly (if far from equilibrium) or weakly; than they would not have the same probabilities as long as they have the same initial and final states. Here the path trajectory a is a sequence of p over some time interval with M sites:

where dt is the coarse graining corresponding to the time scale of experimental observations [4].

The partition sum and the constitutive equation of motion have the relations:

The forward and backward components of the time and ensemble averaged fluxes are:

where m = X_ijc_ij and 2 m = ln   ( N i j f / N i j b ).

The entropy production of a microscopic path a is [2]:

By using the most likely probability on the path space p_a (given in Equation (10)) in Equation (6), the maximum information entropy as a function of the force X becomes:

where W (〈A(X)〉) is the density of paths, W is the number of paths, and (〈A(X)〉) is the average path action.

The entropy curvature (response) matrix and the probability distribution for the time averaged flux J̄ are [2]:

In near equilibrium regime that is for small X, the maximum path information becomes:

The first part on the right side of the equation above is the logarithm of the total number of paths for uniform probability distribution, while the second term is the entropy production. In the MEP, the assumption is that the number of paths W should be an increasing function of the averaged action [24–26]. Here, for the higher value of entropy production, the value of S_I is lowered [2].

MEP principle states that if thermodynamic force X_i is preset, then the true thermodynamic flow J_i satisfying the condition for the local rate of entropy production σ_τ:

yields the maximum value of the σ_τ(J). This can be written using the Lagrange multiplier λ and fixed force:

In the meantime, the relationship between the fluxes and forces:

indicates that this relationship can be both linear and nonlinear [23].

The same entropy production can be both maximum and minimum depending on the constraints used in the entropy production variation. However, it is widely published that the MEP principle may be a critical link in the explanation of the direction of the biological evolution under the imposed constraints of the environment [22,23,26]. If X is fixed, the MEP leads to maximum J that is the selection of fastest process. MEP principle has proved to be valuable for understanding and describing of various nonequilibrium systems in physics, biology, and environment. The local equilibrium hypothesis of a nonequilibrium system and the representation of the entropy production as a bilinear form of flows and forces are mandatory conditions for the use of MEP principle [23–31].

In the cortex, populations of neurons continuously receive input signals from other neurons, interpret their ongoing activity, and generate output signals destined for other neurons. This information processing and transmission is limited by the repertoire of different activated configurations available to the population. The extent of this repertoire may be estimated by its entropy H characterizing the information capacity as the upper limit on the aspects of information processing of the population. When the information transmitted from the input to the output by a population that has only two states in its repertoire (H = 1 bit), then regardless the information the input contains, the output information content cannot exceed 1 bit. Therefore, a network with low entropy population may limit information transmission. Activity in the cortex depends on the ratio of fast excitatory E_x to inhibitory I_n synaptic signals to neurons. The E_x/I_n ratio remains fixed at an average in various events during highly fluctuating activity levels; yet a small E_x/I_n ratio (caused, for example, by weak excitation drive) may reduce the correlations as well as the overall level of activity [7].

For a number of unique binary patterns, the entropy is:

where p_i is the probability that pattern i occurs. Equation (23) estimates the occurrence probability for each pattern. Maximization of entropy may be an organizing principle of neural information processing systems [7]. The information capacity I_C in binary units may be expressed as a function of the probability p:

where Ω is the number of possibilities, p∘ is the probability at equilibrium (i.e., no knowledge), and p is the probability when some information are available about the system. Information here is used as a measure of structure [1,16].

Genes are sequences of deoxyribonucleic acids (DNA) nucleotides composed of four bases (guanine G, adenine A, cytosine C, thymine T). Genes carry and transmit the information to specify 20 amino acids for protein synthesis. The genome refers collectively to the total genetic information coded in a cell. Ribonucleic acid (RNA) molecules transfer information from DNA to the site of protein synthesis [8–14]:

During transcription, one of the strands of the DNA acts as a template and determines the sequence of RNA nucleotides. RNA also contains the base uracil U in place of thymine T. The transcribed RNA binds to ribosomes in the cytoplasm where proteins are synthesized from the encoded information. Only exon segments actually code for amino acids, and a continuous sequence of exons becomes mRNA, which binds to a ribosome. In translation, each ribosome is composed of proteins and ribosomal RNA called rRNA. Transfer RNA (tRNA) is the link between an amino acid and its mRNA codon. Each amino acid brought to a tRNA molecule is linked by a peptide bond to the end of the growing protein chain. The completed protein then undergoes folding. Signals from within or outside the cell can turn on or off the transcription of genes. Homeostatic control systems regulate compensatory responses in which the set point of a variable is controlled. In case of a continuing perturbation, the error signal (negative or positive feedback) may lead to resetting the set point [8–14].

A stimulus is a detectable change in the internal or external environment. A receptor detects the stimulus and produces a signal that is relayed through various pathways to the integrating center, which receives signals from many receptors. The output of the integrating center reflects the net effect of the total afferent input. The output is sent through the efferent pathway to an effector.

Some of the intercellular chemical messengers are hormones, neurotransmitters, and paracrine agents, which are synthesized by cells and diffuse to target cells upon receipt of an appropriate stimulus. Autocrine agents are local chemical messengers that act upon the same cells that secreted them. Many homeostatic systems add and remove different chemicals in response to a particular stress; it is mostly reversible without the involvement of genetic change.

Receptors are glycoproteins on plasma membranes of target cells and bind specific messengers. The binding leads to response through the signal transduction pathways that would vary in different cell types. For example, neurons generate electric signals that pass from one cell to another and release chemical messengers called neurotransmitters to communicate with other cells. A synapse is a junction between two neurons and its influence can be either excitatory or inhibitory. If many presynaptic cells affect a single postsynaptic cell it allows information from many sources to influence the activity of one cell. On the other hand, if a single presynaptic cell affects many postsynaptic cells it allows one information source to affect multiple pathways [8,10].

DNA is a code, and codes from sequence alone do not reveal information. DNA replication starts with an initiator protein, which unwinds a short stretch where another protein known as “helicase” attaches to and breaks apart the hydrogen bonds between the bases on the DNA strands. The nonconditional entropy for DNA sequence or proteins is about two bits per base; a random protein would have log₂(20) = 4.32 bits of entropy per site. Due to repetitions, pair, and triplet correlations the actual entropy would be lower [5]. This entropy per symbol only allows us to estimate our uncertainty about the sequence identity; it will not reveal the “function” of the genes. Estimates of the entropy reduction and information aspects of a living cell are becoming possible in the context of cell’s metabolic activity and hence its maintenance of the nonequilibrium state [10–14,40,41].

In equilibrium thermodynamics, isolated systems have the maximum entropy and there are no correlations between the states; hence there is no information. The information as the amount of correlations between two systems stored in living system (biological genomes) points out that they are far away from equilibrium. Consequently, the information theory becomes a part of nonequilibrium thermodynamics in living cells. Information measures the amount of entropy shared between two systems. Also, information enables us to make predictions about other systems, since only in reference to another ensemble entropy can become information. Therefore, what is described by the correlations between the sequences stores information not the sequence itself. On the other hand, what information a genomic sequence represents depends on the interpreter environment. Therefore, if a sequence means something it can create a function (response) necessary for its environment [6,14].

The information theory introduced “functional information” that leads to self-organizing capabilities of living systems, and “instructional information” that is a physical array. However, linkages with the field of semiotics established a much more compatible approach to biological information [8]. Within this trend “control information” is defined as the capacity to control the acquisition, disposition, and utilization of matter, energy, and information flows functionally.

Functional information I(E_x) represents the probability that an arbitrary configuration of a system (i.e., complex biological systems of many interacting components) will achieve a specific function to a specified degree; therefore, it is as a measure of system complexity. For a given system and function x (e.g., a folded RNA sequence that binds to guanosine triphosphate GTP), and degree of function E_x (e.g., the RNA-GTP binding energy), functional information is I(E_x) = − log₂[F(E_x)], where F(E_x) is the fraction of all possible configurations of the system [14,32–34]. Functional information, for example, analyzes RNA structures that bind target ligands and RNA structures that catalyze specific reactions. Thus, the degree of function E_x of these linear sequences of RNA letters (the four bases adenine, cytosine, guanine, and uracil or thymine A, C, G, and U or T) can be defined quantitatively as the binding energy to a particular molecule or the catalytic activity increase in a specific reaction rate [10,14].

Instead of a single RNA nucleotide, a minimum sequence length (n_min nucleotides) would achieve any significant degree of ribozyme function, E_x > E_min. Increasing the number of nucleotides (n > n_min) will generally lead to more functional sequences. Consequently, for a maximum possible degree of specific function E_max there will be an optimal RNA sequence of length n_opt possessing the maximum functional information of I_max (E_max) = −log₂ [1/(∑_1−nopt(4ⁿ))]. For E_x < E_max, an intermediate functional information shows [I(E_x) < I_max(E_max)] [14].

Each position on the genome is four-base code and the uncertainty at each position is two bits; then the maximum entropy becomes:

since p(i) = ¼. The actual entropy is obtained from the actual probabilities p_j(i) for each position j on the sequence. In N sequences, we have p_j(i) = n_j (i)/N by counting the number of n_j(i) occurrences of nucleotide i at position j (this will be done for all positions j = 1,…,M on the sequence length M). When we ignore correlations between positions j, the information stored in the sequence becomes:

where H = - ∑ j = 1 M ∑ i = G , C , A , T p j ( i )   log 2   p j ( i ).

The thermodynamics of protein structures implies that sequence and structure are related. If a structural entropy of proteins H(str) is obtained for a given chain length and environment, the mutual entropy between structure and sequence becomes [8]:

where H(seq) is the entropy of sequences of length M and H(seq/str) is the entropy of sequences based given the structure. If the environment requires a certain structure that will be functional in that environment then H (seq/str) ≈ H (seq/env) and I (seq:str) is approximately equal to the physical complexity. Assuming that any given sequence produces an exact structure H(str/seq) = 0, then Equation (28) becomes:

Therefore, thermodynamic entropy of a protein structure is limited by the amount of information about the environment coded by the sequence. This may imply that sequences that encode more information about the environment may be more functional.

One of the consequences of the Human Genome project has proved that biology is an “informational science” [5,9]. The communication in living cells is based on the signals, such as electromagnetic-light, mechanical-touch, and chemicals received. In the signal-transduction pathway, a signal on a cell surface is converted into a specific cellular response in a series of functional steps [7]. This suggests that information is conceived as the communication of a form from object to interpreter through the sign. The evolution of ways of storing, transmitting, and interpreting information can be seen a major step in the increased capacity for collective behavior and robustness in living systems [10,11,29]. Information can be introduced into, or extracted from through the agencies intentionally. A sophisticated information organizing capacity of biological origin emerges from the uncertainty inherent in the complexity of all living systems [12–14].

In semiotic understanding of living systems, interpreters of signs and information will often be an interpreter-dependent objective process. Genes should be regarded as signs in DNA, which can only have any effect on a cell function through a triadic-dependent process that is the communication of a form from the object to the interpretant through the mediation of the sign [8]. The object of sign in DNA is a functional, folded, and chemically configured protein production; when a particular gene product is necessary, a signal from the environment activates the expression of a certain gene. The cell as an interpreter alters its internal states triggered by a collective signal transduction pathway to establish the boundary conditions to processes and perform functional process with the genetic material [8,14].

There are two aspects of molecular and supramolecular information transduction: (i) sphingomyelinase activity leading to a lipid-mediated cross-communication between the sphingomyelinase and phospholipase A₂ pathways, and (ii) the compositionally driven lateral organization of whole glial and neuronal membrane interfaces leading to the differential responses. The interfacial properties may express and regulate the generation, transduction, and amplification of molecular and submolecular properties on a temporal and structural scale mainly at mesoscopic level. However, the transduction of information mainly occurs between the microscopic (<10⁻⁶ s, <1 nm) and the macroscopic (>10⁻¹ s, >500 nm) ranges [42].

A physiological mechanism may exist to ensure fast transmission of information to that hemisphere which is more efficient in its processing [43]. This shows how functional information is transmitted by parts to produce a response that serves a purpose for the whole. This is the result of coordinated awareness between the parts processing particular information and the parts receiving the transmitted information for preparing a functional output (response) for the whole.

Signal transduction includes the interrelation and information exchange between phosphohydrolytic pathways producing lipid mediators of signal transduction whose formation modifies the interfacial composition. At the molecular level, there exists cross-talk between a glial cell and the axonal membrane surface. There also exists the subtle intercommunication and response of functional living cells to information contained in the molecular organization of contacting cellular membranes [42].

Thermodynamic coupling [16,17,32,44–49] materialized by certain coupling mechanisms (i.e., configurations, ATPase) has the ability to process information and hence the lower the entropy production σ = ∑_iJ_iX_i > 0. This may lead to self-organization because a process with σ₁ < 0 (unnatural) is thermodynamically coupled with another process with σ₂ >> 0 (natural) to achieve a desired output function; therefore, the coupled system satisfies the second law of thermodynamics [(σ₁ < 0) + (σ₂ >> 0)] > 0. Here, the thermodynamic coupling refers that a flux J occurs without its primary thermodynamic driving force X, or opposite to the direction imposed by the driving force. This is consistent with the second law that states that a finite amount of organization may be obtained at the expense of a greater amount of disorganization in a series of coupled spontaneous processes [16,48,49]. The information processed in the thermodynamic coupling is the functional information, which in turn may lower the total entropy [32]. The “complexity” in (biological) systems that display some of the examples of thermodynamic coupling probably refers to the mechanisms enabling the discrete parts of the whole to be aware of each other hence leading to a coherent and collective behavior to achieve the best degree of desired function [1,3,16,44]. This also indicates that entropy productions of various coupled processes are additive.

Biochemical reactions coupled with diffusion of species can lead to molecular pumps and biochemical cycles, such as the adenosine triphosphate (ATP) synthesis coupled to the respiratory electron transport. This shows a functional process leading to organized structures where the ATP synthesis (σ < 0) has been made possible and the coupled processes satisfy ∑ > 0 [17,31–35]. The ATP synthesis, in turn, is matched and synchronized to cellular ATP utilization processes, since energy and matter flows must be directed by the information for them to be functional and serve a purpose. For living systems, the information is the cause and the entropy reduction is the result [34–36,40–47]:

The general approach for incorporating thermodynamics into the information theory has been to derive probability distributions for nonequilibrium steady states by employing the variational principle. However, composing the appropriate constraints to be used in the variational principle is not clear, since there is no definite extremum quantity to characterize the state space of such steady nonequilibrium states. In the vicinity of equilibrium only, the linear phenomenological laws may be useful in that respect [4,16,17]. Therefore, a natural question is that how useful such an approach would be to describe the information processing in functionally coupled and self-organized biochemical cycles of living systems that are mainly far from equilibrium? The probabilistic measure of information derived from Jaynes’ information theory formalism of statistical mechanics is mainly indifferent to meaning [30].

The unified theory of evolution attempts to explain the origin of biological order as a manifestation of the flows of energy, matter, and information on various spatial and temporal scales. Genes originates the information to synthesize the required enzymes, regulatory and structural proteins. The genome is the source of cellular codes; also any cellular structure such as lipids and polysaccharides may store and transmit information. Besides, thermodynamic forces in the form of transmembrane gradients of H⁺, Na⁺, K⁺, Ca²⁺ and consequent electric potential cause significant displacements from equilibrium, and are therefore, potential sources of information. Genome-protein system may be a component of a large ensemble of cellular structures, which store, encode, and transmit the information [5–10].

Le Chatelier’s principle may be applied to analyze how a protein-signaling network at equilibrium returns to its equilibrium state after being slightly perturbed [20]. For a single cell or small cell colony, cell to cell perturbations are small, while the unperturbed state of a single cell may be unstable in the presence of many other cells. Experiments permit observations of the covariance in the fluctuations and evolution of these fluctuations of different proteins when a single cell is perturbed in the presence of other cells. The information theory helps analyze this covariance to understand the network of interacting proteins [36]. Signal transduction inside and across the cells is complex and cooperative, and occurs in a far-from thermodynamic equilibrium. It is also driven by the activation kinetics strongly dependent on the local energetics [7,20,45].

The composite immediate object of a protein coding gene is the sequence of amino acids of a polypeptide, which can be folded in different ways in different cellular contexts and represents dynamical objects. So sign is a sequence of nucleotides in DNA and becomes effective information only when it is used by an active, coordinated, and coupled system of interpretation in the cell. This means that a range of sequence of amino acids required by the environment (cell) has been constructed [8].

The use of maximum entropy formalism in biology is expanding [23–31] toward detecting expression patterns in signal transduction. At the maximum entropy, the probabilities of the different proteins are not equal; each protein will be present in proportion to its partition function, which is the effective thermodynamic weight of a species at thermal equilibrium. MEP may be the unifying optimization for living systems and ecosystem function, in which entropy production might be a general objective function [24–31,46]. It is stated that “the nonequilibrium state of MEP is the most probable as it can be realized microscopically in a greater number of ways than any other nonequilibrium state.” This statement is not universally accepted [46]. In this sense, MEP is a statistical principle, rather than a physical principle open to experimental validation. MEP may predict the behavior of optimal living system from the perspective of natural selection as well as offer a novel statistical reinterpretation of that behavior that is the survival of the likeliest [29].

For a multicomponent fluid system under mechanical equilibrium with n species and N_r number of chemical reactions, the rate of energy dissipation due to local rate of entropy production σ_τ is [16,49,50]:

where J_i is the vector of mass fluxes, μ_i is the chemical potential of species i, and A is the chemical affinity A = −∑ν_iμ_i. The local mass balance of chemical species i is [16,49]:

For a steady state system, we have ∇ · j_i = ∑_i,jν_ijJ_rj allowing the dissipation to be expressed fully in terms of chemical affinity [29] of many coupled chemical reactions of biochemical cycles.

Considering N number of linear flux-force system J_i = ∑ L_ikX_k [49–54] expressed in matrix form −J = LX in a biochemical cycle, the local rate of entropy production in terms of forces X becomes [16]:

Onsager’s reciprocal relations state that the coefficient matrix L is symmetrical. The L will have N × N elements. The number of elements representing the cross coefficients responsible for the induced effects because of the thermodynamic coupling would be (N²−N)/2. Through the coupling-uncoupling mechanisms, these cross coefficients may be on and off based on the biochemical path and its environment, and leads to (N²−N)/2 bit information. In the absence of pertinent symmetries or invariances, all types of cross-couplings are possible and lead to nonvanishing cross coefficients. If the structure of the system is invariant with respect to some or all of the orthogonal transformations, then the invariance will eliminate certain cross-couplings and their cross-coefficients will vanish.

Thermodynamic coupling may lead to (N²−N)/2 number of possibilities of coupled-uncoupled structures emerging in an N-flux-force system in a biochemical cycle depending on the environmental interpretations. This, in turn, brings out the challenge of implementing the trajectories belonging two or much more coupled processes (correlated with mutual information) with different initial and nonequilibrium states into the fluctuation and information theories.