System Entropy Measurement of Stochastic Partial Differential Systems

Abstract: System entropy describes the dispersal of a system’s energy and is an indication of the disorder of a physical system. Several system entropy measurement methods have been developed for dynamic systems. However, most real physical systems are always modeled using stochastic partial differential dynamic equations in the spatio-temporal domain. No efficient method currently exists that can calculate the system entropy of stochastic partial differential systems (SPDSs) in consideration of the effects of intrinsic random fluctuation and compartment diffusion. In this study, a novel indirect measurement method is proposed for calculating of system entropy of SPDSs using a Hamilton–Jacobi integral inequality (HJII)-constrained optimization method. In other words, we solve a nonlinear HJII-constrained optimization problem for measuring the system entropy of nonlinear stochastic partial differential systems (NSPDSs). To simplify the system entropy measurement of NSPDSs, the global linearization technique and finite difference scheme were employed to approximate the nonlinear stochastic spatial state space system. This allows the nonlinear HJII-constrained optimization problem for the system entropy measurement to be transformed to an equivalent linear matrix inequalities (LMIs)-constrained optimization problem, which can be easily solved using the MATLAB LMI-toolbox (MATLAB R2014a, version 8.3). Finally, several examples are presented to illustrate the system entropy measurement of SPDSs.


Introduction
Information entropy is considered a measure of uncertainty and its maximization guarantees the best solutions for the maximal uncertainty [1][2][3][4][5].Information entropy characterizes uncertainty caused by random parameters of a random system and measurement noise in the environment [6].Entropy has been used for information retrieval such as systemic parametric and nonparametric estimation based on real data, which is an important topic in advanced scientific disciplines such as econometrics [1,2], financial mathematics [4], mathematical statistics [3,4,6], control theory [5,7,8], signal processing [9], and mechanical engineering [10,11].Methods developed within this framework consider model parameters as random quantities and employ the informational entropy maximization principle to estimate these model parameters [6,9].
System entropy describes disorder or uncertainty of a physical system and can be considered to be a significant system property [12].Real physical systems are always modeled using stochastic partial differential dynamic equation in the spatio-temporal domain [12][13][14][15][16][17].The entropy of thermodynamic systems has been discussed in [18][19][20].The maximum entropy generation of irreversible open systems was discussed in [20][21][22].The entropy of living systems was discussed in [19,23].The system entropy of stochastic partial differential systems (SPDSs) can be measured as the logarithm of system randomness, which can be obtained as the ratio of output signal randomness to input signal randomness from the entropic point of view.Therefore, if system randomness can be measured, the system entropy can be easily obtained from its logarithm.The system entropy of biological systems modeled using ordinary differential equations was discussed in [24].However, since many real physical and biological systems are modeled using partial differential dynamic equations, in this study, we will discuss the system entropy of SPDSs.In general, we can measure the system entropy from the system characteristics of a system without measuring the system signal or input noise.For example, a low-pass filter, which is a system characteristic, can be determined from its transfer function or system's frequency response without measuring its input/output signal.Hence, in this study, we will measure the system entropy of SPDSs from the system's characteristics.Actually, many real physical and biological systems are only nonlinear, such as the large-scale systems [25][26][27][28], the multiple time-delay interconnected systems [29], the tunnel diode circuit systems [30,31], and the single-link rigid robot systems [32].Therefore, we will also discuss the system entropy of nonlinear system as a special case in this paper.
However, because direct measurement of the system entropy of SPDSs in the spatio-temporal domain using current methods is difficult, in this study, an indirect method for system entropy measurement was developed through the minimization of its upper bound.That is, we first determined the upper bound of the system entropy and then decreased it to the minimum possible value to achieve the system entropy.For simplicity, we first measure the system entropy of linear stochastic partial differential systems (LSPDSs) and then the system entropy of nonlinear stochastic partial differential systems (NSPDSs) by solving a nonlinear Hamilton-Jacobi integral inequality (HJII)-constrained optimization problem.We found that the intrinsic random fluctuation of SPDSs will increase the system entropy.
To overcome the difficulty in solving the system entropy measurement problem due to the complexity of the nonlinear HJII, a global linearization technique was employed to interpolate several local LSPDSs to approximate a NSPDS; a finite difference scheme was employed to approximate a partial differential operator with a finite difference operator at all grid points.Hence, the LSPDSs at all grid points can be represented by a spatial stochastic state space system and the system entropy of the LSPDSs can be measured by solving a linear matrix inequalities (LMIs)-constrained optimization problem using the MATLAB LMI toolbox [12].Next, the NSPDSs at all grid points can be represented by an interpolation of several local linear spatial state space systems; therefore, the system entropy of NSPDSs can be measured by solving the LMIs-constrained optimization problem.
Finally, based on the proposed systematic analysis and measurement of the system entropy of SPDSs, two system entropy measurement simulation examples of heat transfer system and biochemical system are given to illustrate the proposed system entropy measurement procedure of SPDSs.

General System Entropy of LSPDSs
For simplicity, we will first calculate the entropy of linear partial differential systems (LPDSs).Then, the result will be extended to the measure of NSPDSs.Consider the following LPDS [15,16]: Bt " κ∇ 2 ypx, tq `Aypx, tq `Bvpx, tq zpx, tq " Cypx, tq, where x " rx 1 x 2 s T P U is the space variable, ypx, tq " ry 1 px, tq, . . ., y n px, tqs T P R n is the state variable, vpx, tq " rv 1 px, tq, . . ., v l px, tqs T P R l is the random input signal, and z px, tq " rz 1 px, tq, . . .z m px, tqs T P R m is the output signal.x and t are the space and time variable, respectively.The space domain U is a two-dimensional bounded domain.The system coefficients are κ P R nˆn , A P R nˆn , B P R nˆl , and C P R mˆn .The Laplace (diffusion) operator ∇ 2 is defined as follows [15,16]: Bx k 2 y 1 px, tq, ..., Bx k 2 y n px, tq ( Suppose that the initial value ypx, 0q :" y 0 pxq.For simplicity, the boundary condition is usually given by the Dirichlet boundary condition, i.e., ypx, tq " a constant on B U, or by the Neumann boundary condition ∇ypx, tq ¨á n " 0 on BU, where á n is a normal vector to the boundary BU [15,16].The randomness of the random input signal is measured by the average energy in the domain U and the entropy of the random input signal is measured by the logarithm of the input signal randomness as follows [1,2,24]: where E denotes the expectation operator and t f denotes the period of the random input signal, i.e., vpx, tq P U ˆr0, t f s.Similarly, the entropy of the random output signal zpx, tq is obtained as: In this situation, the system entropy S of the LPDS given in Equation ( 1) can be obtained from the differential entropy between the output signal and input signal, i.e., input signal entropy minus output signal entropy, or the net signal entropy of the LPDS [33]: ) . ( Let us denote the system randomness as the following normalized randomness: Then, the system entropy S " logS 0 .That is, if system randomness can be obtained, the system entropy can be determined from the logarithm of the system randomness.Therefore, our major work of measuring the entropy of the LPDS given in Equation (1) first involves the calculation of the system randomness S 0 given in Equation ( 4).However, it is not easy to directly calculate the normalized randomness S 0 in Equation (4) in the spatio-temporal domain.Suppose there exists an upper bound of S 0 as follows: and we will determine the condition with that S 0 has an upper bound S 0 .Then, we will decrease the value of the upper bound S 0 as small as possible to approach S 0 , and then obtain the system entropy using S " logS 0 .
Remark 1. (i) From the system entropy of LPDS Equation (1), if the randomness of the input signal vpx, tq is larger than the randomness of the output signal zpx, tq, i.e.,: then S 0 ă 1 and S ă 0. A negative system entropy implies that the system can absorb external energy to increase the structure order of the system.All the biological systems are of this type, and according to Schrödinger's viewpoint, biological systems consume negative entropy, leading to construction and maintenance of their system structures, i.e., life can access negative entropy to produce high structural order.(ii) If the randomness of the output signal zpx, tq is larger than the randomness of the input signal vpx, tq, i.e.,: then S 0 ą 1 and S ą 0. A positive system entropy indicates that the system structure disorder increases and the system can disperse entropy to the environment.(iii) If the randomness of the input signal vpx, tq is equal to the randomness of the system signal zpx, tq, i.e.,: then S 0 " 1 and S " 0. In this case, the system structure order is maintained constantly with zero system entropy.(iv) If the initial value y 0 pxq ‰ 0, then the system randomness S 0 in Equation ( 4) should be modified as: for a positive Lyapunov function Vpypx, tqq ą 0, and the randomness due to the initial condition y 0 pxq ‰ 0 should be considered a type of input randomness.
Based on the upper bound S 0 of the system randomness as given in Equation ( 5), we get the following result: Proposition 1.For the LPDS in Equation ( 1), if the following HJII holds for a Lyapunov functio Vpypx, tqq ą 0 and with Vp0q " 0: then the system randomness S 0 has an upper bound S 0 as given in Equation (5).

Proof. See Appendix A.
Since S 0 is the upper bound of S 0 , it can be calculated by solving the following HJII-constrained optimization problem: S 0 " min Vpypx,tqqą0 S 0 subject to the HJII in Equation (10).(11) Consequently, we can calculate the system entropy using S " logS 0 .
Remark 2. If the system in Equation ( 1) is free of a partial differential term ∇ 2 ypx, tq, i.e., in the case of the following conventional linear dynamic system: then the system entropy of linear dynamic system in Equation ( 12) is written as [24]: Therefore, the result of Proposition 1 is modified as the following corollary.

Corollary 1.
For the linear dynamic system in Equation ( 12), if the following Riccati-like inequality holds for a positive definite symmetric P ą 0: or equivalently (by the Schur complement [12]): then the system randomness S 0 of the linear dynamic system in Equation ( 12) has an upper bound S 0 .
Proof.See Appendix B.
Thus, the randomness S 0 of the linear dynamic system in Equation ( 12) is obtained by solving the following LMI-constrained optimization problem: subject to the LMI in Equation (15).( 16) Hence, the system entropy of the linear dynamic system in Equation ( 12) can be calculated using S " logS 0 .The LMI-constrained optimization problem given in Equation ( 16) is easily solved by decreasing S 0 until no positive definite solution P exists for the LMI given in Equation (15), which can be solved using the MATLAB LMI toolbox [12].Substituting S 0 into S 0 in Equation ( 14), we get: The right hand side of Equation ( 17) can be considered as an indication of the system stability.If the eigenvalues of A are more negative (more stable), i.e., the right hand side is more large, then S 0 and the system entropy S, are smaller.Obviously, the system entropy is inversely related to the stability of the dynamic system.If A is fixed, then the increase in input signal coupling B may increase S 0 and S. Remark 3. If the LPDS in Equation (1) suffers from the following intrinsic random fluctuation:
For the LSPDS in Equation ( 19), we get the following result.
then the system randomness S 0 has an upper bound S 0 as given in (5).

Proof. See Appendix C.
Since S 0 is the upper bound of S 0 , it could be calculated by solving the following HJII-constrained optimization problem: S 0 subject to the HJII in Equation ( 20).
Hence, the system entropy of LSPDS in Equations ( 18) or ( 19) could be obtained using S " logS 0 , where S 0 is the system randomness solved from Equations (21).(20) with the HJII in Equation (10) and replacing S 0 with S 0 , we find that Equation (20) has an extra positive term p1{2qy T px, tqH T pB 2 Vpypx, tqq{By 2 q T Hypx, tq due to the intrinsic random parametric fluctuation given in Equation (18).To maintain the left-hand side of Equation (20) as negative, the system randomness S 0 in Equation ( 20) must be larger than the randomness S 0 in Equation (10), i.e., the system entropy of the LPDS in Equations (18) or (19) is larger than that of the LPDS in Equation (1) because the intrinsic random parametric variation Hpypx, tqqwpx, tq in Equation ( 18) can increase the system randomness and the system entropy.

Remark 4. Comparing the HJII in Equation
Remark 5.If the LSPDS in Equation ( 18) is free of the partial differential term ∇ 2 ypx, tq, i.e., in the case of the conventional linear dynamic system: dyptq dt " Ayptq `Bvptq `Hyptqwptq or the following Itô form: dyptq " pAyptq `Bvptqq dt `HyptqdWptq zptq " Cyptq, then we modify Proposition 2 as the following corollary.
Corollary 2. For the linear dynamic system in Equations ( 22) or (23), if the following Riccati-like inequality holds for a positive definite symmetric P " P T ą 0: or equivalently: then the system randomness S 0 of the linear dynamic system in Equations ( 22) or ( 23) has a upper bound S 0 .
Proof.See Appendix D. Therefore, the system randomness S 0 of the linear stochastic system in Equations ( 22) or ( 23) can be obtained by solving the following LMI-constrained optimization problem: S 0 " min Pą0 S 0 subject to the LMI in Equation ( 25), (26) Hence, the system entropy Equation (13) of the linear stochastic system in Equations ( 22) or (23) can be calculated using S " logS 0 , where the system randomness S 0 is the optimal solution of Equation (26).
By substituting S 0 calculated by Equation (26) into Equation (24), we can get: Remark 6. Comparing Equation (27) with Equation ( 17), it can be seen that the term H T PH due to the intrinsic random parametric fluctuation Hyptqwptq in Equation ( 22) can increase the system randomness S 0 which consequently increases the system entropy S.

The System Entropy Measurement of LSPDSs via a Semi-Discretization Finite Difference Scheme
Even though the entropy of the linear systems in Equations ( 12) and ( 22) can be easily measured by solving the optimization problem in Equations ( 16) and (26), respectively, using the LMI toolbox in MATLAB, it is still not easy to solve the HJII-constraint optimization problem in Equations ( 11) and (21) for the system entropy of the LPDS in Equation ( 1) and the LSPDS in Equation (18), respectively.To simplify this system entropy problem, the main method is obtaining a more suitable spatial state space model to represent the LPDSs.For this purpose, the finite difference method and the Kronecker product are used together in this study.The finite difference method is employed to approximate the partial differential term ∇ 2 ypx, tq in Equation (1) in order to simplify the measurement procedure of entropy [14,16].
the Kronecker product are used together in this study.The finite difference method is employed to approximate the partial differential term 2 ( , ) ∇ y x t in Equation (1) in order to simplify the measurement procedure of entropy [14,16].
Consider a typical mesh grid as shown in Figure 1.The state variable ( , ) y x t is represented by x , and the finite difference approximation scheme for the partial differential operator can be written as follows [14,16]: ( , ) .
x x y t y t y t y t y t y t y x t (28) Based on the finite difference approximation in Equation ( 28), the LPDS in Equation ( 1) can be represented by the following finite difference system: Let us denote: )], then we get: κ For the simplification of entropy measurement for the LPDS in Equation (1), we will define a spatial state vector , ( )∈ n k l y t at all grid node in Figure 1.For the Dirichlet boundary conditions [16], the values of , ( ) k l y t at the boundary are fixed.For example, ( , ) 0 = y x t , where ∈∂ x U. We have . Therefore, the spatial state vector ( )∈  nN y t for state variables at all grid nodes is defined as follows: × N N is the number of grid nodes.For example, let  Based on the finite difference approximation in Equation ( 28), the LPDS in Equation ( 1) can be represented by the following finite difference system: x ry k`1,l ptq `yk´1,l ptq `yk,l`1 ptq `yk,l´1 ptq ´4y k,l ptqs `Ay k,l ptq `Bv k,l ptq, k " 1, . . .
where y k,l ptq " ypx, tq ˇˇx"x k,l , v k,l ptq " vpx, tq ˇˇx Let us denote: x ry k`1,l ptq `yk´1,l ptq `yk,l`1 ptq `yk,l´1 ptq ´4y k,l ptqs, then we get: For the simplification of entropy measurement for the LPDS in Equation ( 1), we will define a spatial state vector y k,l ptq P R n at all grid node in Figure 1.For the Dirichlet boundary conditions [16], the values of y k,l ptq at the boundary are fixed.For example, ypx, tq " 0, where x P BU.We have y k,l ptq " 0 at k " 0, N 1 `1 or l " 0, N 2 `1.Therefore, the spatial state vector yptq P R nN for state variables at all grid nodes is defined as follows: where N :" N 1 ˆN2 .Note that N is the dimension of the vector y k,l ptq for each grid node and N 1 ˆN2 is the number of grid nodes.For example, let N 1 " 2 and N 2 " 2, then we have yptq " ry T 1,1 ptq, y T 1,2 ptq,y T 2,1 ptq, y T 2,2 ptqs T P R 4n .To simplify the index of the node y k,l ptq P R n in the spatial state vector yptq P R nN , we will denote the symbol y j ptq P R n to replace y k,l ptq.Note that the index j is from 1 to N, i.e.,: y 1 ptq :" y 1,1 ptq, y 2 ptq :" y 2,1 ptq, ..., y j ptq :" y k,l ptq, ..., y N ptq :" y N 1 ,N 2 ptq, where j " pl ´1qN 1 `k in Equation (32).Thus, the linear difference model of two indices in Equation ( 31) could be represented with only one index as follows: d dt y j ptq " κT j y j ptq `Ay j ptq `Bv j ptq, j " 1, 2, ..., N z j ptq " Cy j ptq, where v j ptq " v k,l ptq with j " pl ´1qN 1 `k and T j is defined as follows: where O n and I N denote the n ˆn zero matrix and N ˆN identity matrix, respectively.
We will collect all states y j ptq of the grid nodes given in Equation ( 33) to the spatial state vector given in Equation (32).The Kronecker product can be used to simplify the representation.Using the Kronecker product, the systems at all grid nodes given in Equation ( 33) can be represented by the following spatial state space system (i.e., the linear dynamic systems of Equation ( 33) at all grid points within domain U in Figure 1 are represented by a spatial state space system [14]): where T " rT T 1 . . .T T N s P R nNˆnN , vptq " pv 1 ptq . . .v N ptqq T P R lN , and I N b κ denotes the Kronecker product between I N and κ.Definition 1. [17,36]: Let M P R aˆb , N P R cˆd .Then the Kronecker product of M and N is defined as the following matrix: Remark 7. Since the spatial state vector yptq in Equation ( 32) is used to represent ypx, tq at all grid points, in this situation, E and E " ş T px, tqvpx, tq dtdxu in the measurement of system randomness in Equations ( 5) or ( 9) could be modified by the temporal forms , and E ) , respectively, for the spatial state space system in (35), where the Lyapunov function Vpyptqq is related to the Lyapunov function Vpypx, tqq as Vpyptqq " ř N j"1 Vpy j ptqq.Therefore, for the spatial state space system in Equation (35), the system randomness in Equations ( 5) or ( 9) is modified as follows: ) ď S 0 (36) or: Hence, our entropy measurement problem of the LPDS in Equation ( 1) becomes the measurement of the entropy of the spatial state system Equation (35), as given below.Proposition 3.For the linear spatial state space system in Equation (35), if the following Riccati-like inequality holds for a positive definite matrix P ą 0: or equivalently: where A " rI N b κsT `rI N b As, B " rI N b Bs, C " rI N b Cs , then the system randomness S 0 in Equations ( 36) or (37) of linear spatial state space system in Equation ( 35) has the upper bound S 0 .
Proof.The proof is similar to the proof of Corollary 1 in Appendix B and can be obtained by replacing A, B, C, and P with A, B, C, and P, respectively.
Therefore, the randomness S 0 of the linear spatial state space system in Equation ( 35) can be obtained by solving the following LMI-constrained optimization problem: S 0 " min Pą0 S 0 subject to the LMI in Equation (39).(40) Hence, the system entropy S of the linear spatial state space system in Equation ( 33) can be calculated using S " logS 0 .

Remark 8. (i)
The Riccati-like inequality in Equation (38) or the LMI in Equation ( 39) is an approximation of the HJII in Equation (10) with the finite difference scheme given in Equation (28).If the finite difference, shown in Equation (28), ∆ x Ñ 0 , then S 0 in Equation (40) will approach S 0 in Equation (11).(ii) Substituting S 0 into Equation (38), we get: If the eigenvalues of A are more negative (more stable), the randomness S 0 as well as the entropy S is smaller.Similarly, the LSPDS in Equation ( 18) can be approximated by the following stochastic spatial state space system via finite difference scheme [14]: where dW " rdW 1 ptq . . .dW N ptqs P R nN , and the Hadamard product of matrices (or vectors) X " rX ij s mˆn and Y " rY ij s mˆn of the same size is the entry-wise product denoted as X ˝Y " rX ij Y ij s mˆn .
Then we can get the following result.
Corollary 3.For the linear stochastic spatial state space system Equation ( 42), if the following Riccati-like inequality holds for a positive definite symmetric P ą 0: or equivalently, the following LMI has a positive definite symmetric solution P ą 0: then the system randomness S 0 of the stochastic state space system in Equation ( 42) has an upper bound S 0 , where H " rI N b Hs.
Proof.The proof is similar to the proof of Corollary 2 in Appendix D. Therefore, the system randomness S 0 of the linear stochastic state space system Equation ( 42) can be obtained by solving the following LMI-constrained optimization problem: subject to the LMI in (44), (45) and hence the system entropy S of the stochastic spatial state space system in Equation ( 42) can be obtained using S " logS 0 .Substituting S 0 into (43), we get: Remark 9. Comparing Equation (41) with Equation (46), because of the term H T PH from the intrinsic random fluctuation, it can be seen that the LSPDS with random fluctuations will lead to a larger S 0 and a larger system entropy S.

System Entropy Measurement of NSPDSs
Most partial dynamic systems are nonlinear; hence, the measurement of the system entropy of nonlinear partial differential systems (NPDSs) will be discussed in this section.Consider the following NPDSs in the domain U: where f pypx, tqq P R n , Cpypx, tqq P R mˆn and gpypx, tqq P R nˆl are the nonlinear functions with f p0q " 0, Cp0q " 0, and gp0q " 0, respectively.The nonlinear diffusion functions κpypx, tqq P R nˆn satisfy κpypx, tqq ě 0, and κp0q " 0. If the equilibrium point of interest is not at the origin, for the convenience of analysis, the origin of the NPDS must be shifted to the equilibrium point (shifted to zero).The initial and boundary conditions are the same as the LPDS in Equation (1); then, we get the following result.Proposition 4. For the NPDS in Equation (47), if the following HJII holds for a Lyapunov function Vpypx, tqq ą 0 with Vp0q " 0: then the system randomness S 0 of the NPDS in Equation ( 47) has an upper bound S 0 as given in Equation (5).
Based on the condition of upper bound S 0 given in Equation (48), the system randomness S 0 could be obtained by solving the following HJII-constrained optimization problem: S 0 subject to the HJII in Equation ( 48).
(49) Hence, the system entropy of NPDS in Equation (47) can be obtained using S " logS 0 .If the NPDS in Equation ( 47) is free of the diffusion operator ∇ 2 ypx, tq as with the following conventional nonlinear dynamic system: then the result of Proposition 4 is reduced to the following corollary.
Corollary 4. For the nonlinear dynamic system Equation (50), if the following HJII holds for a positive Lyapunov function Vpyptqq ą 0 with Vp0q " 0: then the system randomness S 0 of the nonlinear dynamic system in Equation (50) has an upper bound S 0 Proof.The proof is similar to that of Proposition 4 without consideration of the diffusion operator ∇ 2 ypx, tq and spatial integration on the domain U.
Remark 10.By comparing the HJII in Equation (48) with the HJII in Equation ( 55), due to the extra term p1{2qy T px, tqH T pypx, tqqpB 2 Vpypx, tqq{By 2 q T Hpypx, tqqypx, tq from the random intrinsic fluctuation Hpypx, tqqypx, tqwpx, tq in Equation (53), it can be seen that the system randomness of the NSPDS in Equation (53) must be larger than the system randomness of the NPDS in Equation (47).Hence, the system entropy of the NSPDS in Equation ( 53) is larger than that of the NPDS in Equation (47).

System Entropy Measurement of NSPDS via Global Linearization and Semi-Discretization Finite Difference Scheme
In general, it is very difficult to solve the HJII in Equations ( 48) or (55) for the system entropy measurement of the NPDS in Equation (47) or the NSPDS in Equation (53), respectively.In this study, the global linearization technique and a finite difference scheme were employed to simplify the entropy measurement of the NPDS in Equation (47) and NSPDS in Equation (53).Consider the following global linearization of the NPDS in Equation (47), which is bounded by a polytope consisting of L vertices [12,37]: where C 0 denotes the convex hull of a polytope with L vertices defined in Equation (57).Then, the trajectories of ypx, tq for the NPDS in Equation (47) will belong to the convex combination of the state trajectories of the following L linearized PDSs derived from the vertices of the polytope in Equation (57): Bypx, tq Bt " κ i ∇ 2 ypx, tq `Ai ypx, tq `Bi vpx, tq, i " 1, 2, ..., L zpx, tq " C i ypx, tq. (58) From the global linearization theory [16,37], if Equation (57) holds, then every trajectory of the NPDS in Equation ( 47) is a trajectory of a convex combination of L linearized PDSs in Equation (58), and they can be represented by the convex combination of L linearized PDSs in Equation (58) as follows:

Bypx, tq
Bt " where the interpolation functions are selected as α i pyq " p1{||y i ´y|| 2 2 q{p ř L i"1 ||y i ´y|| 2 2 q and they satisfy 0 ď α i pyq ď 1 and ř L i"1 α i pyq " 1.That is, the trajectory of the NPDS in Equation ( 47) can be approximated by the trajectory of the interpolated local LPDS given in Equation (59).
Following the semi-discretization finite difference scheme in Equations ( 28)-( 34), the spatial state space system of the interpolated PDS in Equation ( 59) can be represented as follows: where yptq and vptq are defined in (35).That is, the NPDS in Equation ( 47) is interpolated through local linearized PDSs in Equation (59) to approximate the NPDS in Equation (47) using global linearization and semi-discretization finite difference scheme.
Remark 10.In fact, there are many interpolation schemes for approximating a nonlinear dynamic system with several local linear dynamic systems such as Equation (60); for example, fuzzy interpolation and cubic spline interpolation methods [13].Then, we get the following result.Proposition 6.For the linear dynamic systems in Equation (60), if the following Riccati-like inequalities hold for a positive definite symmetric P ą 0: or equivalently: where A i , B i , and C i are defined as A i " rI N b κ i sT `rI N b A i s, B i " rI N b B i s, and C i " rI N b C i s, respectively, then the system randomness S 0 of the NPDSs in Equation (47) or the interpolated dynamic systems in Equation (60) have an upper bound S 0 .
Proof.See Appendix G. Therefore, the system randomness S 0 of the NPDSs in Equation (47) or the interpolated dynamic systems in Equation (60) can be obtained by solving the following LMIs-constrained optimization problem: S 0 " min Pą0 S 0 subject to the LMI in Equation ( 62).
(63) Hence, the system entropy S of the NPDSs in Equation (47) or the interpolated dynamic systems in Equation (60) can be obtained using S " logS 0 .By substituting S 0 into the Riccati-like inequalities in Equation (61), we can obtain: Obviously, if the eigenvalues of local system matrices A i are more negative (more stable), the randomness S 0 is smaller and the corresponding system entropy S is also smaller, and vice versa.
The NSPDs given in Equation ( 54) can be approximated using the following global linearization technique [12,37]: Then, the NSPDs with the random intrinsic fluctuation given in Equation ( 53) can be approximated by the following interpolated spatial state space system [14]: i.e., we could interpolate L local interpolated stochastic spatial state space systems to approximate the NSPDs in Equation (53).Then, we get the following result.
Proposition 7.For the NSPDs in Equation (54) or the linear interpolated stochastic spatial state space systems in (66), if the following Riccati-like inequalities hold for a positive definite symmetric P ą 0: or equivalently: where H i " rI N b H i s , then the system randomness S 0 of the NSPDs in Equation (53) or the interpolated stochastic systems in Equation (66) can be obtained by solving the following LMIs-constrained optimization problem: S 0 " min Pą0 S 0 subject to the LMIs in Equation p68q.
(69) Then, the system entropy S of NSPD in Equation (53) or the interpolated stochastic systems in Equation (66) could be obtained as S " logS 0 .

Proof. See Appendix H.
Substituting S 0 into in Equation (67), we get: Comparing (64) with Equation (70), S 0 of the NSPDS in Equation ( 53) is larger than S 0 of the NPDS in Equation (47), i.e., the random intrinsic fluctuation Hpypx, tqqypx, tqwpx, tq will increase the system entropy of the NSPDS.Based on the above analysis, the proposed system entropy measurement procedure of NSPDSs is given as follows: Step 1: Given the initial value of state variable, the number of finite difference grids, the vertices of the global linearization, and the boundary condition.

Step 2:
Construct the spatial state space system in Equation (60) by finite difference scheme.

Step 3:
Construct the interpolated state space system Equation (66) by global linearization method.

Step 4:
If the error between the original model Equation ( 54) and the approximated model Equation ( 66) is too large, we could adjust the density of grid nodes of finite difference scheme and the number of vertices of global linearization technique and return to Step 1.

Step 5:
Solve the eigenvalue problem in Equation (69) to obtain P and S 0 , and then system entropy S " logS 0 .

Computational Example
Based on the aforementioned analyses for the system entropy of the considered PDSs, two computational examples are given below for measuring the system entropy.
Example 1.Consider a heat transfer system in a 1 m ˆ0.5 m thin plate with a surrounding temperature of 0 ˝C as follows [38]: Bypx, tq Bt " κ∇ 2 ypx, tq `Aypx, tq `Bvpx, tq zpx, tq " Cypx, tq, ypx, 0q " 20 ˆep´10ˆ|0.5´x 1 |´0.6738qˆep´30ˆ|0.5´2x 2 |q and y(x,t) = 0 ˝C, @ t, @ x on the boundary of Here, y(x,t) is the temperature function, location x is in meters, time t is in s, κ " 10 ´4 m 2 {s is the thermal diffusivity [4-7,9], and the term Aypx, tq with A " ´0.1 s ´1 denotes the thermal dissipation when the temperature of the plate is greater than the surrounding temperature, i.e., ypx, tq ą 0 ˝C, or the thermal absorption when the temperature on the plate is less than the surrounding temperature, i.e., ypx, tq ă 0 ˝C.The output coupling C " 1. Bvpx, tq is the environmental thermal fluctuation input with B " 0.1.We can estimate the system entropy of the heat transfer system in Equation (71).Based on Proposition 3 and the LMI-constrained optimization problem Equation ( 40), we can calculate the system entropy of the heat transfer system in Equation (71) as S " logS 0 " logp0.0046q" ´2.3372.In this calculation of the system entropy, the grid spacing ∆ x of the finite difference scheme was chosen as 0.125m such that there are N " 7 ˆ3 " 21 interior grid points and 24 boundary points in U.The temperature distributions ypx, tq of the heat transfer system in Equation (71) at t " 1, 10, 30 and 50 s are shown in Figure 2 with vpx, tq" 30sinptq.Due to the diffusion term κ∇ 2 ypx, tq, the heat temperature of transfer system Equation (71) will be uniformly distributed gradually.Even if the thin plate has initial value (heat source) or some other influences like input signal and intrinsic random fluctuation, the temperature of the thin plate will gradually achieve a uniform distribution to increase the system entropy.This phenomenon can be seen in Figures 2-5.
Δ x of the finite difference scheme was chosen as 0.125m such that there are κ∇ y x t , the heat temperature of transfer system Equation (71) will be uniformly distributed gradually.Even if the thin plate has initial value (heat source) or some other influences like input signal and intrinsic random fluctuation, the temperature of the thin plate will gradually achieve a uniform distribution to increase the system entropy.This phenomenon can be seen in Figures 2-5.κ∇ y x t , the temperature of heat system will be uniformly distributed gradually to increase the system entropy.
Suppose that the heat transfer system in Equation (71) suffers from the following random intrinsic fluctuation:   Figure 2. The temperature distribution ypx, tq of the heat transfer system given in Equation (71) at t " 1, 10, 30 and 50 s.Due to the diffusion term κ∇ 2 ypx, tq, the temperature of heat system will be uniformly distributed gradually to increase the system entropy.
Entropy 2016, 18, 99 18 of 30 the system entropy of the stochastic heat transfer system in Equation ( 72) is larger than the heat transfer system in Equation (71) without intrinsic random fluctuation.and 50 s.Obviously, the temperature distribution of stochastic heat transfer system in Equation ( 72) is with more random fluctuations and with more system entropy than the heat transfer system in Equation ( 71).The temperature distribution is also uniformly distributed gradually to increase the system entropy as time goes on.In general, the temperature in Figure 3 is more random than Figure 2, i.e., with more system randomness and entropy.

Example 2.
A biochemical enzyme system is used to describe the concentration distribution of the substrate  72) at t " 1, 10, 30 and 50 s.Obviously, the temperature distribution of stochastic heat transfer system in Equation ( 72) is with more random fluctuations and with more system entropy than the heat transfer system in Equation (71).The temperature distribution is also uniformly distributed gradually to increase the system entropy as time goes on.In general, the temperature in Figure 3 is more random than Figure 2, i.e., with more system randomness and entropy.
Suppose that the heat transfer system in Equation (71) suffers from the following random intrinsic fluctuation:

Bypx, tq
Bt " κ∇ 2 ypx, tq `Aypx, tq `Bvpx, tq `Hypx, tqwpx, tq zpx, tq " Cypx, tq, where the term Hypx, tqwpx, tq with H " 0.02 is due to the random parameter variation of the term Ay px, tq.Then, the temperature distributions ypx, tq of the heat transfer system in Equation (72) at t " 1, 10, 30 and 50 s are shown in Figure 3. Based on the Corollary 3 and the LMI-constrained optimization problem in Equation ( 45), we can calculate the system entropy of the stochastic heat transfer system in Equation ( 72) as S " logS 0 " logp0.0339q" ´1.4698.Obviously, it can be seen that the system entropy of the stochastic heat transfer system in Equation ( 72) is larger than the heat transfer system in Equation ( 71) without intrinsic random fluctuation.
Entropy 2016, 18, 99 19 of 30 Clearly, the approximated system based on the global linearization technique and finite difference scheme efficiently approach the nonlinear function.Based on Proposition 6 and the LMIs-constrained optimization given in Equation ( 63), we can obtain P as shown in detail in Appendix J and calculate the system entropy of the enzyme system in Equation ( 73 73) and the approximated system in Equation (60).Obviously, the approximated system based on finite difference scheme and global linearization method can approximate the biochemical enzyme system quite well.
Therefore, it is clear that the approximated system in Equation ( 60) can efficiently approximate biochemical enzyme system in Equation (73).In this simulation, 0.125 Δ = x .Suppose that the biochemical system in Equation (73) suffers from the following random intrinsic fluctuation:    73) and the approximated system in Equation (60).Obviously, the approximated system based on finite difference scheme and global linearization method can approximate the biochemical enzyme system quite well.
Example 2. A biochemical enzyme system is used to describe the concentration distribution of the substrate in a biomembrane.For the enzyme system, the thickness of the artificial biomembrane is 1 µm.The concentration of the substrate is uniformly distributed inside the artificial biomembrane.
Since the biomembrane is immersed in the substrate solution, the reference axis is chosen to be perpendicular to the biomembrane.The biochemical system can be formulated as follows [13]:

Bypx, tq
Bt " κ pypx, tqq ∇ 2 ypx, tq ´VM ypx, tq K M `ypx, tq `y2 px, tq{K S `g pypx, tqq vpx, tq zpx, tq "Cypx, tq, where ypx, tq is the concentration of the substrate in the biomembrane, κ is the substrate diffusion coefficient, V M is the maximum activity in one unit of the biomembrane, K M is the Michaelis constant, and K S is the substrate inhibition constant.The parameters of the biochemical enzyme system are given by κpypx, tqq " e ypx,tq , V M " 0.5, K M " 1, K S " 1 and the output coupling C " 1.Note that the equilibrium point in Example 2 is at zero.The concentration of the initial value of the substrate is given by y 0 pxq " 0.3sinpπxq.The boundary conditions used to restrict the concentration are zero at x " 0 and x " 1, i.e., yp0, tq " 0, yp1, tq " 0. A more detailed discussion about the enzyme can be found in [13].Suppose that the biochemical enzyme system is under the effect of an external signal vpx, tq.For the convenience of computation, the external signal vptq is assumed as a zero mean Gaussian noise with a unit variance.The influence function of external signal is defined as gpypx, tqq " 0.5ypx, tq at x " 4{9, 5{9, and 6{9 (µm).Based on the global linearization in Equation (57), we get A 1 ´A3 and B 1 ´B3 , as shown in detail in Appendix I.The concentration distributions ypx, tq of the real system and approximated system are given in Figure 4 with ∆ x " 0.125, i.e., y i ptq " ry 1 ptq, y 2 ptq, ..., y 9 ptqs " ry 1 p0, tq, y 2 p0.125, tq, y 3 p0.375,tq, y 4 p0.5, tq, y 5 p0.625, tq, y 6 p0.75, tq, y 7 p0.875, tq, y 8 p0.125, tq, y 9 p1, tqs.Clearly, the approximated system based on the global linearization technique and finite difference scheme efficiently approach the nonlinear function.Based on Proposition 6 and the LMIs-constrained optimization given in Equation (63), we can obtain P as shown in detail in Appendix J and calculate the system entropy of the enzyme system in Equation (73) as S " logS 0 " logp7.6990ˆ10 ´7q " ´6.1136.
Therefore, it is clear that the approximated system in Equation (60) can efficiently approximate biochemical enzyme system in Equation (73).In this simulation, ∆ x " 0.125.Suppose that the biochemical system in Equation (73) suffers from the following random intrinsic fluctuation:
Clearly, because of the intrinsic random parameter fluctuation, the system entropy of the stochastic enzyme system given in Equation ( 74) is larger than that of the enzyme system given in Equation (73).
The computation complexities of the proposed LMI-based indirect entropy measurement method is about Oprnpn `1q{2q in solving LMIs, where n is the dimension of P, r is the number of global interpolation points.We also calculate the elapsed time of the simulations examples by using MATLAB.The computation times including the drawing of the corresponding figures to solve the LMI constrained optimization problem are given as follows: in Example 1, the case of heat transfer system in Equation ( 71) is 183.9 s; the case of heat transfer system with random fluctuation in Equation ( 72) is 184.6 s.In Example 2, the case of biochemical system in Equation ( 73) is 17.7 s, the case of biochemical system with random fluctuation in Equation ( 74) is 18.6 s.The RAM of the computer is 4.00 GB, the CPU we used is AMD A4-5000 CPU with Radeon(TM) HD Graphics, 1.50 GHz.The results are reasonable.Because the dimension of grid nodes in Example 1 is 45 ˆ45 and the dimension of grid nodes in Example 2 is 9 ˆ9, obviously, the computation time in Example 1 is much larger than in Example 2. Further, the time spent of the system without the random fluctuation is slightly faster than the system with the random fluctuation.The conventional algorithms of calculating entropy have been applied in image processing, digital signal processing, and particle filters, like in [39][40][41].The conventional algorithms for calculating entropy just can be used in linear discrete systems, but in fact many systems are nonlinear and continuous.The indirect entropy measurement method we proposed can deal with the nonlinear stochastic continuous systems.Though the study in [24] is about the continuous nonlinear stochastic system, many physical systems are always modeled using stochastic partial differential dynamic equation in the spatio-temporal domain.The indirect entropy measurement method we proposed can be employed to solve the system entropy measurement in nonlinear stochastic partial differential system problem.74) and the approximated system in Equation (66).Obviously, the approximated system in Equation (66) could approximate the real system in Equation (74) quite well.
Clearly, because of the intrinsic random parameter fluctuation, the system entropy of the stochastic enzyme system given in Equation ( 74) is larger than that of the enzyme system given in Equation (73).
The computation complexities of the proposed LMI-based indirect entropy measurement method is about 1 ( ( )/ 2) + O rn n in solving LMIs, where n is the dimension of P , r is the number of global interpolation points.We also calculate the elapsed time of the simulations examples by using MATLAB.The computation times including the drawing of the corresponding figures to solve the LMI constrained optimization problem are given as follows: in Example 1, the case of heat transfer  74) and the approximated system in Equation (66).Obviously, the approximated system in Equation (66) could approximate the real system in Equation (74) quite well.

Conclusions
In this study, the system entropy of stochastic partial differential systems (SPDSs) was introduced as the difference between input signal entropy and output signal entropy and was found to be the logarithm of the output signal randomness-to-input signal randomness ratio.We found that the system stability was inversely related to the system entropy and that intrinsic random fluctuation could increase the system entropy.If the eigenvalues of the system matrices are further in the left-hand side of the s-complex domain, then the SPDS has lower system entropy, and vice versa.If the output and the input signal randomness values are equal and the system is independent of the initial value, then the system entropy is zero.To estimate the system entropy of nonlinear stochastic partial differential systems (NSPDSs), the global linearization technique and finite difference scheme were employed to represent the NSPDS using the spatial state space system given in Equation (66).Therefore, the system entropy measurement problem of NPDSs became the problem of solving the HJII-constrained optimization problem given in Equation ( 55), which can be replaced by a simple LMIs-constrained optimization problem given in Equation (69).Hence, using the LMI-toolbox of MATLAB, we could easily calculated the system entropy of NSPDS.Finally, two examples were provided to illustrate the measurement procedure of the system entropy and to confirm that the PDSs with intrinsic random fluctuation possess greater system entropy.

Appendixes
Lemma 1. [12]: For any matrices (or vectors) X, Y, and a symmetric matrix P " P T ą 0 with appropriate dimensions, we have: for any positive constant ξ.
Lemma 2. [42]: Let M i be any matrix with appropriate dimension and α i pzq be the interpolation function for the i th local system and P " P T ą 0 .Then, we have: With Lemma 2, the LMI-constrained optimization in Equations ( 62) or (68) can be solved efficiently.

Appendix A.
Proof. of Proposition 1.
From the fact that Vpypx, t f qq ě 0, we have: `ˆBV pzpx, tqq Bt ˙T `κ∇ 2 ypx, tq `Aypx, tq `Bvpx, tq ˘ff dtdx In this situation, we will follow the proof procedure in Appendix A: From the fact that Vpypx, t f qq ě 0 and E tdWpx, tqu " 0, substituting Equation (C1) into Equation (C2), we get: Therefore, if the HJII given in Equation ( 20) holds, then the inequality of system randomness in Equation ( 9) holds.If the initial condition y 0 pxq " 0, then Vpy 0 pxqq " 0, and the inequality of the system randomness in Equation ( 5) holds.

E
From the fact that

Proof. of Proposition 5
For the NPDS given in Equation ( 54 If the HJII in Equation (55) holds, then the system randomness S 0 of the NSPDSs in Equations ( 53) or (54) has an upper bound S 0 as Equation (5) or Equation (9).
From the fact of the following inequality: from Lemma 2, and the choice of Vpyptqq " y T ptqPyptq, we get: If the inequalities in Equations ( 61) or (62) holds, then we get Equation (36) if yp0q " 0 or Equation (37) if yp0q ‰ 0; i.e., S 0 has an upper bound S 0 as shown in Equations ( 36) or (37).

Figure 1 .
Figure 1.Finite difference grids on the spatio-domain U.

Figure 1 .
Figure 1.Finite difference grids on the spatio-domain U.

Figure 2 .
Figure 2. The temperature distribution ( , ) y x t of the heat transfer system given in Equation (71) at = t 1, 10, 30 and 50 s.Due to the diffusion term

,
Ay x t .Then, the temperature distributions ( , ) y x t of the heat transfer system in Equation (72) at = t 1, 10, 30 and 50 s are shown in Figure3.Based on the Corollary 3 and the LMI-constrained optimization problem in Equation (45), we can calculate the system entropy of the stochastic heat transfer system in Equation (72

Figure 3 .
Figure3.The temperature distribution ypx, tq of heat transfer system in Equation (72) at t " 1, 10, 30 and 50 s.Obviously, the temperature distribution of stochastic heat transfer system in Equation (72) is with more random fluctuations and with more system entropy than the heat transfer system in Equation (71).The temperature distribution is also uniformly distributed gradually to increase the system entropy as time goes on.In general, the temperature in Figure3is more random than Figure2, i.e., with more system randomness and entropy.

Figure 4 .
Figure 4. (a) Spatial-time profiles of the real biochemical system in Equation (73); (b) Spatial-time profiles of the approximated system in Equation (60) based on the finite difference scheme and global linearization technique; (c) The error between the real biochemical system in Equation (73) and the approximated system in Equation (60).Obviously, the approximated system based on finite difference scheme and global linearization method can approximate the biochemical enzyme system quite well.

Figure 4 .
Figure 4. (a) Spatial-time profiles of the real biochemical system in Equation (73); (b) Spatial-time profiles of the approximated system in Equation (60) based on the finite difference scheme and global linearization technique; (c) The error between the real biochemical system in Equation (73) and the approximated system in Equation (60).Obviously, the approximated system based on finite difference scheme and global linearization method can approximate the biochemical enzyme system quite well.

Entropy 2016 ,Figure 5 .
Figure 5. (a) Spatial-time profiles of the real biochemical system in Equation (74); (b) Spatial-time profiles of the approximated system in Equation (66) based on the finite difference scheme and global linearization technique; (c) The error between the real biochemical system in Equation (74) and the approximated system in Equation (66).Obviously, the approximated system in Equation (66) could approximate the real system in Equation (74) quite well.

Figure 5 .
Figure 5. (a) Spatial-time profiles of the real biochemical system in Equation (74); (b) Spatial-time profiles of the approximated system in Equation (66) based on the finite difference scheme and global linearization technique; (c) The error between the real biochemical system in Equation (74) and the approximated system in Equation (66).Obviously, the approximated system in Equation (66) could approximate the real system in Equation (74) quite well.