On the Symmetry Importance in a Relative Entropy Analysis for Some Engineering Problems

Abstract

This paper aims at certain theoretical studies and additional computational analysis on symmetry and its lack in Kullback-Leibler and Jeffreys probabilistic divergences related to some engineering applications. As it is known, the Kullback-Leibler distance in between two different uncertainty sources exhibits a lack of symmetry, while the Jeffreys model represents its symmetrization. The basic probabilistic computational implementation has been delivered in the computer algebra system MAPLE 2019^®, whereas engineering illustrations have been prepared with the use of the Finite Element Method systems Autodesk ROBOT^® & ABAQUS^®. Determination of the first two probabilistic moments fundamental in the calculation of both relative entropies has been made (i) analytically, using a semi-analytical approach (based upon the series of the FEM experiments), and (ii) the iterative generalized stochastic perturbation technique, where some reference solutions have been delivered using (iii) Monte-Carlo simulation. Numerical analysis proves the fundamental role of computer algebra systems in probabilistic entropy determination and shows remarkable differences obtained with the two aforementioned relative entropy models, which, in some specific cases, may be neglected. As it is demonstrated in this work, a lack of symmetry in probabilistic divergence may have a decisive role in engineering reliability, where extreme and admissible responses cannot be simply replaced with each other in any case.

1. Introduction

Uncertainty analysis is one of the most important aspects of a solution to real engineering problems [1], in which geometrical, and material imperfections as well as a statistical scattering of the environmental actions may play a decisive role. Such an analysis may concern the theoretical or numerical determination of probabilistic characteristics of the structural response (expectations, variances, etc.) or finding a difference (distance) of maximum (extreme) responses to their admissible counterparts. This second problem is fundamental for reliability assessment [2], in which engineering structures or systems are verified in the context of deformations, vibrations, stresses, temperatures, or also other physical quantities. Both extreme responses and also their admissible counterparts may have an uncertain nature so that a realistic mathematical problem would be to find the probabilistic distance between two different probability density functions, where one is discrete, and the opposite–most frequently–has continuous character. Therefore, it would be quite natural to apply the probabilistic divergence [3] (relative entropy) to measure the safety margin of both existing as well as newly designed structures and systems. This idea has been explored yet in the literature a few times and may also concern some other research issues like detecting a difference in-between the experimental and numerical results, which is a subject of the specific calibration procedures in deterministic analyses. Some other applications may be seen in eigenfrequency and the induced frequency in forced vibrations of the given system, the extreme and phase change temperatures in the given material, etc. One of the very specific ideas is looking for a distance between original material characteristics and their effective counterparts resulting from some homogenization procedures.

Relative entropy [4] (probabilistic distance) applicable in this context may be defined for univariate and multivariate random sources but always demands a precise definition of two probability distributions (or at least their histograms biased with relatively small numerical errors). Most mathematical models in this area show full symmetry, which means that a choice of primary and target distribution makes no difference. The only exception is the so-called Kullback-Leibler distance [3], which is affected by this choice and which exhibits a certain lack of symmetry and which could be very sensitive to contrasted distribution types and parameters. Mathematical proofs and considerations have been demonstrated in the literature; nevertheless, the importance of real engineering systems and their reliability has not been precisely studied. Such a lack of symmetry may result in some underestimation (or overestimation) of structural safety, which leads to non-optimal design decisions or even to designing over the realistic limits and demands; such an effect is an unwanted situation in both cases.

This asymmetry is the main research problem studied in this work, in which a two-fold methodology has been proposed. The first approach is based upon symbolic integration inherent in the Kullback-Leibler entropy (KL) definition assuming for a brevity of presentation two different Gaussian probability distributions with given intervals of their parameter variations. It is very rare in real engineering problems that the given system response may be described with a specific probability density function and its parameters. That is why the second method of the KL entropy determination is the most common application of the Monte-Carlo simulation as well as the generalized stochastic perturbation technique together with the Finite Element Method (FEM) [5] program. Such a two-fold methodology guarantees a huge variety of possible applications, especially because all probabilistic procedures employed in this study have been programmed by the author in the numerical environment of the system MAPLE. The few computational experiments attached here confirm an importance of the KL entropy asymmetry in both engineering and applied sciences problems while studying a distance between extreme and admissible structural response functions.

2. Historical and Mathematical Background

Probabilistic divergence (also known as the relative entropy [6]) has directly followed the foundations of thermodynamics, where Gibbs (1878) entropy of a given thermodynamical system has been introduced in the statistical context as

$$S=-k_B{\displaystyle \sum _i{p}_i\ln \left(p_i\right)}$$

(1)

where $k_B$ is the Boltzmann constant (1872), while $p_i$ denotes the probabilities of all the microstates. Johann von Neumann (1927) modified this definition to the following form:

$$S=-k_{B}Tr\left(\mathsf{\rho } \ln \left(\mathsf{\rho }\right)\right)$$

(2)

with $\mathsf{\rho }$ being the density matrix of the given quantum mechanical system. Finally, Shannon (1948) [7] proposed this entropy as

$$H=-{\displaystyle \sum _i{p}_i\ln \left(p_i\right)}$$

(3)

which, with some small modifications and extensions toward continuous probability distributions, has been used until now. Lower index i counts here and further the given populations of random accidents. Quite independently, probabilistic divergence itself has been invented and developed more than a hundred years ago (Hellinger, 1905 [8]) to measure a distance in-between two different probability distributions. It has received a lot of attention in mathematical studies, but even more in applied sciences and engineering applications. Some of them concern quantum mechanics, fracture analysis and some other issues exhibiting statistical scattering. One of the most famous models is the so-called Kullback-Leibler (KL) model [9], whose principal deficiency is a lack of symmetry while analyzing a divergence of two given distributions p and q according to its discrete version.

$$D_{KL}\left(p,q\right)={\displaystyle \sum _i{p}_i\log \left(\frac{p_i}{q_i}\right)}$$

(4)

In general cases, such as $D_{KL}\left(p,q\right)\ne D_{KL}\left(q,p\right)$, Jeffreys [10] have proposed a symmetrization of this divergence by simply adding

$$D_J\left(p,q\right)=D_{KL}\left(p,q\right)+D_{KL}\left(q,p\right)$$

(5)

A lack of symmetry in the KL divergence is of paramount importance while analyzing engineering reliability problems, where p may denote structural effort and q means structural strength, for instance. Now, the area of relative entropy includes more advanced models adjacent to both univariate and multivariate PDFs as Jensen-Shannon [11,12], Rényi [13], Bhattacharyya [14], Mahalanobis [15], Tsallis [16], and others have proposed. Nevertheless, symmetry, its possible lack, and its consequences for the probabilistic divergence remain one of the most important theoretical and numerical topics.

It is well known that KL divergence has a huge number of well-documented applications in various branches of engineering starting from energy analysis [17,18], marine engineering [19], mechanical problems of failure [20,21,22] as well as in civil engineering analyses [23]. Some recent mathematical works concern the maximum entropy principle [24] or multivariate distributions [25], while physicists are interested in i.e., non-equilibrium random motion [26]. A comparison of various probabilistic divergence models is also available in the literature and is documented for Hellinger and Kullback-Leibler theories [27], Jensen-Shannon and KL distances [28] as well as for a relation of the latter model of Tsallis statistics [29].

The main aim of this work is to compare two closely related probabilistic relative entropies proposed by Kullback-Leibler and Jeffreys [30], and this is done to study an impact of a lack of symmetry in the first theory. The numerical study proposed here concerns three different examples, and it starts from the analytical symbolic determination of both divergences in the case of two different combinations of neighboring Gaussian distributions. The first computational experiments is based upon symbolic integration inherent in the definitions of two relative entropies under consideration. Further, a reliability study relevant to some steel Pratt trusses is presented, in which KL and Jeffreys divergence is used to detect a distance in between the PDFs of extreme and admissible structural deformations. The extreme deformations are determined using the stochastic finite element method approach realized with (i) probabilistic extension of traditional deterministic designing procedures, (ii) the Monte-Carlo simulation analysis and (iii) the iterative generalized stochastic perturbation method, while the admissible deformations are given due to the engineering codes.

The final example deals with the homogenization procedure of some particulate composite, in which divergence between real and homogenized composite characteristics is under investigation. The computational analysis presented in this paper exhibits that a lack of symmetry in the KL divergence may play a remarkable role in some engineering problems. As has been demonstrated here, it is highly important whether the Kullback-Leibler distance is measured in between the admissible and extreme deformations or vice versa. This aspect is completely absent in reliability analysis, for instance, where an absolute value of the difference of admissible and extreme structural behavior mean values (scaled by its variance) is analyzed. A more remarkable effect is seen while approximating the analogous distance between the original and homogenized composite materials characteristics. Some further idea could be a discussion of probabilistic distance between the designed and real deformations or stresses in the erected civil engineering structures or mechanical systems. This aspect has not been discussed until now in the literature, but it may have some importance, especially in highly nonlinear problems with uncertainty when reliability assessment is decisive for structural safety and durability.

3. Relative Entropies and Probabilistic Numerical Methods

Relative entropy equations applied and discussed here follow probabilistic divergence models invented by Kullback and Leibler as well as by Jeffreys [30]. Their definitions for two different probability distributions p(x) and q(x) are formulated in turn as follows [9]:

$$D_{KL}(p,q)=-{\displaystyle \int p(x)\log (q(x))dx+{\displaystyle \int p(x)\log \left(p(x)\right)dx}}$$

(6)

$$D_J(p,q)=D_{KL}\left(p,q\right)+D_{KL}\left(q,p\right)$$

(7)

As a specific case, one may consider two different Gaussian probability distributions: $p\left(x\right)\equiv N\left({\mu }_1,{\sigma }_1\right)$ $q\left(x\right)\equiv N\left({\mu }_2,{\sigma }_2\right)$. Then, the KL relative entropy equals to

$$D_{KL}\left(p,q\right)=\log \left(\frac{{\sigma }_2}{{\sigma }_1}\right)+\frac{{\sigma }_1^2+{\left({\mu }_1-{\mu }_2\right)}^2}{2{\sigma }_2^2}-\frac{1}{2}$$

(8)

whereas the Jeffreys entropy is calculated as

$$D_J(p,q)=\log \left(\frac{{\sigma }_2}{{\sigma }_1}\right)+\frac{{\sigma }_1^2+{\left({\mu }_1-{\mu }_2\right)}^2}{2{\sigma }_2^2}+\log \left(\frac{{\sigma }_1}{{\sigma }_2}\right)+\frac{{\sigma }_2^2+{\left({\mu }_2-{\mu }_1\right)}^2}{2{\sigma }_1^2}-1$$

(9)

Quite similarly, one may derive the relative entropies for two Weibull distributions having the parameters $p\left(x\right)\equiv W\left(k_1,l_1\right)$ and $q\left(x\right)\equiv W\left(k_2,l_2\right)$

$$D_{KL}\left(p,q\right)=\log \left(\frac{k_1}{l_1{}^{k_1}}\right)-\log \left(\frac{k_2}{l_2{}^{k_2}}\right)+\left(k_1-k_2\right)\left(\log \left(l_1\right)-\frac{\gamma }{k_1}\right)+{\left(\frac{l_1}{l_2}\right)}^{k_2}\Gamma \left(\frac{k_2}{k_1}+1\right)-1$$

(10)

As it is seen, reliable determination of the first two probabilistic moments is necessary to study the probabilistic divergence of two Gaussian distributions. This is the reason to engage not only the Monte-Carlo simulation (MCS) approach but also some faster techniques, such as the stochastic semi-analytical method (SAM) and the iterative stochastic perturbation method (SPT) [31,32,33]. These two last methodologies enable remarkable reduction of a solution of the problems with uncertainty, whose computational effort becomes closer to the deterministic origin rather than the MCS time consumption. A computational implementation may be delivered with the use of the MCS approach, in which statistical estimation usually follows the well-known statistical estimators introduced as

$$E\left[y\left(b\right)\right]=\frac{1}{M}{\displaystyle \sum _{m=1}^M{y}_m\left(b\right)}=\frac{1}{M}{\displaystyle \sum _{m=1}^M{\displaystyle \sum _{p=0}^P{A}_p{b}_m^p}}$$

(11)

$$Var\left(y\left(b\right)\right)=\frac{1}{M-1}{\displaystyle \sum _{m=1}^M{\left\{y_m\left(b\right)-E\left[y\left(b\right)\right]\right\}}^2}=\frac{1}{M-1}{\displaystyle \sum _{m=1}^M{\displaystyle \sum _{p=0}^P{\left\{A_p{b}_m^p-E\left[y\left(b\right)\right]\right\}}^2}}$$

(12)

where b is the input uncertainty source, b_m means its mth random numerical realization, and y(b) is the given structural output. Alternatively, the iterative generalized stochastic perturbation technique is also employed. Its fundamental step is the following nth order Taylor expansion of the same response function y(b) [31,32]:

$$y\left(b\right)=y^0\left(b^0\right)+{\displaystyle \sum _{i=1}^n\frac{{\epsilon }^i}{i!}{\frac{{\partial }^{i}y\left(b\right)}{\partial b^i}|}_{b=b^0}{\left(\Delta b\right)}^i}$$

(13)

It concerns the function f of the given random input parameter b about its mean value b⁰ so that Δb is the first variation of this parameter about its given mean. This expansion is embedded next into the classical integral definition of the kth central probabilistic moment as

$$\begin{array}{ll}{\mu }_k\left(y\left(b\right)\right)& ={\displaystyle \underset{-\infty }{\overset{+\infty }{\int }}{\left(y\left(b\right)-E\left[y\left(b\right)\right]\right)}^k p_b\left(x\right)dx}\\ & ={\displaystyle \underset{-\infty }{\overset{+\infty }{\int }}{\left\{y^0\left(b^0\right)+{\displaystyle \sum _{i=1}^n\frac{{\epsilon }^i}{i!}{\frac{{\partial }^{i}y\left(b\right)}{\partial b^i}|}_{b=b^0}{\left(\Delta b\right)}^i-E\left[y\left(b\right)\right]}\right\}}^k{p}_b\left(x\right)dx}\end{array}$$

(14)

Further transforms are possible after a choice of the specific PDF type as well as its upper and lower bounds. It should be mentioned here that the perturbation method is still frequently applied in many mathematical studies focused on dynamical systems and nonlinear phenomena [34,35].

The semi-analytical approach directly uses the definitions of the central probabilistic moments enriched with the polynomial bases as

$$\begin{array}{ll}{\mu }_k\left(y\left(b\right)\right)& ={\displaystyle \underset{-\infty }{\overset{+\infty }{\int }}{\left(y\left(b\right)-E\left[y\left(b\right)\right]\right)}^k p_b\left(x\right)dx}\\ & ={\displaystyle \underset{-\infty }{\overset{+\infty }{\int }}{\left\{{\displaystyle \sum _{i=1}^N{A}_i{\left(b^0\right)}^i}-E\left[{\displaystyle \sum _{i=1}^N{A}_i{\left(b^0\right)}^i}\right]\right\}}^k{p}_b\left(x\right)dx}\end{array}$$

(15)

The purpose of the first numerical experiment was to investigate analytically probabilistic divergence for two different Gaussian distributions $N_1\left(E_1,{\sigma }_1\right)$ $N_2\left(E_2,{\sigma }_2\right)$, and it was entirely done in the computer algebra system MAPLE 2019.2. This purpose has been achieved by introducing two additional parameters being a difference between the coefficients of variation of these two distributions, ${\alpha }_1$ and ${\alpha }_2$, as well as between their expectations, accordingly

$$\Delta \left(\alpha \right)={\alpha }_2-{\alpha }_1=\frac{{\sigma }_2}{E_2}, \Delta \left(E\right)=E_2-E_1$$

(16)

Two different combinations of the input data have been adopted to check some initial parametric sensitivity of both Kullback-Leibler and Jeffreys relative entropies. The first test corresponds to the following data: E₁ = 2.0 & α₁ = 0.01, whereas the second one uses E₁ = 1.0 and α₁ = 0.01, respectively. The resulting entropy surfaces have been collected in Figure 1 (the first input combination) as well as in Figure 2 (the second test).

The first general observation is that the Jeffreys model returns a few times larger entropy than the Kullback-Leibler theory so a lack of symmetry matters a lot. The second model exhibits significant sensitivity to the input data, while in the Jeffreys model, this effect is almost negligible. The KL approach seems to be more sensitive to a difference in-between coefficients of variation of two random inputs and then, to their expectations. Symmetrized Jeffreys theory shows even higher changes of the entropy while modifying expectations difference, while sensitivity to statistical scattering of the input data is either very small or may be just postponed. Nevertheless, as one could expect, the larger the difference in input expectations, the higher the resulting relative entropy (a distance between both PDFs). It directly follows a deterministic case of two functions and their mean values. Quite a different observation can be made for two input coefficients of variation–a distance for the PDFs under consideration becomes smaller while increasing their difference. It is because the concentration of the PDF about its mean value is smaller while increasing this coefficient so that the PDF splashes and two different bell-shaped curves are effectively closer. Finally, one may conclude that symmetry (and its possible lack) in the relative entropy computations may play a very important role in applied sciences and cannot be simply neglected.

4. Reliability Structural Analysis and Probabilistic Divergence of the Elastic Pratt Truss Structure

4.1. Numerical Model

The next computational experiment has documented the usage of two aforementioned relative entropies to study probabilistic divergence between the extreme (E) and admissible (R) deformation of some civil engineering structures. This divergence is the basis of the so-called limit function g(R,E) = R − E widely employed in the reliability theory [36]. A specific case is investigated, where both R and E have random nature. A simple plane steel Pratt truss with 12 segments is analyzed for this purpose (Figure 3), whose height has been adopted as h = 1.0 m, and its span l is treated as the Gaussian variable having the expectation equal to 18.00 m. The stochastic finite element method (SFEM) analysis of this structure in the context of Bhattacharyya entropy has been analyzed in [37]. The first two probabilistic moments of structural displacements have been determined here via the series of traditional FEM experiments [38], where polynomial bases connecting extreme deformations with the input uncertainty source have been recovered. All these experiments have been carried out here in the civil engineering system Autodesk ROBOT, while the probabilistic part has been completed in the computer algebra system MAPLE 2019.2. This truss FEM model has been uploaded with the constant load q redistributed throughout the upper chord nodal points as the concentrated forces. Its characteristic value has been set as q_k = 15 kN/m, while the design load has been proposed for an illustration as q_d = 20 kN/m; a dead load of the truss is taken into account separately. All the cross-sections of structural components have been assumed as hot-finished square-hollow steel profiles made of structural steel S235, whose material parameters have been assumed according to the design code Eurocode 3 [39]. The following profiles have been found here as the most optimal solution: (i) upper chord–SHS 140 × 140 × 8, (ii) diagonals–SHS 90 × 90 × 8, and also (iii) lower chord–SHS 120 × 120 × 8. The FEM discretization has been completed using 49 linear two-noded truss finite elements, where horizontal and vertical displacements have been fixed at lower left edge; a vertical displacement has been blocked at the right lower corner. A single finite element represents here a column and a diagonal, while upper and lower chords have been represented by the set of single finite elements from node to node, respectively.

Numerical experiments have been completed assuming geometrical nonlinearities and p-delta effect in the structural behavior. The incremental Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithm has been employed for this purpose with the following parameters:

• load increment number has been fixed as 5;
• maximum iteration number for a single increment equals 40;
• increment length reduction number was set as 3;
• increment length reduction factor was equal to 0.5;
• the maximum number of line searches equals 0;
• control parameter for the line search method is set as 0.5;
• the maximum number of the BFGS corrections is set as 10;
• relative tolerance for the residual forces and displacements is taken as 0.0001.

4.2. Numerical Results and Discussion

The basic results of numerical studies have been collected in

Table 1. Relative entropies versus output uncertainty in the Pratt truss structure.

Numerical Method	Data Type	α = 0.025	α = 0.050	α = 0.075	α = 0.100	α = 0.125	α = 0.150
Probabilistic analytical method	α(u)	0.0914	0.1827	0.2736	0.3639	0.4533	0.5418
	DKL	6369.278	1706.486	846.114	548.691	415.147	347.052
	DJ	6403.883	1716.821	851.966	552.988	418.740	350.277
Stochastic perturbation technique	α(u)	0.0914	0.1827	0.2736	0.3639	0.4533	0.5417
	DKL	6369.278	1706.486	846.114	548.691	415.147	347.052
	DJ	6403.884	1716.821	851.966	552.988	418.740	350.277
Monte-Carlo simulation approach	α(u)	0.0914	0.1818	0.2722	0.3618	0.4507	0.5385
	DKL	6367.928	1702.557	842.733	545.431	411.813	343.520
	DJ	6402.542	1712.961	848.615	549.743	415.412	346.747

below as the functions of an increasing coefficient of variation of the uncertain truss length ($\alpha \left(l\right)\in \left[0.00,0.15\right]$). They are obtained with three different numerical methods described briefly in the preceding section and they include (i) the resulting coefficient of variation of the extreme vertical displacement $\alpha \left(u\right)$ obtained on the symmetry axis of the structure, (ii) Kullback-Leibler (D_KL) relative entropy, and (iii) Jeffreys probabilistic divergence (D_J). First of all, it is seen that a lack of symmetry in probabilistic divergence measured here as a difference D_J−D_KL naturally increases together with an increase of the input coefficient α. Nevertheless, it is rather not larger than 1% of the D_J value at the very upper end of the input α range, which is rather negligible.

The very important result is that all three stochastic numerical methods coincide here almost perfectly, which enables significant reduction of computational time by replacing the MCS with SPT or SAM techniques. Further, both entropies exponentially decrease together with a linear decrease of the input COV, which is quite an expected effect. As one may see, particular numerical values of these entropies cannot be directly used in structural reliability assessment as they are with a few ranges larger than the reliability indices listed in Eurocode 0. Let us mention that some rescaling procedure has been proposed for this purpose in [37].

5. Probabilistic Divergence Application to the Homogenization of some Particulate Composite

5.1. Composite Material Numerical Model

The last numerical case study was prepared to study a probabilistic divergence between the random distribution of some mechanical characteristics of the given composite material and its homogenized characteristics. This experiment has been focused on a particulate composite consisting of a single spherical particle centrally embedded into a cubic matrix volume, see Figure 4. A perfect interface between the particle and the matrix is assumed here, and initial deformation has been completely neglected. Its discretization [40] was made in the preprocessor of the ABAQUS system and is shown in Figure 5.

The homogenization procedure performed and analyzed here is a widely known mathematical-numerical approach invented for both classical composite materials as well as for some multiscale heterogeneous structures. The so-called effective characteristics for these complex materials/structures are determined via analytical calculus or in some FEM computer analyses. All such approximations are based upon an assumption that the deformation energy of the real and of the homogenized media are equal to each other. Closed-form equations leading to these effective characteristics are derived from the specific boundary conditions imposed on the composite representative volume element (RVE) [40].

A composite consisting of the rubber particle embedded into the polymeric matrix is analyzed, whose elastic parameters of the particle have been taken as E_p = 1.0 MPa, ν_p = 0.4888, while for the polymer matrix as equal to E_m = 4.0 GPa and ν_m = 0.34. Both Young moduli of this composite have been adopted here in turn as Gaussian variables with given expectations and some variability interval for their coefficients of variation to see an influence of the input uncertainty ($\alpha \in \left[0.00,0.20\right]$). Polynomial approximating functions relating the effective elasticity tensor components and Young moduli of the particle and of the matrix have been recovered numerically via certain series of the FEM experiments and the least squares method implemented in the computer algebra system MAPLE 2019.2 as before.

5.2. Numerical Results and Discussion

Numerical results of such analysis have been contained in Figure 6, Figure 7 and Figure 8–for $C_{1111}^{\left(eff\right)}$, $C_{1122}^{\left(eff\right)}$ and $C_{1212}^{\left(eff\right)}$ while randomizing particle Young modulus (left graphs) and, separately, matrix Young modulus (the right column). Kullback-Leibler and Jeffreys probabilistic divergences have been compared here, and a general observation one can make is that a lack of symmetry is more remarkable in the right graphs, where the absolute extreme distance of the real and homogenized characteristics is a few orders smaller. The opposite case–particle uncertainty–shows practically the same results for two entropy models.

The next observation of a quite general character is that all relative entropies decay very fast while increasing the input coefficient of variation; they are close to 0 for about $\alpha \cong 0.20$. The extreme values marked on the vertical axes allow us to compare the distance of the PDF for particle and for matrix Young moduli to the given effective tensor components. As one may expect, the elasticity modulus of the matrix is many times closer to any component of the tensor $C_{ijkl}^{(eff)}$ than the elastic modulus of the rubber particle. Looking for the expected values of the effective tensor and comparing them with Young moduli of both components, it is seen that particle contribution to the overall elastic characteristics is marginal in this context. It is further seen that a relative entropy could be used in composites engineering (not only in the context of the homogenization method) to find out the most distant random parameters from the given limit state, whose uncertainty may be simply postponed in the model. Other parameters having especially the smallest distance to that limit state may be decisive in reliability assessment, for instance, for the given composite material.

The last conclusion is that various relative entropies may result in totally different numerical values; however, a general character of the function H = H(α) remains the same (very similar to engineering reliability indices). The extreme values result in these graphs from the Jeffreys model, and then from the Kullback-Leibler model, entropy is almost the same or slightly smaller for the given COV value.

6. Concluding Remarks

Symmetrical and non-symmetrical probabilistic divergencies have been presented in this work and contrasted with each other to check the possible lack of symmetry in relative entropy and its possible significance in numerical simulation and specific engineering problems. It has been found that this lack of symmetry may have a remarkable influence on relative entropy values in some specific mathematical analyses, but it seems to be less important in engineering analyses, especially when completed thanks to the stochastic finite element methods. Such a conclusion may be drawn for two Gaussian distributions, and further analyses should be provided for other distributions and their pairs having more important and frequent applications in applied sciences and engineering.

Another important research finding reported in this work is that a lack of symmetry in Kullback-Leibler probabilistic divergence may lead to some imprecision while analyzing the reliability of even relatively simple engineering structures. This imprecision may be higher for large-scale structures, and it deserves further numerical studies. The homogenization analysis presented above shows that Kullback-Leibler divergence asymmetry may be very large while computing a distance between the original and homogenized elastic characteristics. This means that a definition proposed by Equation (6) returns completely different results while counting the distance between the original and homogenized tensors components or vice versa. Symmetric probabilistic divergence models are recommended in this context for further studies.

It should be noted that engineering practice also welcomes time-dependent probabilistic divergence analysis, where two distributions of various natures representing extreme effort and the corresponding limit change their parameters in the exploitation time. Then, this divergence may additionally depend on time in some unknown way, which deserves further studies, both practical and numerical. Let us note that some interesting applications would be a determination of the relative probabilistic entropy in conjunction with the probability transformation method (PTM) presented in [41].

Another interesting research avenue would be to contrast the statistics of the deformation computed via the SFEM models with those resulting from laboratory experiments or some in-situ observations. Then, a relative probabilistic entropy could serve as some calibration measure to improve numerical models and methods themselves. The same observation concerns effective material characteristics, which can be determined also in an experimental manner via uniaxial, and biaxial tensile tests. There is no doubt that modern advanced engineering of solid continua, such as optical fiber [42] behavior or specifically coupled and uncoupled fluid flow phenomena [43,44], may need a determination of probabilistic entropy and also probabilistic divergence. Such an application may be very attractive due to the final single graph of the relative entropy allowing for full identification of the resulting uncertainty. An application of the methodology presented in computer analysis of some acoustics and hydrodynamic phenomena [45] may find its importance in biomedical engineering and it deserves further research attention.