Next Article in Journal
Relationship between Organic Geochemistry and Reservoir Characteristics of the Wufeng-Longmaxi Formation Shale in Southeastern Chongqing, SW China
Previous Article in Journal
Calculating the Inrush Current of Superconducting Transformers
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

High-Order Deterministic Sensitivity Analysis and Uncertainty Quantification: Review and New Developments

by
Dan Gabriel Cacuci
Center for Nuclear Science and Energy, Department of Mechanical Engineering, University of South Carolina, Columbia, SC 29208, USA
Energies 2021, 14(20), 6715; https://doi.org/10.3390/en14206715
Submission received: 20 August 2021 / Revised: 26 September 2021 / Accepted: 7 October 2021 / Published: 15 October 2021

Abstract

:
This work reviews the state-of-the-art methodologies for the deterministic sensitivity analysis of nonlinear systems and deterministic quantification of uncertainties induced in model responses by uncertainties in the model parameters. The need for computing high-order sensitivities is underscored by presenting an analytically solvable model of neutron scattering in a hydrogenous medium, for which all of the response’s relative sensitivities have the same absolute value of unity. It is shown that the wider the distribution of model parameters, the higher the order of sensitivities needed to achieve a desired level of accuracy in representing the response and in computing the response’s expectation, variance, skewness and kurtosis. This work also presents new mathematical expressions that extend to the sixth-order of the current state-of-the-art fourth-order formulas for computing fourth-order correlations among computed model response and model parameters. Another novelty presented in this work is the mathematical framework of the 3rd-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Nonlinear Systems (3rd-CASAM-N), which enables the most efficient computation of the exact expressions of the 1st-, 2nd- and 3rd-order functional derivatives (“sensitivities”) of a model’s response to the underlying model parameters, including imprecisely known initial, boundary and/or interface conditions. The 2nd- and 3rd-level adjoint functions are computed using the same forward and adjoint computer solvers as used for solving the original forward and adjoint systems. Comparisons between the CPU times are also presented for an OECD/NEA reactor physics benchmark, highlighting the fact that finite-difference schemes would not only provide approximate values for the respective sensitivities (in contradistinction to the 3rd-CASAM-N, which provides exact expressions for the sensitivities) but would simply be unfeasible for computing sensitivities of an order higher than first-order. Ongoing work will generalize the 3rd-CASAM-N to a higher order while aiming to overcome the curse of dimensionality.

1. Introduction

The functional derivatives of the results produced by models with respect to the underlying model parameters are customarily called the model “response sensitivities”. The various methods for computing first-order sensitivities can be grouped into two broad categories: statistical methods and deterministic methods. There is a plethora of statistical methods, but all are subject to the “curse of dimensionality” [1], in that the number of computations increases exponentially with the number of parameters and order of sensitivity. Additionally, none of the statistical methods can compute sensitivities exactly. On the other hand, there are just three deterministic methods for computing sensitivities: (i) finite-differences, which can produce approximate results within the order of approximation of the respective finite-difference formulas and are also subject to the curse of dimensionality; (ii) the forward variational method, which obtains exact expressions for the sensitivities but is also subject to the curse of dimensionality, and (iii) the adjoint sensitivity analysis method, which also produces exact expressions for the sensitivities but can overcome the curse of dimensionality, as will be discussed in this work.
The field of nuclear science and engineering has provided the earliest use [2] of adjoint operators for performing a sensitivity analysis of a large-scale linear system, modeled by the linear Boltzmann transport equation and comprising many model parameters. The large system analyzed by Wigner [2] was the model of linear neutron transport through the core of a nuclear reactor. The reactor’s multiplication factor was the “response”, and the various neutron cross-sections influencing the reactor’s behavior were the “model parameters”. Wigner’s ideas were generalized and set in a rigorous mathematical (functional analysis) framework for generic nonlinear systems by Cacuci [3,4]. Several works [5,6,7,8,9] in the field of nuclear engineering pioneered the computation of specific 2nd-order sensitivities of reaction rates and/or multiplication factors for nuclear reactors modeled by forward and adjoint linear Boltzmann or diffusion equations. These works generally indicated that, for a system comprising T P model parameters, the computation of 2nd-order sensitivities of responses to model parameters required O ( T P 2 ) large-scale computations per response. Therefore, only the 2nd-order sensitivities that were deemed a priori to be “important” were actually computed, and these 2nd-order sensitivities turned out to be considerably smaller than the corresponding 1st-order sensitivities. These findings likely gave rise to the generally held opinion that “2nd-order sensitivities are generally insignificant in reactor physics”. This opinion was likely responsible for the diminishing interest in computing 2nd-order response sensitivities, which practically vanished in the nuclear engineering field in the 1990s. On the other hand, interest in computing 2nd-order sensitivities became evident in other fields, such as structural mechanics [10,11,12,13,14,15], parametric circuit analysis and optimization [16,17] and the atmospheric and earth sciences; see, e.g., References [18,19,20,21,22].
All of the 2nd-order adjoint methods mentioned above were developed for specific applications for which they usually estimated rather than computed exactly and exhaustively as 2nd-order response sensitivities to the model’s parameters. Cacuci [23,24,25] conceived the generally applicable “Second-Order Adjoint Sensitivity Analysis Methodology” for both linear [23] and nonlinear [24,25] systems, which enables the most efficient computation of the exact expressions of all of the second-order functional derivatives of model responses to model parameters while overcoming the curse of dimensionality. This methodology has been applied [26,27,28,29,30,31] to compute the exact expressions of the first- and second-order sensitivities to the parameters of the leakage response of an OECD/NEA reactor physics benchmark [32]. This OECD/NEA benchmark was modeled using the neutron transport code PARTISN [33]. As has been shown in References [26,27,28,29,30,31], this reactor physics benchmark comprises 21,976 model parameters, which give rise to 21,976 first-order sensitivities (of which 7477 have nonzero values) and 241,483,276 second-order sensitivities (of which 27,956,503 have nonzero values).
The 2nd-order methodology developed in References [23,24,25] has been extended by Cacuci [34,35] to a 3rd order and has been applied [36,37,38] to perform a 3rd-order sensitivity and uncertainty analysis of the OECD/NEA benchmark [32]. Recently, Cacuci [39] developed the “Fourth-Order Comprehensive Adjoint Sensitivity Analysis (4th-CASAM) of Response-Coupled Linear Forward/Adjoint Systems”, which has been applied to perform [40,41] an initial 4th-order sensitivity analysis of the OECD/NEA benchmark [32].
As is well-known, model responses of linear systems may involve the solutions of both the forward and the adjoint linear models that correspond to the respective physical ones. Since nonlinear operators do not admit bona fide adjoint operators (only a linearized form of a nonlinear operator admits an adjoint operator), the responses of nonlinear models can depend only on the forward functions. Hence, linear systems cannot be treated as particular cases of nonlinear systems but need to be treated separately; for this reason, the adjoint sensitivity analysis methodology for linear systems will not be reviewed in this work.
The mathematical formulation of the equations that underlies the computational model of a generic nonlinear physical system will be presented in Section 2, along with the definition of a generic response that depends on the model’s state variables and parameters, which are considered to be uncertain (i.e., not known precisely). Besides the initial conditions and correlations, the model parameters are also considered to include geometrical parameters that describe the system’s boundaries and internal interfaces. Section 3 presents the sixth-order Taylor series expansion of a model’s response in the model’s uncertain parameters. Section 3 also illustrates the need for computing high-order sensitivities by means of a paradigm physical model of neutron scattering in an infinite hydrogenous medium, which admits a closed form solution for all orders of the sensitivities of the model’s response (neutron flux) to the model’s scattering cross-section. This model was deliberately chosen, because the relative sensitivities of all orders of the model’s response with respect to the scattering cross-section are unity. The sixth-order Taylor series expansion presented in Section 3 provides the basis for the new formulas, accurate up to and including the sixth order in the parameters’ standard deviations, for the quantification of uncertainties induced in the model’s response by the uncertainties in the model. These new formulas are presented in Section 4, along with additional new formulas that provide the expressions for the correlations up to the fourth order among the model responses and model parameters. By means of a paradigm physical model of neutron scattering in an infinite hydrogenous medium, Section 4 also illustrates the need for high-order uncertainty quantification.
Section 5 presents the novel Third-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Nonlinear Systems (3rd-CASAM-N). Section 5 also presents illustrative comparisons of the CPU times needed by the 3rd-CASAM versus finite-difference formulas for computing sensitivities of various orders for the OECD/NEA benchmark [32]. These comparisons underscore the fact that finite-difference schemes would not only provide approximate values for the respective sensitivities (in contradistinction to the 3rd-CASAM-N, which provides exact expressions for the sensitivities) but would simply be unfeasible for computing sensitivities of an order higher than first-order. Since statistical methods require orders of magnitude more computational time and resources than finite-differences, it is evident that the statistical methods could not even compute the first-order sensitivities of the benchmark’s response to the benchmark’s 21,976 model parameters.
The concluding discussion presented in Section 5 highlights the fact that the 3rd-CASAM-N presented in this work provides a fundamentally important step in the quest to overcome the “curse of dimensionality” in the sensitivity analysis, uncertainty quantification and predictive modeling.

2. Mathematical Description of a Generic Nonlinear Physical System with Uncertain Parameters and Boundaries

The modeling of a physical system and/or the result of an indirect experimental measurement includes the following modeling components:
(a)
A mathematical model comprising independent variables (e.g., space, time, etc.); dependent variables (aka “state functions”, e.g., temperature, mass, momentum, etc.) and various parameters (appearing in correlations, coordinates of physical boundaries, etc.), which are all interrelated by equations (linear and/or nonlinear in the state functions) that usually represent conservation laws.
(b)
Model parameters, which usually stem from processes that are external to the system under consideration and are seldom, if ever, known precisely. The known characteristics of the model parameters may include their nominal (expected/mean) values and, possibly, higher-order moments or cumulants (i.e., variance/covariances, skewness and kurtosis), which are usually determined from experimental data and/or processes external to the physical system under consideration. Occasionally, only inequality and/or equality constraints that delimit the ranges of the system’s parameters are known.
(c)
One or several computational results, customarily called “system responses” (or objective functions or indices of performance), which are computed using the mathematical model.
(d)
Experimentally measured values of the responses under consideration, which may be used to infer nominal (expected) values and uncertainties (variances, covariances, skewness, kurtosis, etc.) of the respective measured responses.
Without loss of generality, the imprecisely known model parameters can be considered to be real-valued scalar quantities. These model parameters will be denoted as α 1 , …, α T P , where T P denotes the “total number of imprecisely known parameters” underlying the model under consideration. For subsequent developments, it is convenient to consider that these parameters are components of a “vector of parameters” denoted as α ( α 1 , , α T P ) E α T P , where E α is also a normed linear space and where T P denotes the T P -dimensional subset of the set of real scalars. The components of the T P -dimensional column vector α T P are considered to include imprecisely known geometrical parameters that characterize the physical system’s boundaries in the phase-space of the model’s independent variables. Matrices will be denoted using capital bold letters while vectors will be denoted using either capital or lower-case bold letters. The symbol “ ” will be used to denote “is defined as” or “is by definition equal to”. Transposition will be indicated by a dagger ( ) superscript.
The model is considered to comprise T I independent variables which will be denoted as x i , i = 1 , , T I , and which are considered to be components of a T I -dimensional column vector denoted as x ( x 1 , , x T I ) T I , where the sub/superscript “TI” denotes the “Total (number of) Independent variables”. The vector x T I of independent variables is considered to be defined on a phase-space domain which will be denoted as Ω ( α ) and which is defined as follows: Ω ( α ) { λ i ( α ) x i ω i ( α ) ; i = 1 , , T I } . The lower boundary-point of an independent variable is denoted as λ i ( α ) and the corresponding upper boundary-point is denoted as ω i ( α ) . A typical example of boundaries that depend on both geometrical parameters and material properties are the “boundaries facing vacuum” in models based on diffusion theory, where conditions are imposed on the “extrapolated boundary” of the respective spatial domain. The “extrapolated boundary” depends both on the imprecisely known physical dimensions of the problem’s domain and also on the medium’s properties, such as atomic number densities and microscopic transport cross sections. The boundary of Ω ( α ) , which will be denoted as Ω ( α ) , comprises the set of all of the endpoints λ i ( α ) , ω i ( α ) , i = 1 , , T I , of the respective intervals on which the components of x are defined, i.e., Ω ( α ) { λ i ( α )   ω i ( α ) , i = 1 , , T I } .
A nonlinear physical system can be generally represented/modeled by means of coupled equations which can be represented in operator form as follows:
N [ u ( x ) , α ] = Q ( x , α )   ,   x Ω x ( α ) .
The quantities which appear in Equation (1) are defined as follows:
  • u ( x ) [ u 1 ( x ) , , u T D ( x ) ] is a T D -dimensional column vector of dependent variables; the abbreviation “ T D ” denotes “Total (number of) Dependent variables.” The functions u i ( x ) , i = 1 , , T D , denote the system’s “dependent variables” (also called “state functions”); u ( x ) E u , where E u is a normed linear space over the scalar field F of real numbers.
  • N [ u ( x ) , α ] [ N 1 ( u , α ) , , N T D ( u , α ) ] denotes a T D -dimensional column vector The components N i ( u , α ) , i = 1 , , T D are operators (including differential, difference, integral, distributions, and/or infinite matrices) acting (usually) nonlinearly on the dependent variables u ( x ) , the independent variables x and the model parameters α ; N ( α , u ) is the mapping N :   D E E Q , where D = D u D α , D u E u , D α E α , E = E u E α . An arbitrary element e E has the form e   =   ( α , u ) .
  • Q ( x , α )   [ q 1 ( x ; α ) , . , q T D ( x ; α ) ] is a T D -dimensional column vector which represents inhomogeneous source terms, which usually depend nonlinearly on the uncertain parameters α ; Q E Q , where E Q is also a normed linear space.
  • The equalities in this work are considered to hold in the weak (“distributional”) sense. The right-sides of Equation (1) and of other various equations to be derived in this work may contain “generalized functions/functionals”, particularly Dirac-distributions and derivatives thereof.
Boundary and/or initial conditions must also be provided if differential operators appear in Equation (1). In an operator form, these boundary and/or initial conditions are represented as follows:
B [ u ( x ) , x ; α ] - C ( x , α ) = 0 ,   x Ω x ( α ) .
In Equation (2), the components B i ( u ; α ) , i = 1 , , T D of B ( u , α ) [ B 1 ( u ; α ) , , B T D ( u ; α ) ] are nonlinear operators of u ( x ) and α , which are defined on the boundary Ω x ( α ) of the model’s domain Ω x ( α ) . The components C i ( u ; α ) , i = 1 , , T D of C ( x , α ) [ C 1 ( u ; α ) , , C T D ( u ; α ) ] comprise inhomogeneous boundary sources which are nonlinear functions of α .
Solving Equations (1) and (2) at the nominal parameter values α 0 [ α 1 0 , , α i 0 , , α T P 0 ] provides “the nominal solution” u 0 ( x ) , i.e., the vectors u 0 ( x ) and α 0 satisfy the following equations:
N [ u 0 ( x ) , α 0 ] = Q ( x , α 0 )   ,   x Ω x ,
B [ u 0 ( x ) , x ; α 0 ] - C ( x , α 0 ) = 0 ,   x Ω x ( α 0 ) .
Responses of particularly important interest are model representations/computations of physical measurements of the model’s state function u ( x p ) at a specific point, x p , in phase-space. Other responses of particular interest are averages over the phase-space domain (or segments thereof). Both point-measurements and “average” responses can be represented generically in the following integral form:
R [ u ( x ) , α ; x ] i = 1 T I λ i ( α ) ω i ( α ) S   [ u ( x ) ; α ] d x i ,   i = 1 T I λ i ( α ) ω i ( α )   [   ] d x i λ 1 ( α ) ω 1 ( α ) λ i ( α ) ω i ( α ) λ T I ( α ) ω T I ( α ) [   ] d x 1 d x 2   d x i d x T I ,
where S [ u ( x ) ; α ] is suitably differentiable nonlinear function of u ( x ) and of α . It is important to note that the components of α also include parameters that may appear specifically just in the definition of the response under consideration, but which might not appear in Equations (1) and (2). In particular, a measurement of a physical quantity can be represented as a response R p [ φ ( x p ) , ψ ( x p ) ; α ] located at a point, x p , which may itself be afflicted by uncertainties. Such a response can be represented mathematically in the form R p [ u ( x p ) ; α ] i = 1 T I λ i ( α ) ω i ( α ) R p [ u ( x ) ; α ] δ ( x x p ) d x i , where δ ( x x p ) denotes the multidimensional Dirac-delta functional. The measurement point x p appears only in the definition of the response but does not appear in Equations (1) and (2). Thus, the (physical) “system” includes not only the computational model but also the model’s response.

3. Sixth-Order Formulas for Sensitivity Analysis of Model Responses to Model Parameters

The system’s response is a function of the system’s parameters α ( α 1 , , α T P ) T P , either directly or indirectly through the state functions. Up to 6th-order, the explicit Taylor-series of a system response, denoted as R k ( α ) , around the expected (or nominal) parameter values α 0 has the following well-known formal expression:
R k ( 0 ) ( α ) = R k ( α 0 ) ,
R k ( 1 ) ( α ) = R k ( 0 ) ( α ) + j 1 = 1 T P R k ( α 0 ) α j 1 ( α j 1 α j 1 0 ) ,
R k ( 2 ) ( α ) = R k ( 1 ) ( α ) + 1 2 j 1 = 1 T P j 2 = 1 T P 2 R k ( α 0 ) α j 1 α j 2 ( α j 1 α j 1 0 ) ( α j 2 α j 2 0 ) ,
R k ( 3 ) ( α ) = R k ( 2 ) ( α )   + 1 3 ! j 1 = 1 T P j 2 = 1 T P j 3 = 1 T P 3 R k ( α 0 ) α j 1 α j 2 α i 3 ( α j 1 α j 1 0 ) ( α j 2 α j 2 0 ) ( α j 3 α j 3 0 ) ,
R k ( 4 ) ( α ) = R k ( 3 ) ( α )   + 1 4 ! j 1 = 1 T P j 2 = 1 T P j 3 = 1 T P j 4 = 1 T P 4 R k ( α 0 ) α j 1 α j 2 α j 3 α j 4 × [ ( α j 1 α j 1 0 ) ( α j 2 α j 2 0 ) ( α j 3 α j 3 0 ) ( α j 4 α j 4 0 ) ] ,
R k ( 5 ) ( α ) = R k ( 4 ) ( α ) + 1 5 ! j 1 = 1 T P j 2 = 1 T P j 3 = 1 T P j 4 = 1 T P j 5 = 1 T P 5 R k ( α 0 ) α j 1 α j 2 α j 3 α j 4 α j 5 × [ ( α j 1 α j 1 0 ) ( α j 2 α j 2 0 ) ( α j 3 α j 3 0 ) ( α j 4 α j 4 0 ) ( α j 5 α j 5 0 ) ] ,
R k ( 6 ) ( α ) = R k ( 5 ) ( α ) + 1 6 ! j 1 = 1 T P j 2 = 1 T P j 3 = 1 T P j 4 = 1 T P j 5 = 1 T P j 6 = 1 T P 6 R k ( α 0 ) α j 1 α j 2 α j 3 α j 4 α j 5 α j 6   ×   [ ( α j 1 α j 1 0 ) ( α j 2 α j 2 0 ) ( α j 3 α j 3 0 ) ( α j 4 α j 4 0 ) ( α j 5 α j 5 0 ) ( α j 6 α j 6 0 ) ] .
As is well known, and as indicated by Equations (6)–(12), the Taylor-series of a function of T P -variables [e.g., R k ( α ) ] comprises T P 1st-order derivatives, T P ( T P + 1 ) / 2 distinct 2nd-order derivatives, T P ( T P + 1 ) ( T P + 2 ) / 6 distinct 3rd-order derivatives, and so on. The computation by conventional methods of the n t h -order functional derivatives (called “sensitivities” in the field of sensitivity analysis) of a response with respect to the T P -parameters it depends on would require at least O ( T P n ) large-scale computations. This exponential increase—with the order of response sensitivities—of the number of large-scale computations needed to determine higher-order sensitivities is the manifestation of the “curse of dimensionality in sensitivity analysis,” by analogy to expression coined by Belmann [1] to express the difficulty of using “brute-force” grid search when optimizing a function with many input variables.

Illustrating the Need for High-Order Sensitivities: Neutron Scattering in an Infinite Hydrogenous Medium

The energy-dependent neutron flux, Φ ( E ) , of scattered of neutrons emitted by an energy-dependent source in a hydrogen-moderated system in which absorption is neglected, is modeled [42,43,44,45,46,47,48] by the following integral equation: Q ( E )
E Φ ( E )   Σ s ( E ) d E E + Q ( E ) = Φ ( E ) Σ s ( E ) .
The quantity Σ s ( E ) in Equation (13) represents the scattering cross section of hydrogen, having the expression Σ s ( E ) = N H σ s ( E ) , where N H denotes the atomic number density of hydrogen while σ s ( E ) denotes the microscopic scattering cross section of hydrogen. The solution of Equation (13) has the following form:
Φ H ( σ ; E ) = Q ( α 0 ) E N H σ s ( E ) ;   Q ( α ) E s ( α ) Q ( α ; E ) d E   ( E )
Although Equation (14) is strictly correct only for an infinite hydrogen medium, it is often used to estimate the energy-dependent flux in water-moderated reactors [18,19,20]. The effects of the scattering cross sections can be quantified by in isolation from the effects of the other model parameters by considering that the source does not vary from its nominal value denoted as Q ( α 0 ) and that the number density of hydrogen, N H , remains unchanged, as well.
Consider that the energy-dependent microscopic total cross section of hydrogen, σ s ( E ) , was measured in a series of measurements at an arbitrary but fixed energy value E m , 0 < E m < , and was found to have a mean value denoted as σ m . The Taylor-series expansion of Φ H ( σ ; E ) around σ m has the following form:
Φ H ( σ ; E ) = Q ( α 0 ) E N H n = 0 1 n ! { d n [ σ s ( E ) ] 1 d [ σ s ( E ) ] n } ( σ m ) [ σ s ( E ) σ m ] n = lim n [ Φ H ( n ) ( σ ; E ) ] ,
where the quantity Φ H ( n ) ( σ ; E ) denotes the “nth-order approximation of the value of the actual flux Φ H ( σ ; E ) ”, and is defined as follows:
  Φ H ( n ) ( σ ; E ) Q ( α 0 ) E N H σ m k = 0 n ( 1 ) n t n ;   t σ s ( E ) σ m σ m .
The Taylor-series in Equation (15) converges only for values of σ s ( E ) confined within the following interval:
( 1 β ) σ m σ s ( E ) ( 1 + β ) σ m ;   0 < β < 1 .
It is especially important to ensure that the values of σ s ( E ) used for constructing statistical properties for Φ H ( σ ; E ) do not fall outside the interval of convergence shown in Equation (17). In particular, it follows from Equation (17) that if a standard deviation, S D [ σ s ( E ) ] , is also provided (in addition to the measured mean value σ m ) for the variate σ s ( E ) , then it must be ensured that the Taylor-series shown in Equation (15) is used only for the following range of values:
β σ m S D [ σ s ( E ) ] β σ m ;   0 < β < 1 .
If a standard deviation is not provided for the measurement of σ s ( E m ) , then the most conservative assumption is to consider that σ s ( E ) is uniformly distributed within the interval provided in Equation (17), in which the Taylor-series of Φ H ( σ ; E ) convergences, since the uniform distribution is the least informative (hence, the least biased) distribution. It will henceforth be assumed that the distribution for σ s ( E ) is uniform in the interval given in Equation (17). Thus, the uniform probability distribution, P [ σ s ( E ) ] , the expectation, E [ σ s ( E ) ] , the variance, V a r [ σ s ( E ) ] , and the standard deviation, S D [ σ s ( E ) ] , of σ s ( E ) have the following expressions:
P [ σ s ( E ) ] = 1 2 β σ m ; E [ σ s ( E ) ] = σ m ; V a r [ σ s ( E ) ] = ( β σ m ) 2 3 ; S D [ σ s ( E ) ] = 1 3 β σ m ;   0 < β < 1   .
It follows from the expression given in Equation (15) that the absolute sensitivities of Φ H ( σ ; E ) , which will be denoted as S n ( E ) ,   n = 1 , 2 , , with respect to the parameter σ s ( E ) at energy E = E m , where σ s ( E m ) = σ m , have the following expressions:
S n ( E ) = Q ( α 0 ) E N H σ m ( 1 ) n [ 1 σ m ] n .
It follows from the expression in Equation (20) that the relative sensitivities of Φ H ( σ ; E ) with respect to the parameter σ s ( E ) , which will be denoted as R n ( E ) ,   n = 1 , 2 , , have the following expressions:
R n ( E ) = S n ( E ) ( σ m ) n Φ H ( σ m ; E ) = ( 1 ) n ,   n = 1 , 2 , .
It is evident from Equation (21) that the relative values of the sensitivities of all orders are all equal to unity; their values do not decrease with increasing order n = 1 , 2 , .
The relative error, ε Φ ( n ) , between the exact value of Φ H ( σ ; E ) provided in Equation (14) and the nth-order approximation   Φ H ( n ) ( σ ; E ) is defined as follows expression:
ε Φ ( n ) 1 Φ H ( σ ; E ) [ Φ H ( σ ; E ) Q ( α 0 ) E N H σ m k = 0 n ( 1 ) k t k ] = ( 1 ) n + 1 t n + 1 .
It follows from Equation (22) that:
| ε Φ ( n ) | = | t n + 1 | = | [ σ s ( E ) σ m σ m ] n + 1 | < 1 .
As an example of the use of the Taylor-series expansion in sensitivity analysis, consider that it is desired to predict the value of Φ H ( σ ; E ) when σ s ( E ) is “one standard deviation away from the mean,” i.e., when:
σ s 1 S D ( E ) = σ m + S D [ σ s ( E ) ] = σ m ( 1 + 1 3 β ) .
Replacing the result obtained in Equation (24) into the definition provided in Equation (16) and taking   β = 1 10 4 (to be extremely close to the outer boundaries of the “one-standard deviation interval”) yields the following result when keeping the first four decimal digits:
t = β / 3 =   ( 1 10 4 ) / 3 0.5773 .
In sensitivity analysis, the value provided in Equation (25) is used in Equation (16) to investigate how accurately the various orders of approximations,   Φ H ( n ) ( σ ; E ) , can predict the actual value of the flux at Φ H ( σ ; E ) . The following results, to four significant decimals, are obtained for   β = 1 10 4 and t = 0.5773 from Equations (16) and (24):
1. The exact value of the flux is:
Φ H ( σ s 1 S D ; E ) = Q ( α 0 ) E N H σ m ( 1 + 1 10 4 3 ) 1 0.6340 Q ( α 0 ) E N H σ m .
2. If only the first-order sensitivities of Φ H ( σ ; E ) with respect to σ s ( E ) are available, then the 1st-order expansion ( n = 1 ) in Equation (16) yields the following result:
  Φ H ( 1 ) ( σ ; E ) Q ( α 0 ) E N H σ m ( 1 t ) = 0.4227 Q ( α 0 ) E N H σ m .
It follows from Equations (26) and (27) that the error of the 1st-order approximation is:
ε Φ ( 1 ) Φ H ( σ s 1 S D ; E ) Φ H ( 1 ) ( σ ; E ) Φ H ( σ s 1 S D ; E ) = 33.33 % .
Thus, the result obtained by keeping only the first-order sensitivities in the Taylor-expansion underestimates the exact flux by 33.34%.
3. If first-order and second-order sensitivities of Φ H ( σ ; E ) with respect to σ s ( E ) are available, then the 2nd-order expansion ( n = 2 ) in Equation (16) yields the following result:
  Φ H ( 2 ) ( σ ; E ) Q ( α 0 ) E N H σ m ( 1 t + t 2 ) = 0.7560 Q ( α 0 ) E N H σ m .
It follows from Equations (29) and (26) that the error of the 2nd-order approximation is as follows:
ε Φ ( 2 ) Φ H ( σ s 1 S D ; E ) Φ H ( 2 ) ( σ ; E ) Φ H ( σ s 1 S D ; E ) = 19.24 % .
Thus, the result obtained by keeping the first- and second-order sensitivities in the Taylor-expansion overestimates the exact flux by 19.24%.
4. If all sensitivities up to and including the third-order sensitivities of Φ H ( σ ; E ) with respect to σ s ( E ) are available, then the 3rd-order expansion ( n = 3 ) in Equation (16) yields the following result:
  Φ H ( 3 ) ( σ ; E ) Q ( α 0 ) E N H σ m ( 1 t + t 2 t 3 ) = 0.5636 Q ( α 0 ) E N H σ m .
It follows from Equations (31) and (26) that the error of the 3rd-order approximation is as follows:
ε Φ ( 3 ) Φ H ( σ s 1 S D ; E ) Φ H ( 3 ) ( σ ; E ) Φ H ( σ s 1 S D ; E ) = 11.11 % .
Thus, the result obtained by keeping the 1st-, 2nd-, and 3rd-order sensitivities in the Taylor-expansion underestimates the exact flux by 11.11%.
Thus, the results obtained by keeping the 1st-, 2nd- and 3rd-order sensitivities in the Taylor-expansion underestimate the exact flux by 11.11%.
5. If all sensitivities up to and including the 4th-order sensitivities of Φ H ( σ ; E ) with respect to σ s ( E ) are available, then the 4th-order expansion ( n = 4 ) in Equation (16) yields the following result:
  Φ H ( 4 ) ( σ ; E ) Q ( α 0 ) E N H σ m ( 1 t + t 2 t 3 + t 4 ) = 0.6747 Q ( α 0 ) E N H σ m .
It follows from Equations (33) and (26) that the error of the 4th-order approximation is as follows:
ε Φ ( 4 ) Φ H ( σ s 1 S D ; E ) Φ H ( 4 ) ( σ ; E ) Φ H ( σ s 1 S D ; E ) = 6.42 % .
As indicated by the results in Equations (28), (30), (32) and (34), the odd-order approximations underestimate the true value, and the even-order approximations overestimate the true value of the flux, while converging to the actual, true value, of the flux Φ H ( σ ; E ) . This behavior is expected since the flux is represented by an alternating series, cf. Equation (16). The results obtained in Equations (28), (30), (32) and (34) are consistent with the error estimate provided in Equation (22). The relation provided in Equation (23) can be used to determine which order of approximation would need to be used to obtain an approximate result which would be within a predetermined error by comparison to the exact result.

4. Sixth-Order Moments of the Response Distribution in the Parameter Phase-Space

In practice, even though the model parameters are not bona fide random quantities, they are considered to be variates distributed according to a multivariate probability distribution function, denoted as p α ( α ) . Although p α ( α ) is unknown, it is considered to be formally defined on a domain D α , so that the various moments of p α ( α ) can be defined in formally as follows:
u ( α ) α D α u ( α ) p α ( α ) d α ,
where u ( α ) is a continuous function of the parameters α . Using the general notation introduced in Equation (35), the moments of p α ( α ) are defined as follows:
1. The expectation of a model parameter α j , is denoted as α j 0 and is defined as follows:
α j 0 α j α D α α j p α ( α ) d α , j = 1 , , T P .
The expected values α j 0 are considered to be the components of the following vector of mean (expected) values:
α 0 ( α 1 0 , , α T P 0 ) .
2. The covariance, cov ( α j 1 , α j 2 ) , of two parameters, α j 1 and α j 2 , is defined as follows:
μ 2 j 1 j 2 ( α ) cov ( α j 1 , α j 2 ) ( α j 1 α j 1 0 ) ( α j 2 α j 2 0 ) α , j 1 , j 2 = 1 , , T P .
The variance, var ( α i ) , of a parameter α i , is defined as follows:
var ( α j ) ( α j α j 0 ) 2 α , j = 1 , , T P .
The standard deviation, σ i , of α i , is defined as follows: σ j var ( α j ) . The correlation, ρ j 1 j 2 , between two parameters α j 1 and α j 2 , is defined as follows:
ρ j 1 j 2 cov ( α j 1 , α j 2 ) / ( σ j 1 σ j 2 ) ; j 1 , j 2 = 1 , , T P .
3. The third-order parameter moment, μ 3 j 1 j 2 j 3 , and the associated third-order correlation t j 1 j 2 j 3 among three parameters, are defined as follows:
μ 3 j 1 j 2 j 3 ( α j 1 α j 1 0 ) ( α j 2 α j 2 0 ) ( α j 3 α j 3 0 ) α t j 1 j 2 j 3 σ j 1 σ j 2 σ j 3 ; j 1 , j 2 , j 3 = 1 , , T P .
4. The fourth-order parameter moment, μ 4 j 1 j 2 j 3 j 4 , is defined as follows:
μ 4 i j k l ( α j 1 α j 1 0 ) ( α j 2 α j 2 0 ) ( α j 3 α j 3 0 ) ( α j 4 α j 4 0 ) α q j 1 j 2 j 3 j 4 σ j 1 σ j 2 σ j 3 σ j 4 ;   j 1 , j 2 , j 3 , j 4 = 1 , , T P ,
where q j 1 j 2 j 3 j 4 denotes the fourth-order correlation among four parameters.
5. The fifth-order parameter moment, μ 5 j 1 j 2 j 3 j 4 j 5 , is defined as follows:
μ 5 j 1 j 2 j 3 j 4 j 5 ( α j 1 α j 1 0 ) ( α j 2 α j 2 0 ) ( α j 3 α j 3 0 ) ( α j 4 α j 4 0 ) ( α j 5 α j 5 0 ) α p j 1 j 2 j 3 j 4 j 5 σ j 1 σ j 2 σ j 3 σ j 4 σ j 5 ;   j 1 , j 2 , j 3 , j 4 , j 5 = 1 , , T P ,
where p i j k l m denotes the fifth-order correlation among five parameters.
6. The sixth-order parameter moment, μ 6 j 1 j 2 j 3 j 4 j 5 j 6 , is defined as follows:
μ 6 j 1 j 2 j 3 j 4 j 5 j 6 ( α j 1 α j 1 0 ) ( α j 2 α j 2 0 ) × × ( α j 5 α j 5 0 ) ( α j 6 α j 6 0 ) α s j 1 j 2 j 3 j 4 j 5 j 6 σ j 1 σ j 2 σ j 3 σ j 4 σ j 5 σ j 6 ,   j 1 , j 2 , j 3 , j 4 , j 5 , j 6 = 1 , , T P ,
where s j 1 j 2 j 3 j 4 j 5 j 6 denotes the sixth-order correlation among six parameters.
Higher-order moments of the parameter distribution are defined similarly to the first six moments defined in Equations (36)–(44). Uncertainties in the model’s parameters will evidently give rise to uncertainties in the model responses R k ( α ) . The approximate moments of the unknown distribution of R k ( α ) are obtained by using the so-called “propagation of errors” methodology, integrating formally, over the parameter distribution, various expressions involving the truncated Taylor-series expansion of the response provided in Equation (12). Using this concept of integrating the truncated Taylor-series expansion of the response, Tukey [49] has presented the most extensive formulas to date for the mean, variance, skewness and kurtosis of the response distribution, which include terms up to 4th-order response sensitivities. The remainder of this Section presents formulas which are consistent up to and including the 6th-order standard deviations, thus generalizing the formulas presented by Tukey [49].

4.1. Expectation Value of a Response

The expectation (value), E ( R k ) , of a response R k ( α ) is defined using Equation (35), as follows:
E ( R k ) R k ( α ) α D α R k ( α ) p α ( α ) d α .
Using Equations (6)–(12) in the definition provided in Equation (45) yields the following expressions for the nth-order approximate expectation, E ( n ) ( R k ) , n = 1 , , 6 , of the response R k ( α ) :
E ( 1 ) ( R k ) = E ( 0 ) ( R k ) R k ( α 0 ) ,
E ( 2 ) ( R k ) = E ( 1 ) ( R k ) + 1 2 j 1 = 1 T P j 2 = 1 T P 2 R k ( α 0 ) α j 1 α j 2 ρ j 1 j 2 σ j 1 σ j 2 ,
E ( 3 ) ( R k ) = E ( 2 ) ( R k ) + 1 6 j 1 = 1 T P j 2 = 1 T P j 3 = 1 T P 3 R k ( α 0 ) α j 1 α j 2 α i 3 t j 1 j 2 j 3 σ j 1 σ j 2 σ j 3 ,
E ( 4 ) ( R k ) = E ( 3 ) ( R k ) + 1 4 ! j 1 = 1 T P j 2 = 1 T P j 3 = 1 T P j 4 = 1 T P 4 R k ( α 0 ) α j 1 α j 2 α j 3 α j 4 q j 1 j 2 j 3 j 4 σ j 1 σ j 2 σ j 3 σ j 4   ,
E ( 5 ) ( R k ) = E ( 4 ) ( R k ) + 1 5 ! j 1 = 1 T P j 2 = 1 T P j 3 = 1 T P j 4 = 1 T P j 5 = 1 T P 5 R k ( α 0 ) α j 1 α j 2 α j 3 α j 4 α j 5 × p j 1 j 2 j 3 j 4 j 5 σ j 1 σ j 2 σ j 3 σ j 4 σ j 5   ,
E ( 6 ) ( R k ) = E ( 5 ) ( R k ) + 1 6 ! j 1 = 1 T P j 2 = 1 T P j 3 = 1 T P j 4 = 1 T P j 5 = 1 T P j 6 = 1 T P 6 R k ( α 0 ) α j 1 α j 2 α j 3 α j 4 α j 5 α j 6 × s j 1 j 2 j 3 j 4 j 5 j 6 σ j 1 σ j 2 σ j 3 σ j 4 σ j 5 σ j 6   .
The radius/domain of convergence of the series in Equation (51) must also be established before using it for any subsequent purpose, since the radius/domain of convergence of the series in Equation (51) may differ from the radius/domain of convergence of the Taylor-series in Equation (12). As indicated in Equation (47), the second-order sensitivities cause the response’s expected value to differ from the response’s computed value, i.e., E ( n ) ( R k ) R k ( α 0 ) , n 2 .

4.2. Response Parameter Covariances

For n = 1 , , 6 , the nth-order approximation of the covariance, cov ( α i , R k ) ( α i α i 0 ) [ R k ( α ) E ( R k ) ] α , between a computed responses and a parameter is defined as follows:
c o v ( α i ,   R k ( n ) ) ( α i α i 0 ) [ R k ( n ) ( α ) E ( n ) ( R k ) ] α   ,   n = 1 , , 6 ; i = 1 , , T P .
Using Equations (6)–(12) in Equation (52) yields the following expressions for i = 1 , , T P :
c o v ( α i ,   R k ( 1 ) ) ( α i α i 0 ) R k ( 1 ) ( α ) α = j 1 = 1 T P R k ( α 0 ) α j 1 ρ i , j 1 σ i σ j 1 ;
c o v ( α i ,   R k ( 2 ) ) ( α i α i 0 ) R k ( 2 ) ( α ) α   =   c o v ( α i ,   R k ( 1 ) ) + 1 2 j 1 = 1 T P j 2 = 1 T P 2 R k ( α 0 ) α j 1 α j 2 t i , j 1 j 2 σ i σ j 1 σ j 2   ;
c o v ( α i ,   R k ( 3 ) ) ( α i α i 0 ) R k ( 3 ) ( α ) α = c o v ( α i ,   R k ( 2 ) ) + 1 6 j 1 = 1 T P j 2 = 1 T P j 3 = 1 T P 3 R k ( α 0 ) α j 1 α j 2 α j 3 q i , j 1 j 2 j 3 σ i σ j 1 σ j 2 σ j 3   ;
c o v ( α i ,   R k ( 4 ) ) ( α i α i 0 ) R k ( 4 ) ( α ) α = c o v ( α i ,   R k ( 3 ) ) + 1 4 ! j 1 = 1 T P j 2 = 1 T P j 3 = 1 T P j 4 = 1 T P 4 R k ( α 0 ) α j 1 α j 2 α j 3 α j 4 p i , j 1 j 2 j 3 j 4 σ i σ j 1 σ j 2 σ j 3 σ j 4   ;
c o v ( α i ,   R k ( 5 ) ) ( α i α i 0 ) R k ( 5 ) ( α ) α = c o v ( α i ,   R k ( 4 ) ) + 1 5 ! j 1 = 1 T P j 2 = 1 T P j 3 = 1 T P j 4 = 1 T P j 5 = 1 T P 5 R k ( α 0 ) α j 1 α j 2 α j 3 α j 4 α j 5 s i , j 1 j 2 j 3 j 4 j 5 σ i σ j 1 σ j 2 σ j 3 σ j 4 σ j 5 .

4.3. Covariance of Two Response

For n = 1 , , 6 , the nth-order approximation, c o v ( R k ( n ) ,   R ( n ) ) , of the covariance, c o v ( R k ,   R ) [ R k ( α ) E ( R k ) ] [ R ( α ) E ( R ) ] α , between two responses R k and R is defined as follows:
{ c o v ( R k ( n ) ,   R ( n ) ) } i n c [ R k ( n ) ( α ) E ( n ) ( R k ) ] [ R ( n ) ( α ) E ( n ) ( R ) ] α .
It will be seen that the result, denoted as { c o v ( R k ( n ) ,   R ( n ) ) } i n c , which is provided by the right-side of, is inconsistent (i.e., incomplete) in the approximation of the highest-order term, which contains the product of standard deviations σ j 1 × σ j 2 × × σ j n . The alternative definition below provides an approximation for a “consistent” covariance, which is denoted as { c o v ( R k ( n ) ,   R ( n ) ) } c o n and which comprises the consistent (i.e., complete) expression for the term containing σ j 1 × σ j 2 × × σ j n , when the terms of O ( σ n + 1 ) are discarded, i.e.,
{ c o v ( R k ( n ) ,   R ( n ) ) } c o n c o v ( R k ( n + 1 ) ,   R ( n + 1 ) ) O ( σ n + 1 ) ,   n = 1 , , 5 .
The standard deviation, S D [ R k ( n ) ( α ) ] , of R k ( n ) ( α ) is obtained by setting k = in either Equation (58) or Equation (59) and taking the square root of the resulting expression, respectively, to obtain:
{ S D ( R k ( n ) ) } i n c { var ( R k ( n ) ,   R k ( n ) ) } i n c + O ( σ n / 2 ) ,   n = 1 , 2 ,
{ S D ( R k ( n ) ) } c o n { var ( R k ( n ) ,   R k ( n ) ) } c o n ,   n = 1 , 2 ,
As indicated by the right-side of Equation (60), the standard deviation { S D ( R k ( n ) ) } i n c corresponding to the inconsistent variance { var ( R k ( n ) ,   R k ( n ) ) } i n c will not comprise the complete number of terms that contain the nth-order parameter standard deviations. In contradistinction to { S D ( R k ( n ) ) } i n c , the consistent standard deviation, { S D ( R k ( n ) ) } c o n , which corresponds to the consistent variance { var ( R k ( n ) ,   R k ( n ) ) } i n c , will comprise all of the terms that contain the nth-order parameter standard deviations.
The expressions for the covariances and standard deviations of various orders of approximation n = 1 , , 5 , are provided below:
  • The 1st-order approximation of the covariance c o v ( R k ( n ) ,   R ( n ) )
    c o v ( R k ( 1 ) ,   R ( 1 ) ) [ R k ( 1 ) ( α ) E ( 1 ) ( R k ) ] [ R ( 1 ) ( α ) E ( 1 ) ( R ) ] α = j 1 = 1 N α j 2 = 1 N α R k ( α 0 ) α j 1 R ( α 0 ) α j 2 ρ j 1 j 2 σ j 1 σ j 2 ,
    Notably, the expression of c o v ( R k ( 1 ) ,   R ( 1 ) ) provided in Equation (62) is consistent both in the highest-order (in this case: first-order) of sensitivities and also in the highest-order of parameter standard deviation (in this case, second-order) since all of the terms involving the product σ j 1 σ j 2 are consistently included (i.e., none are missing) in the expression provided. Therefore, the standard deviation will also be correct to first-order in the standard deviations of the parameters, i.e.,
    S D ( R k ( 1 ) ) = { j 1 = 1 T P j 2 = 1 T P R k ( α 0 ) α j 1 R ( α 0 ) α j 2 ρ j 1 j 2 σ j 1 σ j 2 } 1 2 .
  • The 2nd-order approximation of the covariance c o v ( R k ( n ) ,   R ( n ) ) :
    (i)
    Using Equation (58) yields the following expression:
    { c o v ( R k ( 2 ) ,   R ( 2 ) ) } i n c = c o v ( R k ( 1 ) ,   R ( 1 ) ) + 1 2 j 1 = 1 T P j 2 = 1 T P j 3 = 1 T P ( 2 R k ( α 0 ) α j 1 α j 2 R ( α 0 ) α j 3 + R k ( α 0 ) α j 1 2 R ( α 0 ) α j 2 α j 3 )   t j 1 j 2 j 3 σ j 1 σ j 2 σ j 3 + 1 4 j 1 = 1 T P j 2 = 1 T P j 3 = 1 T P j 4 = 1 T P 2 R k ( α 0 ) α j 1 α j 2 2 R ( α 0 ) α j 3 α j 4 ( q j 1 j 2 j 3 j 4 ρ j 1 j 2 ρ j 3 j 4 ) σ j 1 σ j 2 σ j 3 σ j 4   .
    (ii)
    Using Equation (59) yields the following expression:
    { c o v ( R k ( 2 ) ,   R ( 2 ) ) } c o n = { c o v ( R k ( 2 ) ,   R ( 2 ) ) } i n c + 1 6 j 1 = 1 T P j 2 = 1 T P j 3 = 1 T P j 4 = 1 T P ( 3 R k ( α 0 ) α j 1 α j 2 α j 3 R ( α 0 ) α j 4 + R k ( α 0 ) α j 1 3 R ( α 0 ) α j 2 α j 3 α j 4 )   q j 1 j 2 j 3 j 4 σ j 1 σ j 2 σ j 3 σ j 4 .
  • Notably, the expression of { c o v ( R k ( 2 ) ,   R ( 2 ) ) } i n c provided in Equation (64) is consistent in the 2nd-order of sensitivities but is inconsistent in the 4th-order of standard deviations of parameters, i.e., O ( σ j 1 σ j 2 σ j 3 σ j 4 )   . On the other hand, the expression { c o v ( R k ( 2 ) ,   R ( 2 ) ) } c o n provided in Equation (65) is consistent in the 4th-order (i.e., σ j 1 σ j 2 σ j 3 σ j 4   ) of parameter standard deviations. Therefore, if the 3rd-order sensitivities are available, then the expression provided in Equation (65) should be used, since it is correct up to, and including, the fourth-order terms containing the products σ j 1 σ j 2 σ j 3 σ j 4   . Consequently, the consistent second-order standard deviation { S D ( R k ( 2 ) ) } c o n { c o v ( R k ( 2 ) ,   R ( 2 ) ) } c o n will not have any 2nd-order errors in the parameter standard deviations. In contradistinction, the 2nd-order inconsistent standard deviation { S D ( R k ( 2 ) ) } i n c { c o v ( R k ( 2 ) ,   R ( 2 ) ) } i n c will have second-order errors in the parameter standard deviations.
  • The 3rd-order approximation of the covariance c o v ( R k ( 3 ) ,   R ( 3 ) ) :
    (i)
    Using Equation (58) yields the following expression:
    { c o v ( R k ( 3 ) ,   R ( 3 ) ) } i n c = 1 12 j 1 = 1 T P j 2 = 1 T P j 3 = 1 T P j 4 = 1 T P j 5 = 1 T P [ 3 R k ( α 0 ) α j 1 α j 2 α j 3 2 R ( α 0 ) α j 4 α j 5 ( p j 1 j 2 j 3 j 4 j 5 t j 1 j 2 j 3 ρ j 4 j 5 ) + 2 R k ( α 0 ) α j 1 α j 2 3 R ( α 0 ) α j 3 α j 4 α j 5 ( p j 1 j 2 j 3 j 4 j 5 ρ j 1 j 2 t j 3 j 4 j 5 ) ] σ j 1 σ j 2 σ j 3 σ j 4 σ j 5   + { c o v ( R k ( 2 ) ,   R ( 2 ) ) } c o n .
    (ii)
    Using Equation (59) yields the following expression:
    { c o v ( R k ( 3 ) ,   R ( 3 ) ) } c o n = { c o v ( R k ( 3 ) ,   R ( 3 ) ) } i n c + 1 24 j 1 = 1 T P j 2 = 1 T P j 3 = 1 T P j 4 = 1 T P j 5 = 1 T P [ 4 R k ( α 0 ) α j 1 α j 2 α j 3 α j 4 R ( α 0 ) α j 5   + R k ( α 0 ) α j 1 4 R ( α 0 ) α j 2 α j 3 α j 4 α j 5 ] p j 1 j 2 j 3 j 4 j 5 σ j 1 σ j 2 σ j 3 σ j 4 σ j 5   .
    The expression of { c o v ( R k ( 3 ) ,   R ( 3 ) ) } i n c provided in Equation (66) is consistent in the third-order sensitivities but is inconsistent in the highest-order of parameter standard deviations, i.e., O ( σ j 1 σ j 2 σ j 3 σ j 4 σ j 5 )   , while the expression { c o v ( R k ( 3 ) ,   R ( 3 ) ) } c o n provided in Equation (67) is consistent in the highest-order of parameter standard deviations. Therefore, if the 4th-order sensitivities are available, then the expression provided in Equation (67) should be used, since it is correct up to, and including, the fifth-order terms containing the products σ j 1 σ j 2 σ j 3 σ j 4   σ j 5 . Also noteworthy is the fact that neither the consistent nor the inconsistent standard deviations computed using the 3rd-order variance approximations provided in Equation (66) or Equation(67), respectively, are correct to third-order (i.e., σ j 1 σ j 2 σ j 3 σ j 4 σ j 5 σ j 6 ) parameter standard deviations. In order to obtain standard deviations which are correct up to and including the 3rd-order terms containing O ( σ j 1 σ j 2 σ j 3 σ j 4 σ j 5 σ j 6 ) , it is necessary to use the 4th-order variance approximations to be provided below.
  • The 4th-order approximation of the covariance c o v ( R k ( n ) ,   R ( n ) ) :
    (i)
    Using Equation (58) yields the following expression:
    { c o v ( R k ( 4 ) ,   R ( 4 ) ) } i n c = { c o v ( R k ( 3 ) ,   R ( 3 ) ) } c o n + 1 36 j 1 = 1 J α j 2 = 1 J α j 3 = 1 J α j 4 = 1 N α j 5 = 1 N α j 6 = 1 N α 3 R k ( α 0 ) α j 1 α j 2 α j 3 3 R l ( α 0 ) α j 4 α j 5 α j 6   × ( s j 1 j 2 j 3 j 4 j 5 j 6 t j 1 j 2 j 3 t j 4 j 5 j 6 ) σ j 1 σ j 2 σ j 3 σ j 4 σ j 5 σ j 6 + 1 48 j 1 = 1 N α j 2 = 1 N α j 3 = 1 N α j 4 = 1 N α j 5 = 1 N α j 6 = 1 N α 4 R k ( α 0 ) α j 1 α j 2 α j 3 α j 4 2 R l ( α 0 ) α j 5 α j 6 × ( s j 1 j 2 j 3 j 4 j 5 j 6 q j 1 j 2 j 3 j 4 ρ j 5 j 6 ) σ j 1 σ j 2 σ j 3 σ j 4 σ j 5 σ j 6 + 1 48 j 1 = 1 J α j 2 = 1 J α j 3 = 1 N α j 4 = 1 N α j 5 = 1 N α j 6 = 1 N α 2 R k ( α 0 ) α j 1 α j 2 4 R l ( α 0 ) α j 3 α j 4 α j 5 α j 6 × ( s j 1 j 2 j 3 j 4 j 5 j 6 ρ j 1 j 2 q j 3 j 4 j 5 j 6 ) σ j 1 σ j 2 σ j 3 σ j 4 σ j 5 σ j 6   .
    (ii)
    Using Equation (59) yields the following expression:
    { c o v ( R k ( 4 ) ,   R ( 4 ) ) } c o n = { c o v ( R k ( 4 ) ,   R ( 4 ) ) } i n c + 1 120 j 1 = 1 T P j 2 = 1 T P j 3 = 1 T P j 4 = 1 T P j 5 = 1 T P j 6 = 1 T P [ 5 R k ( α 0 ) α j 1 α j 2 α j 3 α j 4 α j 5 × R l ( α 0 ) α j 6 + R k ( α 0 ) α j 1 5 R l ( α 0 ) α j 2 α j 3 α j 4 α j 5 α j 6 ] s j 1 j 2 j 3 j 4 j 5 j 6 σ j 1 σ j 2 σ j 3 σ j 4 σ j 5 σ j 6   .
The expression of { c o v ( R k ( 4 ) ,   R ( 4 ) ) } i n c provided in Equation (68) is consistent in the 4th-order of sensitivities but is inconsistent in the highest-order (6th-order) of parameter standard deviations, while the expression { c o v ( R k ( 3 ) ,   R ( 3 ) ) } c o n provided in Equation (67) is consistent in the 6th-order of parameter standard deviations σ j 1 σ j 2 σ j 3 σ j 4   σ j 5   σ j 6 . Therefore, if the 5th-order sensitivities are available, then the expression provided in Equation (67) should be used, since it is correct up to, and including, the fifth-order terms containing the products σ j 1 σ j 2 σ j 3 σ j 4   σ j 5   σ j 6 . The standard deviation { S D ( R k ( 4 ) ) } c o n { c o v ( R k ( 4 ) ,   R ( 4 ) ) } c o n would comprise all of the 3rd-order standard deviations of the form σ j 1 σ j 2 σ j 3 σ j 4   σ j 5   σ j 6 .

4.4. Triple Correlations among Responses and Parameters

  • The triple-correlations (or third-order moment of the distribution of responses), denoted as μ 3 ( R k , R , R m ) , among three responses R k ( α ) , R ( α ) and R m ( α ) , are defined as follows:
    μ 3 ( R k , R , R m ) [ R k ( α ) E ( R k ) ] [ R ( α ) E ( R ) ] [ R m ( α ) E ( R m ) ] α .
    • If only first-order response sensitivities are available, the following 1st-order approximate expression, denoted as μ 3 ( 1 ) ( R k , R , R m ) , can be obtained:
      μ 3 ( 1 ) ( R k , R , R m ) = j 1 = 1 T P j 2 = 1 T P j 3 = 1 T P R k ( α 0 ) α j 1 R ( α 0 ) α j 2 R m ( α 0 ) α j 3 t j 1 j 2 j 3 σ j 1 σ j 2 σ j 3 .
    • If 1st- and 2nd-order response sensitivities are available, the following 2nd-order approximate expression, denoted as μ 3 ( 2 ) ( R k , R , R m ) , can be obtained:
      μ 3 ( 2 ) ( R k , R , R m ) = μ 3 ( 1 ) ( R k , R , R m ) + μ 3 ( 2 ) ( σ 5 ) + μ 3 ( 2 ) ( σ 6 ) + 1 2 j 1 = 1 N α j 2 = 1 N α j 3 = 1 N α j 4 = 1 N α [ R k ( α 0 ) α j 1 R ( α 0 ) α j 2 2 R m ( α 0 ) α j 3 α j 4 ( q j 1 j 2 j 3 j 4 ρ j 1 j 2 ρ j 3 j 4 ) + R k ( α 0 ) α j 1 2 R ( α 0 ) α j 2 α j 3 R m ( α 0 ) α j 4 ( q j 1 j 2 j 3 j 4 ρ j 1 j 4 ρ j 2 j 3 ) ] σ j 1 σ j 2 σ j 3 σ j 4 ,
      where the quantity μ 3 ( 2 ) ( σ 5 ) contains products of the form σ j 1 σ j 2 σ j 3 σ j 4 σ j 5 and is defined as follows:
      μ 3 ( 2 ) ( σ 5 ) 1 4 j 1 = 1 T P j 2 = 1 T P j 3 = 1 T P j 4 = 1 T P j 5 = 1 T P [ R k ( α 0 ) α j 1 2 R ( α 0 ) α j 2 α j 3 2 R m ( α 0 ) α j 4 α j 5 × ( p j 1 j 2 j 3 j 4 j 5 t j 1 j 2 j 3 ρ j 4 j 5 t j 1 j 4 j 5 ρ j 2 j 3 ) + 2 R k ( α 0 ) α j 1 α j 2 R ( α 0 ) α j 3 2 R m ( α 0 ) α j 4 α j 5 × ( p j 1 j 2 j 3 j 4 j 5 ρ j 1 j 2 t j 3 j 4 j 5 t j 1 j 2 j 3 ρ j 4 j 5 ) + 2 R k ( α 0 ) α j 1 α j 2 2 R ( α 0 ) α j 3 α j 4 R m ( α 0 ) α j 5 × ( p j 1 j 2 j 3 j 4 j 5 ρ j 1 j 2 t j 3 j 4 j 5 t j 1 j 2 j 5 ρ j 3 j 4 ) ] σ j 1 σ j 2 σ j 3 σ j 4 σ j 5   ,
      while the quantity μ 3 ( 2 ) ( σ 6 ) contains products of the form σ j 1 σ j 2 σ j 3 σ j 4 σ j 5 σ j 6 and is defined as follows:
      μ 3 ( 2 ) ( σ 6 ) 1 8 j 1 = 1 T P j 2 = 1 T P j 3 = 1 T P j 4 = 1 T P j 5 = 1 T P j 6 = 1 T P 2 R k ( α 0 ) α j 1 α j 2 2 R ( α 0 ) α j 3 α j 4 2 R m ( α 0 ) α j 5 α j 6 σ j 1 σ j 2 σ j 3 σ j 4 σ j 5 σ j 6 ( s j 1 j 2 j 3 j 4 j 5 j 6 + 2 ρ j 1 j 2 ρ j 3 j 4 ρ j 5 j 6 ρ j 1 j 2 q j 3 j 4 j 5 j 6   q j 1 j 2 j 5 j 6 ρ j 3 j 4 q j 1 j 2 j 3 j 4 ρ j 5 j 6 )   .
      As indicated in Equation (74), the expression of μ 3 ( 2 ) ( R k , R , R m ) consistently contains all of the 1st- and 2nd-order response sensitivities, and all of the 5th-order terms in standard deviations (of the form σ j 1 σ j 2 σ j 3 σ j 4 σ j 5 ) but does not contain all of the 6th-order terms in standard deviations (of the form σ j 1 σ j 2 σ j 3 σ j 4 σ j 5 σ j 6 ) The 6th-order terms in standard deviations, containing the remaining terms of the form σ j 1 σ j 2 σ j 3 σ j 4 σ j 5 σ j 6 are provided below.
    • The availability of the third-order response sensitivities enables the computation of additional terms that comprise 6th-order terms containing the products σ j 1 σ j 2 σ j 3 σ j 4 σ j 5 σ j 6 . The following 3rd-order (in sensitivities) approximate expression, denoted as μ 3 ( 3 ) ( R k , R , R m ) , can be obtained:
      μ 3 ( 3 ) ( R k , R l , R m ) 1 12   j 1 = 1 T P j 2 = 1 T P j 3 = 1 T P j 4 = 1 T P j 5 = 1 T P j 6 = 1 T P [ R k ( α 0 ) α j 1 2 R l ( α 0 ) α j 2 α j 3 3 R m ( α 0 ) α j 4 α j 5 α j 6 × ( s j 1 j 2 j 3 j 4 j 5 j 6 t j 1 j 2 j 3 t j 4 j 5 j 6 q j 1 j 4 j 5 j 6 ρ j 2 j 3 ) + R k ( α 0 ) α i 1 3 R l ( α 0 ) α i 2 α i 3 α i 4 2 R m ( α 0 ) α i 5 α i 6 × ( s j 1 j 2 j 3 j 4 j 5 j 6 t j 1 j 5 j 6 t j 2 j 3 j 4 q j 1 j 2 j 3 j 4 ρ j 5 j 6 ) + 2 R k ( α 0 ) α j 1 α j 2 R l ( α 0 ) α j 3 3 R m ( α 0 ) α j 4 α j 5 α j 6 × ( s j 1 j 2 j 3 j 4 j 5 j 6 t j 1 j 2 j 3 t j 4 j 5 j 6 ρ j 1 j 2 q j 3 j 4 j 5 j 6 ) + 2 R k ( α 0 ) α j 1 α j 2 3 R l ( α 0 ) α j 3 α j 4 α j 5 R m ( α 0 ) α j 6   × ( s j 1 j 2 j 3 j 4 j 5 j 6 t j 1 j 2 j 6 t j 3 j 4 j 5 ρ j 1 j 2 q j 3 j 4 j 5 j 6 ) + 3 R l ( α 0 ) α j 1 α j 2 α j 3 R l ( α 0 ) α j 4 2 R m ( α 0 ) α j 5 α j 6 × ( s j 1 j 2 j 3 j 4 j 5 j 6 t j 1 j 2 j 3 t j 4 j 5 j 6 q j 1 j 2 j 3 j 4 ρ j 5 j 6 ) + 3 R l ( α 0 ) α j 1 α j 2 α j 3 2 R l ( α 0 ) α j 4 α j 5 R m ( α 0 ) α j 6 × ( s j 1 j 2 j 3 j 4 j 5 j 6 t j 1 j 2 j 3 t j 4 j 5 j 6 q j 1 j 2 j 3 j 6 ρ j 4 j 5 ) ] σ j 1 σ j 2 σ j 3 σ j 4 σ j 5 σ j 6 + μ 3 ( 2 ) ( R k , R , R m ) .
    • If fourth-order response sensitivities are also available, the following 4th-order (in sensitivities) approximate expression μ 3 ( 4 ) ( R k , R l , R m ) can be obtained:
      μ 3 ( 4 ) ( R k , R l , R m ) 1 24 j 1 = 1 T P j 2 = 1 T P j 3 = 1 T P j 4 = 1 T P j 5 = 1 T P j 6 = 1 T P [ R k ( α 0 ) α j 1 R l ( α 0 ) α j 2 4 R m ( α 0 ) α j 3 α j 4 α j 5 α j 6   × ( s j 1 j 2 j 3 j 4 j 5 j 6 ρ j 1 j 2 q j 3 j 4 j 5 j 6 ) + R k ( α 0 ) α j 1 4 R l ( α 0 ) α j 2 α j 3 α j 4 α j 5 R m ( α 0 ) α j 6 × ( s j 1 j 2 j 3 j 4 j 5 j 6 ρ j 1 j 6 q j 2 j 3 j 4 j 5 ) + 4 R k ( α 0 ) α j 1 α j 2 α j 3 α j 4 R l ( α 0 ) α j 5 R m ( α 0 ) α j 6 × ( s j 1 j 2 j 3 j 4 j 5 j 6 q j 1 j 2 j 3 j 4 ρ j 5 j 6 ) ] σ j 1 σ j 2 σ j 3 σ j 4 σ j 5 σ j 6   + μ 3 ( 3 ) ( R k , R l , R m ) .
      As can be observed from Equation (76), the quantity μ 3 ( 4 ) ( R k , R l , R m ) contains all of the terms involving 6th-order products σ j 1 σ j 2 σ j 3 σ j 4 σ j 5 σ j 6 of standard deviations.
  • The triple-correlations among one parameter, α i , and two responses, R k ( α ) and R ( α ) , are defined as follows:
    μ 3 ( α i , R k , R ) ( α i α i 0 ) [ R k ( α ) E ( R k ) ] [ R ( α ) E ( R ) ] α .
    Using the Taylor series up to and including the 6th-order in standard deviations, the expression of μ 3 ( α i , R k , R ) is as follows:
    μ 3 ( α i , R k , R ) = j 1 = 1 T P j 2 = 1 T P R j ( α 0 ) α j 1 R k ( α 0 ) α j 2 t j 1 j 2 i σ j 1 σ j 2 σ i + 1 2 j 1 = 1 T P j 2 = 1 T P j 3 = 1 T P [ R j ( α 0 ) α j 1 2 R k ( α 0 ) α j 2 α j 3 × ( q j 1 j 2 j 3 i ρ j 1 i ρ j 2 j 3 ) + 2 R j ( α 0 ) α j 1 α j 2 R k ( α 0 ) α j 3 ( q j 1 j 2 j 3 i ρ j 1 j 2 ρ j 3 i ) ] σ j 1 σ j 2 σ j 3 σ i + μ 5 ( 5 ) ( α i , R k , R ) + μ 5 ( 6 ) ( α i , R k , R ) ,
    where the quantity μ 5 ( 5 ) ( α i , R k , R ) comprises terms in 5th-order standard deviations involving products of the form σ j 1 σ j 2 σ j 3 σ j 4 σ i , and is defined as follows:
    μ 3 ( 5 ) ( α i , R k , R ) 1 4 j 1 = 1 T P j 2 = 1 T P j 3 = 1 T P j 4 = 1 T P 2 R j ( α 0 ) α j 1 α j 2 2 R k ( α 0 ) α j 3 α j 4 ( p j 1 j 2 j 3 j 4 i t j 1 j 2 i ρ j 3 j 4 ρ j 1 j 2 t j 3 j 4 i ) × σ j 1 σ j 2 σ j 3 σ j 4 σ i + 1 6 j 1 = 1 T P j 2 = 1 T P j 3 = 1 T P j 4 = 1 T P [ R j ( α 0 ) α j 1 3 R k ( α 0 ) α j 2 α j 3 α j 4 ( p j 1 j 2 j 3 j 4 i ρ j 1 i t j 2 j 3 j 4 ) + 3 R j ( α 0 ) α j 1 α j 2 α j 3 R k ( α 0 ) α j 4 × ( p j 1 j 2 j 3 j 4 i t j 1 j 2 j 3 ρ j 4 i ) ] σ j 1 σ j 2 σ j 3 σ j 4 σ i ,
    while the quantity μ 5 ( 6 ) ( α i , R k , R ) comprises terms in 6th-order standard deviations involving products of the form σ j 1 σ j 2 σ j 3 σ j 4 σ j 5 σ i , and is defined as follows:
    μ 5 ( 6 ) ( α i , R k , R ) 1 12 j 1 = 1 T P j 2 = 1 T P j 3 = 1 T P j 4 = 1 T P j 5 = 1 T P [ 2 R j ( α 0 ) α j 1 α j 2 3 R k ( α 0 ) α j 3 α j 4 α j 5 × ( s j 1 j 2 j 3 j 4 j 5 i t j 1 j 2 i t j 3 j 4 j 5 ρ j 1 j 2 q j 3 j 4 j 5 i ) + 3 R j ( α 0 ) α j 1 α j 2 α j 3 2 R k ( α 0 ) α j 4 α j 5 × ( s j 1 j 2 j 3 j 4 j 5 i t j 1 j 2 j 3 t j 4 j 5 i q j 1 j 2 j 3 i ρ j 4 j 5 ) ] σ j 1 σ j 2 σ j 3 σ j 4 σ j 5 σ i + 1 24 j 1 = 1 T P j 2 = 1 T P j 3 = 1 T P j 4 = 1 T P j 5 = 1 T P [ R j ( α 0 ) α j 1 4 R k ( α 0 ) α j 2 α j 3 α j 4 α j 5 ( s j 1 j 2 j 3 j 4 j 5 i ρ j 1 i q j 2 j 3 j 4 j 5 ) + 4 R j ( α 0 ) α j 1 α j 2 α j 3 α j 4 R k ( α 0 ) α j 5 ( s j 1 j 2 j 3 j 4 j 5 i q j 1 j 2 j 3 j 4 ρ j 5 i ) ] σ j 1 σ j 2 σ j 3 σ j 4 σ j 5 σ i .
  • The triple-correlations among two parameters α i , α j and one response R k ( α ) are defined as follows:
    μ 3 ( α i , α j , R k ) ( α i α i 0 ) ( α j α j 0 ) [ R k ( α ) E ( R k ) ] α
    Using the Taylor series up to and including the 6th-order in standard deviations, the expression of μ 3 ( α i , α j , R k ) is as follows:
    μ 3 ( α i , α j , R k ) = j 1 = 1 T P R k ( α 0 ) α j 1 t j 1 i j σ j 1 σ i σ j + 1 2 j 1 = 1 T P j 2 = 1 T P 2 R k ( α 0 ) α j 1 α j 2 ( q j 1 j 2 i j ρ j 1 j 2 ρ i j ) σ j 1 σ j 2 σ i σ j + 1 6 j 1 = 1 T P j 2 = 1 T P j 3 = 1 T P 3 R k ( α 0 ) α j 1 α j 2 α j 3 ( p j 1 j 2 j 3 i j t j 1 j 2 j 3 ρ i j ) σ j 1 σ j 2 σ j 3 σ i σ j + 1 24 j 1 = 1 T P j 2 = 1 T P j 3 = 1 T P j 4 = 1 T P 4 R k ( α 0 ) α j 1 α j 2 α j 3 α j 4 ( s j 1 j 2 j 3 j 4 i j q j 1 j 2 j 3 j 4 ρ i j ) σ j 1 σ j 2 σ j 3 σ j 4 σ i σ j .

4.5. Quadruple Correlations among Responses and Parameters

  • The quadruple-correlations (or fourth-order moment of the distribution of responses), denoted as μ 4 ( R k , R , R m , R n ) , among four responses R k ( α ) , R ( α ) , R m ( α ) and R n ( α ) , are defined as follows:
    μ 4 ( R k , R , R m , R n ) [ R k ( α ) E ( R k ) ] [ R ( α ) E ( R ) ] × [ R m ( α ) E ( R m ) ] [ R n ( α ) E ( R n ) ] α .
    • If only first-order response sensitivities are available, the following 1st-order approximate expression, denoted as μ 4 ( 1 ) ( R k , R l , R m , R n ) , can be obtained:
      μ 4 ( 1 ) ( R k , R l , R m , R n ) [ R k ( 1 ) ( α ) E ( 1 ) ( R k ) ] [ R ( 1 ) ( α ) E ( 1 ) ( R ) ] × [ R m ( 1 ) ( α ) E ( 1 ) ( R m ) ] [ R n ( 1 ) ( α ) E ( 1 ) ( R n ) ] α = j 1 = 1 T P j 2 = 1 T P j 3 = 1 T P j 4 = 1 T P R k ( α 0 ) α j 1 R l ( α 0 ) α j 2 R m ( α 0 ) α j 3 R n ( α 0 ) α j 4 q j 1 j 2 j 3 j 4 σ j 1 σ j 2 σ j 3 σ j 4   .
      The expression of μ 4 ( 1 ) ( R k , R l , R m , R n ) is consistent in that it comprises all of the 1st-order sensitivities and all of the terms involving 4th-order products of standard deviations of the form σ j 1 σ j 2 σ j 3 σ j 4 .
    • If all 1st- and 2nd-order response sensitivities are available, the following approximation of the quadruple-correlations among four responses R k ( α ) , R ( α ) , R m ( α ) and R n ( α ) can be obtained:
      μ 4 ( 2 ) ( R k , R , R m , R n ) μ 4 ( 1 ) ( R k , R , R m , R n ) + μ 4 ( 2 , a ) ( σ 6 ) + μ 4 ( 2 , b ) ( σ 6 ) + 1 2 j 1 = 1 T P j 2 = 1 T P j 3 = 1 T P j 4 = 1 T P j 5 = 1 T P [ R k ( α 0 ) α j 1 R ( α 0 ) α j 2 R m ( α 0 ) α j 3 2 R n ( α 0 ) α j 4 α j 5 × ( p j 1 j 2 j 3 j 4 j 5 t j 1 j 2 j 3 ρ j 4 j 5 ) + R k ( α 0 ) α j 1 R ( α 0 ) α j 2 2 R m ( α 0 ) α j 3 α j 4 R n ( α 0 ) α j 5 ( p j 1 j 2 j 3 j 4 j 5 t j 1 j 2 j 5 ρ j 3 j 4 ) + R k ( α 0 ) α j 1 2 R ( α 0 ) α j 2 α j 3 R m ( α 0 ) α j 4 R n ( α 0 ) α j 5 ( p j 1 j 2 j 3 j 4 j 5 t j 1 j 4 j 5 ρ j 2 j 3 ) + 2 R k ( α 0 ) α j 1 α j 2 R ( α 0 ) α j 3 R m ( α 0 ) α j 4 R n ( α 0 ) α j 5 ( p j 1 j 2 j 3 j 4 j 5 ρ j 1 j 2 t j 3 j 4 j 5 ) ] σ j 1 σ j 2 σ j 3 σ j 4 σ j 5 ,
      where the quantities μ 4 ( 2 , a ) ( σ 6 ) and μ 4 ( 2 , b ) ( σ 6 ) contain the terms involving 1st-order and 2nd-order sensitivities but 6th-order products of standard deviations of the form σ j 1 σ j 2 σ j 3 σ j 4 σ j 5 σ j 6 , and are defined as follows:
      μ 4 ( 2 , a ) ( σ 6 ) 1 4 j 1 = 1 T P j 2 = 1 T P j 3 = 1 T P j 4 = 1 T P j 5 = 1 T P j 6 = 1 T P [ R k ( α 0 ) α j 1 R ( α 0 ) α j 2 2 R m ( α 0 ) α j 3 α j 4 2 R n ( α 0 ) α j 5 α j 6 × ( s j 1 j 2 j 3 j 4 j 5 j 6 + ρ j 1 j 2 ρ j 3 j 4 ρ j 5 j 6 q j 1 j 2 j 5 j 6 ρ j 3 j 4 q j 1 j 2 j 3 j 4 ρ j 5 j 6 ) + R k ( α 0 ) α j 1 2 R ( α 0 ) α j 2 α j 3 R m ( α 0 ) α j 4 2 R n ( α 0 ) α j 5 α j 6 × ( s j 1 j 2 j 3 j 4 j 5 j 6 + ρ j 1 j 4 ρ j 2 j 3 ρ j 5 j 6 q j 1 j 4 j 5 j 6 ρ j 2 j 3 q j 1 j 2 j 3 j 4 ρ j 5 j 6 ) + R k ( α 0 ) α j 1 2 R ( α 0 ) α j 2 α j 3 2 R m ( α 0 ) α j 4 α j 5 R n ( α 0 ) α j 6 × ( s j 1 j 2 j 3 j 4 j 5 j 6 + ρ j 1 j 6 ρ j 2 j 3 ρ j 4 j 5 q j 1 j 4 j 5 j 6 ρ j 2 j 3 q j 1 j 2 j 3 j 6 ρ j 4 j 5 ) ]   σ j 1 σ j 2 σ j 3 σ j 4 σ j 5 σ j 6   ,
      μ 4 ( 2 , b ) ( σ 6 ) 1 4 j 1 = 1 T P j 2 = 1 T P j 3 = 1 T P j 4 = 1 T P j 5 = 1 T P j 6 = 1 T P [ 2 R k ( α 0 ) α j 1 α j 2 R ( α 0 ) α j 3 R m ( α 0 ) α j 4 2 R n ( α 0 ) α j 5 α j 6 × ( s j 1 j 2 j 3 j 4 j 5 j 6 + ρ j 1 j 2 ρ j 3 j 4 ρ j 5 j 6 ρ j 1 j 2 q j 3 j 4 j 5 j 6 q j 1 j 2 j 3 j 4 ρ j 5 j 6 ) + 2 R k ( α 0 ) α j 1 α j 2 R ( α 0 ) α j 3 2 R m ( α 0 ) α j 4 α j 5 R n ( α 0 ) α j 6 × ( s j 1 j 2 j 3 j 4 j 5 j 6 + ρ j 1 j 2 ρ j 3 j 6 ρ j 4 j 5 ρ j 1 j 2 q j 3 j 4 j 5 j 6 q j 1 j 2 j 3 j 6 ρ j 4 j 5 ) + 2 R k ( α 0 ) α j 1 α j 2 2 R l ( α 0 ) α j 3 α j 4 R m ( α 0 ) α j 5 R n ( α 0 ) α j 6 × ( s j 1 j 2 j 3 j 4 j 5 j 6 + ρ j 1 j 2 ρ j 3 j 4 ρ j 5 j 6 ρ j 1 j 2 q j 3 j 4 j 5 j 6 q j 1 j 2 j 5 j 6 ρ j 3 j 4 ) ] σ j 1 σ j 2 σ j 3 σ j 4 σ j 5 σ j 6   .
    • If third-order response sensitivities are also available, then the remaining terms containing 6th-order products of standard deviations of the form σ j 1 σ j 2 σ j 3 σ j 4 σ j 5 σ j 6 can also be computed, to obtain the following approximation consistent in 6th-order products of standard deviations, denoted as μ 4 ( 3 ) ( R k , R l , R m , R n ) :
      μ 4 ( 3 ) ( R k , R , R m , R n ) μ 4 ( 2 ) ( R k , R , R m , R n ) + 1 6 j 1 = 1 T P j 2 = 1 T P j 3 = 1 T P j 4 = 1 T P j 5 = 1 T P j 6 = 1 T P [ R k ( α 0 ) α j 1 R l ( α 0 ) α j 2 R m ( α 0 ) α j 3 3 R n ( α 0 ) α j 4 α j 5 α j 6 × ( s j 1 j 2 j 3 j 4 j 5 j 6 t j 1 j 2 j 3 t j 4 j 5 j 6 ) + R k ( α 0 ) α j 1 R l ( α 0 ) α j 2 3 R l ( α 0 ) α j 3 α j 4 α j 5 R n ( α 0 ) α j 6 × ( s j 1 j 2 j 3 j 4 j 5 j 6 t j 1 j 2 j 6 t j 3 j 4 j 5 ) + R k ( α 0 ) α j 1 3 R l ( α 0 ) α j 2 α j 3 α j 4 R m ( α 0 ) α j 5 R n ( α 0 ) α j 6 × ( s j 1 j 2 j 3 j 4 j 5 j 6 t j 1 j 5 j 6 t j 2 j 3 j 4 ) + 3 R k ( α 0 ) α j 1 α j 2 α j 3 R l ( α 0 ) α j 4 R m ( α 0 ) α j 5 R n ( α 0 ) α j 6 × ( s j 1 j 2 j 3 j 4 j 5 j 6 t j 1 j 2 j 3 t j 4 j 5 j 6 ) ] σ j 1 σ j 2 σ j 3 σ j 4 σ j 5 σ j 6 .
  • The quadruple-correlations denoted as μ 4 ( α i , R j , R k , R ) , among one parameter, α i , and three responses R j ( α ) , R k ( α ) , and R ( α ) , are defined as follows:
    μ 4 ( α i , R j , R k , R ) ( α i α i 0 ) [ R j ( α ) E ( R j ) ] [ R k ( α ) E ( R k ) ] [ R ( α ) E ( R ) ] α .
    • If first- and second-order response sensitivities are available, the following 2nd-order approximate expression, denoted as μ 4 ( 2 ) ( α i , R j , R k , R ) , can be obtained:
      μ 4 ( 2 ) ( α i , R j , R k , R ) = j 1 = 1 T P j 2 = 1 T P j 3 = 1 T P R j ( α 0 ) α j 1 R k ( α 0 ) α j 2 R l ( α 0 ) α j 3 q j 1 j 2 j 3 i σ j 1 σ j 2 σ j 3 σ i + 1 2 j 1 = 1 T P j 2 = 1 T P j 3 = 1 T P j 4 = 1 T P [ R j ( α 0 ) α j 1 R k ( α 0 ) α j 2 2 R l ( α 0 ) α j 3 α j 4 ( p j 1 j 2 j 3 j 4 i t j 1 j 2 i ρ j 3 j 4 ) + R j ( α 0 ) α j 1 2 R k ( α 0 ) α j 2 α j 3 R l ( α 0 ) α j 4 ( p j 1 j 2 j 3 j 4 i ρ j 2 j 3 t j 1 j 4 i ) + 2 R j ( α 0 ) α j 1 α j 2 R l ( α 0 ) α j 3 R l ( α 0 ) α j 4 × ( p j 1 j 2 j 3 j 4 i ρ j 1 j 2 t j 3 j 4 i ) ] σ j 1 σ j 2 σ j 3 σ j 4 σ i + 1 4 j 1 = 1 T P j 2 = 1 T P j 3 = 1 T P j 4 = 1 T P j 5 = 1 T P [ R j (