Advances in High-Order Sensitivity Analysis for Uncertainty Quantification and Reduction in Nuclear Energy Systems

The computational models of physical systems comprise parameters, independent and dependent variables. Since most of the model’s parameters stem from experimental procedures subject to uncertainties, the results predicted by computational models are also imprecise. The functional derivatives (called “sensitivities”) of results (called “responses”) produced by computational models are needed for many purposes, including: (i) ranking the importance of the various model parameters, which is essential to understanding the model; (ii) eliminating unimportant parameters and/or processes to obtain “reduced-order” models; (iii) “propagating” the model uncertainties, using sensitivities, to quantify uncertainties induced in the model’s response; (iv) comparing computations to experiments to perform “model validation,” addressing the question “does the model represent reality?”; (v) prioritizing improvements in the model; (vi) performing forward “predictive modeling,” including data assimilation and model calibration, to obtain best-estimate predicted results with reduced predicted uncertainties; (vii) performing inverse “predictive modeling”; (viii) designing and optimizing the system.

Evidently, the field of sensitivity analysis is widely interdisciplinary and the computational tools that have been used to estimate sensitivities encompass both deterministic and statistical methods. There are numerous publications that report computations of first-order sensitivities. It is well-known that for a model that comprises a total number of TP parameters, the computation of 1st-order sensitivities using conventional statistical or finite difference methods requires at least 2TP large-scale computations. Furthermore, these conventional methods can produce only approximate, rather than exact, values for the 1st-order sensitivities.

A fundamental breakthrough in the first-order sensitivity analysis of problems of interest to nuclear reactor physics, which are modeled by the linear Boltzmann transport equation, was achieved by Wigner [1], who pioneered the use of the adjoint Boltzmann neutron transport equation to compute first-order perturbations (and, hence, first-order sensitivities) in the reactor’s multiplication factor response arising from perturbations in the model parameters (nuclear cross sections). Wigner’s method for the linear Boltzmann equation requires a single large-scale solving of the adjoint Boltzmann neutron transport equation for obtaining all of the first-order sensitivities of the reactor’s multiplication factor to the reactor’s imprecisely known parameters (nuclear cross sections), regardless of the number of model parameters. However, Wigner’s procedure, which was developed within the specific context of the linear neutron transport equation, cannot be directly applied to nonlinear models since nonlinear operators do not admit adjoint operators. The rigorous 1st-order adjoint sensitivity analysis methodology for generic large-scale nonlinear (as opposed to linearized) systems involving generic nonlinear operator responses has been conceived and developed by Cacuci [2,3], who is also credited (see, e.g., [4,5]) for having introduced these principles to the atmospheric and other sciences. Cacuci’s first-order adjoint sensitivity analysis methodology for nonlinear systems [2,3] is just as efficient for computing 1st-order sensitivities as Wigner’s method is for linear systems, requiring a single large-scale (adjoint) computation for obtaining all the first-order sensitivities, regardless of the number of model parameters. In contradistinction, the number of large-scale computations that would be required by conventional methods (e.g., finite differences, statistical) to estimate approximately the 1st-order sensitivities would be at least “twice the number of model parameters.”

The computation of higher-order sensitivities by conventional methods is subject to the “curse of dimensionality,” a term coined by Belmann [6] to describe phenomena in which the number of computations increases exponentially in the respective phase-space. In the particular case of sensitivity analysis, the number of large-scale computations increases exponentially in the parameter phase-space as the order of sensitivities increases if conventional methods (e.g., finite-differences) are used. In particular, the computation of 2nd-order sensitivities using finite-differences would require at least (3TP)² computations; the computation of the nth-order sensitivities using finite-differences would requires at least [(n+1)TP]ⁿ large-scale computations. Of course, such high-order sensitivities cannot be computed by using statistical methods.

A methodological breakthrough that enabled the efficient computation of the exact values of 2nd-order sensitivities was achieved by Cacuci [7,8,9]. The unparalleled efficiency of this 2nd-order adjoint sensitivity analysis methodology [7,8,9] was demonstrated in [10,11,12,13,14,15] by applying this methodology to an OECD/NEA reactor physics benchmark [16], enabling the exact computation of the 21,976 first-order sensitivities and 482,944,576 second-order sensitivities of this benchmark’s leakage response with respect to the benchmark’s 21,976 parameters (including isotopic number densities, microscopic cross sections, fission spectrum parameters, neutron sources, and detector response functions). This benchmark is modeled by using the six-dimensional integro-differential neutron transport (Boltzmann) equation; solving numerically this equation is representative of a “large-scale computation” involving many parameters. The results obtained in [10,11,12,13,14,15] indicated that many 2nd-order sensitivities were much larger than the largest 1st-order ones, which implies that the consideration of only the first-order sensitivities is insufficient for making credible predictions regarding the expected values and uncertainties (variances, covariances, skewness) of calculated and predicted/adjusted responses.

The findings reported in [10,11,12,13,14,15] motivated the computation [17,18,19,20] of the largest-valued 3rd- and 4th-order sensitivities for the OECD/NEA reactor physics benchmark [16], many of which were found to be even larger than the 2nd-order ones. These findings [17,18,19,20] have motivated the conception by Cacuci [21] of the “nth-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Response-Coupled Forward/Adjoint Linear Systems” (abbreviated as “nth-CASAM-L”). The nth-CASAM-L enables the efficient computation of exactly-determined expressions of arbitrarily high-order sensitivities of a generic system response—which can simultaneously depend on both the forward and adjoint state functions—with respect to all of the parameters that characterize the physical system. The qualifier “comprehensive” is employed in order to highlight that the model parameters considered within the framework of the nth-CASAM-L includes the system’s uncertain boundaries and internal interfaces in phase-space. The “nth-CASAM-L” mathematical framework was developed specifically for linear systems because the most important model responses produced by such systems are various Lagrangian functionals which depend simultaneously on both the forward and adjoint state functions governing the respective linear system (see, e.g., [22]). Included among such functionals are the Raleigh quotient for computing eigenvalues and/or separation constants (when solving linear partial differential equations), and the Schwinger [23] and Rousopoulos [24] functionals, which play fundamental roles in optimization and control procedures, derivation of numerical methods for solving equations (differential, integral, integro-differential), and sensitivity analysis. Such responses, which simultaneously depend on the model’s forward and adjoint state functions, cannot occur for nonlinear models since nonlinear operators do not admit adjoint operators.

In parallel with the aforementioned developments, Cacuci [25] has extended his original work [2,3] on nonlinear systems by conceiving the nth-Order Comprehensive Adjoint Sensitivity Analysis Methodology for Nonlinear Systems (nth-CASAM-N), which enables the most efficient computation of the exact expressions of arbitrarily-high-order sensitivities of responses of a nonlinear model with respect to the model’s imprecisely known parameters, boundaries and/or internal interfaces. The nth-CASAM-L can be treated as a particular case on the nth-CASAM-N in the case when the response of interest of a linear model depends only on the forward or adjoint state function, but not simultaneously on both.

The nth-CASAM-L and nth-CASAM-N methodologies share the same fundamental principles, both being formulated in linearly increasing higher-dimensional Hilbert spaces, as opposed to exponentially increasing parameter-dimensional spaces. Therefore, both the nth-CASAM-L and the nth-CASAM-N methodologies share the following common features:

(i)

For a scalar-valued response associated with a model comprising $TP$ model parameters, the 1st-CASAM-N/L methodologies (i.e., the nth-CASAM for n = 1) requires 1 adjoint computation for computing exactly all of the 1st-order response sensitivities; this is in contradistinction to all other methods, which would require at least $TP$ forward computations, i.e., as least as many computations as there are model parameters.

(ii)

For every lower-order sensitivity of interest, the nth-CASAM-N/L methodologies obtain the exact expressions of the $TP$ “next-higher-order” sensitivities using a single adjoint computation performed in a linearly increasing higher-dimensional Hilbert space. In this sense, the nth-CASAM-N/L methodologies overcome the curse of dimensionality [6] in sensitivity analysis because if conventional methods were used, the number of large-scale computations needed to compute the nth-order sensitivities would increase exponentially with the sensitivities’ order (n).

(iii)

The nth-CASAM-N/L methodologies are applicable to any model (deterministic, statistical, etc.).

(iv)

For linear models, the higher-level adjoint functions are computed using the same forward and adjoint solvers (i.e., computer codes) as used for solving the original forward and adjoint systems. For nonlinear models, the solver used for solving the 1st-level adjoint system is also used for computing the higher-level adjoint functions. For either linear or nonlinear models, the computation of the various-order sensitivities require relatively minor additional software development.

(v)

The larger the number of model parameters, the more efficient the nth-CASAM-N/L methodologies become for computing arbitrarily high-order response sensitivities.

Of course, computing sensitivities by using finite-difference formulas in conjunction with re-computations using altered parameter values could be a “solution of first or last resort” for selected sensitivities. However, using finite-differences becomes not only computationally unfeasible for obtaining higher-order sensitivities (because of the “curse of dimensionality”), but finding the optimal step-size to minimize the error between the finite-difference result and the exact result is practically impossible to achieve unless one knew before-hand what the exact result is (which would be possible only by using the nth-CASAM-N/L methodologies). As has been demonstrated in [26,27], a step-size that is “too small” produces large errors, while a step-size which is “too large” produces even larger errors or non-convergence. However, unless one knows ahead of time the exact result, it is not possible even to know that the finite-difference formula has produced a correct or an erroneous result.

The information provided by the 1st-order sensitivities usually indicates which 2nd-order sensitivities are important and which could be neglected. Therefore, it is useful to prioritize the computation of the 2nd-order sensitivities by using the rankings of the relative magnitudes of the 1st-order sensitivities as a “priority indicator”: the larger the magnitude of the relative 1st-order sensitivity, the higher the priority for computing the corresponding 2nd-order sensitivities. In addition, since vanishing 1st-order sensitivities may indicate critical points of the response in the phase-space of model parameters, it is also of interest to compute the 2nd-order sensitivities that correspond to vanishing 1st-order sensitivities. In practice, only those 2nd-order partial sensitivities which are deemed important would need to be computed. Each successively higher-order sensitivities are computed using the same principles as used for computing the 2nd-order sensitivities, i.e., the computation of the 3rd-order sensitivities can be prioritized by using the magnitudes/importances of the 1st-order and 2nd-order sensitivities as guiding indicators, and so on. In many instances, models may have same-order relative sensitivities which have the same numerical values (e.g., a large number of 1st-order relative sensitivities which all have values of 1.0), as illustrated by the paradigm models analyzed in [28]. The precise and efficient computation of relative sensitivities that have identical values does not pose any difficulties to the nth-CASAM-N/L methodologies, which can compute exactly all sensitivities. This situation is in contradistinction to conventional statistical methods, which generally yield differing “sensitivity indices” if parameters having identical relative sensitivities would be characterized by numerically distinct relative standard deviations (since statistical methods cannot compute the actual parameter sensitivities, but only amalgamations of parameter sensitivities and standard deviations).

The roles played by high-order sensitivities in formulas that propagate the uncertainties in model parameters to produce uncertainties in the model results have been originally illustrated by Tukey [29], who presented “error propagation formulas” which included response sensitivities (with respect to model parameters) up to 4th-order. Generalizations of Tukey’s formulas [29], up to and including the 6th-order standard deviations, are presented in [28]. These formulas [28,29] indicate that the first-order sensitivities contribute the leading terms to the second-, third-, and fourth-order moments of the response distribution, thus providing the leading contributions to the responses’ variance/covariances, skewness, and kurtosis. Obtaining the exact and complete set of first-order sensitivities of responses to model parameters is evidently of paramount importance for any analysis of a computational model. Since the 1st-order sensitivities do not contribute to the response’s expected value, it is also paramount to compute the second-order sensitivities, which contribute the leading correction terms to the response’s expected value, thus causing it to differ from the response’s computed value. If the parameters follow a symmetric (e.g., Gaussian) multivariate distribution, the contributions to the response’s third-order moment which stem from the 1st-order sensitivities vanish, and the second-order sensitivities contribute the leading terms. Therefore, neglecting the second-order response sensitivities to symmetrically distributed parameters would erroneously nullify the third-order response correlations. Evidently, at least 2nd-order sensitivities must be used in order to estimate the expectation values and the third-order moment (and hence the skewness) of the response distribution. Skewness indicates the direction and relative magnitude of a distribution’s deviation from the normal distribution while kurtosis indicates the propensity of the predicted response distribution to have heavy tails and/or outliers. With pronounced skewness, standard statistical inference procedures such as constructing a confidence interval for the mean (expectation) of a computed/predicted model response will be not only incorrect, in the sense that the true coverage level will differ from the nominal (e.g., 95%) level, but the error probabilities will be unequal on each side of the predicted mean. The fourth-order moment of the response distribution yields the response’s kurtosis, which indicates the propensity of the predicted response distribution to have heavy tails and/or outliers. As a matter of course, therefore, at least the second-order response sensitivities should be computed and, if any of the 2nd-order sensitivities turn out to be important, the corresponding 3rd-order sensitivities should also be computed and investigated appropriately; and so on.

Higher-order sensitivities have also been incorporated into a predictive modeling formalism [17], which combines experimental and computational information in the joint phase-space of responses and model parameters, including not only the 1st-order response sensitivities, but also the complete hessian matrix of 2nd-order second-sensitivities and also the 3rd-order sensitivities. The predictive modeling methodology presented in [17] thus generalizes to third-order the “BERRU Predictive Modeling: Best Estimate Results with Reduced Uncertainties” methodology presented in [30]. Both of these predictive modeling methodologies use the maximum entropy principle to eliminate the need for introducing and “minimizing” a user-chosen “cost functional quantifying the discrepancies between measurements and computations.” These predictive modeling methodologies [17,30] also incorporate correlations among the imprecisely known model parameters and computed model responses, thereby considerably generalizing the so-called “4DVAR” data assimilation methodologies (see, e.g., [31]), which do not include correlations between model responses and model parameters and/or 3rd-order sensitivities. These predictive modeling methodologies [17,30] can be formulated in either the phase-space of model parameters or the phase-space of model responses, thus enabling the inversion of the smallest-possible matrix (either in the parameters’ or the responses’ phase-space, whichever is the smallest), while also providing a quantitative metric, constructed from sensitivities and covariance matrices, for determining the degree of agreement among the various computational and experimental data while identifying and subsequently eliminating discrepant information.

Although this article has used illustrative examples from the field of nuclear energy systems, the nth-CASAM-N/L methodologies are generally applicable to linear and/or nonlinear models in any field, enabling the practical computation of arbitrarily-high-order sensitivities and subsequent decisions regarding their importance to the respective model response. Such comparisons enable investigations of the convergence of the (multivariate) Taylor series expansion of the response in terms of parameter variations, as well as investigations of the actual validity of expressions (e.g., response variances/covariance, skewness, kurtosis, etc.) that are derived from the Taylor-expansion of the response as a function of the model’s parameters. The larger the number of model parameters, the more efficient the nth-CASAM-N/L methodologies become for computing arbitrarily high-order response sensitivities.

The application of the nth-CASAM-N/L methodologies is expected to revolutionize all of the fields of activities which require response sensitivities, including the fields of uncertainty quantification, model validation, optimization, data assimilation, model calibration, sensor fusion, reduced-order modeling, inverse problems, and predictive modeling. Ongoing work is aimed at addressing open issues such developing a “high-order predictive modelling methodology” by incorporating the high-order sensitivities now made available by applying the nth-CASAM-N/L methodologies into the concepts underlying the “BERRU-predictive modelling” methodology [17,30]. Further advancements are expected by incorporating artificial intelligence (AI) concepts, which could potentially lead to a ground-breaking methodology aimed at “Intelligent High-Order Forward and Inverse Predictive Modelling” in order to achieve the following goals: (i) quantify the impact of ignorance, such as missing phenomena in the computational model and/or experimental information; (ii) reduce uncertainties in predicted model responses/results; (iii) improve the fidelity of computational models, by reducing uncertainties in imprecisely known model parameters and incorporating missing phenomena, thus reducing the impact of prior ignorance.