Demonstrative Application to an OECD/NEA Reactor Physics Benchmark of the 2nd-BERRU-PM Method—II: Nominal Computations Apparently Inconsistent with Measurements

Abstract

This work presents illustrative applications of the 2nd-BERRU-PM (second-order best-estimate results with reduced uncertainties predictive modeling) methodology to the leakage response of a polyethylene-reflected plutonium OECD/NEA reactor physics benchmark, which is modeled using the neutron transport Boltzmann equation. The 2nd-BERRU-PM methodology simultaneously calibrates responses and parameters while simultaneously reducing the predicted standard deviation values of these quantities. The situations analyzed in this work pertain to the values of measured responses that appear to be inconsistent with the computed response values, in that the standard deviation values of the measured responses do not initially overlap with the standard deviation values of the computed responses. It is shown that the inconsistency diminishes as higher-order sensitivities are progressively included, thus illustrating their significant impact. In all cases, the 2nd-BERRU-PM methodology yields predicted best-estimate standard deviation values that are smaller than both the computed and the experimentally measured values of the standard deviation for the model response under consideration.

1. Introduction

This work continues the presentations commenced in [1] of illustrative results obtained by applying the 2nd-BERRU-PM methodology [2,3] to a polyethylene-reflected plutonium (PERP) OECD/NEA reactor physics benchmark [4]. The response of interest of this benchmark is the leakage of neutrons through the benchmark’s outer surface. As has been described in [1], this benchmark is modeled using the neutron transport Boltzmann equation, involving 21,976 imprecisely known parameters, the solution of which is representative of “large-scale computations”. It has been shown that the largest sensitivities are the benchmark’s response sensitivities to the 180 microscopic total cross-sections [5]. The 2nd-BERRU-PM methodology (second-order best-estimate results with reduced uncertainties predictive modeling) [2,3] was applied in [1] to reduce the predicted standard deviation values of the best-estimate predicted leakage response in situations where the standard deviation of the measured response overlapped with the standard deviation of the computed response, thus indicating apparent “consistency” between measurements and computations. Continuing the work presented in [1], Section 2 of this work describes the formulas of the 2nd-BERRU-PM methodology [2,3], which are applied in Section 3 to situations where the standard deviation of the measured response does not initially overlap with the standard deviation of the computed response, thereby indicating apparent “inconsistency” between measurements and computations. The discussion presented in Section 4 offers conclusions regarding the uncertainty-reducing abilities of the 2nd-BERRU-PM methodology and indicates ongoing extensions of this methodology designed to overcome its current limitations.

2. Method: Second-Order Predictive Modeling Methodology (2nd-BERRU-PM)

As presented in [1], the 2nd-BERRU-PM methodology yields the following expression for the best-estimate (optimal) mean values, $r_{be}$, of the predicted responses:

$$r_{be}=r_e+C_e^{rr}{\left(C_e^{rr}+C_c^{rr}\right)}^{-1}\left[E_c\left(r\right)-r_e\right]$$

(1)

and it also yields the following expression for the best-estimate (optimal) covariance values, $C_{be}^{rr}$, of the best-estimate predicted responses:

$$C_{be}^{rr}=C_e^{rr}-C_e^{rr}{\left(C_e^{rr}+C_c^{rr}\right)}^{-1}{C}_e^{rr}.$$

(2)

The quantities appearing in Equations (1) and (2) are defined as follows:

(i)

Matrices are denoted using capital bold letters, while vectors are denoted using either capital or lower-case bold letters.

(ii)

The components $r_k^{be},k=1,\dots ,TR$ of the vector $r_{be}\triangleq {\left(r_1^{be},\dots ,r_{TR}^{be}\right)}^{†}$ denote the best-estimate values of the responses. The quantity “TR” denotes the “total number of responses” under consideration. The symbol “$\triangleq$” is used to denote “is defined as” or “is by definition equal to”. Transposition is indicated by a dagger $\left(†\right)$ superscript.

(iii)

The components $r_k^e,k=1,\dots ,TR$ of the vector $r_e\triangleq {\left(r_1^e,\dots ,r_{TR}^e\right)}^{†}$ denote the expected values of the experimentally measured responses. The letter “e” is used either as a superscript or a subscript to indicate experimentally measured quantities.

(iv)

The quantity $C_e^{rr}\triangleq {\left[\mathrm{cov}\left(r_i^e,r_j^e\right)\right]}_{TR\times TR}$ denotes the covariance matrix of experimentally measured responses. The component covariance values stem from measurements.

(v)

The quantity $E_c\left(r\right)\triangleq {\left[E_c\left(r_1\right),\dots ,E_c\left(r_{TR}\right)\right]}^{†}$ denotes the “vector of expected values of computed responses”, and its components, $E_c\left(r_k\right)$, are obtained by determining the expectation value of the multivariate Taylor expansion of the computed response $r_k\left(\alpha \right)$ with respect to the benchmark model’s uncertain parameters, evaluated around the parameters’ mean values. The complete expression of the expected value $E_c\left(r_k\right)$ of a generic response $r_k\left(\alpha \right)$ is presented in [1]. The particular response considered in this work is the neutron leakage out of the benchmark’s outer surface, and the impact of the standard deviation of the benchmark’s total cross-sections is illustrated by considering that these cross-sections are uncorrelated and normally distributed. Under these circumstances, the mean value, $E(L^c)$, of the computed leakage response takes the following form (up to fourth-order sensitivities):

$$E(L^c)=L^c\left({\alpha }^0\right)+\frac{1}{2}{\displaystyle \sum _{j_1=1}^{TP}\frac{{\partial }^{2}L\left({\alpha }^0\right)}{{\left(\partial {\alpha }_{j_1}\right)}^2}{\sigma }_{j_1}^2+\frac{1}{8}{\displaystyle \sum _{j_1=1}^{TP}\frac{{\partial }^{4}L\left({\alpha }^0\right)}{{\left(\partial {\alpha }_{j_1}\right)}^4}{\sigma }_{j_1}^4}},$$

(3)

where $L^c\left({\alpha }^0\right)=1.7648\times {10}^6$ neutrons/sec denotes the value of the computed leakage response at the nominal parameter values and where $TP=180$ denotes the “total number” of microscopic total cross-sections comprised in the benchmark’s computational model. As has been discussed in [1,5], the largest relative sensitivities of the leakage response are with respect to the benchmark’s 180 microscopic total cross-sections, and the contributions stemming from the benchmark’s remaining parameters are negligible, their combined contribution to the expectation value and variance of the computed leakage response being of the order of 5%.

(vi)

The quantity $C_c^{rr}\triangleq {\left[\mathrm{cov}\left(r_j,r_k\right)\right]}_{TR\times TR}$ in Equation (1) denotes the covariance matrix for the computed responses. The complete expression of the covariance $\mathrm{cov}\left(r_j,r_k\right)$ between two computed responses $r_j\left(\alpha \right)$ and $r_k\left(\alpha \right)$ is provided in [1]. In this section, only the neutron leakage response will be analyzed, and the 180 microscopic total cross-sections, which will be considered the benchmark’s imprecisely known parameters, are assumed to be uncorrelated and normally distributed. Under these circumstances, the matrix $C_c^{rr}\triangleq {\left[\mathrm{cov}\left(r_j,r_k\right)\right]}_{TR\times TR}$ reduces to the (scalar) variance, $\mathrm{var}(L^c)$, of the computed leakage response, $L^c$, having the following simplified form of the corresponding expression provided in [1]:

$$\begin{array}{ll}\mathrm{var}(L^c)& ={\displaystyle \sum _{j_1=1}^{TP}{\left[\frac{\partial L\left({\alpha }^0\right)}{\partial {\alpha }_{j_1}}\right]}^2{\sigma }_{j_1}^2}+\frac{1}{2}{\displaystyle \sum _{j_1=1}^{TP}{\left[\frac{{\partial }^{2}L\left({\alpha }^0\right)}{{\left(\partial {\alpha }_{j_1}\right)}^2}\right]}^2}{\sigma }_{j_1}^4+{\displaystyle \sum _{j_1=1}^{TP}\left[\frac{{\partial }^{3}L\left({\alpha }^0\right)}{{\left(\partial {\alpha }_{j_1}\right)}^3}\times \frac{\partial L\left({\alpha }^0\right)}{\partial {\alpha }_{j_1}}\right]{\sigma }_{j_1}^4}\\ & +\frac{15}{36}{\displaystyle \sum _{j_1=1}^{TP}{\left[\frac{{\partial }^{3}L\left({\alpha }^0\right)}{{\left(\partial {\alpha }_{j_1}\right)}^3}\right]}^2{\sigma }_{j_1}^6}+\frac{1}{2}{\displaystyle \sum _{j_1=1}^{TP}\left[\frac{{\partial }^{4}L\left({\alpha }^0\right)}{{\left(\partial {\alpha }_{j_1}\right)}^4}\times \frac{{\partial }^{2}L\left({\alpha }^0\right)}{{\left(\partial {\alpha }_{j_1}\right)}^2}\right]{\sigma }_{j_1}^6}.\end{array}$$

(4)

3. Results

As has been mentioned above, the response of interest for the PERP benchmark [4] is the total leakage of neutrons (i.e., neutrons exiting the outer surface of the benchmark). The computed nominal value of the total leakage response for this benchmark is $L^c=1.7648\times {10}^6$ neutrons/sec. The superscript “c” will be used to denote “computed quantities”. The situations to be analyzed in this section will consider a measured response with a nominal value of $L^e=3.0\times {10}^6$ neutrons/sec, with a relative standard deviation with a ”moderate” value of $S{D}^{(e)}=5\%$. The impact of the precision of the uncertain parameters will be quantified by considering relative standard deviation values (for the parameters) of 10% (low precision), 5% (medium precision), and 2% (high precision), respectively.

3.1. Measured Response $L^e=3.0\times {10}^6$ n/sec; Relative $S{D}^{(e)}=5\%$; Parameters’ Relative SD = 5%

The results presented in this subsection all consider that the experimentally measured value of the leakage response is $L^e=3.0\times {10}^6$ neutrons/sec, with a relative standard deviation of $S{D}^{(e)}=5\%$, which is a usual (medium) value for this type of measurement. The nominal value of the computed leakage response is $L^c\left({\alpha }^0\right)=1.7648\times {10}^6$ neutrons/sec. The model parameters are the microscopic total cross-sections, which are considered here to be uncorrelated and normally distributed, having a relative standard deviation of 5%. As has been discussed in [1], relative standard deviation values of 5% for the model parameters are “borderline” values regarding the convergence (or non-convergence) of the Taylor series expansion of the leakage response in terms of the model parameters (total cross-sections). The numerical values for the computed quantities $E(L^c)\pm S{D}^{\left(i\right)}$ (i.e., computed response mean value plus/minus computed standard deviation) and $L^{be}\pm S{D}^{\left(be,i\right)}$ (i.e., best-estimate predicted response value plus/minus predicted standard deviation) for $i=1,\dots ,4$ are presented in Table 1. For $i=1$, only the contributions stemming from the first-order sensitivities are included; for $i=2$, the contributions stemming from the first- and second-order sensitivities are included; for $i=3$, the contributions stemming from the first-, second-, and third-order sensitivities are included; and for $i=4$, all contributions stemming from the first- to fourth-order sensitivities are included. The numerical results presented in Table 1 are depicted in Figure 1 below.

Table 1. Numerical values of $E(L^c)\pm S{D}^{\left(i\right)}$ and $L^{be}\pm S{D}^{\left(be,i\right)}$ ($i=1,\dots ,4$) for measured response $L^e=3.0\times {10}^6$ neutrons/sec with $S{D}^{(e)}=5\%$. Parameters’ relative standard deviation = 5%.

Order of Sensitivities Considered	$\mathbf{Value}\mathbf{of}{\mathit{L}}^{\mathit{e}} \pm \mathit{S}{\mathit{D}}^{(\mathit{e})}$ (Neutrons/sec)	$\mathbf{Value}\mathbf{of}\mathit{E}({\mathit{L}}^{\mathit{c}}) \pm \mathit{S}{\mathit{D}}^{\left(\mathit{i}\right)}$ (Neutrons/sec)	$\mathbf{Value}\mathbf{of}{\mathit{L}}^{\mathit{b}\mathit{e}} \pm \mathit{S}{\mathit{D}}^{\left(\mathit{b}\mathit{e},\mathit{i}\right)}$ (Neutrons/sec)
1st-order ($i=1$)	3.000 × 106 ± 1.500 × 105	1.765 × 106 ± 9.246 × 105	2.968 × 106 ± 1.481 × 105
1st- + 2nd-order ($i=2$)		2.914 × 106 ± 1.629 × 106	2.969 × 106 ± 1.494 × 105
1st- + 2nd- + 3rd-order ($i=3$)		2.914 × 106 ± 5.103 × 106	2.999 × 106 ± 1.499 × 105
1st- + 2nd- + 3rd- + 4th-order ($i=4$)		6.681 × 106 ± 7.386 × 106	3.002 × 106 ± 1.499 × 105

Figure 1. Comparison of $E(L^c)\pm S{D}^{\left(i\right)}$ (in green), $L^{be}\pm S{D}^{\left(be,i\right)}$ (in red), and $L^e\pm S{D}^{(e)}$ (in blue), for measured response $L^e=3.0\times {10}^6$ neutrons/sec with $S{D}^{(e)}=5\%$. Parameters’ relative standard deviation = 5%. (a) Only the 1st-order ($i=1$) sensitivities are considered. (b) The 1st- and 2nd-order ($i=2$) sensitivities are included. (c) The 1st-, 2nd-, and 3rd-order ($i=3$) sensitivities are included. (d) The 1st-, 2nd-, + 3rd-, and + 4th-order ($i=4$) sensitivities are included.

The following conclusions follow from the results presented in Table 1 and Figure 1:

(i)

When considering only the first-order sensitivities, the standard deviation $S{D}^{(e)}$ of the measured response does not overlap with the standard deviation of the computed response. Thus, when considering just the first-order sensitivities, the measured response value appears to be “inconsistent” (“discrepant”) with the computed nominal value of the response. Nevertheless, the best-estimate predicted response value, $L^{be}$, is very close to the measured value, with a standard deviation that is also close to (and smaller than) the measured value.

(ii)

As expected from Equations (3) and (4), the values of both $E(L^c)$ and $S{D}^{\left(i\right)}$ increase progressively as the contributions from the higher-order sensitivities are progressively taken into account for $i=1,\dots ,4$. When the second-order sensitivities are considered, the computational results appear to become consistent with the measured values, since the respective measured and computed standard deviation values overlap as soon as the second-order sensitivities are considered.

(iii)

The best-estimate predicted response value, $L^{be}$, remains close to the measured value, even though the expected response value continues to increase divergently as higher-order sensitivities are considered.

3.2. Measured Response $L^e=3.0\times {10}^6$ n/sec; Relative $S{D}^{(e)}=5\%$; Parameters’ Relative SD = 10%

The results presented in this subsection all consider that the experimentally measured value of the leakage response is $L^e=3.0\times {10}^6$ neutrons/sec, with a relative standard deviation of $S{D}^{(e)}=5\%$, which is a usual (medium) value for this type of measurement. The model parameters are the microscopic total cross-sections, which are considered to be uncorrelated and normally distributed and have a relative standard deviation of 10%. As has been discussed in [1], the Taylor series expansion of the model’s leakage response in terms of the total cross-section parameters diverges for the relative standard deviation of 10% of these parameters. Thus, the numerical values for $E(L^c)\pm S{D}^{\left(i\right)}$ and $L^{be}\pm S{D}^{\left(be,i\right)}$ for $i=1,\dots ,4$, which are presented in Table 2 and depicted in Figure 2, are computed based on a truncated non-convergent Taylor series expansion.

Table 2. Numerical values of $E(L^c)\pm S{D}^{\left(i\right)}$ and $L^{be}\pm S{D}^{\left(be,i\right)}$ ($i=1,\dots ,4$) for measured response $L^e=3.0\times {10}^6$ neutrons/sec with $S{D}^{(e)}=5\%$. Parameters’ relative standard deviation = 10%.

Order of Sensitivities Considered	$\mathbf{Value}\mathbf{of}{\mathit{L}}^{\mathit{e}} \pm \mathit{S}{\mathit{D}}^{\left(\mathit{e}\right)}$ (Neutrons/sec)	$\mathbf{Value}\mathbf{of}\mathit{E}({\mathit{L}}^{\mathit{c}}) \pm \mathit{S}{\mathit{D}}^{\left(\mathit{i}\right)}$ (Neutrons/sec)	$\mathbf{Value}\mathbf{of}{\mathit{L}}^{\mathit{b}\mathit{e}} \pm \mathit{S}{\mathit{D}}^{\left(\mathit{b}\mathit{e},\mathit{i}\right)}$ (Neutrons/sec)
1st-order ($i=1$)	3.000 × 106 ± 1.500 × 105	1.765 × 106 ± 1.849 × 106	2.992 × 106 ± 1.495 × 105
1st- + 2nd-order ($i=2$)		6.363 × 106 ± 5.675 × 106	3.002 × 106 ± 1.499 × 105
1st- + 2nd- + 3rd-order ($i=3$)		6.363 × 106 ± 3.561 × 107	3.001 × 106 ± 1.499 × 105
1st- + 2nd- + 3rd- + 4th-order ($i=4$)		6.662 × 107 ± 5.562 × 107	3.001 × 106 ± 1.499 × 105

Figure 2. Comparison of $E(L^c)\pm S{D}^{\left(i\right)}$ (in green), $L^{be}\pm S{D}^{\left(be,i\right)}$ (in red), and $L^e\pm S{D}^{(e)}$ (in blue), for measured response $L^e=3.0\times {10}^6$ neutrons/sec with $S{D}^{(e)}=5\%$. Parameters’ relative standard deviation = 10%. (a) Only the 1st-order ($i=1$) sensitivities are considered. (b) The 1st- and 2nd-order ($i=2$) sensitivities are included. (c) The 1st-, 2nd-, and 3rd-order ($i=3$) sensitivities are included. (d) The 1st-, 2nd-, 3rd-, and 4th-order ($i=4$) sensitivities are included.

The following conclusions follow from the results presented in Table 2 and Figure 2:

(i)

The computed results appear to be consistent with the measured results.

(ii)

The divergent nature of the computations for the values of $E(L^c)$ and $S{D}^{\left(i\right)}$ are apparent, as they increase significantly when the contributions from the higher-order sensitivities are progressively taken into account for $i=1,\dots ,4$. Nevertheless, the best-estimate predicted response value, $L^{be}$, remains close to the measured value, even if only the first-order sensitivities are considered.

3.3. Measured Response $L^e=3.0\times {10}^6$ n/sec; Relative $S{D}^{(e)}=5\%$; Parameters’ Relative SD = 2%

The results presented in this subsection all consider that the experimentally measured value of the leakage response is $L^e=3.0\times {10}^6$ neutrons/sec, with a relative standard deviation of $S{D}^{(e)}=5\%$, which is a usual (medium) value for this type of measurement. The model parameters are the microscopic total cross-sections, considered to be uncorrelated and normally distributed, and have a relative standard deviation of 2%. As has been discussed in [1], the Taylor series expansion of the computed leakage response in terms of the model’s total cross-section converges with a uniform relative standard deviation of 3% or less (for the model parameters). Thus, the numerical values of the quantities $E(L^c)\pm S{D}^{\left(i\right)}$ and $L^{be}\pm S{D}^{\left(be,i\right)}$ for $i=1,\dots ,4$, which are presented in Table 3 and depicted in Figure 3, are computed based on a convergent Taylor series, as is the case when the model parameters all have a relative standard deviation of 2%.

Table 3. Numerical values of $E(L^c)\pm S{D}^{\left(i\right)}$ and $L^{be}\pm S{D}^{\left(be,i\right)}$ ($i=1,\dots ,4$) for measured response $L^e=3.0\times {10}^6$ neutrons/sec with $S{D}^{(e)}=5\%$. Parameters’ relative standard deviation = 2%.

Order of Sensitivities Considered	$\mathbf{Value}\mathbf{of}{\mathit{L}}^{\mathit{e}} \pm \mathit{S}{\mathit{D}}^{\left(\mathit{e}\right)}$ (Neutrons/sec)	$\mathbf{Value}\mathbf{of}\mathit{E}({\mathit{L}}^{\mathit{c}}) \pm \mathit{S}{\mathit{D}}^{\left(\mathit{i}\right)}$ (Neutrons/sec)	$\mathbf{Value}\mathbf{of}{\mathit{L}}^{\mathit{b}\mathit{e}} \pm \mathit{S}{\mathit{D}}^{\left(\mathit{b}\mathit{e},\mathit{i}\right)}$ (Neutrons/sec)
1st-order ($i=1$)	3.000 × 106 ± 1.500 × 105	1.765 × 106 ± 3.698 × 105	2.826 × 106 ± 1.390 × 105
1st- + 2nd-order ($i=2$)		1.949 × 106 ± 4.276 × 105	2.885 × 106 ± 1.415 × 105
1st- + 2nd- + 3rd-order ($i=3$)		1.949 × 106 ± 6.288 × 105	2.943 × 106 ± 1.459 × 105
1st- + 2nd- + 3rd- + 4th-order ($i=4$)		2.045 × 106 ± 7.157 × 105	2.960 × 106 ± 1.468 × 105

Figure 3. Comparison of $E(L^c)\pm S{D}^{\left(i\right)}$ (in green), $L^{be}\pm S{D}^{\left(be,i\right)}$ (in red), and $L^e\pm S{D}^{(e)}$ (in blue), for measured response $L^e=3.0\times {10}^6$ neutrons/sec with $S{D}^{(e)}=5\%$. Parameters’ relative standard deviation = 2%. (a) Only the 1st-order ($i=1$) sensitivities are considered. (b) The 1st- + 2nd-order ($i=2$) sensitivities are included. (c) The 1st-, 2nd-, and 3rd-order ($i=3$) sensitivities are included. (d) The 1st-, 2nd-, 3rd-, and 4th-order ($i=4$) sensitivities are included.

The following conclusions follow from the results presented in Table 3 and Figure 3:

(i)

The computed results appear to be inconsistent with the measured results because the small standard deviation considered for the parameters yields small values for the standard deviation of the computed response. This fact is particularly evident when only the first-order sensitivities are considered, which is a situation similar to the situation depicted in Figure 1a.

(ii)

As higher-order sensitivities are progressively considered for $i=1,\dots ,4$, the agreement between the measured and the computed quantities is progressively improved, indicating a possible convergence if sensitivities of orders higher than four were to be included.

(iii)

In all cases, however, the predicted best-estimate values $L^{be}\pm S{D}^{\left(be,4\right)}$ remain consistent with the measured values. The best-estimate predicted response value, $L^{be}$, moves progressively closer to the measured value as progressively higher-order sensitivities are considered, while the best-estimate standard deviation $S{D}^{\left(be,i\right)}$ is, in all cases, close to, albeit smaller than, the experimentally measured standard deviation $S{D}^{\left(e\right)}$.

4. Discussion

This work considered situations in which the measurements and the computations of the leakage response may appear to be inconsistent with one another in that their respective standard deviation values do not initially overlap. However, this inconsistency diminishes as higher-order sensitivities are progressively included, since the standard deviation of the computed response increases. In all cases, the 2nd-BERRU-PM methodology yields a predicted mean value for the response that is closer to the measured response—which is more accurately known than the computed response—in the illustrative situations analyzed in this work. Furthermore, in all cases, the 2nd-BERRU-PM methodology yields predicted best-estimate standard deviation values that are smaller than both the computed and the experimentally measured values of the standard deviation for the leakage response. This reduction in the predicted best-estimate uncertainties (standard deviation values) was also evident in [1], wherein the illustrative situation analyzed involved “consistent” measurements and computations. This unique property of the 2nd-BERRU-PM methodology of reducing the predicted standard deviation values of the predicted results to values smaller than either the experimentally measured or the computed values of the respective standard deviation stems from the information-theoretical foundation of the 2nd-BERRU-PM methodology, which ensures that the incorporation of additional (consistent) information reduces the predicted uncertainties.

The illustrative examples presented in [1] and in this work were restricted to analyzing best-estimate predicted responses. However, as has been mentioned in [1], the 2nd-BERRU-PM methodology also yields best-estimate predicted values for the model’s parameters, as well as response–parameter correlations, all of which are essential for performing model validation and calibration. Ongoing work aims at presenting additional illustrative examples. Currently, the 2nd-BERRU-PM methodology is limited to producing first- and second-order statistics (i.e., predicted mean values and predicted covariances) for the posterior joint distribution of best-estimate responses and parameters. Further ongoing work aims at extending the 2nd-BERRU-PM to the fourth order, thereby enabling the computation of third-order correlations (skewness) and fourth-order correlations (kurtosis) of the posterior joint distribution of best-estimate predicted model responses and parameters.