# Introducing Robust Statistics in the Uncertainty Quantification of Nuclear Safeguards Measurements

## Abstract

**:**

## 1. Introduction

- Suitable breakdown point: they have the capacity to handle a consistent proportion of outliers in the data before returning an incorrect result;
- High efficiency: they provide estimates with desirable properties even when there are no outliers in the data;
- Closed expression: they do not depend on optimization algorithms and are rather easy to calculate.

## 2. Measurement Error Model and the Classical Estimation of Its Variance Components

- -
- the distribution of the ratio $\frac{{O}_{jk}-{I}_{jk}}{{O}_{jk}}$ is extremely close to a normal distribution;
- -
- it provides an accurate approximation of the target variable.

## 3. Robust Estimation of Unbalanced One-Way Random Effect Models

- (a)
- Anomalous values of the group error${\varsigma}_{j}$, also called outlying blocks;
- (b)
- Anomalous values of the individual measurement error${\rho}_{jk}$, also called outlying measurements within blocks.

- High asymptotic efficiency: defining the asymptotic efficiency as the ratio between the variances of asymptotic distributions of the robust estimators and those of the classical estimators, [16] proved that the asymptotic efficiency of ${\tilde{\sigma}}_{\tau}$ and ${\tilde{\sigma}}_{\rho ,2}$ in a balanced experiment is at least 82%, and can even exceed 90% depending on the values of n and ${\sigma}_{\varsigma}/{\sigma}_{\rho}$. The asymptotic efficiency of ${\tilde{\sigma}}_{\rho ,1}$, instead, starts at 37% for $n=2$ and increase up to 86% when n increases.
- Closed expression:${\tilde{\sigma}}_{\tau}$, ${\tilde{\sigma}}_{\rho ,1}$ and ${\tilde{\sigma}}_{\rho ,2}$ are the direct outcome of an explicit formula applied on a set of data. This means that they are not the result of iterative procedures and/or optimization algorithms that may be difficult to calculate and, more importantly, rather time consuming. The only complication of the proposed estimators that might lead to a sensible raise in the execution time of quartile calculation is the growth of the cardinality of ${D}_{+}^{2}$ and ${D}_{-}^{2}$ when N and g increase. However, for moderate values of N and g as the ones considered in this article, this does not seem to be an issue. Moreover, ${\tilde{\sigma}}_{\tau}$, ${\tilde{\sigma}}_{\rho ,1}$ and ${\tilde{\sigma}}_{\rho ,2}$ do not depend on the estimation of any location parameter.

## 4. Small Sample Comparison of Classical and Robust Estimates

- (i)
- the total number of observations $N={\sum}_{j=1}^{g}{n}_{j}$;
- (ii)
- the number of groups g, that in nuclear safeguards context corresponds to the number of calibration or inspection periods;
- (iii)
- balanced and unbalanced samples. The values of ${n}_{j}$ that describe the unbalanced cases are provided in Table 1;
- (iv)
- the ratio between the systematic variance component and the total variance: ${\psi}^{2}={\sigma}_{\varsigma}^{2}/{\sigma}_{\tau}^{2}$. Actually, looking at expression (4), the value of ${\psi}^{2}$ represents the correlation between observations in the same group. Fixing the value of the total variance to 1 (that is, ${\sigma}_{\tau}^{2}={\sigma}_{\rho}^{2}+{\sigma}_{\varsigma}^{2}=1$), we will consider three different cases:
- -
- ${\psi}^{2}=0.25$, that implies ${\sigma}_{\varsigma}=0.500$ and ${\sigma}_{\rho}=0.866$;
- -
- ${\psi}^{2}=0.5$, that implies ${\sigma}_{\rho}={\sigma}_{\varsigma}=0.707$;
- -
- ${\psi}^{2}=0.75$, that implies ${\sigma}_{\varsigma}=0.866$ and ${\sigma}_{\rho}=0.500$.

- (A)
- contamination of 10% of the random errors ${\rho}_{jk}$ in different groups replaced by $6{\sigma}_{\rho}$;
- (B)
- contamination of 10% of the short-term systematic errors ${\varsigma}_{j}$, replaced by $6{\sigma}_{\varsigma}$. For this scheme, we skip the case $g=3$, and when $g=6$, we contaminate one systematic error in each simulation, so slightly more than 10%;
- (C)
- 10% of the random errors ${\rho}_{jk}$ in different groups replaced by $6{\sigma}_{\rho}$ and at least 10% of the short-term systematic errors ${\varsigma}_{j}$, whose random errors are not contaminated, replaced by $6{\sigma}_{\varsigma}$. Whenever possible, the two contamination schemes do not involve the same group. As before, we skip the case $g=3$, and when $g=6$ we contaminate one systematic error in each simulation, so slightly more than 10%;

N | g | ${\mathit{n}}_{\mathit{j}}$ |
---|---|---|

30 | 3 | 4,10,16 |

30 | 6 | 3,3,5,5,7,7 |

60 | 6 | 4,4,8,12,16,16 |

60 | 10 | 4,4,5,5,6,6,7,7,8,8 |

100 | 10 | 4,4,7,7,10,10,13,13,16,16 |

100 | 20 | 3,3,3,3,4,4,4,4,5,5,5,5,6,6,6,6,7,7,7,7 |

200 | 20 | 4,4,4,4,7,7,7,7,10,10,10,10,13,13,13,13,16,16,16,16 |

200 | 40 | 3,3,3,3,3,3,3,3,4,4,4,4,4,4,4,4,5,5,5,5,5,5,5,5,6,6,6,6,6,6,6,6,7,7,7,7,7,7,7,7 |

#### 4.1. Simulation Results—No Contamination

#### 4.2. Simulation Results—Contaminated Samples

## 5. Empirical Application

- classical estimates for ${\sigma}_{\tau}$ are always larger than robust ones;
- differently from classical one, robust estimation does not detect any relevant short-term systematic error component.

## 6. Conclusions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## Abbreviations

UQ | Uncertainty quantification |

ANOVA | Analysis-Of-Variance |

MAB | Mean Absolute Bias |

## Appendix A. MATLAB Codes and Implementation Details

- 1.
`MonteCarlo.m`- -
- INPUT: (i)
`Ng`: a ($g\times 1$) vector of integers representing the number of observation in each group; (ii)`sigmaRand`: the value of ${\sigma}_{\rho}$; (iii)`sigmaSyst`: the value of ${\sigma}_{\rho}$; (iv)`contamination`: the kind of contamination (A, B, C or empty for no contamination); (v)`nsim`: the number of simulation to generate. - -
- OUTPUT: (i)
`outClassical`: (nsim $\times 3$) matrix containing the values of $\left[{\widehat{\sigma}}_{\tau}\phantom{\rule{0.277778em}{0ex}}{\widehat{\sigma}}_{\rho}\phantom{\rule{0.277778em}{0ex}}{\widehat{\sigma}}_{\varsigma}\right]$ obtained in each simulation; (ii)`outRobust1`: (nsim $\times 3$) matrix containing the values of $\left[{\tilde{\sigma}}_{\tau}\phantom{\rule{0.277778em}{0ex}}{\tilde{\sigma}}_{\rho ,1}\phantom{\rule{0.277778em}{0ex}}{\tilde{\sigma}}_{\varsigma ,1}\right]$ obtained in each simulation; (iii)`outRobust2`: (nsim $\times 3$) matrix containing the values of $\left[{\tilde{\sigma}}_{\tau}\phantom{\rule{0.277778em}{0ex}}{\tilde{\sigma}}_{\rho ,2}\phantom{\rule{0.277778em}{0ex}}{\tilde{\sigma}}_{\varsigma ,2}\right]$ obtained in each simulation.

- 2.
`Gestimator.m`- -
- INPUT: (i)
`y`: $(N\times 1)$ vector of observations; (ii)`g`: $(N\times 1)$ vector of integers representing that group of each observation. - -
- OUTPUT: (i)
`sigmaTot1`: the value obtained for ${\tilde{\sigma}}_{\tau}$; (ii)`sigmaRand1`: the value obtained for ${\tilde{\sigma}}_{\rho ,1}$; (iii)`sigmaSyst1`: the value obtained for ${\tilde{\sigma}}_{\varsigma ,1}$; (iv)`sigmaRand2`: the value obtained for ${\tilde{\sigma}}_{\rho ,2}$; (v)`sigmaSyst2`: the value obtained for ${\tilde{\sigma}}_{\varsigma ,2}$;

## References

- Walsh, S.; Burr, T.; Martin, K. Discussion of the IAEA error approach to producing variance estimates for use in material balance evaluation and the international target values, and comparison to metrological definitions of precision. J. Nucl. Mater. Manag.
**2017**, 45, 4–14. [Google Scholar] - Bonner, E.; Burr, T.; Guzzardo, T.; Krieger, T.; Norman, C.; Zhao, K.; Beddingfield, D.; Geist, W.; Laughter, M.; Lee, T. Ensuring the effectiveness of safeguards through comprehensive uncertainty quantification. J. Nucl. Mater. Manag.
**2016**, 44, 53–61. [Google Scholar] - Burr, T.L.; Knepper, P.L. A study of the effect of measurement error in predictor variables in nondestructive assay. Appl. Radiat. Isot.
**2000**, 53, 547–555. [Google Scholar] [CrossRef] - Scheffe, H. The Analysis of Variance; John Wiley & Sons: Hoboken, NJ, USA, 1999; Volume 72. [Google Scholar]
- Grubbs, F.E. On estimating precision of measuring instruments and product variability. J. Am. Stat. Assoc.
**1948**, 43, 243–264. [Google Scholar] [CrossRef] - Burr, T.; Croft, S.; Krieger, T.; Martin, K.; Norman, C.; Walsh, S. Uncertainty quantification for radiation measurements: Bottom-up error variance estimation using calibration information. Appl. Radiat. Isot.
**2016**, 108, 49–57. [Google Scholar] [CrossRef] - Burr, T.; Krieger, T.; Norman, C.; Zhao, K. The impact of metrology study sample size on uncertainty in IAEA safeguards calculations. EPJ Nucl. Sci. Technol.
**2016**, 2, 36. [Google Scholar] [CrossRef] - Burr, T.; Croft, S.; Dale, D.; Favalli, A.; Weaver, B.; Williams, B. Emerging applications of bottom-up uncertainty quantification in nondestructive assay. ESARDA Bull.
**2015**, 53, 54–61. [Google Scholar] - Burr, T.; Croft, S.; Jarman, K.; Nicholson, A.; Norman, C.; Walsh, S. Improved uncertainty quantification in nondestructive assay for nonproliferation. Chemom. Intell. Lab. Syst.
**2016**, 159, 164–173. [Google Scholar] [CrossRef] - Burr, T.; Martin, K.; Norman, C.; Zhao, K. Analysis of variance for item differences in verification data with unknown groups. Sci. Technol. Nucl. Install.
**2019**, 2019, 1769149. [Google Scholar] [CrossRef] - Zhou, J.; Zhu, H. Robust estimation and design procedures for the random effects model. Can. J. Stat.
**2003**, 31, 99–110. [Google Scholar] [CrossRef] - Burr, T.; Bonner, E.; Krzysztoszek, K.; Norman, C. Setting Alarm Thresholds in Measurements with Systematic and Random Errors. Stats
**2019**, 2, 259–271. [Google Scholar] [CrossRef] - Uhlig, S. Robust estimation of variance components with high breakdown point in the 1-way random effect model. In Industrial Statistics; Springer: Berlin/Heidelberg, Germany, 1997; pp. 65–73. [Google Scholar]
- Rousseeuw, P.J.; Croux, C. Alternatives to the median absolute deviation. J. Am. Stat. Assoc.
**1993**, 88, 1273–1283. [Google Scholar] [CrossRef] - Krishnamoorthy, K.; Mathew, T. One-sided tolerance limits in balanced and unbalanced one-way random models based on generalized confidence intervals. Technometrics
**2004**, 46, 44–52. [Google Scholar] [CrossRef] - Müller, C.H.; Uhlig, S. Estimation of variance components with high breakdown point and high efficiency. Biometrika
**2001**, 88, 353–366. [Google Scholar] [CrossRef] - Wellmann, J.; Gather, U. Identification of outliers in a one-way random effects model. Stat. Pap.
**2003**, 44, 335–348. [Google Scholar] [CrossRef] - Rocke, D.M. Robust statistical analysis of interlaboratory studies. Biometrika
**1983**, 70, 421–431. [Google Scholar] [CrossRef] - Fellner, W.H. Robust estimation of variance components. Technometrics
**1986**, 28, 51–60. [Google Scholar] [CrossRef] - Richardson, A.M.; Welsh, A.H. Robust restricted maximum likelihood in mixed linear models. Biometrics
**1995**, 51, 1429–1439. [Google Scholar] [CrossRef] - Huggins, R. A robust approach to the analysis of repeated measures. Biometrics
**1993**, 49, 715–720. [Google Scholar] [CrossRef] - Wellmann, J. Robustness of an S-Estimator in the One-Way Random Effects Model. Biom. J. J. Math. Methods Biosci.
**2000**, 42, 215–221. [Google Scholar] [CrossRef] - Gervini, D.; Yohai, V.J. Robust estimation of variance components. Can. J. Stat.
**1998**, 26, 419–430. [Google Scholar] [CrossRef] - Bickel, P.J.; Lehmann, E.L. Descriptive statistics for nonparametric models. III. Dispersion. In Selected Works of EL Lehmann; Springer: Berlin/Heidelberg, Germany, 2012; pp. 499–518. [Google Scholar]

**Figure 1.**Effect of the introduction of one single outlier (highlighted in red) on the simulated case study of [12] (dashed lines represent the average of the 10 observations in each group). (

**a**) Clean data; (

**b**) One outlier.

**Figure 2.**$100\times $MAB of the variance components estimates obtained in the 100,000 simulations

**without contamination**

^{*}. (

**a**) $100\times $MAB of ${\widehat{\sigma}}_{\tau}$ (classical estimator, solid line) and ${\tilde{\sigma}}_{\tau}$ (robust estimator, dashed line); (

**b**) $100\times $MAB of ${\widehat{\sigma}}_{\rho}$ (classical estimator, solid line), ${\tilde{\sigma}}_{\rho ,1}$ (robust estimator, dashed line) and ${\tilde{\sigma}}_{\rho ,2}$ (robust estimator, dotted line).

^{*}In all simulations ${\sigma}_{\tau}=1$. The labels on the x-axis are in the form N–g.

**Figure 3.**Estimated efficiency of the robust estimators ${\tilde{\sigma}}_{\tau}$ (solid line), ${\tilde{\sigma}}_{\rho ,1}$ (dashed line) and ${\tilde{\sigma}}_{\rho ,2}$ (dotted line) calculated on the 100,000 simulations

**without contamination**

^{*}.

^{*}In all simulations ${\sigma}_{\tau}=1$. The labels on the x-axis are in the form N–g.

**Figure 4.**$100\times $ MAB of the variance components estimates obtained in the 100,000 simulations without contamination (gray lines) and under

**contamination (A)**(black lines)

^{*}. (

**a**) $100\times $MAB of ${\widehat{\sigma}}_{\tau}$ (classical estimator, solid line) and ${\tilde{\sigma}}_{\tau}$ (robust estimator, dashed line); (

**b**) $100\times $MAB of ${\widehat{\sigma}}_{\rho}$ (classical estimator, solid line), ${\tilde{\sigma}}_{\rho ,1}$ (robust estimator, dashed line) and ${\tilde{\sigma}}_{\rho ,2}$ (robust estimator, dotted line).

^{*}In all simulations ${\sigma}_{\tau}=1$. The labels on the x-axis are in the form N–g.

**Figure 5.**$100\times $MAB of the variance components estimates obtained in the 100,000 simulations without contamination (gray lines) and under

**contamination (B)**(black lines)

^{*}. In panel Figure 5b, black and grey lines overlap. (

**a**) $100\times $MAB of ${\widehat{\sigma}}_{\tau}$ (classical estimator, solid line) and ${\tilde{\sigma}}_{\tau}$ (robust estimator, dashed line); (

**b**) $100\times $MAB of ${\widehat{\sigma}}_{\rho}$ (classical estimator, solid line), ${\tilde{\sigma}}_{\rho ,1}$ (robust estimator, dashed line) and ${\tilde{\sigma}}_{\rho ,2}$ (robust estimator, dotted line).

^{*}In all simulations ${\sigma}_{\tau}=1$. The labels on the x-axis are in the form N–g.

**Figure 6.**$100\times $MAB of the variance components estimates obtained in the 100,000 simulations without contamination (gray lines) and under

**contamination (C)**(black lines)

^{*}. (

**a**) $100\times $MAB of ${\widehat{\sigma}}_{\tau}$ (classical estimator, solid line) and ${\tilde{\sigma}}_{\tau}$ (robust estimator, dashed line); (

**b**) $100\times $MAB of ${\widehat{\sigma}}_{\rho}$ (classical estimator, solid line), ${\tilde{\sigma}}_{\rho ,1}$ (robust estimator, dashed line) and ${\tilde{\sigma}}_{\rho ,2}$ (robust estimator, dotted line).

^{*}In all simulations ${\sigma}_{\tau}=1$. The labels on the x-axis are in the form N–g.

**Figure 7.**Simulated observations of a balanced sample with $N=60$ and $g=6$ under

**contamination (A)**.

**Solid line**: $3\sigma $ threshold calculated with the real value of the mean and of the total standard error ${\sigma}_{\tau}$.

**Dotted line**: $3\sigma $ threshold based on the real value of the mean and on the classical estimation ${\widehat{\sigma}}_{\tau}$ calculated with the data of the first five inspections.

**Dashed line**: $3\sigma $ threshold based on the real value of the mean and on the robust estimation ${\tilde{\sigma}}_{\tau}$.

**Figure 8.**Classical and Robust estimation of ${\sigma}_{\tau}$ on a set of empirical data concerning impure Plutonium items measured in attended mode.

**Dotted line**: $3\sigma $ threshold based on the sample mean and on the classical estimation ${\widehat{\sigma}}_{\tau}$.

**Dashed line**: $3\sigma $ threshold based on the sample median and on the robust estimation ${\tilde{\sigma}}_{\tau}={\tilde{\sigma}}_{\rho ,1}$.

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2022 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Cerasa, A.
Introducing Robust Statistics in the Uncertainty Quantification of Nuclear Safeguards Measurements. *Entropy* **2022**, *24*, 1160.
https://doi.org/10.3390/e24081160

**AMA Style**

Cerasa A.
Introducing Robust Statistics in the Uncertainty Quantification of Nuclear Safeguards Measurements. *Entropy*. 2022; 24(8):1160.
https://doi.org/10.3390/e24081160

**Chicago/Turabian Style**

Cerasa, Andrea.
2022. "Introducing Robust Statistics in the Uncertainty Quantification of Nuclear Safeguards Measurements" *Entropy* 24, no. 8: 1160.
https://doi.org/10.3390/e24081160