# Model Uncertainty in Operational Risk Modeling Due to Data Truncation: A Single Risk Case

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Model Uncertainty

#### 2.1. Motivation

#### 2.2. Typical Models

#### 2.2.1. Empirical Model

#### 2.2.2. Parametric Models

**Truncated Approach**: The truncated approach uses the observed data ${X}_{1},\dots ,{X}_{n}$ and fully recognizes its distributional properties. That is, it takes into account Equation (2) and derives maximum likelihood estimator (MLE) values by maximizing the following log-likelihood function:

**Naive Approach**: The naive approach uses the observed data ${X}_{1},\dots ,{X}_{n}$, but ignores the presence of threshold t. That is, it bypasses Equation (2) and derives MLE values by maximizing the following log-likelihood function:

**Shifted Approach**: The shifted approach uses the observed data ${X}_{1},\dots ,{X}_{n}$ and recognizes threshold t by first shifting the observations by t. Then, it derives parameter MLEs by maximizing the following log-likelihood function:

#### 2.3. Parametric VaR Estimation

#### 2.3.1. Example 1: Exponential Distribution

#### 2.3.2. Example 2: Lomax Distribution

- Severity distribution $\mathrm{Lomax}(\alpha =3.5,{\theta}_{1})$: ${\theta}_{1}=1$ (for $F\left(t\right)=0$), ${\theta}_{1}=890,355$ (for $F\left(t\right)=0.5$), ${\theta}_{1}=209,520$ (for $F\left(t\right)=0.9$).
- Threshold: $t=0$ (for $F\left(t\right)=0$) and $t=195,000$ (for $F\left(t\right)=0.5,\phantom{\rule{3.33333pt}{0ex}}0.9$).
- Complete sample size: $N=100$ (for $F\left(t\right)=0$); $N=200$ (for $F\left(t\right)=0.5$); $N=1000$ (for $F\left(t\right)=0.9$). The average observed sample size is $n=100$.
- Number of simulation runs: $10,000$.

## 3. Real-Data Example

#### 3.1. Data

#### 3.2. Model Fitting

#### 3.3. Model Validation

#### 3.4. VaR Estimates

#### 3.5. Model Predictions

## 4. Concluding Remarks

- The naive and empirical approaches are inappropriate for determining VaR estimates.
- The shifted approach—although fundamentally flawed (simply because it assumes that operational losses below the data collection threshold are impossible)—has the flexibility to adapt to data well and successfully pass standard model validation tests.
- The truncated approach is theoretically sound when appropriate fits data well, and (in our examples) produces lower VaR-based capital estimates than those of the shifted approach.

## Acknowledgments

## Author Contributions

## Conflicts of Interest

## Appendix A

#### Appendix A.1 Generalized Pareto Distribution

- Choosing $1/\gamma =\alpha $, $\sigma /\gamma =\theta $, and $\mu =\theta $ leads to what actuaries call a single-parameter Pareto distribution, with the scale parameter $\theta >0$ (usually treated as known deductible) and shape $\alpha >0$.
- Choosing $1/\gamma =\alpha $, $\sigma /\gamma =\theta $, and $\mu =0$ yields the Lomax distribution with the scale parameter $\theta >0$ and shape $\alpha >0$. This is also known as a Pareto II distribution.

#### Appendix A.2 Asymptotic Theorems

**Theorem**

**A1**

**.**Let $0<{\beta}_{1}<\cdots <{\beta}_{k}<1$, with $k>1$, and suppose that pdf g is continuous, as discussed above. Then, the k-variate vector of sample quantiles $\left({X}_{\left(\lceil n{\beta}_{1}\rceil \right)},\dots ,{X}_{\left(\lceil n{\beta}_{k}\rceil \right)}\right)$ is $\mathcal{AN}$ with the mean vector $\left({G}^{-1}\left({\beta}_{1}\right),\dots ,{G}^{-1}\left({\beta}_{k}\right)\right)$ and the covariance–variance matrix ${\left[{\sigma}_{ij}^{2}\right]}_{i,j=1}^{k}$ with the entries

**Theorem**

**A2**

**.**Suppose pdf g is indexed by k unknown parameters, $({\theta}_{1},\dots ,{\theta}_{k})$, and let $\left({\widehat{\theta}}_{1},\dots ,{\widehat{\theta}}_{k}\right)$ denote the MLE of those parameters. Then, under the regularity conditions mentioned above,

**Theorem**

**A3**

**.**Suppose that $\left({\widehat{\theta}}_{1},\dots ,{\widehat{\theta}}_{k}\right)$ is $\mathcal{AN}$ with the parameters specified in Theorem A2. Let the real-valued functions ${h}_{1}\left({\theta}_{1},\dots ,{\theta}_{k}\right),\dots ,{h}_{m}\left({\theta}_{1},\dots ,{\theta}_{k}\right)$ represent m different risk measures, tail probabilities, or other functions of model parameters. Then, under some smoothness conditions on functions ${h}_{1},\dots ,{h}_{m}$, the vector of MLE-based estimators

## References

- AMA Group. 2013. AMA Quantification Challenges: AMAG Range of Practice and Observations on “The Thorny LDA Topics”. Munich: Risk Management Association. [Google Scholar]
- Arnold, Barry C. 2015. Pareto Distributions, 2nd ed. London: Chapman & Hall. [Google Scholar]
- Basel Coordination Committee. 2014. Supervisory guidance for data, modeling, and model risk management under the operational risk advanced measurement approaches. Basel Coordination Committee Bulletin 14: 1–17. [Google Scholar]
- Baud, Nicolas, Antoine Frachot, and Thierry Roncalli. 2002. Internal Data, External Data and Consortium Data for Operational Risk Measurement: How to Pool Data Properly? Working Paper, Groupe de Recherche Opérationnelle, Crédit Lyonnais, France. [Google Scholar]
- Brazauskas, Vytaras, Bruce L. Jones, and Ričardas Zitikis. 2015. Trends in disguise. Annals of Actuarial Science 9: 58–71. [Google Scholar] [CrossRef]
- Cavallo, Alexander, Benjamin Rosenthal, Xiao Wang, and Jun Yan. 2012. Treatment of the data collection threshold in operational risk: A case study with the lognormal distribution. Journal of Operational Risk 7: 3–38. [Google Scholar] [CrossRef]
- Chau, Joris. 2013. Robust Estimation in Operational Risk Modeling. Master’s thesis, Department of Mathematics, Utrecht University, Utrecht, The Netherland. [Google Scholar]
- Chernobai, Anna S., Svetlozar T. Račev, and Frank J. Fabozzi. 2007. Operational Risk: A Guide to Basel II Capital Requirements, Models, and Analysis. Hoboken: Wiley. [Google Scholar]
- Cope, Eric. 2011. Penalized likelihood estimators for truncated data. Journal of Statistical Planning and Inference 141: 345–58. [Google Scholar] [CrossRef]
- Cruz, Marcelo G. 2002. Modeling, Measuring and Hedging Operational Risk. Hoboken: Wiley. [Google Scholar]
- Cruz, Marcelo G., Gareth W. Peters, and Pavel V. Shevchenko. 2015. Fundamental Aspects of Operational Risk and Insurance Analytics: A Handbook of Operational Risk. Hoboken: Wiley. [Google Scholar]
- De Fontnouvelle, Patrick, Virginia Dejesus-Rueff, John S. Jordan, and Eric S. Rosengren. 2006. Capital and risk: New evidence on implications of large operational losses. Journal of Money, Credit, and Banking 38: 1819–46. [Google Scholar] [CrossRef]
- Ergashev, Bakhodir, Konstantin Pavlikov, Stan Uryasev, and Evangelos Sekeris. 2016. Estimation of truncated data samples in operational risk modeling. Journal of Risk and Insurance 83: 613–40. [Google Scholar] [CrossRef]
- Horbenko, Nataliya, Peter Ruckdeschel, and Taehan Bae. 2011. Robust estimation of operational risk. Journal of Operational Risk 6: 3–30. [Google Scholar] [CrossRef]
- Klugman, Stuart A., Harry Panjer, and Gordon E. Willmot. 2012. Loss Models: From Data to Decisions, 4th ed. Hoboken: Wiley. [Google Scholar]
- Luo, Xiaolin, Pavel V. Shevchenko, and John B. Donnelly. 2007. Addressing the impact of data truncation and parameter uncertainty on operational risk estimates. Journal of Operational Risk 2: 3–26. [Google Scholar] [CrossRef]
- Moscadelli, Marco, Anna Chernobai, and Svetlozar T. Rachev. 2005. Treatment of missing data in the field of operational risk: The impacts on parameter estimates, EL and UL figures. Operational Risk 6: 28–34. [Google Scholar]
- Office of the Comptroller of the Currency. 2011. Supervisory guidance on model risk management. SR Letter 11: 1–21. [Google Scholar]
- Opdyke, John Douglas. 2014. Estimating operational risk capital with greater accuracy, precision, and robustness. Journal of Operational Risk 9: 3–79. [Google Scholar] [CrossRef]
- Opdyke, John Douglas, and Alexander Cavallo. 2012. Estimating operational risk capital: The challenges of truncation, the hazards of maximum likelihood estimation, and the promise of robust statistics. Journal of Operational Risk 7: 3–90. [Google Scholar] [CrossRef]
- Serfling, Robert J. 1980. Approximation Theorems of Mathematical Statistics. Hoboken: Wiley. [Google Scholar]
- Shevchenko, Pavel V., and Grigory Temnov. 2009. Modeling operational risk data reported above a time-varying threshold. Journal of Operational Risk 4: 19–42. [Google Scholar] [CrossRef]

**Figure 1.**Truncated, naive, and shifted $Exponential\phantom{\rule{0.166667em}{0ex}}\left(\sigma \right)$ and $Lomax\phantom{\rule{0.166667em}{0ex}}(\alpha =3.5,{\theta}_{1})$ probability density functions. Data collection threshold $t=195,000$, with 50% of data unobserved. Parameters $\sigma $ and ${\theta}_{1}$ are chosen to match those in Tables 2 and 3 (see Section 2.3).

**Figure 2.**Fitted-versus-observed log-losses for exponential (

**top row**) and Lomax (

**bottom row**) distributions, using truncated (

**left**), naive (

**middle**), and shifted (

**right**) approaches.

**Table 1.**Function $H\left(c\right)$ evaluated for various combinations of c, confidence level $\beta $, proportion of unobserved data $F\left(t\right)$, and severity distributions with varying degrees of tail heaviness ranging from light- and moderate-tailed to heavy-tailed. The sample size is $n=100$.

c | $\mathit{\beta}$ | $\mathit{F}\left(\mathit{t}\right)=0$ | $\mathit{F}\left(\mathit{t}\right)=0.5$ | $\mathit{F}\left(\mathit{t}\right)=0.9$ | ||||||
---|---|---|---|---|---|---|---|---|---|---|

Light | Moderate | Heavy | Light | Moderate | Heavy | Light | Moderate | Heavy | ||

1 | 0.95 | 0.500 | 0.500 | 0.500 | 0.944 | 0.925 | 0.874 | 1.000 | 1.000 | 0.981 |

0.995 | 0.500 | 0.500 | 0.500 | 0.688 | 0.672 | 0.638 | 0.949 | 0.884 | 0.738 | |

0.999 | 0.500 | 0.500 | 0.500 | 0.587 | 0.579 | 0.563 | 0.767 | 0.703 | 0.612 | |

1.2 | 0.95 | 0.085 | 0.178 | 0.331 | 0.585 | 0.753 | 0.824 | 1.000 | 1.000 | 0.978 |

0.995 | 0.226 | 0.349 | 0.444 | 0.398 | 0.551 | 0.612 | 0.811 | 0.840 | 0.734 | |

0.999 | 0.331 | 0.424 | 0.475 | 0.414 | 0.517 | 0.550 | 0.615 | 0.668 | 0.610 | |

1.5 | 0.95 | 0.000 | 0.010 | 0.138 | 0.032 | 0.326 | 0.726 | 0.968 | 0.996 | 0.975 |

0.995 | 0.030 | 0.167 | 0.362 | 0.083 | 0.364 | 0.571 | 0.403 | 0.756 | 0.727 | |

0.999 | 0.137 | 0.317 | 0.437 | 0.191 | 0.424 | 0.532 | 0.358 | 0.613 | 0.606 | |

2 | 0.95 | 0.000 | 0.000 | 0.015 | 0.000 | 0.009 | 0.523 | 0.056 | 0.930 | 0.968 |

0.995 | 0.000 | 0.026 | 0.240 | 0.001 | 0.127 | 0.501 | 0.017 | 0.577 | 0.715 | |

0.999 | 0.014 | 0.170 | 0.376 | 0.025 | 0.280 | 0.500 | 0.073 | 0.516 | 0.600 |

**Table 2.**Functions ${H}_{T}\left(c\right)$, ${H}_{N}\left(c\right)$, ${H}_{S}\left(c\right)$ evaluated for various combinations of c, confidence level $\beta $, and proportion of unobserved data $F\left(t\right)$. (The sample size is $n=100$.)

c | $\mathit{\beta}$ | $\mathit{F}\left(\mathit{t}\right)=0$ | $\mathit{F}\left(\mathit{t}\right)=0.5$ | $\mathit{F}\left(\mathit{t}\right)=0.9$ | ||||||
---|---|---|---|---|---|---|---|---|---|---|

T | N | S | T | N | S | T | N | S | ||

1 | 0.95 | 0.500 | 0.500 | 0.500 | 0.500 | 1.000 | 0.990 | 0.500 | 1.000 | 1.000 |

0.995 | 0.500 | 0.500 | 0.500 | 0.500 | 1.000 | 0.905 | 0.500 | 1.000 | 1.000 | |

0.999 | 0.500 | 0.500 | 0.500 | 0.500 | 1.000 | 0.842 | 0.500 | 1.000 | 1.000 | |

1.2 | 0.95 | 0.023 | 0.023 | 0.023 | 0.023 | 1.000 | 0.623 | 0.023 | 1.000 | 1.000 |

0.995 | 0.023 | 0.023 | 0.023 | 0.023 | 1.000 | 0.245 | 0.023 | 1.000 | 0.991 | |

0.999 | 0.023 | 0.023 | 0.023 | 0.023 | 1.000 | 0.159 | 0.023 | 1.000 | 0.909 | |

1.5 | 0.95 | 0.000 | 0.000 | 0.000 | 0.000 | 0.973 | 0.004 | 0.000 | 1.000 | 0.996 |

0.995 | 0.000 | 0.000 | 0.000 | 0.000 | 0.973 | 0.000 | 0.000 | 1.000 | 0.257 | |

0.999 | 0.000 | 0.000 | 0.000 | 0.000 | 0.973 | 0.000 | 0.000 | 1.000 | 0.048 | |

2 | 0.95 | 0.000 | 0.000 | 0.000 | 0.000 | 0.001 | 0.000 | 0.000 | 1.000 | 0.010 |

0.995 | 0.000 | 0.000 | 0.000 | 0.000 | 0.001 | 0.000 | 0.000 | 1.000 | 0.000 | |

0.999 | 0.000 | 0.000 | 0.000 | 0.000 | 0.001 | 0.000 | 0.000 | 1.000 | 0.000 |

**Table 3.**Functions ${H}_{T}\left(c\right)$, ${H}_{N}\left(c\right)$, ${H}_{S}\left(c\right)$ evaluated for various combinations of c, confidence level $\beta $, and proportion of unobserved data $F\left(t\right)$. The average sample size is $n=100$.

c | $\mathit{\beta}$ | $\mathit{F}\left(\mathit{t}\right)=0$ | $\mathit{F}\left(\mathit{t}\right)=0.5$ | $\mathit{F}\left(\mathit{t}\right)=0.9$ | ||||||
---|---|---|---|---|---|---|---|---|---|---|

T | N | S | T | N | S | T | N | S | ||

1 | 0.95 | 0.453 | 0.453 | 0.453 | 0.459 | 0.951 | 0.982 | 0.547 | 0.908 | 1.000 |

0.995 | 0.433 | 0.433 | 0.433 | 0.435 | 0.692 | 0.734 | 0.444 | 0.891 | 0.998 | |

0.999 | 0.426 | 0.426 | 0.426 | 0.437 | 0.149 | 0.624 | 0.331 | 0.867 | 0.944 | |

1.2 | 0.95 | 0.131 | 0.131 | 0.131 | 0.095 | 0.945 | 0.791 | 0.356 | 0.904 | 0.999 |

0.995 | 0.247 | 0.247 | 0.247 | 0.184 | 0.208 | 0.518 | 0.170 | 0.889 | 0.993 | |

0.999 | 0.297 | 0.297 | 0.297 | 0.272 | 0.059 | 0.484 | 0.121 | 0.845 | 0.864 | |

1.5 | 0.95 | 0.009 | 0.009 | 0.009 | 0.002 | 0.626 | 0.270 | 0.112 | 0.879 | 0.998 |

0.995 | 0.097 | 0.097 | 0.097 | 0.044 | 0.044 | 0.278 | 0.021 | 0.875 | 0.872 | |

0.999 | 0.178 | 0.178 | 0.178 | 0.123 | 0.016 | 0.313 | 0.019 | 0.843 | 0.708 | |

2 | 0.95 | 0.000 | 0.000 | 0.000 | 0.000 | 0.032 | 0.010 | 0.002 | 0.865 | 0.984 |

0.995 | 0.025 | 0.025 | 0.025 | 0.004 | 0.004 | 0.090 | 0.000 | 0.851 | 0.563 | |

0.999 | 0.075 | 0.075 | 0.075 | 0.032 | 0.002 | 0.147 | 0.001 | 0.224 | 0.459 |

**Table 4.**Parameter maximum likelihood estimators (MLEs, with variance and covariance estimates in parentheses) of the exponential and Lomax models, using truncated, naive, and shifted approaches.

Model | Truncated | Naive | Shifted |
---|---|---|---|

Exponential | $\widehat{\sigma}=351,021$ $\left(2.28\times {10}^{9}\right)$ | $\widehat{\sigma}=546,021$ $\left(5.52\times {10}^{9}\right)$ | $\widehat{\sigma}=351,021$ $\left(2.28\times {10}^{9}\right)$ |

Lomax | $\widehat{\alpha}=1.91$ $\left(0.569\right)$ | $\widehat{\alpha}=22.51$ $\left(5,189.86\right)$ | $\widehat{\alpha}=1.91$ $\left(0.569\right)$ |

$\widehat{\theta}=151,234$ $\left(3.84\times {10}^{10}\right)$ | $\widehat{\theta}=11,735,899$ $\left(1.54\times {10}^{15}\right)$ | $\widehat{\theta}=346,234$ $\left(3.84\times {10}^{10}\right)$ | |

$\left(\widehat{cov}(\widehat{\alpha},\widehat{\theta})=138,934\right)$ | $\left(\widehat{cov}(\widehat{\alpha},\widehat{\theta})=2.82\times {10}^{9}\right)$ | $\left(\widehat{cov}(\widehat{\alpha},\widehat{\theta})=138,934\right)$ |

**Table 5.**Values of KS and AD statistics (with p-values in parentheses) for the fitted models, using truncated, naive, and shifted approaches.

Model | Kolmogorov–Smirnov | Anderson–Darling | ||||
---|---|---|---|---|---|---|

Truncated | Naive | Shifted | Truncated | Naive | Shifted | |

Exponential | $0.186$ (0.004) | $0.307$ (0.000) | $0.186$ (0.004) | $3.398$ (0.000) | $4.509$ (0.000) | $3.398$ (0.000) |

Lomax | $0.072$ (0.632) | $0.316$ (0.000) | $0.072$ (0.631) | $0.272$ (0.671) | $4.696$ (0.000) | $0.272$ (0.678) |

**Table 6.**Value-at-risk ($\mathrm{VaR}\left)\right(\beta )$ estimates (with 95% confidence intervals in parentheses), measured in millions and based on the fitted models, using truncated, naive, and shifted approaches.

Model | $\mathit{\beta}$ | Truncated | Naive | Shifted |
---|---|---|---|---|

Exponential | 0.95 | 1.052 (0.771; 1.332) | 1.636 (1.199; 2.072) | 1.247 (0.966; 1.527) |

0.995 | 1.860 (1.364; 2.356) | 2.893 (2.121; 3.665) | 2.055 (1.559; 2.551) | |

0.999 | 2.425 (1.778; 3.071) | 3.772 (2.766; 4.778) | 2.620 (1.973; 3.266) | |

Lomax | 0.95 | 0.576 (0.071; 1.160) | 1.670 (1.134; 2.206) | 1.514 (0.978; 2.755) |

0.995 | 2.281 (0.413; 4.758) | 3.114 (2.257; 5.023) | 5.417 (2.213; 20.604) | |

0.999 | 5.504 (1.100; 13.627) | 4.214 (3.019; 8.586) | 12.797 (3.649; 89.992) |

142,774.19 | 146,875.00 | 151,000.00 | 160,000.00 | 176,000.00 | 182,435.12 | 191,070.31 |

143,000.00 | 150,411.29 | 153,592.54 | 165,000.00 | 176,000.00 | 185,000.00 | 192,806.74 |

145,500.50 | 150,930.39 | 157,083.00 | 165,000.00 | 180,000.00 | 186,330.00 | 193,500.00 |

**Table 8.**Model-based predictions (with 95% confidence intervals in parentheses) of several statistics for the unobserved losses between $150,000 and $175,000.

Model | Truncated | Naive | ||||
---|---|---|---|---|---|---|

Number of Losses | Average Loss | Total Loss | Number of Losses | Average Loss | Total Loss | |

Exponential | 4.2 | 162,352 | 685,108 | 2.6 | 162,405 | 426,197 |

(3.0; 5.5) | (162,312; 162,391) | (452,840; 917,376) | (1.9; 3.4) | (162,379; 162,430) | (141,592; 710,802) | |

Lomax | 9.9 | 162,017 | 1,609,649 | 2.7 | 162,397 | 441,155 |

(3.3; 16.5) | (161,647; 162,388) | (543,017; 2,676,281) | (1.8; 3.7) | (162,343; 162,451) | (288,324; 593,985) |

© 2017 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Yu, D.; Brazauskas, V. Model Uncertainty in Operational Risk Modeling Due to Data Truncation: A Single Risk Case. *Risks* **2017**, *5*, 49.
https://doi.org/10.3390/risks5030049

**AMA Style**

Yu D, Brazauskas V. Model Uncertainty in Operational Risk Modeling Due to Data Truncation: A Single Risk Case. *Risks*. 2017; 5(3):49.
https://doi.org/10.3390/risks5030049

**Chicago/Turabian Style**

Yu, Daoping, and Vytaras Brazauskas. 2017. "Model Uncertainty in Operational Risk Modeling Due to Data Truncation: A Single Risk Case" *Risks* 5, no. 3: 49.
https://doi.org/10.3390/risks5030049