Uncertainy’s Indices Assessment for Calibrated Energy Models
Abstract
:1. Introduction and Motivation for the Work
Summary of the ZEC Methodology
2. List of a Number of Error Metrics
2.1. Bias Error Indices
2.2. Uncertainty Indices Based on Absolute Deviations
2.3. Uncertainty Indices Based on Square Deviations
2.4. GoodnessofFit Metrics
2.5. Efficiency Criteria
2.6. Indices for Model Discrimination
2.7. Proximity Measures
3. Cases Studies, Building Description, and Models’ Preparation
4. Methodology to Evaluate Energy Models: Analysis of Case Studies
 In the first group, the indices whose maximum correlation is reached at $\lambda =0.25$ for a synthetic model and $\lambda =1.45$ for a real model are measures calculated by the absolute value of the distances.
 The second group reaches the maximum at $\lambda =0.3$ and $\lambda =0.45$ for a synthetic model and real model, respectively, and they are calculated with squared distances.
 In the third group, the value of $\lambda $ varies from $\lambda =0.05$ to $\lambda =1.35$ for a synthetic model and from $\lambda =0.3$ to $\lambda =1.75$ for a real model. They are not related to a specific distance measure.
5. Conclusions and Future Research
Author Contributions
Funding
Acknowledgments
Conflicts of Interest
Abbreviations
Arg  Argument 
$AE$  Absolute Error 
$AIC$  Akaike Information Criterion 
$BES$  Building Energy Simulation 
$BE$  Bias Error 
$BIC$  Bayesian Information Criterion 
$b{R}^{2}$  Multiplied by the coefficient of the regression line (b) 
$CV\left(RMSE\right)$  Coefficient of Variation of $RMSE$ 
d  Index of agreement 
$GoF$  Goodnessoffit index 
$M\&V$  Measurement and Verification process 
$MAE$  Mean Absolute Error 
$MAPE$  Mean Absolute Percent Error 
$MBE$  Mean Bias Error 
$md$  Modified index of agreement 
$mNSE$  Modified Nash–Sutcliffe efficiency 
$MSE$  Mean Squared Error 
$NMBE$  Normalized Mean Bias Error 
$NSE$  Nash–Sutcliffe Efficiency 
$PBIAS$  Percent Bias 
${R}^{2}$  Coefficient of determination 
$rd$  Relative index of agreement 
$RMSE$  Root Mean Squared Error 
$rNSE$  Relative Nash–Sutcliffe Efficiency 
$rSD$  Ratio of Standard Deviations 
$UI$  Uncertainty Index 
$ZEC$  Zero Energy for Calibration 
References
 Arcipowska, A.; Anagnostopoulos, F.; Mariottini, F.; Kunkel, S. Energy Performance Certificates across the EU; A Mapping of National Approaches; The buildings Performance Institute Europe (BPIE): Bruxelles, Belgium, 2014. [Google Scholar]
 Lewry, A.J.; Ortiz, J.; Nabil, A.; Schofield, N.; Vaid, R.; Hussain, S.; Davidson, P. Bridging the Gap Between Operational and Asset Ratings–The UK Experience and the Green Deal Tool; BRE Group: Watford, UK, 2013. [Google Scholar]
 IPMVP Committee. International Performance Measurement and Verification Protocol: Concepts and Options for Determining Energy and Water Savings Volume I; Technical Report; Efficiency Valuation Organization: Washington, DC, USA, 2012. [Google Scholar]
 Fernández Bandera, C.; Ramos Ruiz, G. Towards a New Generation of Building Envelope Calibration. Energies 2017, 10, 2102. [Google Scholar] [CrossRef]
 Florita, A.; Henze, G. Whole Building Fault Diagnostics—A Bayesian Approach; In Proceedings of the Intelligent Building Operations Workshop, Boulder, CO, USA, 20–22 June 2013.
 Haberl, J.; BouSaada, T. Procedures for calibrating hourly simulation models to measured building energy and environmental data. J. Sol. Energy Eng. 1998, 120, 193–204. [Google Scholar] [CrossRef]
 Dhar, A.; Reddy, T.; Claridge, D. A Fourier series model to predict hourly heating and cooling energy use in commercial buildings with outdoor temperature as the only weather variable. J. Sol. Energy Eng. 1999, 121, 47–53. [Google Scholar] [CrossRef]
 Haberl, J.S.; Sreshthaputra, A.; Claridge, D.E.; Kissock, J.K. Inverse model toolkit: Application and testing. ASHRAE Trans. 2003, 109, 435. [Google Scholar]
 Kissock, J.K.; Reddy, T.A.; Claridge, D.E. Ambienttemperature regression analysis for estimating retrofit savings in commercial buildings. J. Sol. Energy Eng. 1998, 120, 168–176. [Google Scholar] [CrossRef]
 Crawley, D.B.; Lawrie, L.K.; Winkelmann, F.C.; Buhl, W.F.; Huang, Y.J.; Pedersen, C.O.; Strand, R.K.; Liesen, R.J.; Fisher, D.E.; Witte, M.J.; et al. EnergyPlus: Creating a newgeneration building energy simulation program. Energy Build. 2001, 33, 319–331. [Google Scholar] [CrossRef]
 Sonderegger, R. Diagnostic Tests Determining the Thermal Response of a House; Lawrence Berkeley National Laboratory: Berkeley, CA, USA, 1977.
 Vuelvas, J.; Ruiz, F.; Gruosso, G. Limiting gaming opportunities on incentivebased demand response programs. Appl. Energy 2018, 225, 668–681. [Google Scholar] [CrossRef][Green Version]
 Communication from the Commission. A Policy Framework for Climate and Energy in the Period From 2020 to 2030; European Commission: Brussels, Belgium, 2014. [Google Scholar]
 SabinaProject. SABINA. SmArt BIDirectional Multi eNergy gAteway. 2016. Available online: https://sabinaproject.eu/ (accessed on 31 May 2019).
 Webster, L.; Bradford, J.; Sartor, D.; Shonder, J.; Atkin, E.; Dunnivant, S.; Frank, D.; Franconi, E.; Jump, D.; Schiller, S.; et al. M&V Guidelines: Measurement and Verification for PerformanceBased Contracts; Version 4.0; Technical Report; U.S. Department of Energy, Federal Energy Management Program: Washington, DC, USA, 2015.
 ASHRAE. ASHRAE Guideline 14–2014, Measurement of Energy, Demand, and Water Savings; ASHRAE: New York, NY, USA, 2014. [Google Scholar]
 Giuliani, M.; Henze, G.P.; Florita, A.R. Modelling and calibration of a highmass historic building for reducing the prebound effect in energy assessment. Energy Build. 2016, 116, 434–448. [Google Scholar] [CrossRef]
 Tahmasebi, F.; Zach, R.; Schuß, M.; Mahdavi, A. Simulation model calibration: An optimizationbased approach. BauSIM 2012, 1, 386–391. [Google Scholar]
 Hong, T.; Lee, S.H. Integrating physicsbased models with sensor data: An inverse modeling approach. Build. Environ. 2019, 154, 23–31. [Google Scholar] [CrossRef][Green Version]
 Cipriano, J.; Mor, G.; Chemisana, D.; Pérez, D.; Gamboa, G.; Cipriano, X. Evaluation of a multistage guided search approach for the calibration of building energy simulation models. Energy Build. 2015, 87, 370–385. [Google Scholar] [CrossRef]
 Bandera, C.F. La Inteligencia Artificial como Inspiración Para la Generación y Diseño de Modelos térmicos de Edificios. Ph.D. Thesis, Universidad de Navarra, Pamplona, Spain, 2016. [Google Scholar]
 Bandera, C.F.; Mardones, A.F.M.; Du, H.; Trueba, J.E.; Ruiz, G.R. Exergy As a Measure of Sustainable Retrofitting of Buildings. Energies 2018, 11, 1–19. [Google Scholar]
 Manfren, M.; Aste, N.; Moshksar, R. Calibration and uncertainty analysis for computer models—A metamodel based approach for integrated building energy simulation. Appl. Energy 2013, 103, 627–641. [Google Scholar] [CrossRef]
 Robertson, J.J.; Polly, B.J.; Collis, J.M. Reducedorder modeling and simulated annealing optimization for efficient residential building utility bill calibration. Appl. Energy 2015, 148, 169–177. [Google Scholar] [CrossRef][Green Version]
 Nguyen, A.T.; Reiter, S.; Rigo, P. A review on simulationbased optimization methods applied to building performance analysis. Appl. Energy 2014, 113, 1043–1058. [Google Scholar] [CrossRef]
 Ruiz, G.R.; Bandera, C.F. Analysis of uncertainty indices used for building envelope calibration. Appl. Energy 2017, 185, 82–94. [Google Scholar] [CrossRef]
 Farhang, T.; Ardeshir, M. Monitoringbased optimizationassisted calibration of the thermal performance model of an office building. In Proceedings of the International Conference on Architecture and Urban Design, Tirana, Albania, 19–21 April 2012. [Google Scholar]
 DoE, U. Energyplus Engineering Reference; The Reference to Energyplus Calculations; EnergyPlus: Washington, DC, USA, 2010.
 Wetter, M.; Wright, J. A comparison of deterministic and probabilistic optimization algorithms for nonsmooth simulationbased optimization. Build. Environ. 2004, 39, 989–999. [Google Scholar] [CrossRef][Green Version]
 Chaudhary, G.; New, J.; Sanyal, J.; Im, P.; O’Neill, Z.; Garg, V. Evaluation of Autotune calibration against manual calibration of building energy models. Appl. Energy 2016, 182, 115–134. [Google Scholar] [CrossRef]
 Ruiz, G.R.; Bandera, C.F.; Temes, T.G.A.; Gutierrez, A.S.O. Genetic algorithm for building envelope calibration. Appl. Energy 2016, 168, 691–705. [Google Scholar] [CrossRef]
 Mustafaraj, G.; Marini, D.; Costa, A.; Keane, M. Model calibration for building energy efficiency simulation. Appl. Energy 2014, 130, 72–85. [Google Scholar] [CrossRef]
 ZambranoBigiarini, M. hydroGOF: GoodnessofFit Functions for Comparison of Simulated And Observed Hydrological Time Series, R Package Version 0.38. 2014.
 Abbaspour, K.C.; Rouholahnejad, E.; Vaghefi, S.; Srinivasan, R.; Yang, H.; Kløve, B. A continentalscale hydrology and water quality model for Europe: Calibration and uncertainty of a highresolution largescale SWAT model. J. Hydrol. 2015, 524, 733–752. [Google Scholar] [CrossRef][Green Version]
 ASHRAE. Fundamentals Handbook, IP ed.; ASHRAE: New York, NY, USA, 2017. [Google Scholar]
 EVO. Uncertainty Assessment for IPMVP—Release of a New IPMVP Application Guide; EVO: Washington, DC, USA, 2018. [Google Scholar]
 Yoon, J.; Lee, E.J.; Claridge, D. Calibration procedure for energy performance simulation of a commercial building. J. Sol. Energy Eng. 2003, 125, 251–257. [Google Scholar] [CrossRef]
 Touzani, S.; Granderson, J.; Fernandes, S. Gradient boosting machine for modeling the energy consumption of commercial buildings. Energy Build. 2018, 158, 1533–1543. [Google Scholar] [CrossRef][Green Version]
 Glasgo, B.; Hendrickson, C.; Azevedo, I.L. Assessing the value of information in residential building simulation: Comparing simulated and actual building loads at the circuit level. Appl. Energy 2017, 203, 348–363. [Google Scholar] [CrossRef]
 Chakraborty, D.; Elzarka, H. Performance testing of energy models: Are we using the right statistical metrics? J. Build. Perform. Simul. 2018, 11, 433–448. [Google Scholar] [CrossRef]
 Coakley, D.; Aird, G.; Earle, S.; Klebow, B.; Conaghan, C. Development of Calibrated Operational Models of Existing Buildings for RealTime Decision Support and Performance Optimisation. In Proceedings of the CIBSE Technical Symposium, Edinburgh, UK, 14–15 April 2016. [Google Scholar]
 Reddy, T.A.; Claridge, D.E. Uncertainty of measured energy savings from statistical baseline models. HVAC R Res. 2000, 6, 3–20. [Google Scholar] [CrossRef]
 Beven, K.J. RainfallRunoff Modelling: The Primer; John Wiley & Sons: Hoboken, NJ, USA, 2011. [Google Scholar]
 Abbaspour, K. User Manual for SWATCUP, SWAT Calibration and Uncertainty Analysis Programs; Swiss Federal Institute of Aquatic Science and Technology, Eawag: Duebendorf, Switzerland, 2007. [Google Scholar]
 Kamali, B.; Abbaspour, K.C.; Yang, H. Assessing the uncertainty of multiple input datasets in the prediction of water resource components. Water 2017, 9, 709. [Google Scholar] [CrossRef]
Index  Equation 

Bias Error 
$$BE=\sum _{i=1}^{n}\left({y}_{i}{\widehat{y}}_{i}\right)$$

Mean Bias Error 
$$MBE={\displaystyle \frac{1}{n}}\sum _{i=1}^{n}\left({y}_{i}{\widehat{y}}_{i}\right)$$

Relative Error 
$$RE=\sum _{i=1}^{n}{\displaystyle \frac{\left({y}_{i}{\widehat{y}}_{i}\right)}{{y}_{i}}}$$

Normalized Mean Bias Error 
$$NMBE={\displaystyle \frac{1}{\overline{y}\left(np\right)}}\sum _{i=1}^{n}\left({y}_{i}{\widehat{y}}_{i}\right)$$

$PBIAS$ 
$$PBIAS={\displaystyle \frac{{\displaystyle \sum _{i=1}^{n}}({y}_{i}\widehat{{y}_{i}})}{{\displaystyle \sum _{i=1}^{n}}{y}_{i}}}\times 100$$

Index  Equation 

Absolute Error 
$$AE=\sum _{i=1}^{n}\left{y}_{i}{\widehat{y}}_{i}\right$$

Mean Absolute Error 
$$MAE={\displaystyle \frac{1}{n}}\sum _{i=1}^{n}\left{y}_{i}{\widehat{y}}_{i}\right$$

Relative Absolute Error 
$$RAE=\sum _{i=1}^{n}{\displaystyle \frac{\left{y}_{i}{\widehat{y}}_{i}\right}{{y}_{i}}}$$

Mean Absolute Percent Error 
$$MAPE={\displaystyle \frac{1}{\overline{y}}}\sum _{i=1}^{n}{\displaystyle \frac{\left{y}_{i}{\widehat{y}}_{i}\right}{{y}_{i}}}\times 100\%$$

$Emax$ 
$$Emax=\underset{1\le i\le n}{max}\left{y}_{i}{\widehat{y}}_{i}\right$$

Index  Equation  Range  Optimal value 

Mean Squared Error 
$$MSE={\displaystyle \frac{1}{n}}\sum _{i=1}^{n}{\left({y}_{i}{\widehat{y}}_{i}\right)}^{2}$$
 $[0,\infty )$  0 
Root Mean Squared Error 
$$RMSE={\left[{\displaystyle \frac{1}{n}}\sum _{i=1}^{n}{\left({y}_{i}{\widehat{y}}_{i}\right)}^{2}\right]}^{\frac{1}{2}}$$
 $[0,\infty )$  0 
RMSEobservation Standard Deviation ratio 
$$RSR={\left[{\displaystyle \frac{{\sum}_{i=1}^{n}{\left({y}_{i}\widehat{{y}_{i}}\right)}^{2}}{{\sum}_{i=1}^{n}{\left({y}_{i}\overline{y}\right)}^{2}}}\right]}^{\frac{1}{2}}$$
 $[0,\infty )$  0 
Coefficient of Variation of RMSE 
$$CV\left(RMSE\right)={\displaystyle \frac{1}{\overline{y}}}{\left[{\displaystyle \frac{{\sum}_{i=1}^{n}{\left({y}_{i}{\widehat{y}}_{i}\right)}^{2}}{np}}\right]}^{\frac{1}{2}}$$
 $[0,\infty )$  0 
$RMSE/MAE$ 
$$\frac{RMSE}{MAE}}={\displaystyle \frac{{\left[n{\displaystyle \sum _{i=1}^{n}}{\left({y}_{i}{\widehat{y}}_{i}\right)}^{2}\right]}^{\frac{1}{2}}}{{\displaystyle \sum _{i=1}^{n}}\left{y}_{i}{\widehat{y}}_{i}\right}$$
 $[1,\sqrt{n}]$  1 
Index  Equation  Range  Optimal Value 

Pearson Correlation Coefficient 
$$r=\frac{{\displaystyle \sum _{i=1}^{n}}\left({y}_{i}\overline{y}\right)\left({\widehat{y}}_{i}\overline{\widehat{y}}\right)}{\sqrt{\left({\displaystyle \sum _{i=1}^{n}}{\left({y}_{i}\overline{y}\right)}^{2}\right)\left({\displaystyle \sum _{i=1}^{n}}{\left({\widehat{y}}_{i}\overline{\widehat{y}}\right)}^{2}\right)}}$$
 $[1,1]$  $\leftr\right=1$ 
Spearman Correlation Coefficient 
$$\rho =\frac{{\displaystyle \sum _{i=1}^{n}}\left(rg\left({y}_{i}\right)\overline{rg\left(y\right)}\right)\left(rg\left({\widehat{y}}_{i}\right)\overline{rg\left(y\right)}\right)}{\sqrt{\left({\displaystyle \sum _{i=1}^{n}}{\left(rg\left({y}_{i}\right)\overline{rg\left(y\right)}\right)}^{2}\right)\left({\displaystyle \sum _{i=1}^{n}}{\left(rg\left({\widehat{y}}_{i}\right)\overline{rg\left(y\right)}\right)}^{2}\right)}}$$
 $[1,1]$  $\left\rho \right=1$ 
Coefficient of Determination 
$${R}^{2}={r}^{2}$$
 $[0,1]$  1 
$b{R}^{2}$ 
$$b{R}^{2}=\left\{\begin{array}{cc}\leftb\right{R}^{2}& if\leftb\right\le 1\hfill \\ {R}^{2}/\leftb\right& if\leftb\right>1\hfill \end{array}\right.$$
 $[0,\infty )$  1 
GoF* 
$$GOF={\left[\frac{1}{2}\left(CV{\left(RMSE\right)}^{2}+NMB{E}^{2}\right)\right]}^{1/2}$$
 $[0,1]$  0 
Index ZEC 
$$\leftNMBE\right+CV\left(RMSE\right)+(1{R}^{2})$$
 $[0,\infty )$  0 
Ratio of Standard Deviations 
$$rSD={\left({\displaystyle \frac{{\sum}_{i=1}^{n}{\left({y}_{i}\overline{y}\right)}^{2}}{{\sum}_{i=1}^{n}{\left(\widehat{{y}_{i}}\overline{\widehat{y}}\right)}^{2}}}\right)}^{\frac{1}{2}}$$
 $[0,\infty )$  1 
Index  Equation  Range  Optimal Value 

Nash–Sutcliffe efficiency 
$$NSE=1{\displaystyle \frac{{\displaystyle \sum _{i=1}^{n}}{\left({y}_{i}{\widehat{y}}_{i}\right)}^{2}}{{\displaystyle \sum _{i=1}^{n}}{\left({y}_{i}\overline{y}\right)}^{2}}}$$
 $(\infty ,1]$  1 
Modified NSE 
$$mNSE=1{\displaystyle \frac{{\displaystyle \sum _{i=1}^{n}}{\left{y}_{i}{\widehat{y}}_{i}\right}^{j}}{{\displaystyle \sum _{i=1}^{n}}{\left{y}_{i}\overline{y}\right}^{j}}}$$
 $(\infty ,1]$  1 
Relative NSE 
$$rNSE=1{\displaystyle \frac{{\displaystyle \sum _{i=1}^{n}}{\left(\frac{{y}_{i}{\widehat{y}}_{i}}{{y}_{i}}\right)}^{2}}{{\displaystyle \sum _{i=1}^{n}}{\left(\frac{{y}_{i}\overline{y}}{\overline{y}}\right)}^{2}}}$$
 $(\infty ,1]$  1 
Logarithmic NSE 
$$\mathrm{log}NSE=1{\displaystyle \frac{{\displaystyle \sum _{i=1}^{n}}{\left(\mathrm{log}{y}_{i}\mathrm{log}{\widehat{y}}_{i}\right)}^{2}}{{\displaystyle \sum _{i=1}^{n}}{\left(\mathrm{log}{y}_{i}\mathrm{log}\overline{y}\right)}^{2}}}$$
 $(\infty ,1]$  1 
Index of Agreement 
$$d=1{\displaystyle \frac{{\displaystyle \sum _{i=1}^{n}}{\left({y}_{i}{\widehat{y}}_{i}\right)}^{2}}{{\displaystyle \sum _{i=1}^{n}}{\left(\left\widehat{{y}_{i}}\overline{y}\right+\left{y}_{i}\overline{y}\right\right)}^{2}}}$$
 $[0,1]$  1 
Modified Index of agreement 
$$md=1{\displaystyle \frac{{\displaystyle \sum _{i=1}^{n}}{\left{y}_{i}{\widehat{y}}_{i}\right}^{j}}{{\displaystyle \sum _{i=1}^{n}}{\left(\left\widehat{{y}_{i}}\overline{y}\right+\left{y}_{i}\overline{y}\right\right)}^{j}}}$$
 $[0,1]$  1 
Relative Index of Agreement 
$$rd=1{\displaystyle \frac{\sum _{i=1}^{n}{\left(\frac{{y}_{i}{\widehat{y}}_{i}}{{y}_{i}}\right)}^{2}}{\sum _{i=1}^{n}{\left(\frac{\left\widehat{{y}_{i}}\overline{y}\right+\left{y}_{i}\overline{y}\right}{\overline{y}}\right)}^{2}}}$$
 $(\infty ,1]$  1 
Coefficient of Persistence 
$$cp=1{\displaystyle \frac{{\sum}_{i=2}^{n}{\left({y}_{i}{\widehat{y}}_{i}\right)}^{2}}{{\sum}_{i=1}^{n1}{\left({y}_{i+1}{y}_{i}\right)}^{2}}}$$
 $(\infty ,1]$  1 
Volumetric Efficiency 
$$VE=1{\displaystyle \frac{{\sum}_{i=1}^{n}\left{y}_{i}{\widehat{y}}_{i}\right}{{\sum}_{i=1}^{n}{y}_{i}}}$$
 $[0,1]$  0 
Index  Equation  Range  Optimal Value 

Akaike Information Criterion 
$$AIC=n\mathrm{log}\left(MSE\right)+2d$$
 $\mathbb{R}$  lower value 
Bayesian Information Criterion 
$$BIC=n\mathrm{log}\left(MSE\right)+d\mathrm{log}\left(n\right)$$
 $\mathbb{R}$  lower value 
Uncertainty Index (UI)  Maximum Correlation between UI and pFactor$\left(\mathit{\lambda}\right)$  Reached at $\mathit{\lambda}$  Correlation between UI and Consumed Energy 

GROUP 1  
VE  0.9907  0.25  0.9055 
MAE  0.9907  0.25  0.9055 
mNSE  0.9906  0.25  0.9055 
md  0.9895  0.25  0.9089 
MAPE  0.9886  0.25  0.9115 
GROUP 2  
CV(RMSE)  0.9923  0.30  0.9001 
NSE  0.9923  0.30  0.9001 
cp  0.9923  0.30  0.9001 
RSR  0.9923  0.30  0.9001 
d  0.9918  0.30  0.9015 
rNSE  0.9897  0.30  0.9056 
logNSE  0.9896  0.30  0.9061 
ZEC_index  0.9898  0.30  0.8933 
$b{R}^{2}$  0.9779  0.30  0.8633 
GoF  0.9906  0.30  0.8818 
r.Spearman  0.9206  0.30  0.9274 
rSD  0.7720  0.30  0.6035 
GROUP 3  
rd  0.9896  0.40  0.9081 
Emax  0.9660  0.40  0.9276 
${R}^{2}$  0.9177  0.40  0.9611 
AIC  0.9923  0.30  0.9001 
$RMSE/MAE$  0.6999  0.05  0.5411 
PBIAS%  0.5802  1.35  0.1628 
$NMBE$  0.5802  1.35  0.1628 
Uncertainty Index (UI)  Maximum Correlation between UI and pFactor$\left(\mathit{\lambda}\right)$  Reached at $\mathit{\lambda}$  Correlation between UI and Consumed Energy 

GROUP 1  
VE  0.9811  1.50  0.8006 
MAE  0.9811  1.50  0.8007 
mNSE  0.9811  1.50  0.8007 
md  0.9831  1.50  0.8056 
MAPE  0.9834  1.45  0.8220 
GROUP 2  
CV(RMSE)  0.9905  1.45  0.8275 
NSE  0.9905  1.45  0.8275 
cp  0.9905  1.45  0.8275 
RSR  0.9905  1.45  0.8275 
d  0.9907  1.45  0.8259 
rNSE  0.9947  1.30  0.8712 
logNSE  0.9968  1.20  0.9137 
ZEC_index  0.9716  1.50  0.7999 
$b{R}^{2}$  0.9456  1.60  0.7575 
GoF  0.9897  1.40  0.8291 
r.Spearman  0.9653  0.45  0.5900 
rSD  0.8189  1.00  0.8556 
GROUP 3  
rd  0.9941  1.35  0.8575 
Emax  0.9877  1.75  0.7260 
${R}^{2}$  0.9484  0.45  0.6663 
AIC  0.9904  1.45  0.8275 
$RMSE/MAE$  0.9155  0.30  0.4085 
PBIAS%  0.9635  0.60  0.5335 
$NMBE$  0.9635  0.60  0.5335 
Model  ZEC_Index  Energy Ranking  Model  Energy Ranking 

P13_M10  1  25  P5_M7  1 
P13_M20  2  35  P5_M1  2 
P5_M4  3  10  P5_M2  3 
P5_M3  4  5  P5_M9  4 
P13_M15  5  27  P5_M3  5 
P5_M8  6  8  P13_M1  6 
P5_M6  12  12  P5_M5  7 
P13_M4  8  14  P5_M8  8 
P5_M9  9  4  P5_M10  9 
P5_M5  10  7  P5_M4  10 
P5_M17  11  16  P5_M13  11 
P13_M1  12  6  P5_M6  12 
P13_M6  13  22  P5_M14  13 
P5_M10  14  9  P13_M4  14 
P5_M2  15  3  P5_M15  15 
P5_M14  16  13  P5_M17  16 
P5_M16  17  20  P5_M11  17 
P13_M9  18  32  P5_M12  18 
P13_M13  19  29  P5_M19  19 
P13_M8  20  30  P5_M16  20 
Model  ZEC_Index  Energy Ranking  Model  Energy Ranking 

P10_M2  1  29  P5_M6  1 
P13_M10  2  19  P9_M8  2 
P13_M5  3  30  P5_M1  3 
P13_M12  4  39  P16_M3  4 
P13_M3  5  45  P16_M5  5 
P13_M4  6  46  P5_M7  6 
P13_M18  7  31  P5_M12  7 
P10_M3  8  84  P5_M3  8 
P16_M4  9  10  P5_M19  9 
P13_M9  10  47  P16_M4  10 
P9_M8  11  2  P5_M10  11 
P13_M1  12  62  P9_M14  12 
P10_M6  13  89  P9_M6  13 
P13_M2  14  56  P9_M5  14 
P14_M4  15  52  P6_M17  15 
P14_M1  16  63  P5_M8  16 
P13_M11  17  81  P9_M1  17 
P13_M14  18  75  P9_M3  18 
P9_M1  19  23  P13_M10  19 
P9_M2  20  70  P6_M10  20 
Model  Energy  MAPE  rNSE  logNSE  rd  pFactor (0.20) 

P5_M4  10  1  1  1  1  100.0% 
P5_M3  5  2  2  2  2  100.0% 
P5_M8  8  3  3  3  3  100.0% 
P5_M6  12  4  4  4  4  100.0% 
P5_M5  7  6  5  5  5  100.0% 
P5_M9  4  5  6  6  6  100.0% 
P5_M17  16  7  7  7  7  100.0% 
P5_M2  3  9  8  8  8  100.0% 
P5_M14  13  10  9  9  9  99.8% 
P5_M16  20  12  10  10  10  99.9% 
P5_M10  9  8  11  11  11  100.0% 
P13_M15  27  13  12  12  12  99.9% 
P5_M7  1  11  13  13  13  100.0% 
P5_M11  17  17  14  14  14  99.9% 
P13_M10  25  18  15  15  15  99.7% 
P5_M18  21  19  16  16  18  100.0% 
P13_M4  14  15  17  17  16  99.5% 
P13_M1  6  14  18  18  17  99.3% 
P5_M19  19  22  19  19  19  99.8% 
P5_M12  18  23  20  20  20  99.9% 
Model  Energy  MAPE  rNSE  logNSE  rd  pFactor (1.00) 

P9_M8  2  1  1  1  1  90.1% 
P9_M14  12  5  2  2  3  87.5% 
P16_M5  5  24  6  3  7  88.9% 
P16_M4  10  3  3  4  2  87.0% 
P9_M6  13  12  5  5  5  86.9% 
P9_M5  14  11  4  6  6  86.9% 
P9_M3  4  66  12  7  8  88.5% 
P16_M7  37  29  11  8  13  87.8% 
P10_M2  29  2  7  9  4  84.7% 
P9_M4  17  16  9  10  10  84.5% 
P9_M3  18  15  10  11  11  84.5% 
P13_M18  31  4  8  12  9  83.3% 
P9_M12  22  21  13  13  14  83.1% 
P9_M9  24  19  15  14  15  82.5% 
P13_M10  19  17  14  15  12  82.1% 
P16_M20  38  25  18  16  16  82.7% 
P16_M18  48  51  20  17  25  84.1% 
P9_M11  27  23  17  18  17  82.8% 
P9_M13  32  20  16  19  23  82.5% 
P16_M17  68  40  28  20  30  82.8% 
© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
González, V.G.; Colmenares, L.Á.; Fidalgo, J.F.L.; Ruiz, G.R.; Bandera, C.F. Uncertainy’s Indices Assessment for Calibrated Energy Models. Energies 2019, 12, 2096. https://doi.org/10.3390/en12112096
González VG, Colmenares LÁ, Fidalgo JFL, Ruiz GR, Bandera CF. Uncertainy’s Indices Assessment for Calibrated Energy Models. Energies. 2019; 12(11):2096. https://doi.org/10.3390/en12112096
Chicago/Turabian StyleGonzález, Vicente Gutiérrez, Lissette Álvarez Colmenares, Jesús Fernando López Fidalgo, Germán Ramos Ruiz, and Carlos Fernández Bandera. 2019. "Uncertainy’s Indices Assessment for Calibrated Energy Models" Energies 12, no. 11: 2096. https://doi.org/10.3390/en12112096