# Sensitivity of Calibrated Parameters and Water Resource Estimates on Different Objective Functions and Optimization Algorithms

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

^{2}, bR

^{2}, NSE, MNS, RSR, SSQR, KGE, and PBIAS) in a SWAT model to calibrate the monthly discharges in two watersheds in Iran. The results show that all three algorithms, using the same objective function, produced acceptable calibration results; however, with significantly different parameter ranges. Similarly, an algorithm using different objective functions also produced acceptable calibration results, but with different parameter ranges. The different calibrated parameter ranges consequently resulted in significantly different water resource estimates. Hence, the parameters and the outputs that they produce in a calibrated model are “conditioned” on the choices of the optimization algorithm and objective function. This adds another level of non-negligible uncertainty to watershed models, calling for more attention and investigation in this area.

## 1. Introduction

^{2}, bR

^{2}, NSE, MNS, RSR, SSQR, KGE, and PBIAS (see Table 1 for a definition of the function). To achieve our objectives, we used the Soil and Water Assessment Tool (SWAT) [23] in the Salman Dam Basin (SDB) and Karkheh River Basin (KRB). For model calibration, we used SWAT-CUP [24], which couples five optimization algorithms to SWAT, allowing the use of different objective functions for SUFI-2 and PSO algorithms.

## 2. Materials and Methods

#### 2.1. Hydrologic Model SWAT

#### 2.2. Calibration/Uncertainty Analysis Programs

#### 2.3. Objective Function

^{2}) and modified R

^{2}(bR

^{2}), Nash-Sutcliffe efficiency (NSE), Modified Nash-Sutcliffe efficiency (MNS), ratio of the standard deviation of observations to root mean square error (RSR), ranked sum of squares (SSQR), Kling-Gupta efficiency (KGE), and percent bias (PBIAS). The formulation of these eight objective functions is presented in Table 1.

#### 2.4. Case Studies

^{2}, with geographic coordinates of 28°26′ N to 29°47′ N and 51°55′ E to 54°19′ E. The elevation of the basin ranges from less than 800 m above sea level in the southern areas to more than 3100 m in the northern areas of the basin. The main river in the SDB is Ghareh-Aghaj, with an annual average discharge of 18 m

^{3}·s

^{−1}. The average annual precipitation is less than 250 mm·year

^{−1}in the central and southern part of the watershed, and >750 mm·year

^{−1}in the northwest regions.

^{2}and lies between 30° N to 35° N and 46° E to 49° E geographic coordinates, with an elevation ranging from less than the mean sea level to more than 3600 m (Figure 1b). The Karkheh river is the third longest river in Iran, with an annual average discharge of 188 m

^{3}·s

^{−1}[31]. The climate is semi-arid in the uplands (north) and arid in the lowlands (south). The precipitation exhibits large spatial and temporal variability. The mean annual precipitation is about 450 mm·year

^{−1}, ranging from 150 mm·year

^{−1}in the lower arid plains to 750 mm·year

^{−1}in the upper mountainous parts [31]. A large multi-purpose earthen embankment dam, Karkheh, was built on the river and has been utilized since 2001 in order to supply irrigation water in the Khuzestan plains (in the lower Karkheh region), and hydropower generation and flood control. Management information relating to Karkheh reservoir operation (i.e., the minimum and maximum daily outflow, reservoir surface area, and spillway conditions) was considered in the SWAT model of KRB [3].

#### 2.5. Model Setup

#### 2.5.1. SDB and KRB Models

^{2}and MNS, we introduced measures based on the results of similar studies [22,38,39], where the satisfactory threshold values for bR

^{2}and MNS were considered greater than or equal to 0.4. No such threshold could be specified for SSQR as the measured and simulated variables are independently ranked and its value depends on the magnitude of the variables being investigated.

#### 2.5.2. Optimization Algorithms

#### 2.5.3. Statistical Analysis

## 3. Results

#### 3.1. Sensitivity of Model Performance to the Objective Functions Used in SUFI-2 Algorithm

#### 3.2. Sensitivity of Model Parameters to Objective Functions

#### 3.3. Sensitivity of Water Resources Components to the Objective Functions

^{−1}in their national model for AET for the same region. In the current study, the minimum and maximum values of the annual average AET were determined by RSR and KGE as being 191 and 295 mm·year

^{−1}, respectively (Figure 4a). These values are within the uncertainty ranges reported by Faramarzi et al. [2]. The results of SW and WYLD in SDB (Figure 4b,c) also corresponded well with the values reported by Faramarzi et al. [2].

#### 3.4. Sensitivity of Calibration Performance and Model Parameters to Optimization Algorithms Using NSE

## 4. Conclusions

^{2}, bR

^{2}, NSE, MNS, RSR, SSQR, KGE, and PBIAS) and optimization algorithms (e.g., SUFI-2, GLUE, and PSO) using SWAT in two watersheds. The following conclusions could be drawn:

- 1)
- In most cases, different objective functions with one optimization algorithm (in this case SUFI-2) led to satisfactory calibration/validation results for river discharges in both case studies. However, the calibrated parameters were significantly different in each case, leading to different water resource estimates.
- 2)
- Different optimization algorithms with one objective function (in this case NSE) also produced satisfactory calibration/validation results for river discharges in both case studies. However, the calibrated parameters were significantly different in each case, resulting in significantly different water resources estimates.

## Acknowledgments

## Author Contributions

## Conflicts of Interest

## References

- Tong, S.T.Y.; Chen, W. Modeling the relationship between land use and surface water quality. J. Environ. Manag.
**2002**, 66, 377–393. [Google Scholar] [CrossRef] - Faramarzi, M.; Abbaspour, K.C.; Schulin, R.; Yang, H. Modelling blue and green water resources availability in Iran. Hydrol. Process.
**2009**, 23, 486–501. [Google Scholar] [CrossRef] - Ashraf Vaghefi, S.; Mousavi, S.J.; Abbaspour, K.C.; Srinivasan, R.; Yang, H. Analyses of the impact of climate change on water resources components, drought and wheat yield in semiarid regions: Karkheh River Basin in Iran. Hydrol. Process.
**2014**, 28, 2018–2032. [Google Scholar] [CrossRef] - Rouholahnejad, E.; Abbaspour, K.C.; Srinivasan, R.; Bacu, V.; Lehmann, A. Water resources of the black sea basin at high spatial and temporal resolution. Water. Resour. Res.
**2014**, 50, 5866–5885. [Google Scholar] [CrossRef] - Thanapakpawin, P.; Richey, J.; Thomas, D.; Rodda, S.; Campbell, B.; Logsdon, M. Effects of landuse change on the hydrologic regime of the Mae Chaem River Basin, NW Thailand. J. Hydrol.
**2007**, 334, 215–230. [Google Scholar] [CrossRef] - Abbaspour, K.C.; Rouholahnejad, E.; Vaghefi, S.; Srinivasan, R.; Yang, H.; Kløve, B. A continental-scale hydrology and water quality model for Europe: Calibration and uncertainty of a high-resolution large-scale SWAT model. J. Hydrol.
**2015**, 524, 733–752. [Google Scholar] [CrossRef] - Arnold, J.G.; Moriasi, D.N.; Gassman, P.W.; Abbaspour, K.C.; White, M.J.; Srinivasan, R.; Santhi, C.; Harmel, R.; Van Griensven, A.; Van Liew, M.W. SWAT: Model use, calibration, and validation. Trans. ASABE
**2012**, 55, 1491–1508. [Google Scholar] [CrossRef] - Montanari, A.; Shoemaker, C.A.; van de Giesen, N. Introduction to special section on uncertainty assessment in surface and subsurface hydrology: An overview of issues and challenges. Water. Resour. Res.
**2009**, 45. [Google Scholar] [CrossRef] - Beven, K.; Binley, A. The future of distributed models: Model calibration and uncertainty prediction. Hydrol. Process.
**1992**, 6, 279–298. [Google Scholar] [CrossRef] - Abbaspour, K.C.; Johnson, C.A.; van Genuchten, M.T. Estimating uncertain flow and transport parameters using a sequential uncertainty fitting procedure. Vadose Zone J.
**2004**, 3, 1340–1352. [Google Scholar] [CrossRef] - Van Griensven, A.; Meixner, T. Methods to quantify and identify the sources of uncertainty for river basin water quality models. Water. Sci. Technol.
**2006**, 53, 51–59. [Google Scholar] [CrossRef] [PubMed] - Eberhart, R.; Kennedy, J. A new optimizer using particle swarm theory. In Proceedings of the Sixth International Symposium on Micro Machine and Human Science, Nagoya, Japan, 4–6 October 1995; pp. 39–43. [Google Scholar]
- Kennedy, J.; Eberhart, R. Particle swarm optimization. In Proceedings of the 1995 IEEE International Conference on Neural Networks, Perth, Australia, 27 November–1 December 1995; Volume 1944, pp. 1942–1948. [Google Scholar]
- Nash, J.E.; Sutcliffe, J.V. River flow forecasting through conceptual models part I—A discussion of principles. J. Hydrol.
**1970**, 10, 282–290. [Google Scholar] [CrossRef] - Moriasi, D.N.; Arnold, J.G.; Van Liew, M.W.; Bingner, R.L.; Harmel, R.D.; Veith, T.L. Model evaluation guidelines for systematic quantification of accuracy in watershed simulations. Trans. ASABE
**2007**, 50, 885–900. [Google Scholar] [CrossRef] - Gupta, H.V.; Kling, H.; Yilmaz, K.K.; Martinez, G.F. Decomposition of the mean squared error and NSE performance criteria: Implications for improving hydrological modelling. J. Hydrol.
**2009**, 377, 80–91. [Google Scholar] [CrossRef] - Yang, J.; Reichert, P.; Abbaspour, K.C.; Xia, J.; Yang, H. Comparing uncertainty analysis techniques for a SWAT application to the Chaohe Basin in China. J. Hydrol.
**2008**, 358, 1–23. [Google Scholar] [CrossRef] - Uniyal, B.; Jha, M.K.; Verma, A.K. Parameter identification and uncertainty analysis for simulating streamflow in a river basin of eastern India. Hydrol. Process.
**2015**, 29, 3744–3766. [Google Scholar] [CrossRef] - Arsenault, R.; Poulin, A.; Côté, P.; Brissette, F. Comparison of stochastic optimization algorithms in hydrological model calibration. J. Hydrol. Eng.
**2014**, 19, 1374–1384. [Google Scholar] [CrossRef] - Wu, H.; Chen, B. Evaluating uncertainty estimates in distributed hydrological modeling for the Wenjing River watershed in China by GLUE, SUFI-2, and ParaSol methods. Ecol. Eng.
**2015**, 76, 110–121. [Google Scholar] [CrossRef] - Wu, Y.; Liu, S. A suggestion for computing objective function in model calibration. Ecol. Inform.
**2014**, 24, 107–111. [Google Scholar] [CrossRef] - Muleta, M.K. Model performance sensitivity to objective function during automated calibrations. J. Hydrol. Eng.
**2012**, 17, 756–767. [Google Scholar] [CrossRef] - Arnold, J.G.; Srinivasan, R.; Muttiah, R.S.; Williams, J.R. Large area hydrologic modeling and assessment part 1: Model development. J. Am. Water Resour. Assoc.
**1998**, 34, 73–89. [Google Scholar] [CrossRef] - Abbaspour, K.C.; Yang, J.; Maximov, I.; Siber, R.; Bogner, K.; Mieleitner, J.; Zobrist, J.; Srinivasan, R. Modelling hydrology and water quality in the pre-alpine/alpine Thur watershed using SWAT. J. Hydrol.
**2007**, 333, 413–430. [Google Scholar] [CrossRef] - Krause, P.; Boyle, D.; Bäse, F. Comparison of different efficiency criteria for hydrological model assessment. Adv. Geosci.
**2005**, 5, 89–97. [Google Scholar] [CrossRef] - van Griensven, A.; Bauwens, W. Multiobjective autocalibration for semidistributed water quality models. Water Resour. Res.
**2003**. [Google Scholar] [CrossRef] - Yapo, P.O.; Gupta, H.V.; Sorooshian, S. Automatic calibration of conceptual rainfall-runoff models: Sensitivity to calibration data. J. Hydrol.
**1996**, 181, 23–48. [Google Scholar] [CrossRef] - Neitsch, S.L.; Arnold, J.G.; Kiniry, J.R.; Williams, J.R. Soil and Water Assessment Tool; Theoretical Documentation: Version 2009; Texas Water Resource Institute: College Station, TX, USA, 2011. [Google Scholar]
- Blasone, R.S.; Vrugt, J.A.; Madsen, H.; Rosbjerg, D.; Robinson, B.A.; Zyvoloski, G.A. Generalized likelihood uncertainty estimation (GLUE) using adaptive Markov Chain Monte Carlo sampling. Adv. Water Resour.
**2008**, 31, 630–648. [Google Scholar] [CrossRef] - Oweis, T.; Siadat, H.; Abbasi, F. Improving On-Farm Agricultural Water Productivity in the Karkheh River Basin; Technical Report; CGIAR Challenge Program on Water and Food: Colombo, Sri Lanka, 2009. [Google Scholar]
- Etemad-Shahidi, A.; Afshar, A.; Alikia, H.; Moshfeghi, H. Total dissolved solid modeling; Karkheh reservoir case example. Int. J. Environ. Res.
**2010**, 3, 671–680. [Google Scholar] - Reuter, H.I.; Nelson, A.; Jarvis, A. An evaluation of void-filling interpolation methods for SRTM data. Int. J. Geogr. Inf. Sci.
**2007**, 21, 983–1008. [Google Scholar] [CrossRef] - Food and Agriculture Organization (FAO). The Digital Soil Map of the World and Derived Soil Properties; FAO: Rome, Italy, 1995. [Google Scholar]
- Hargreaves, G.L.; Hargreaves, G.H.; Riley, J.P. Agricultural benefits for Senegal River Basin. J. Irrig. Drain. Eng.
**1985**, 111, 113–124. [Google Scholar] [CrossRef] - Ritchie, J.T. Model for predicting evaporation from a row crop with incomplete cover. Water Resour. Res.
**1972**, 8, 1204–1213. [Google Scholar] [CrossRef] - Wang, X.; Melesse, A.; Yang, W. Influences of potential evapotranspiration estimation methods on SWAT’s hydrologic simulation in a northwestern Minnesota watershed. Trans. ASAE
**2006**, 49, 1755–1771. [Google Scholar] [CrossRef] - Ghobadi, Y.; Pradhan, B.; Sayyad, G.A.; Kabiri, K.; Falamarzi, Y. Simulation of hydrological processes and effects of engineering projects on the Karkheh River Basin and its wetland using SWAT 2009. Quat. Int.
**2015**, 374, 144–153. [Google Scholar] [CrossRef] - Mehdi, B.; Ludwig, R.; Lehner, B. Evaluating the impacts of climate change and crop land use change on streamflow, nitrates and phosphorus: A modeling study in Bavaria. J. Hydrol.
**2015**, 4, 60–90. [Google Scholar] [CrossRef] - Akhavan, S.; Abedi-Koupai, J.; Mousavi, S.F.; Afyuni, M.; Eslamian, S.S.; Abbaspour, K.C. Application of SWAT model to investigate nitrate leaching in Hamadan–Bahar watershed, Iran. Agric. Ecosyst. Environ.
**2010**, 139, 675–688. [Google Scholar] [CrossRef] - Thiemig, V.; Rojas, R.; Zambrano-Bigiarini, M.; De Roo, A. Hydrological evaluation of satellite-based rainfall estimates over the Volta and Baro-Akobo Basin. J. Hydrol.
**2013**, 499, 324–338. [Google Scholar] [CrossRef] - Rouholahnejad, E.; Abbaspour, K.C.; Vejdani, M.; Srinivasan, R.; Schulin, R.; Lehmann, A. A parallelization framework for calibration of hydrological models. Environ. Model. Softw.
**2012**, 31, 28–36. [Google Scholar] [CrossRef]

**Figure 1.**Location of the study areas: (

**a**) Salman Dam Basin (SDB) and (

**b**) Karkheh River Basin (KRB).

**Figure 2.**Calibration (1990–2008) and validation (1977–1988) results of monthly simulated discharges showing performance of the best (NSE) and the worst (PBIAS) objective functions for the T.Karzin station in Salman Dam Basin (SDB).

**Figure 3.**Uncertainty ranges of calibrated parameters using different objective functions in (Top) SDB and (bottom) KRB. The points in each line show the best value of parameters, r_ refers to a relative change where the current values are multiplied by (one plus a factor from the given parameter range), and v_ refers to the substitution by a value from the given parameter range [24]).

**Figure 4.**Uncertainty ranges of annual average (

**a**) actual evapotranspiration (mm·year

^{−1}); (

**b**) soil water (mm·year

^{−1}); and (

**c**) water yield (mm·year

^{−1}) derived by different objective functions in Salman Dam Basin (SDB).

**Figure 5.**Distribution of annual average (top) actual evapotranspiration (mm·year

^{−1}), (middle) soil water (mm·year

^{−1}), and water yield (mm·year

^{−1}) modeled by SWAT using different objective functions in calibration in Salman Dam Basin (SDB).

**Figure 6.**Calibration (1990–2008) and validation (1977–1989) results of the monthly simulated discharges using the three optimization algorithms (SUFI-2, GLUE, and PSO), with NSE as the objective function in the T.Karzin station in Salman Dam Basin (SDB).

**Figure 7.**Uncertainty ranges of the parameters based on all three methods applied in Salman Dam Basin (SDB). The points in each line show the best value of the parameters, r_ refers to a relative change where the current values are multiplied by one plus a factor from the given parameter range, and v_ refers to the substitution by a value from the given parameter range [24]).

Objective Function | Reference | Formulation * |
---|---|---|

Modified Coefficient of determination (bR^{2}) | [25] | $b{R}^{2}=\{\begin{array}{c}|b|.{R}^{2}\mathrm{for}b\le 1\\ {|b|}^{-1}.{R}^{2}\mathrm{for}b1\end{array}$ |

Coefficient of determination (R^{2}) | - | ${R}^{2}=\frac{{[{\sum}_{\mathrm{i}}({\mathrm{Q}}_{\mathrm{i},\mathrm{o}}-{\overline{\mathrm{Q}}}_{\mathrm{o}})({\mathrm{Q}}_{\mathrm{i},\mathrm{s}}-{\overline{\mathrm{Q}}}_{\mathrm{s}})]}^{2}}{{\sum}_{\mathrm{i}}{({\mathrm{Q}}_{\mathrm{i},\mathrm{o}}-{\overline{\mathrm{Q}}}_{\mathrm{o}})}^{2}{\sum}_{\mathrm{i}}{({\mathrm{Q}}_{\mathrm{i},\mathrm{s}}-{\overline{\mathrm{Q}}}_{\mathrm{s}})}^{2}}$ |

Nash-Sutcliffe efficiency (NSE) | [14] | $NSE=1-\left[\frac{{\sum}_{\mathrm{i}=1}^{\mathrm{n}}{({\mathrm{Q}}_{\mathrm{i},\mathrm{o}}-{\mathrm{Q}}_{\mathrm{i},\mathrm{s}})}^{2}}{{\sum}_{\mathrm{i}=1}^{\mathrm{n}}{({\mathrm{Q}}_{\mathrm{i},\mathrm{o}}-{\overline{Q}}_{o})}^{2}}\right]$ |

Modified Nash-Sutcliffe efficiency (MNS) | [25] | $MNS=1-\frac{{\sum}_{\mathrm{i}=1}^{\mathrm{n}}{|{\mathrm{Q}}_{\mathrm{i},\mathrm{o}}-{\mathrm{Q}}_{\mathrm{i},\mathrm{s}}|}^{\mathrm{j}}}{{\sum}_{\mathrm{i}=1}^{\mathrm{n}}{|{\mathrm{Q}}_{\mathrm{i},\mathrm{o}}-\overline{{\mathrm{Q}}_{\mathrm{o}}}|}^{\mathrm{j}}}\mathrm{with}\mathrm{j}\in \mathrm{N}$ |

Ratio of standard deviation of observations to root mean square error (RSR) | [15] | $RSR=\frac{RMSE}{STDE{V}_{\mathrm{o}}}=\frac{\left[\sqrt{{\sum}_{\mathrm{i}=1}^{\mathrm{n}}{({\mathrm{Q}}_{\mathrm{i},\mathrm{o}}-{\mathrm{Q}}_{\mathrm{i},\mathrm{s}})}^{2}}\right]}{\left[\sqrt{{\sum}_{\mathrm{i}=1}^{\mathrm{n}}{({\mathrm{Q}}_{\mathrm{i},\mathrm{o}}-{\mathrm{Q}}_{\mathrm{m}})}^{2}}\right]}$ |

Ranked sum of squares (SSQR) | [26] | $SSQR=\frac{1}{\mathrm{n}}{\sum}_{\mathrm{i}=1}^{\mathrm{n}}{[{\mathrm{Q}}_{\mathrm{i},\mathrm{o}}-{\mathrm{Q}}_{\mathrm{i},\mathrm{s}}]}^{2}$ |

Kling-Gupta efficiency (KGE) | [16] | $KGE=1-\sqrt{{(R-1)}^{2}+{(\mathsf{\alpha}-1)}^{2}+{(\mathsf{\beta}-1)}^{2}}$ |

Percent bias (PBIAS) | [27] | $PBIAS=100\ast \left[\frac{{\sum}_{\mathrm{i}=1}^{\mathrm{n}}({\mathrm{Q}}_{\mathrm{i},\mathrm{o}}-{\mathrm{Q}}_{\mathrm{i},\mathrm{s}})}{{\sum}_{\mathrm{i}=1}^{\mathrm{n}}{\mathrm{Q}}_{\mathrm{i},\mathrm{o}}}\right]$ |

_{i,o}and Q

_{i,s}are the ith observed and simulated values, respectively; ${\overline{\mathrm{Q}}}_{\mathrm{o}},{\overline{\mathrm{Q}}}_{\mathrm{s}}$ are the mean observed and simulated values, respectively; n is total number of observations; α = σ

_{s}/σ

_{m}and β = µ

_{s}/µ

_{m}where σ

_{m}and σ

_{s}are the standard deviation of the observed and simulated data, respectively, and μ

_{m}and μ

_{s}are the mean of observed and simulated data, respectively.

Data Type | Source |
---|---|

Digital Elevation Maps (DEM) (resolution 90 m) | The Shuttle Radar Topography Mission (SRTM by NASA) [32] |

Soil data (resolution 10 km) | The Food and Agriculture Organization of the United Nations [33] |

Landuse data | Satellite images (IRS-P6 LISS-IV and IRS-P5-Pan satellite images, ETM + 2001 Landsat) |

Weather data (minimum and maximum daily air temperature and daily precipitation) | Iranian ministry of Energy, The Iranian Meteorological Organization, and WFDEI_CRU data (0.5° × 0.5°) |

River discharge | Iranian ministry of Energy |

Digital river network and geological position of reservoirs and dams | Iranian ministry of Energy |

Management information of Karkheh reservoir operation (i.e., minimum and maximum daily outflow, reservoir surface area, and spillway conditions) | Iranian ministry of Energy |

**Table 3.**Initial ranges and descriptions of the parameters used for calibrating the SWAT models in the Salman Dam Basin (SDB) and Karkheh River Basin (KRB).

Parameter | Description | Parameter Range | |||
---|---|---|---|---|---|

SDB | KRB | ||||

min | max | min | max | ||

* r_CN2.mgt | SCS runoff curve number | −0.2 | 0.2 | −0.3 | 0.3 |

** v_CH_N2.rte | Manning’s “n” value for main channel | 0 | 0.3 | 0 | 0.3 |

v_ALPHA_BF.gw | Baseflow alpha factor (days) | 0 | 0.3 | 0 | 1 |

r_SOL_BD.sol | Moist bulk density | −0.5 | 0.5 | −0.5 | 0.5 |

v_GW_DELAY.gw | Groundwater delay (days) | 30 | 450 | 0 | 500 |

v_SMFMX.bsn | Max. melt rate for snow during year | 0 | 10 | 0 | 20 |

v_SFTMP.bsn | Snowfall temperature | −5 | 5 | ||

v_SMTMP.bsn | Snow melt base temperature | −5 | 5 | ||

v_SMFMN.bsn | Minimum melt rate for snow during year | 0 | 10 | ||

v_TIMP.bsn | Snow pack temperature lag factor | 0 | 1 | ||

v_ESCO.hru | Soil evaporation compensation factor | 0.7 | 1 | ||

v_CH_K2.rte | Effective hydraulic conductivity in channel | 5 | 130 | ||

r_SOL_AWC.sol | Available water capacity | −0.4 | 0.4 | ||

r_SOL_K.sol | Saturated hydraulic conductivity | −0.8 | 0.8 | ||

v_ALPHA_BNK.rte | Baseflow alpha factor for bank storage | 0 | 1 | ||

v_GWQMN.gw | Threshold depth of water in the shallow aquifer | 0 | 5000 | ||

r_OV_N.hru | Manning’s “n” value for overland flow | −1 | 1 | ||

v_GW_REVAP.gw | Groundwater “revap” coefficient | 0 | 0.2 |

Performance Rating | R^{2} | NSE | RSR | PBIAS | KGE |
---|---|---|---|---|---|

Very good | 0.75 < R^{2} ≤ 1 | 0.75 < NSE ≤ 1 | 0 ≤ RSR ≤ 0.5 | PBIAS < ±10 | 0.9 ≤ KGE ≤ 1 |

Good | 0.65 < R^{2} ≤ 0.75 | 0.65 < NSE ≤ 0.75 | 0.5 < RSR ≤ 0.6 | ±10 ≤ PBIAS < ±15 | 0.75 ≤ KGE < 0.9 |

Satisfactory | 0.5 < R^{2} ≤ 0.65 | 0.5 < NSE ≤ 0.65 | 0.6 < RSR ≤ 0.7 | ±15 ≤ PBIAS < ±25 | 0.5 ≤ KGE < 0.75 |

Unsatisfactory | R^{2} ≤ 0.5 | NSE ≤ 0.5 | RSR > 0.7 | PBIAS ≥ ±25 | KGE < 0.5 |

**Table 5.**Calibration and validation (in parentheses) results by eight different objective functions using the SUFI-2 optimization algorithm.

Station | bR^{2} | R^{2} | NSE | MNS | RSR | SSQR | KGE | PBIAS |
---|---|---|---|---|---|---|---|---|

- | Salman Dam Basin (SDB) | |||||||

B.Bahman | 0.57 (0.52) | 0.64 (0.57) | 0.57 (0.48) | 0.46 (0.38) | 0.66 (0.7) | 6.3 (3) | 0.76 (0.7) | 37.8 (34.6) |

Ali abad | 0.62 (0.56) | 0.79 (0.71) | 0.65 (0.7) | 0.53 (0.5) | 0.6 (0.53) | 11 (3.8) | 0.76 (0.75) | 9.6 (−19.4) |

Barak | 0.57 (0.15) | 0.67 (0.36) | 0.64 (0.14) | 0.41 (0.13) | 0.61 (0.88) | 0.93 (2.1) | 0.65 (0.05) | −6.9 (−5.5) |

T.karzin | 0.62 (0.61) | 0.76 (0.61) | 0.74 (0.57) | 0.53 (0.42) | 0.52 (0.65) | 13 (17) | 0.84 (0.62) | −40.5 (−9) |

- | Karkheh River Basin (KRB) | |||||||

Aran | 0.51 (0.57) | 0.61 (0.57) | 0.73 (0.51) | 0.49 (0.49) | 0.81 (0.7) | 8.1 (11) | 0.48 (0.49) | −43.00 (58.5) |

Polchehr | 0.59 (0.54) | 0.62 (0.45) | 0.55 (0.5) | 0.47 (0.42) | 0.66 (0.76) | 38 (110) | 0.75 (0.74) | −7.30 (−9.5) |

Ghorbaghestan | 0.68 (0.71) | 0.69 (0.71) | 0.67 (0.66) | 0.53 (0.49) | 0.56 (0.6) | 14 (85) | 0.82 (0.72) | 4.70 (10.6) |

Haleilan | 0.66 (0.69) | 0.71 (0.58) | 0.65 (0.62) | 0.52 (0.5) | 0.58 (0.64) | 170 (130) | 0.79 (0.8) | 3.10 (0.2) |

Tang saz | 0.65 (0.73) | 0.72 (0.69) | 0.66 (0.54) | 0.53 (0.45) | 0.58 (0.66) | 250 (240) | 0.82 (0.8) | −1.40 (−4.5) |

Afarineh | 0.51 (0.37) | 0.67 (0.54) | 0.56 (0.42) | 0.41 (0.49) | 0.65 (0.78) | 180 (740) | 0.67 (0.48) | 22.20 (32.4) |

Jelogir | 0.67 (0.67) | 0.71 (0.62) | 0.66 (0.59) | 0.50 (0.5) | 0.57 (0.64) | 480 (980) | 0.83 (0.81) | 4.80 (7.2) |

Payepol | 0.39 (0.56) | 0.43 (0.6) | 0.13 (0.27) | 0.16 (0.4) | 0.94 (0.85) | 730 (4400) | 0.56 (0.67) | 14.30 (14.6) |

**Table 6.**Correlation coefficients of the objective functions based on the best simulation in the calibration period using the SUFI-2 algorithm.

Case Study | Objective Function | bR^{2} | R^{2} | NSE | MNS | RSR | SSQR | KGE | PBIAS |
---|---|---|---|---|---|---|---|---|---|

Salman Dam Basin (SDB) | bR^{2} | 1.00 | 0.88 | 0.96 | 0.93 | 0.96 | 0.94 | 0.95 | 0.58 |

R^{2} | - | 1.00 | 0.93 | 0.93 | 0.95 | 0.83 | 0.87 | 0.61 | |

NSE | - | - | 1.00 | 0.98 | 0.98 | 0.96 | 0.98 | 0.56 | |

MNS | - | - | - | 1.00 | 0.96 | 0.95 | 0.97 | 0.54 | |

RSR | - | - | - | - | 1.00 | 0.93 | 0.94 | 0.55 | |

SSQR | - | - | - | - | - | 1.00 | 0.99 | 0.46 | |

KGE | - | - | - | - | - | - | 1.00 | 0.50 | |

PBIAS | - | - | - | - | - | - | - | 1.00 | |

Karkheh River Basin (KRB) | bR^{2} | 1.00 | 0.97 | 0.95 | 0.96 | 0.96 | 0.84 | 0.97 | 0.79 |

R^{2} | - | 1.00 | 0.98 | 0.99 | 0.98 | 0.81 | 0.98 | 0.74 | |

NSE | - | - | 1.00 | 0.98 | 1.00 | 0.78 | 1.00 | 0.70 | |

MNS | - | - | - | 1.00 | 0.98 | 0.83 | 0.98 | 0.76 | |

RSR | - | - | - | - | 1.00 | 0.78 | 1.00 | 0.69 | |

SSQR | - | - | - | - | - | 1.00 | 0.79 | 0.95 | |

KGE | - | - | - | - | - | - | 1.00 | 0.71 | |

PBIAS | - | - | - | - | - | - | - | 1.00 |

**Table 7.**Results of Tukey’s post-hoc test to determine if parameters obtained by different objective functions were statistically different or similar.

Case Study | Parameter | bR^{2} | R^{2} | NSE | MNS | RSR | SSQR | KGE | PBIAS |
---|---|---|---|---|---|---|---|---|---|

Salman Dam Basin (SDB) | r_CN2.mgt (B.Bahman) | - | - | A1 | - | - | A1 | A1 | - |

r_CN2.mgt (Ali abad) | B1 | - | B2 | - | B2 | B3 | B1 | B3 | |

r_CN2.mgt (barak) | - | - | C1 | C1 | - | - | C2 | C2 | |

r_CN2.mgt (T.Karzin ) | - | - | D1 | D1 | - | - | D1 | - | |

v_ESCO.hru | - | - | - | - | - | E1 | E1 | - | |

v_ALPHA_BNK.rte | F1 | F2 | - | - | F1 | F3 | F3 | F2 | |

r_SOL_BD.sol | - | - | - | G1 | G1 | - | - | - | |

v_GW_DELAY.gw | - | - | H1 | - | - | - | - | H1 | |

v_CH_K2.rte | - | I1 | I2 | I1 | I2 | - | I2 | - | |

v_CH_N2.rte | J1 | J2 | J2 | - | J2 | - | - | J1 | |

r_SOL_AWC.sol | - | - | K1 | - | K1 | - | - | - | |

r_SOL_K.sol | - | - | - | L1 | L2 | L2 | L1 | l1 | |

v_ALPHA_BF.gw | - | - | M1 | - | - | - | - | M1 | |

Karkheh River Basin (KRB) | r_CN2.mgt | - | A1 | A2 | - | A2 | - | A1 | - |

v_CH_N2.rte | B1 | B1 | B2 | - | B2 | - | - | - | |

v_ALPHA_BF.gw | C1 | C1 | C2 | C3 | C2 | - | C3 | C3 | |

r_SOL_BD.sol | - | - | D1 | - | D1 | - | - | D1 | |

v_GW_REVAP.gw | - | - | E1 | - | E1 | E1 | - | - | |

v_GWQMN.gw | - | F1 | F2 | F1 | F2 | - | F2 | - | |

v_GW_DELAY.gw | - | - | G1 | G1 | G1 | - | - | - | |

r_OV_N.hru | H1 | - | H2 | - | H2 | H1 | H2 | - |

**Table 8.**Performance of the optimization algorithms and the number of behavioral parameter ranges for the calibration and/validation periods in Salman Dam Basin (SDB) and Karkheh River Basin (KRB).

Case Study | Performance | SUFI-2 | GLUE | PSO | |||
---|---|---|---|---|---|---|---|

Cal. | Val. | Cal. | Val. | Cal. | Val. | ||

SDB | P-factor | 0.84 | 0.88 | 0.65 | 0.68 | 0.57 | 0.58 |

d-factor | 1.22 | 1.83 | 0.93 | 1.44 | 0.61 | 0.88 | |

Best NSE value | 0.65 | 0.47 | 0.7 | 0.45 | 0.62 | 0.45 | |

No. of behavioral parameter sets | 214/480 | - | 283/1440 | - | 477/1440 | - | |

KRB | P-factor | 0.55 | 0.6 | 0.15 | - | - | - |

d-factor | 0.67 | 0.78 | 0.22 | - | - | - | |

Best NSE value | 0.53 | 0.51 | 0.5 | - | 0.47 | - | |

No. of behavioral parameter sets | 103/480 | - | 3/2400 | - | 0/2400 | - |

**Table 9.**Correlation coefficient among the best simulation of discharges obtained by all optimization techniques in all stations at Salman Dam Basin (SDB).

Optimization Technique | SUFI-2 | GLUE | PSO |
---|---|---|---|

SUFI-2 | 1.00 | 0.99 | 0.98 |

GLUE | - | 1.00 | 0.98 |

PSO | - | - | 1.00 |

© 2017 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Kouchi, D.H.; Esmaili, K.; Faridhosseini, A.; Sanaeinejad, S.H.; Khalili, D.; Abbaspour, K.C.
Sensitivity of Calibrated Parameters and Water Resource Estimates on Different Objective Functions and Optimization Algorithms. *Water* **2017**, *9*, 384.
https://doi.org/10.3390/w9060384

**AMA Style**

Kouchi DH, Esmaili K, Faridhosseini A, Sanaeinejad SH, Khalili D, Abbaspour KC.
Sensitivity of Calibrated Parameters and Water Resource Estimates on Different Objective Functions and Optimization Algorithms. *Water*. 2017; 9(6):384.
https://doi.org/10.3390/w9060384

**Chicago/Turabian Style**

Kouchi, Delaram Houshmand, Kazem Esmaili, Alireza Faridhosseini, Seyed Hossein Sanaeinejad, Davar Khalili, and Karim C. Abbaspour.
2017. "Sensitivity of Calibrated Parameters and Water Resource Estimates on Different Objective Functions and Optimization Algorithms" *Water* 9, no. 6: 384.
https://doi.org/10.3390/w9060384