# Evaluation of Different Objective Functions Used in the SUFI-2 Calibration Process of SWAT-CUP on Water Balance Analysis: A Case Study of the Pursat River Basin, Cambodia

^{1}

^{2}

^{3}

^{4}

^{5}

^{6}

^{*}

## Abstract

**:**

## 1. Introduction

^{2}), modified coefficient of determination (bR

^{2}), Nash–Sutcliffe efficiency (NSE), modified Nash–Sutcliffe efficiency (MNS), ratio of standard deviation of observations to root mean square error (RSR), ranked sum of squares (SSQR), Kling–Gupta efficiency (KGE), and percent bias (PBIAS). We also aimed to identify a proper objective function which produced reasonable calibration results, calibrated parameter sets, and water resources estimations which reflected the characteristics of this river basin.

## 2. Study Area

^{2}[21] (Figure 1a). The river flows for approximately 150 km from the Southwest at the Cardamom Mountains to the Northeast direction until the Tonle Sap Lake with a basin elevation ranging between 6 and 1717 m above sea level. There is an operating hydrological station named Bak Trakuon with a drainage area of 4245 km

^{2}[22] where this study focused on (Figure 1b). More than 75% of the river basin encompasses a hilly terrain, with an elevation greater than 30 m above sea level and is covered by forested land of varying density, while the remaining low-lying land is occupied by agriculture [23]. Those over 75% of the forest cover are mainly concentrated in Bak Trakuon drainage boundary (Figure 1b) where the major soil types are Dystric Leptosols and Dystric Leptosols/Dystric Cambisols (LPd and LPd/CMd) and Gleyic Acrisols/Plinthic Acrisols (ACg/ACp) [22]. The climate in Pursat River Basin is influenced by tropic monsoon systems with distinct wet and dry seasons. The wet season, which extends from May to November, receives approximately 90% of the total annual rainfall. The dry season, which extends from December to April, is characterized by the prevalence of hot and dry air [22]. The rainfall in the area increases with elevation, but the annual totals vary considerably from year to year with the average ranging from 1200 to 1700 mm [24]. Daily maximum temperatures vary between 36 °C during the hottest months (April–May) and 32 °C during the coldest months (December–January). Daily minimum temperatures vary between 25 and 17 °C. The annual average temperature is approximately 28 °C. The monthly mean relative humidity ranges from 66% in the dry season to 71% in the wet season with a mean annual humidity of 70% [22].

## 3. Materials and Methods

#### 3.1. SWAT Input Datasets

#### 3.2. SWAT Model Setup

#### 3.3. SWAT-CUP with the SUFI-2 Algorithm

#### 3.3.1. Parameterization

#### 3.3.2. Objective Functions

^{2}), modified coefficient of determination (bR

^{2}), Nash–Sutcliffe efficiency (NSE), modified Nash–Sutcliffe efficiency (MNS), ratio of the standard deviation of observations to the root mean square error (RSR), ranked sum of squares (SSQR), Kling–Gupta efficiency (KGE), and percent bias (PBIAS). The equations and references for these objective functions are presented in Table 3. For NSE and RSE, it is expected that their simulation results are similar as they are equivalent objective functions given that NSE = 1 − RSR

^{2}according to their equations in Table 3.

#### 3.3.3. Model Calibration, Validation, and Evaluation

^{2}and MNS were considered greater than or equal to 0.4, which was adopted from Kouchi et al. [20], who introduced measures based on the results of the studies of Muleta [37], Mehdi et al. [38], and Akhavan et al. [39]. For the SSQR, a satisfactory value could not be specified because the measured and simulated variables were independently ranked, and the value depends on the magnitude of the variables being investigated. Satisfactory values of the other indices were based on Moriasi et al. [32] and Thiemig et al. [40].

#### 3.4. Evaluation of Each Objective Function

^{2}) and root mean squared deviation (RMSD) were used for the evaluation. The R

^{2}, which was calculated as shown in Table 3, determines how much variance the two variables share, and its value varies between 0 (no correlation) and 1 (perfect correlation). However, when the model systematically over- or under-predicts all the time, the R

^{2}value is still close to 1. To cope with this problem, the slope and intercept of the regression on which R

^{2}is based were taken into account. For a good agreement, the slope and intercept should be close to 1 and 0, respectively. The RMSD represents the mean deviation of the predicted values with respect to the observed values [43,44] and can be calculated as

## 4. Results

#### 4.1. Simulation Results

^{2}objective function, the calibrated model usually overestimated peak flows. When bR

^{2}was used, the peak flow estimation became even more overestimated compared to the observed data. This originated from the equations of these objective functions. Both are defined by a minimization of total errors from a linear regression model and the length from the average value, which are not direct errors of measured and estimated data and could have the effect on the calibrated parameters. According to Legates and McCabe [45], owing to the squared differences in their equations, they are oversensitive to high extreme values and insensitive to additive and proportional differences between model predictions and measured data.

#### 4.2. Model Performance

^{2}objective function failed to satisfy the PBIAS statistical index during the validation period. For the result of the parameter set by SSQR, it failed to satisfy the MNS statistical index during both calibration and validation periods. Of the worst, the result of the parameter set by the bR

^{2}objective function failed to satisfy NSE, MNS, and RSR statistical indices during the calibration period, whereas the SSQR statistical value of this objective function during the calibration period was quite high compared to that of other objective functions. For the PBIAS objective function, the result of the parameter set during the calibration period failed to satisfy the NSE, MNS, and RSR statistical indices.

## 5. Discussion

#### 5.1. Discharge Process Estimations

^{2}objective function. The result showed that the estimated water balance components and water yields of the NSE and RSR objective functions were the same.

#### 5.2. Best Parameter Sets and Sensitivity Rank

^{2}objective function, the relative value of the parameter change for CN2 was high (+13%), whereas the existing parameter value of CN2 of the initial model was (between 55 and 92) approximately 78 on average, which was already high for the land-use and soil types of this study area. This led to a big runoff (surface and lateral flow) amount, as shown in Figure 5a, and high simulated peak flows, as shown in Figure 3. This objective function produced a small threshold value of GWQMN of 1680.42 mm, which led to more groundwater flow, and a small coefficient of GW_REVAP of 0.03, which generated a lower evapotranspiration rate and revap because of the limitation of movement of water from the shallow aquifer to the root zone. Moreover, these three parameters were among the five most sensitive parameters, which highly controlled the simulation results of the R

^{2}objective function. Even worst, the bR

^{2}objective function produced a higher relative value of the parameter change for CN2 of +15%, a smaller threshold value of GWQMN of 291.86, a low coefficient of GW_REVAP of 0.07, and a large value of ESCO, which then led to a large runoff, greater groundwater flow, smaller evapotranspiration, and lower revap (Figure 5a). Consequently, this objective function highly overestimated the peak flows as shown in Figure 3. Furthermore, while the value of the initial model of SOL_AWC was small between 84 and 371 mm (approximately 186 mm on average), the fitted value of the relative change was still underestimated at +16%, and also, the fitted value of SLSUBBSN was overestimated at 68.03 m. As a result, a huge surface runoff and a neglected lateral flow occurred for this objective function. Additionally, the calibrated value of the average slope steepness (HRU_SLP) of bR

^{2}was small. However, this parameter was the least sensitive, which may not have as a considerable effect as the earlier mentioned six parameters (they were the top six sensitive parameters).

#### 5.3. Hydrograph Components Estimation

^{2}values, better slope (optimum value of 1) and intercept (optimum value of 0), and smaller RMSD value. The simulated base flows of the MNS objective function achieved an R

^{2}of 0.45, slope of 0.48, intercept of 3.59, and RMSD of 7.78, whereas the simulation results of the NSE objective function provided a value of R

^{2}of 0.48, slope of 0.41, intercept of 2.84, and RMSD of 10.76.

^{2}of 0.21 at most and large intercepts; however, the correlation slopes of some of them reached a value that was greater than 0.6. Again, NSE objective function provided the closest simulation results (Figure 9) with R

^{2}, slope, intercept, and RMSD values of 0.21, 0.67, 35.16, and 63.97, respectively.

^{2}, R

^{2}, and KGE objective functions attained slightly higher R

^{2}values compared to the results of the other objective functions at slightly larger than 0.50, whereas those of the remaining objective functions were a little less than 0.50. However, the regression slopes of the results obtained from the NSE and MNS objective functions reached a satisfactory value of above 0.70, and those obtained from the other objective functions were between 0.54 and 0.64. The intercepts obtained from all objective functions were between 76.82 and 102.54, and the RMSD values were between 106.67 and 127.61.

^{2}of 0.79 with a good slope of 1.01, a small intercept of −9.62, and a RMSD of 37.38. The NSE and KGE objective functions performed similarly but were lower with an R

^{2}of approximately 0.78 and RMSD of approximately 41. However, the regression slope and intercept obtained from the NSE objective function were at 0.97 and −14.46, respectively, whereas those obtained from KGE were 0.90 and −5.52, respectively.

^{2}, bR

^{2}, SSQR, and PBIAS were among the more poor objective functions, especially for the simulation during low flow periods. For NSE, the differences between the observed and predicted values were calculated as squared values. As a result, larger values in a time series are strongly overestimated, whereas lower values are neglected [45]. Additionally, runoff peaks will tend to be underestimated when NSE is used in the optimization [34]. However, NSE is good for use with continuous long-term simulations and can be used to determine how well a model simulates trends for the output response of concern [46,50]. Because the calibration duration of this study was 14 years, it is likely that the NSE objective function could capture this long-term trend of the discharge. For the MNS objective function, which was the modified form of NSE with the modified factor of p = 1 used in this study, it can be expected that the modified forms are more sensitive to significant over- or under-prediction than the squared forms [30]. However, in this study, the performance of the MNS objective function was only slightly better than the performance of the NSE objective function when we simulated the falling limbs, but it always slightly performed worse than the NSE objective function when we simulated the other components of the hydrograph.

^{2}objective function is widely used in hydrological modeling studies, but it is oversensitive to high extreme values and insensitive to additive and proportional differences between model predictions and measured data [45]. For the bR

^{2}objective function, the under- or over-predictions are quantified together with the dynamics, which results in a more comprehensive reflection of model results [30]. The SSQR objective function aims at fitting the distribution of the flows, ensuring that the full range of the flows is represented but without considering the time of occurrence of a given value of the flows [33]. Perhaps due this characteristic, the estimated recession curves (falling limbs) were typically flatter than the observed data, and the estimated base flows were shortened (Figure 3). As a result, the simulation performances of the falling limbs (Figure 11) and base flows (Figure 8) of this objective function were relatively low. For the PBIAS objective function, it is useful for continuous long-term simulations and can be used to determine how well the model simulates the average magnitudes for the output response of interest [46]. PBIAS can provide a deceptive rating of model performance when the model over-predicts as much as it under-predicts, in which case PBIAS will be close to zero even though the model simulation is poor [46]. This may be why there were several sudden peak and drop points of the simulated result of this objective function during the base flow and rising limb periods (Figure 3), leading to poor model performances in simulating the base flows and rising limbs, as shown in Figure 6 and Figure 7, respectively.

#### 5.4. Objective Functions Corresponding to the Characteristics of the River Basin

## 6. Conclusions

## Author Contributions

## Funding

## Acknowledgments

## Conflicts of Interest

## References

- Gosain, A.K.; Rao, S.; Basuray, D. Climate change impact assessment on hydrology of Indian river basins. Curr. Sci.
**2006**, 90, 9. [Google Scholar] - Rostamian, R.; Jaleh, A.; Afyuni, M.; Mousavi, S.F.; Heidarpour, M.; Jalalian, A.; Abbaspour, K.C. Application of a SWAT model for estimating runoff and sediment in two mountainous basins in central Iran. Hydrol. Sci. J.
**2008**, 53, 977–988. [Google Scholar] [CrossRef] - Huang, T.; Lo, K. Effects of Land Use Change on Sediment and Water Yields in Yang Ming Shan National Park, Taiwan. Environments
**2015**, 2, 32–42. [Google Scholar] [CrossRef] [Green Version] - Näschen, K.; Diekkrüger, B.; Evers, M.; Höllermann, B.; Steinbach, S.; Thonfeld, F. The Impact of Land Use/Land Cover Change (LULCC) on Water Resources in a Tropical Catchment in Tanzania under Different Climate Change Scenarios. Sustainability
**2019**, 11, 7083. [Google Scholar] [CrossRef] [Green Version] - Cambien, N.; Gobeyn, S.; Nolivos, I.; Forio, M.A.E.; Arias-Hidalgo, M.; Dominguez-Granda, L.; Witing, F.; Volk, M.; Goethals, P.L.M. Using the Soil and Water Assessment Tool to Simulate the Pesticide Dynamics in the Data Scarce Guayas River Basin, Ecuador. Water
**2020**, 12, 696. [Google Scholar] [CrossRef] [Green Version] - Arnold, J.G.; Srinivasan, R.; Muttiah, R.S.; Williams, J.R. Large area hydrologic modeling and assessment, Part I: Model development. J. Am. Water Resour. Assoc.
**1998**, 34, 73–89. [Google Scholar] [CrossRef] - Arnold, J.G.; Moriasi, D.N.; Gassman, P.W.; Abbaspour, K.C.; White, M.J.; Srinivasan, R.; Santhi, C.; Harmel, R.D.; Van Griensven, A.; Van Liew, M.W.; et al. SWAT: Model Use, Calibration, and Validation. Trans. ASABE
**2012**, 55, 1491–1508. [Google Scholar] [CrossRef] - Neitsch, S.L.; Arnold, J.G.; Kiniry, J.R.; Williams, J.R. Soil and Water Assessment Tool—Theoretical Documentation (Version 2005); Agricultural Research Service: Temple, TX, USA, 2005.
- Neitsch, S.L.; Arnold, J.G.; Kiniry, J.R.; Williams, J.R. Soil and Water Assessment Tool Theoretical Documentation Version 2009; Texas Water Resources Institute Technical Report No. 406; Texas A&M University System: College Station, TX, USA, 2011. [Google Scholar]
- Gassman, P.W.; Reyes, M.R.; Green, C.H.; Arnold, J.G. The Soil and Water Assessment Tool: Historical Development, Applications, and Future Research Directions. Trans. ASABE
**2007**, 50, 1211–1250. [Google Scholar] [CrossRef] [Green Version] - Abbaspour, K.C.; Rouholahnejad, E.; Vaghefi, S.; Srinivasan, R.; Yang, H.; Kløve, B. A continental-scale hydrology and water quality model for Europe: Calibration and uncertainty of a high-resolution large-scale SWAT model. J. Hydrol.
**2015**, 524, 733–752. [Google Scholar] [CrossRef] [Green Version] - Abbaspour, K.C. SWAT-CUP: SWAT Calibration and Uncertainty Programs—A User Manual; Swiss Federal Institute of Aquatic Science and Technology: Zurich, Switzerland, 2015. [Google Scholar]
- Ha, L.T.; Bastiaanssen, W.G.M.; van Griensven, A.; van Dijk, A.I.J.M.; Senay, G.B. SWAT-CUP for Calibration of Spatially Distributed Hydrological Processes and Ecosystem Services in a Vietnamese River Basin Using Remote Sensing. Hydrol. Earth Syst. Sci. Discuss.
**2017**, 1–35. [Google Scholar] [CrossRef] [Green Version] - Roth, V.; Nigussie, T.K.; Lemann, T. Model parameter transfer for streamflow and sediment loss prediction with SWAT in a tropical watershed. Environ. Earth Sci.
**2016**, 75, 1321. [Google Scholar] [CrossRef] [Green Version] - Gholami, A.; Habibnejad Roshan, M.; Shahedi, K.; Vafakhah, M.; Solaymani, K. Hydrological stream flow modeling in the Talar catchment (central section of the Alborz Mountains, north of Iran): Parameterization and uncertainty analysis using SWAT-CUP. J. Water Land Dev.
**2016**, 30, 57–69. [Google Scholar] [CrossRef] - Abbaspour, K.C.; Vejdani, M.; Haghighat, S. SWAT-CUP Calibration and Uncertainty Programs for SWAT; Modelling and Simulation Society of Australia and New Zealand: Christchurch, New Zealand, 2007; pp. 1596–1602. [Google Scholar]
- Khoi, D.N.; Thom, V.T. Parameter uncertainty analysis for simulating streamflow in a river catchment of Vietnam. Glob. Ecol. Conserv.
**2015**, 4, 538–548. [Google Scholar] [CrossRef] [Green Version] - Shivhare, N.; Dikshit, P.K.S.; Dwivedi, S.B. A Comparison of SWAT Model Calibration Techniques for Hydrological Modeling in the Ganga River Watershed. Engineering
**2018**, 4, 643–652. [Google Scholar] [CrossRef] - Sloboda, M.; Swayne, D. Autocalibration of Environmental Process Models Using a PAC Learning Hypothesis. In Environmental Software Systems. Frameworks of eEnvironment; IFIP Advances in Information and Communication Technology; Springer: Berlin/Heidelberg, Germany, 2011; Volume 359, pp. 528–534. ISBN 978-3-642-22284-9. [Google Scholar]
- Kouchi, D.H.; Esmaili, K.; Faridhosseini, A.; Sanaeinejad, S.H.; Khalili, D.; Abbaspour, K.C. Sensitivity of Calibrated Parameters and Water Resource Estimates on Different Objective Functions and Optimization Algorithms. Water
**2017**, 9, 384. [Google Scholar] [CrossRef] [Green Version] - Ashwell, D.; Lic, V.; Loeung, K.; Maltby, M.; McNaughton, A.; Mulligan, B.; Oum, S.; Starr, A. Baseline Assessment and Recommendations for Improved Natural Resources Management and Biodiversity Conservation in the Tonle Sap Basin, Cambodia; Fauna & Flora International: Phnom Penh, Cambodia, 2011; 92p. [Google Scholar]
- Cambodia National Mekong Committee (CNMC). Basin Development Plan Program (Ed.): Profile of the Tonle Sap Sub-Area (SA-9C); CNMC: Phnom Penh, Cambodia, 2012. [Google Scholar]
- Japan International Cooperation Agency (JICA). Report on Examination of Impact of New Dam Plans on the West Tonle Sap Irrigation Rehabilitation Project in Pursat River Basin; Ministry of Water Resources and Meteorology (MOWRAM): Phnom Penh, Cambodia, 2011. [Google Scholar]
- Japan International Cooperation Agency (JICA). Brief Progress Report on the Water Balance Examination Study for Pursat and Baribor River Basins; Ministry of Water Resources and Meteorology (MOWRAM): Phnom Penh, Cambodia, 2013. [Google Scholar]
- Abbaspour, K.C.; van Genuchten, M.T.; Schulin, R.; Schläppi, E. A sequential uncertainty domain inverse procedure for estimating subsurface flow and transport parameters. Water Resour. Res.
**1997**, 33, 1879–1892. [Google Scholar] [CrossRef] [Green Version] - Abbaspour, K.C.; Johnson, C.A.; Van Genuchten, M.Th. Estimating Uncertain Flow and Transport Parameters Using a Sequential Uncertainty Fitting Procedure. Vadose Zone J.
**2004**, 3, 1340–1352. [Google Scholar] [CrossRef] - Rossi, C.G.; Srinivasan, R.; Jirayoot, K.; Duc, T.L.; Souvannabouth, P.; Binh, N.; Gassman, P.W. Hydrologic Evaluation of the Lower Mekong River Basin with the Soil and Water Assessment Tool Model. Int. Agric. Eng. J.
**2009**, 18, 1–13. [Google Scholar] - Ly, S.; Oeurng, C. Climate change and water governance in Stung Chrey Bak Cathment of Tonle Sap Great Lake Basin in Cambodia. In Proceedings of the 15th Annual Conference of the Science Council of Asia (SCA), Siem Reap, Cambodia, 17–19 May 2015; pp. 134–138. [Google Scholar]
- Ang, R.; Oeurng, C. Simulating streamflow in an ungauged catchment of Tonlesap Lake Basin in Cambodia using Soil and Water Assessment Tool (SWAT) model. Water Sci.
**2018**, 32, 89–101. [Google Scholar] [CrossRef] [Green Version] - Krause, P.; Boyle, D.P.; Bäse, F. Comparison of different efficiency criteria for hydrological model assessment. Adv. Geosci.
**2005**, 5, 89–97. [Google Scholar] [CrossRef] [Green Version] - Nash, J.E.; Sutcliffe, J.V. River flow forecasting through conceptual models part I—A discussion of principles. J. Hydrol.
**1970**, 10, 282–290. [Google Scholar] [CrossRef] - Moriasi, D.N.; Arnold, J.G.; Liew, M.W.V.; Bingner, R.L.; Harmel, R.D.; Veith, T.L. Model Evaluation Guidelines for Systematic Quantification of Accuracy in Watershed Simulations. Trans. ASABE
**2007**, 50, 885–900. [Google Scholar] [CrossRef] - Van Griensven, A.; Bauwens, W. Multiobjective autocalibration for semidistributed water quality models: Autocalibrations for water quality models. Water Resour. Res.
**2003**, 39, 1348. [Google Scholar] [CrossRef] - Gupta, H.V.; Kling, H.; Yilmaz, K.K.; Martinez, G.F. Decomposition of the mean squared error and NSE performance criteria: Implications for improving hydrological modelling. J. Hydrol.
**2009**, 377, 80–91. [Google Scholar] [CrossRef] [Green Version] - Yapo, P.O.; Gupta, H.V.; Sorooshian, S. Automatic calibration of conceptual rainfall-runoff models: Sensitivity to calibration data. J. Hydrol.
**1996**, 181, 23–48. [Google Scholar] [CrossRef] - James, L.D.; Burges, S.J. Selection, Calibration, and Testing of Hydrologic Models. In Hydrologic Modeling of Small Watersheds; American Society of Agricultural Engineers: St. Joseph, MI, USA, 1982; pp. 437–470. [Google Scholar]
- Muleta, M.K. Model Performance Sensitivity to Objective Function during Automated Calibrations. J. Hydrol. Eng.
**2012**, 17, 756–767. [Google Scholar] [CrossRef] - Mehdi, B.; Ludwig, R.; Lehner, B. Evaluating the impacts of climate change and crop land use change on streamflow, nitrates and phosphorus: A modeling study in Bavaria. J. Hydrol. Reg. Stud.
**2015**, 4, 60–90. [Google Scholar] [CrossRef] [Green Version] - Akhavan, S.; Abedi-Koupai, J.; Mousavi, S.-F.; Afyuni, M.; Eslamian, S.-S.; Abbaspour, K.C. Application of SWAT model to investigate nitrate leaching in Hamadan–Bahar Watershed, Iran. Agric. Ecosyst. Environ.
**2010**, 139, 675–688. [Google Scholar] [CrossRef] - Thiemig, V.; Rojas, R.; Zambrano-Bigiarini, M.; De Roo, A. Hydrological evaluation of satellite-based rainfall estimates over the Volta and Baro-Akobo Basin. J. Hydrol.
**2013**, 499, 324–338. [Google Scholar] [CrossRef] - Running, S.; Mu, Q.; Zhao, M. MOD16A2 MODIS/Terra Net Evapotranspiration 8-Day L4 Global 500m SIN Grid V006 [Data Set]; NASA EOSDIS Land Processes DAAC: Sioux Falls, SD, USA, 2017.
- Busetto, L.; Ranghetti, L. MODIStsp: An R package for automatic preprocessing of MODIS Land Products time series. Comput. Geosci.
**2016**, 97, 40–48. [Google Scholar] [CrossRef] [Green Version] - Kobayashi, K.; Salam, M.U. Comparing Simulated and Measured Values Using Mean Squared Deviation and its Components. Agron. J.
**2000**, 92, 345–352. [Google Scholar] [CrossRef] - Gauch, H.G.; Hwang, J.T.G.; Fick, G.W. Model Evaluation by Comparison of Model-Based Predictions and Measured Values. Agron. J.
**2003**, 95, 1442–1446. [Google Scholar] [CrossRef] [Green Version] - Legates, D.R.; McCabe, G.J. Evaluating the use of “goodness-of-fit” Measures in hydrologic and hydroclimatic model validation. Water Resour. Res.
**1999**, 35, 233–241. [Google Scholar] [CrossRef] - Harmel, R.D.; Smith, P.K.; Migliaccio, K.W. Modifying Goodness-of-Fit Indicators to Incorporate Both Measurement and Model Uncertainty in Model Calibration and Validation. Trans. ASABE
**2010**, 53, 55–63. [Google Scholar] [CrossRef] - Shimizu, A.; Suzuki, M.; Sawano, S.; Kabeya, N.; Nobuhiro, T.; Tamai, K.; Tsuboyama, Y.; Chann, S.; Keth, N. Water Resources Observation and Large-scale Model Estimation in Forested Areas in Mekong River Basin. Jpn. Agric. Res. Q.
**2010**, 44, 179–186. [Google Scholar] [CrossRef] - Japan International Cooperation Agency (JICA). The Study on Groundwater Development in Southern Cambodia; Ministry of Rural Development: Phnom Penh, Cambodia, 2002; 309p.
- Rafiei Emam, A.; Kappas, M.; Hoang Khanh Nguyen, L.; Renchin, T. Hydrological Modeling in an Ungauged Basin of Central Vietnam Using SWAT Model. Hydrol. Earth Syst. Sci. Discuss.
**2016**, 1–33. [Google Scholar] [CrossRef] [Green Version] - Harmel, R.D.; Smith, P.K. Consideration of measurement uncertainty in the evaluation of goodness-of-fit in hydrologic and water quality modeling. J. Hydrol.
**2007**, 337, 326–336. [Google Scholar] [CrossRef] - Subramanya, K. Engineering Hydrology; Tata McGraw-Hill: New York, NY, USA, 2008; ISBN 978-0-07-133748-9. [Google Scholar]
- Waring, R.H.; Running, S.W. Forest Ecosystems: Analysis at Multiple Scales, 3rd ed.; Elsevier/Academic Press: Amsterdam, Boston, 2007; ISBN 978-0-12-370605-8. [Google Scholar]
- Chapin, F.S.; Matson, P.A.; Vitousek, P.M. Principles of Terrestrial Ecosystem Ecology; Springer: New York, NY, USA, 2011; ISBN 978-1-4419-9503-2. [Google Scholar]
- Bonan, G.B. Forests and Climate Change: Forcings, Feedbacks, and the Climate Benefits of Forests. Science
**2008**, 320, 1444–1449. [Google Scholar] [CrossRef] [Green Version] - Sun, G.; Alstad, K.; Chen, J.; Chen, S.; Ford, C.R.; Lin, G.; Liu, C.; Lu, N.; McNulty, S.G.; Miao, H.; et al. A general predictive model for estimating monthly ecosystem evapotranspiration. Ecohydrology
**2011**, 4, 245–255. [Google Scholar] [CrossRef] - Sun, G.; Lockaby, B.G. Water Quantity and Quality at the Urban-Rural Interface. In Urban-Rural Interfaces; American Society of Agronomy, Soil Science Society of America, Crop Science Society of America, Inc.: Madison, WI, USA, 2012; pp. 29–48. ISBN 978-0-89118-616-8. [Google Scholar]

**Figure 1.**(

**a**) Location of the Pursat River Basin and (

**b**) land-use map in 2003 with monitoring stations.

**Figure 3.**Monthly simulated and validated discharges of different objective functions compared with observed data.

**Figure 5.**(

**a**) Results of the water balance components and (

**b**) water yields of different objective functions using the best parameter sets from the calibration process.

**Figure 6.**Average monthly (

**left**) and annual evapotranspiration (

**right**) in Pursat River basin (drainage area at Bak Trakuon outlet) from 2001 to 2015 obtained from MODIS ET.

**Figure 7.**Linear regression between monthly average evapotranspiration obtained from the simulation results (ET Sim.) of different objective functions and monthly average evapotranspiration obtained from MODIS ET.

**Figure 11.**Scatter plots of the simulated versus observed falling limbs for different objective functions.

**Figure 12.**Monthly average hydrograph at Bak Trakuon Station during the calibration period from 1995 to 2008 (excluding 1997 and 1998 when rainfall data were missing).

Data | Description | Year/Period | Source |
---|---|---|---|

Digital Elevation Model (DEM) | Shuttle Radar Topography Mission (SRTM) Global: Raster resolution of 30 m | - | OpenTopography (https://www.opentopography.org/) |

Land-use map | Raster resolution of 250 m | 2003 | Cambodia National Mekong Committee |

Soil data | Raster resolution of 250 m | - | Cambodia National Mekong Committee |

Weather data | Daily rainfall at Kravanh station | 1994–2015 | Pursat Provincial Department of Water Resources and Meteorology |

Daily maximum and minimum temperature at Pursat station | 2001–2015 | Pursat Provincial Department of Water Resources and Meteorology | |

Hydrological data | Daily discharge at Bak Trakuon station | 1995–2015 | Pursat Provincial Department of Water Resources and Meteorology |

Parameter | Extension | Method | Description | Initial Range | |
---|---|---|---|---|---|

Min | Max | ||||

CN2 | .mgt | Relative | SCS runoff curve number | −0.25 | 0.25 |

SOL_AWC () | .sol | Relative | Available water capacity | −0.25 | 0.25 |

ESCO | .hru | Replace | Soil evaporation compensation factor | 0.01 | 1 |

OV_N | .hru | Replace | Manning’s “n” value for overland flow | 0.01 | 30 |

HRU_SLP | .hru | Replace | Average slope steepness | 0 | 1 |

SLSUBBSN | .hru | Replace | Average slope length | 10 | 150 |

GWQMN | .gw | Replace | Threshold depth of water in the shallow aquifer | 0 | 5000 |

GW_REVAP | .gw | Replace | Groundwater “revap *” coefficient | 0.02 | 0.2 |

REVAPMN | .gw | Replace | Threshold depth of water in the shallow aquifer for “revap *” to occur | 0 | 500 |

Objective Functions | Equation | Reference |
---|---|---|

Coefficient of determination | ${\mathrm{R}}^{2}=\frac{{\left[{{\displaystyle \sum}}_{\mathrm{i}}\left({\mathrm{Q}}_{\mathrm{m},\mathrm{i}}-{\overline{\mathrm{Q}}}_{\mathrm{m}}\right)\left({\mathrm{Q}}_{\mathrm{s},\mathrm{i}}-{\overline{\mathrm{Q}}}_{\mathrm{s}}\right)\right]}^{2}}{{{\displaystyle \sum}}_{\mathrm{i}}{\left({\mathrm{Q}}_{\mathrm{m},\mathrm{i}}-{\overline{\mathrm{Q}}}_{\mathrm{m}}\right)}^{2}{{\displaystyle \sum}}_{\mathrm{i}}{\left({\mathrm{Q}}_{\mathrm{s},\mathrm{i}}-{\overline{\mathrm{Q}}}_{\mathrm{s}}\right)}^{2}}$ | [30] |

Modified coefficient of determination | ${\mathrm{bR}}^{2}=\{\begin{array}{c}\left|\mathrm{b}\right|{\mathrm{R}}^{2}\mathrm{if}\left|\mathrm{b}\right|\le 1\\ {\left|\mathrm{b}\right|}^{-1}{\mathrm{R}}^{2}\mathrm{if}\left|\mathrm{b}\right|1\end{array}$ | [30] |

Nash–Sutcliffe efficiency | $\mathrm{NSE}=1-\frac{{{\displaystyle \sum}}_{\mathrm{i}}{\left({\mathrm{Q}}_{\mathrm{m}}-{\mathrm{Q}}_{\mathrm{s}}\right)}_{\mathrm{i}}^{2}}{{{\displaystyle \sum}}_{\mathrm{i}}{\left({\mathrm{Q}}_{\mathrm{m},\mathrm{i}}-{\overline{\mathrm{Q}}}_{\mathrm{m}}\right)}^{2}}$ | [31] |

Modified Nash–Sutcliffe efficiency | $\mathrm{MNS}=1-\frac{{{\displaystyle \sum}}_{\mathrm{i}}{\left|{\mathrm{Q}}_{\mathrm{m}}-{\mathrm{Q}}_{\mathrm{s}}\right|}_{\mathrm{i}}^{\mathrm{p}}}{{{\displaystyle \sum}}_{\mathrm{i}}{\left|{\mathrm{Q}}_{\mathrm{m},\mathrm{i}}-{\overline{\mathrm{Q}}}_{\mathrm{m}}\right|}_{\mathrm{i}}^{\mathrm{p}}}$ | [30] |

Ratio of the standard deviation of observations to root mean square error | $\mathrm{RSR}=\frac{\sqrt{{{\displaystyle \sum}}_{\mathrm{i}=1}^{\mathrm{n}}{\left({\mathrm{Q}}_{\mathrm{m}}-{\mathrm{Q}}_{\mathrm{s}}\right)}_{\mathrm{i}}^{2}}}{\sqrt{{{\displaystyle \sum}}_{\mathrm{i}=1}^{\mathrm{n}}{\left({\mathrm{Q}}_{\mathrm{m},\mathrm{i}}-{\overline{\mathrm{Q}}}_{\mathrm{m}}\right)}^{2}}}$ | [32] |

Ranked sum of squares | $\mathrm{SSQR}=\frac{1}{\mathrm{n}}{\displaystyle \sum}_{\mathrm{j}=1}^{\mathrm{n}}{\left({\mathrm{Q}}_{\mathrm{j},\mathrm{m}}-{\mathrm{Q}}_{\mathrm{j},\mathrm{s}}\right)}^{2}$ | [33] |

Kling–Gupta efficiency | $\mathrm{KGE}=1-\sqrt{{\left(\mathrm{r}-1\right)}^{2}+{\left(\mathsf{\alpha}-1\right)}^{2}+{\left(\mathsf{\beta}-1\right)}^{2}}$ | [34] |

Percent bias | $\mathrm{PBIAS}=100\frac{{{\displaystyle \sum}}_{\mathrm{i}=1}^{\mathrm{n}}{\left({\mathrm{Q}}_{\mathrm{m}}-{\mathrm{Q}}_{\mathrm{s}}\right)}_{\mathrm{i}}}{{{\displaystyle \sum}}_{\mathrm{i}=1}^{\mathrm{n}}{\mathrm{Q}}_{\mathrm{m},\mathrm{i}}}$ | [35] |

Indices | R^{2} | bR^{2} | NSE | MNS | RSR | SSQR | KGE | PBIAS |
---|---|---|---|---|---|---|---|---|

Range | 0 to 1 | 0 to 1 | $-\infty $ to 1 | $-\infty $ to 1 | 0 to $\infty $ | 0 to $\infty $. | $-\infty $ to 1 | $-\infty $ to $\infty $ |

Optimal Value | 1 | 1 | 1 | 1 | 0 | 0 | 1 | 0 |

Satisfactory Value | >0.5 | ≥0.4 | >0.5 | ≥0.4 | ≤0.7 | - | ≥0.5 | <±25 |

**Table 5.**Fitted values and sensitivity ranks (values in parentheses) of the calibrated parameters obtained from different objective functions during the calibration process.

Parameter | Fitted Parameter Values and Parameter Sensitivity Ranks (Values in Parentheses) by Different Objective Functions | |||||||
---|---|---|---|---|---|---|---|---|

R^{2} | bR^{2} | NSE | MNS | RSR | SSQR | KGE | PBIAS | |

r__CN2.mgt * | 13% | 15% | 3% | −1% | 3% | 6% | 8% | 14% |

(4) | (5) | (2) | (2) | (2) | (4) | (5) | (1) | |

r__SOL_AWC().sol * | 45% | 16% | 45% | 37% | 45% | 18% | 38% | −10% |

(6) | (4) | (3) | (5) | (3) | (5) | (6) | (2) | |

v__ESCO.hru | 0.76 | 0.96 | 0.76 | 0.57 | 0.76 | 0.95 | 0.72 | 0.11 |

(3) | (1) | (4) | (4) | (4) | (3) | (4) | (3) | |

v__OV_N.hru | 14.84 | 10.82 | 22.38 | 12.37 | 22.38 | 29.61 | 22.70 | 27.39 |

(9) | (7) | (7) | (9) | (7) | (8) | (8) | (6) | |

v__HRU_SLP.hru | 0.83 | 0.22 | 0.74 | 0.91 | 0.74 | 0.96 | 0.89 | 0.81 |

(7) | (9) | (5) | (3) | (5) | (9) | (9) | (5) | |

v__SLSUBBSN.hru | 13.53 | 68.03 | 13.21 | 11.00 | 13.21 | 37.25 | 11.55 | 118.23 |

(1) | (6) | (1) | (1) | (1) | (7) | (2) | (4) | |

v__GWQMN.gw | 1680.42 | 291.86 | 2696.49 | 2381.61 | 2696.49 | 3261.89 | 1731.04 | 3006.24 |

(2) | (3) | (8) | (6) | (8) | (1) | (3) | (8) | |

v__GW_REVAP.gw | 0.03 | 0.07 | 0.11 | 0.14 | 0.11 | 0.04 | 0.07 | 0.18 |

(5) | (2) | (6) | (8) | (6) | (2) | (1) | (9) | |

v__REVAPMN.gw | 380.62 | 301.52 | 133.67 | 416.76 | 133.67 | 5.75 | 349.10 | 462.03 |

(8) | (8) | (9) | (7) | (9) | (6) | (7) | (7) |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Sao, D.; Kato, T.; Tu, L.H.; Thouk, P.; Fitriyah, A.; Oeurng, C.
Evaluation of Different Objective Functions Used in the SUFI-2 Calibration Process of SWAT-CUP on Water Balance Analysis: A Case Study of the Pursat River Basin, Cambodia. *Water* **2020**, *12*, 2901.
https://doi.org/10.3390/w12102901

**AMA Style**

Sao D, Kato T, Tu LH, Thouk P, Fitriyah A, Oeurng C.
Evaluation of Different Objective Functions Used in the SUFI-2 Calibration Process of SWAT-CUP on Water Balance Analysis: A Case Study of the Pursat River Basin, Cambodia. *Water*. 2020; 12(10):2901.
https://doi.org/10.3390/w12102901

**Chicago/Turabian Style**

Sao, Davy, Tasuku Kato, Le Hoang Tu, Panha Thouk, Atiqotun Fitriyah, and Chantha Oeurng.
2020. "Evaluation of Different Objective Functions Used in the SUFI-2 Calibration Process of SWAT-CUP on Water Balance Analysis: A Case Study of the Pursat River Basin, Cambodia" *Water* 12, no. 10: 2901.
https://doi.org/10.3390/w12102901