Assessing the Impact of CFSR and Local Climate Datasets on Hydrological Modeling Performance in the Mountainous Black Sea Catchment

Abstract

Precise representation of precipitation input is one of the predominant factors affecting the simulation of hydrological processes in catchments. Choosing the representative climate datasets is crucial to obtain accurate model results, especially in mountainous regions. Hence, this study assesses the suitability of the Climate Forecasting System Reanalysis (CFSR) and local climate data to simulate the streamflow at multiple gauges in the data-scarce mountainous Black Sea catchment. Moreover, the applicability of using the elevations band in the model is also tested. The Soil and Water Assessment Tool (SWAT) is used as a hydrological simulator. Calibration and uncertainty analysis are performed by using SWAT-CUP with the Sequential Uncertainty Fitting (SUFI-2) algorithm based on monthly streamflow data at six different hydrometric stations located at different altitudes. The results reveal that the CFSR dataset provides quite reasonable agreements between the simulated and the observed streamflow at the gauge stations compared to the local dataset. However, SWAT simulations with both datasets result in poor performance for the upstream catchments of the study area. Considering orographic precipitation by applying elevation bands to the local climate dataset using CFSR data leads also to significant improvements to the model’s performance. Model results obtained with both climate datasets result in similar objective metrics, and larger uncertainty with a coefficient variation (CV) ranging from 73% to 107%. This paper mainly highlights that (i) global climate datasets (i.e., CFSR) can be a good alternative especially for data-scarce regions, (ii) elevation band application can improve the model performance for the catchments with high elevation gradients, and iii) CFSR data can be used to determine precipitation lapse rate in data scarce-regions.

1. Introduction

Hydrological models can be valuable tools to improve our understanding of environmental system behaviors and to solve long-term water resource management problems. Precipitation is mainly considered as the most predominant driving force for simulating the hydrological processes [1,2,3], and sediment and nutrient fluxes in watersheds [4,5,6]. The data needed for hydrological modeling depend on the type of hydrological model. Distributed hydrological models mainly necessitate more data then lumped models [7]. Distributed hydrological modeling requires representative spatial and temporal data to simulate the hydrologic behavior of the watersheds [8]. In developing countries/regions especially, and particularly in mountainous areas, recorded spatial rainfall data is subject to larger uncertainties due to sparse networks of rain gauges [9]. Therefore, the availability of representative climate data is crucial for accurate water resources modeling in mountain environments [10].

To overcome the limitations of data scarcity and poor-quality observations, numerous approaches were conducted that include different interpolation techniques, radar data, and remotely sensed data to achieve satisfactory simulations [11]. With the continuous improvement of modern, powerful computing technologies, gridded observational precipitation data for global and regional domains under different temporal and spatial resolutions were developed by different institutions around the world. A large number of climate databases are available from various sources, for instance, Precipitation Estimation from Remote-Sensing Information using Artificial Neural Network (PERSIANN) [12], Tropical Rainfall Measuring Mission (TRMM) [13], Towards the Evaluation of Water Resources (APHRODITE) [14], Climate Forecasting System Reanalysis from The National Oceanic and Atmospheric Administration’s (NOAA) National Centers for Environmental Prediction (CFSR) [15], and Système d’Analyse Fournissant des Renseignements Adaptés à la Nivologie) from Météo-France, the French weather agency (SAFRAN) [16].

Several studies have compared gridded climate databases with local datasets to evaluate their suitability by using different hydrological models [17,18,19]. Among these models, the semi-distributed hydrological model, the Soil and Water Assessment Tool (SWAT) [20], has been extensively used for small-scale to continental-scale studies under different climatic conditions by using different climate databases [21,22]. The Climate Forecast System Reanalysis (CFSR) is publicly available on the internet (https://globalweather.tamu.edu/) and formatted as input to SWAT on a daily basis (for the years between 1979 and 2014), providing time series of precipitation, minimum and maximum temperature, wind, relative humidity, and solar radiation with a spatial resolution of approximately 0.3˚.

Several studies have been conducted to compare CFSR data with local climate datasets. In a general overview of their research, Dile and Srinivasan [23] obtained better simulation results with conventional climate datasets than CFSR in the Upper Blue Nile basin. Unsatisfactory results were achieved by Roth and Lemann [24] in three catchments in the Ethiopian Highlands by using the CFSR weather input. Their research suggests not using CFSR data in regions where no preliminary analysis is possible [24]. Auerbach et al. [11] improved model simulation, which was conducted with CFSR climate datasets by using spatial interpolation techniques in the tropics. Alemayehu et al. [25] tested CFSR and ground-based weather observations to simulate streamflow in the Mara Basin. Their streamflow simulations with CFSR resulted in poor performance compared to the local rainfall dataset.

Fuka et al. [26] showed that CFSR data generally resulted in better streamflow predictions, even CFSR stations located more than 10 km away from the watershed. Grusson et al. [27] obtained better results by using two different reanalysis precipitation data (CFSR and SAFRAN) rather than using local climate observations.

Even though the CFSR data set resulted in poor performance in some cases, some studies in the literature suggested that evaluating and using CFSR data is valuable in watershed modeling. Representability of both CFSR climate data and local climate observations were evaluated in various hydrological modeling studies under different geographic conditions and varying data availability. Comparing climate datasets in mountainous catchments with scarce local climate observations still remains understudied. Therefore, there is a need to test CFSR data in the Black Sea Region catchments having high elevation gradients with a few local climate observations. This study is expected to provide useful information to fill this gap in the literature.

Meteorological parameters such as precipitation and temperature are significantly influenced by elevation gradients [28]. The elevation band approach for the SWAT model developed by Fontaine et al. [29] was used to consider the orographic effects on precipitation and temperature in mountainous regions. In numerous studies, the elevation band approach was used and evaluated to predict the stream flows by hydrological models in mountainous catchments. The majority of these studies reported that using the elevation band improved the accuracy of streamflow predictions [29,30,31,32,33,34]. The elevation band approach requires precipitation and temperature lapse rate as an input parameter. In the case of scarce meteorological data, the determination of these parameters might become difficult [28]. In order to determine the lapse rates, most of the studies using the elevation band approach defined lapse rates as calibration parameters [27,30,34,35,36,37,38,39,40], whereas several other studies estimated precipitation lapse rate by using local climate observations [29,31,41]. In addition, two different studies conducted by Zhang et al. [28] and Zhang et al. [33] in the Yellow River Basin, China, used the precipitation lapse rate given by Fontaine et al. [29] for the Upper Wind River Basin, Wyoming in the USA. Previous studies have provided valuable information on the advantages of using elevation bands by using different methodologies to estimate lapse rates. However, the applicability of CFSR climate data to determine the precipitation lapse rate in a mountainous region has not been discussed in the literature.

Hence, the uniqueness of this study originates from investigating the usefulness of the application of the CFSR dataset and comparing the streamflow simulation results in the mountainous Black Sea catchment with the spatially scarce local climate data. Furthermore, this research examines the prediction of precipitation lapse by using the CFSR dataset for data-scarce region and the role of using this rate for the local climate observations to predict streamflow simulations by using the SWAT model.

2. Materials and Methods

2.1. Study Area

The Melen Watershed covers an area of 2445 km² and is located in the western Black Sea Coast of Turkey (Figure 1). The climate of the basin is characterized by the Black Sea. The Black Sea region receives high and evenly distributed precipitation throughout the year. The mean annual precipitation (1995–2012) in the central part of the basin is around 800 mm. In the Black Sea region, summers are warm and humid, winters are cool and damp. The mean annual temperature in the Duzce Plain is 13.5 °C. The hottest and coldest months are July and December, respectively. The Melen Watershed generally consists of mountains, plains, and plateaus. The mountains are cut in deep valleys and they rise from South to North and from West to East. The altitudes of the watershed range from 0 to 1952 m. Duzce Plain is approximately 120 m above sea level, surrounded by the Bolu Mountains from the East, the Orhan Mountains from the North, and the Samanlı Mountains from the South. The Great Melen River reaches sea level by decreasing altitudes with a moderate slope from the North and the East of the watershed (Figure 1).

According to the Corine (2006) database, the land use of the watershed is summarized in five classes (

Table 1. Land-use summary of the Melen Watershed.

Land Use	Percentage (%)
Forest	62.87
Agriculture	33.20
Urban	2.15
Pasture	1.56
Water Bodies	0.22

). Agricultural activity is quite essential for the region. Hazelnut production is one of the main sources of income, followed by corn and wheat production. The watershed is considered as the most feasible alternative water resources for Istanbul to supply water demand in the future [42]. The Melen Watershed has the highest water potential (1.5 billion m³ year⁻¹) among the other watersheds of Istanbul and it provides almost 45% of the total water resources [43]. The flowrate of the Great Melen River (Discharge Station No: D13A059 in Figure 1) ranges between 4 m³ sec ⁻¹ and 156 m³ sec⁻¹ with a mean flow rate of 44 m³ sec⁻¹. It reaches its highest values in March due to the precipitation and snow melt in the mountainous regions.

2.2. Soil and Water Assessment Tool (SWAT) Model and Data Sources

The SWAT (Soil and Water Assessment Tool) is a continuous-time, semi-distributed, and process-based hydrological model. The model is used to simulate the quality and quantity of surface and groundwater and predict the environmental impact of land use, land-management practices, and climate change. SWAT is widely used in assessing soil erosion prevention and control, non-point source pollution control, and regional management in watersheds [44]. It uses spatially distributed data on topography, soils, land cover, land management, and climate data to predict water, sediment, nutrient and pesticide [45]. The main hydrological processes include infiltration, runoff, evapotranspiration, lateral flow, and percolation. Water balance computation is performed at the level of hydrological response units (HRU), which are derived by a unique combination of land use, management, topographical, and soil characteristics, aggregated at the subbasins level and routed towards the reaches and the catchment outlet. More details about the SWAT model can be found in the theoretical documentation and from the relevant literature [20,44,45,46,47].

SWAT model setup and watershed delineations were conducted by using ArcSWAT2012 on ArcGIS 10.3 Desktop version. To delineate the watershed and its stream network, the digital elevation model SRTM 90 m digital elevation data was used from CGIAR (Consultative Group for International Agricultural Research). Soil data was compiled from FAO/UNESCO (Food and Agriculture Organization / The United Nations Educational, Scientific and Cultural Organization) Soil Map of the World [48] whereas, the land-use map was obtained from the CORINE 2006 Land Cover datasets (http://www.eea.europa.eu/data-and-maps/data/corine-land-cover-2006-raster-3).

The SWAT model setup for Melen Watershed was mostly guided by a modeling study performed by Cuceloglu et al. [43] in the region. Considering the stream network and the discharge monitoring stations located in the basin, the Melen Watershed was divided into 21 subbasins and 478 HRUs.

The local climate dataset was obtained from the Turkish State Meteorological Service (MGM). Climate stations measure temperature (maximum, minimum, and average), precipitation, wind as well as the humidity. These parameters were available on a daily basis. For the quality control of measured data, MGM follows the procedures given by the World Meteorological Organization. Sonmez [49] provided detailed instructions about the quality control methods applied to the meteorological data. Selected meteorological stations from MGM and related information are summarized in

Table 2. Local climate stations and related information.

Station Name	Station No	Data Period	Elevation (m asl)
Duzce	17072	1995–2012	146
Akcakoca	17015	1995–2012	743
Bolu	17070	1995–2012	10

. MGM has been observing meteorological parameters since 1960 for these stations. However, based on the available model setup data such as observed discharge data, land use, and the global weather data (CFSR), the appropriate time interval was chosen for the period between 1995 and 2012. The time series in local meteorological stations has many data gaps in terms of humidity, wind, and radiation. For this reason, min-max temperatures and precipitation data, which are common for both datasets, were used in hydrological modeling studies.

Climate Forecast System Analysis (CFSR) dataset spans the period 1979–2014. CFSR data includes daily precipitation, min-max temperatures, relative humidity, wind speed, and solar radiation data. CFSR data is obtained with coupled atmospheric, oceanic, and surface-modeling components with a resolution of ~38 km [15] so that it has various spatial and temporal interpolations. Six CFSR data grids were obtained for the study area by using a bounding box: latitude 40.36–41.6 N and longitude 30.48–31.48 E.

Except for the climate datasets, two similar models were set up in terms of the number of subbasin and HRUs, soil data, land use, etc. to evaluate the impact of CFSR and local climate datasets on hydrological modeling performance by SWAT model. Since local observations have min-max temperatures and precipitation data, similar parameters of CFSR data (for the period 1995–2012) were used. Therefore, the Hargreaves method [50], which requires only min-max temperatures, was used to estimate the potential evapotranspiration. The Hargreaves equation has already been successfully applied in the region [43], and also in other local and regional studies [51,52,53]. The SWAT simulations were run for the period between 1995 and 2012 (used for both climate databases). Five years were chosen as a warm-up period for the model.

2.3. Model Calibration and Evaluation

SWAT-CUP is a standalone and freely available software that allows calibration and uncertainty assessment of SWAT model. The program has five different optimization schemes as well as the parallel processing options for the SUFI-2 optimization algorithm for a higher performance calibration purpose [54]. The SWAT-CUP program was used for the calibration/validation as well as the uncertainty analysis of the model with uncertainty analysis by using the SUFI-2 algorithm [55,56] for the 2000–2012 period. Monthly discharge data of five different gauge stations located at different altitudes were used for model calibration, validation, and uncertainty assessment. Nash–Sutcliffe efficiency was used as an objective function in this study. Percent bias (PBIAS) and R² of the calibration/validation results were also measured. In addition, P-factor and the R-factor were used to evaluate the model performance and model output uncertainty. These statistics and their equations are briefly explained in this section.

Nash–Sutcliffe efficiency (NSE) is a normalized statistic that determines the relative magnitude of the residual variance compared to the measured data variance [57]. NSE values can range from–∞ to 1. An NSE value of 1 corresponds to a perfect match of observed streamflow to simulated streamflow. An NSE value between 0 and 1 is considered an acceptable level of performance, whereas an NSE value ≤ 0 suggests that the observed average is a better predictor than the model. NSE value is computed as follows:

$$\mathrm{NSE} = 1- \left[\frac{{{\displaystyle \sum }}_{\mathrm{i}=1}^{\mathrm{n}}{{(\mathrm{Q}}_{\mathrm{obs}}^{\mathrm{i}}{- \mathrm{Q}}_{\mathrm{sim}}^{\mathrm{i}})}^2}{{{\displaystyle \sum }}_{\mathrm{i}=1}^{\mathrm{n}}{{(\mathrm{Q}}_{\mathrm{obs}}^{\mathrm{i}}{- \mathrm{Q}}_{\mathrm{obs}}^{\mathrm{mean}})}^2}\right]$$

(1)

where ${ \mathrm{Q}}_{\mathrm{obs}}^{\mathrm{i}}$. is the observed streamflow, ${ \mathrm{Q}}_{\mathrm{sim}}^{\mathrm{i}}$. is the simulated streamflow, whereas the ${ \mathrm{Q}}_{\mathrm{obs}}^{\mathrm{mean}}$. is the mean observed streamflow. n shows the total number of observations. Percent bias (PBIAS) compares the average tendency of the simulated data to the corresponding observed data. The optimal value of PBIAS is 0. A positive value indicates that the model leads to underestimation and a negative value indicates an overestimation [58]. PBIAS is given as:

$$\mathrm{PBIAS} = \frac{{{\displaystyle \sum }}_{\mathrm{i}=1}^{\mathrm{n}}{(\mathrm{Q}}_{\mathrm{obs}}^{\mathrm{i}}{-\mathrm{Q}}_{\mathrm{sim}}^{\mathrm{i}}) \mathrm{x} 100}{{{\displaystyle \sum }}_{\mathrm{i}=1}^{\mathrm{n}}{(\mathrm{Q}}_{\mathrm{obs}}^{\mathrm{i}})}$$

(2)

In Equation (2), where ${ \mathrm{Q}}_{\mathrm{obs}}^{\mathrm{i}}$. and ${ \mathrm{Q}}_{\mathrm{sim}}^{\mathrm{i}}$. represent the observed and simulated streamflow at the ith time step, respectively, and n shows the total number of observations. The R² measures how well the observed data is correlated to the simulated data and varies from 0 to 1 and it is calculated as:

$${\mathrm{R}}^2 = \frac{{{\displaystyle \sum }}_{\mathrm{i}=1}^{\mathrm{n}}\left[{(\mathrm{Q}}_{\mathrm{obs}}^{\mathrm{i}}{- \mathrm{Q}}_{\mathrm{obs}}^{\mathrm{mean}}{\left)\right( \mathrm{Q}}_{\mathrm{sim}}^{\mathrm{i}}{- \mathrm{Q}}_{\mathrm{sim}}^{\mathrm{mean}})\right]}{{{\displaystyle \sum }}_{\mathrm{i}=1}^{\mathrm{n}}\left[{\left({\mathrm{Q}}_{\mathrm{obs}}^{\mathrm{i}}-{ \mathrm{Q}}_{\mathrm{obs}}^{\mathrm{mean}}\right)}^2\right]{{\displaystyle \sum }}_{\mathrm{i}=1}^{\mathrm{n}}\left[{\left({\mathrm{Q}}_{\mathrm{sim}}^{\mathrm{i}}-{ \mathrm{Q}}_{\mathrm{sim}}^{\mathrm{mean}}\right)}^2\right]}$$

(3)

where ${ \mathrm{Q}}_{\mathrm{obs}}^{\mathrm{i}}$. and ${ \mathrm{Q}}_{\mathrm{sim}}^{\mathrm{i}}$. are the observed and simulated streamflow at the ith time step, respectively; whereas the ${ \mathrm{Q}}_{\mathrm{obs}}^{\mathrm{mean}}$. and ${ \mathrm{Q}}_{\mathrm{sim}}^{\mathrm{mean}}$ are the mean of the observed and simulated streamflow respectively; and n is the total number of observations.

The degree of uncertainty in the calibrated/validated model was quantified using the P-factor and R-factor. The P-factor represents the percentage of observations bracketed by the 95% prediction uncertainty (95PPU) while the R-factor is the average width of the 95PPU band. The 95PPU is calculated at the 2.5 and 97.5% confidence intervals of observed streamflow obtained through Latin hypercube sampling [56]. The range of the P-factor ranges from 0 to 1, with values close to 1 indicating a better model performance, while the R-factor is the average width of the 95PPU band divided by the standard deviation of the measured variable and varies between the 0 and 1 [56,59]. The R-factor is calculated as [59]:

$$\mathrm{r}-\mathrm{factor} = \frac{\frac{1}{\mathrm{n}} {{\displaystyle \sum }}_{{\mathrm{t}}_{\mathrm{i}}=1}^{\mathrm{n}}\left({\mathrm{y}}_{{\mathrm{t}}_{\mathrm{i}},97.5\%}^{\mathrm{M}}-{\mathrm{y}}_{{\mathrm{t}}_{\mathrm{i}},97.5\%}^{\mathrm{M}}\right)}{{\mathsf{\sigma }}_{\mathrm{obs}}}$$

(4)

where ${\mathrm{y}}_{{\mathrm{t}}_{\mathrm{i}},97.5\%}^{\mathrm{M}}$ and ${\mathrm{y}}_{{\mathrm{t}}_{\mathrm{i}},97.5\%}^{\mathrm{M}}$ represent the upper and lower boundaries of the 95PPU, and ${\mathsf{\sigma }}_{\mathrm{obs}}$ indicates the standard deviation of the measured data. The P- and R-factors are closely related to each other, as it can be seen from the definition that higher P-factor can be obtained for larger R-factor. Thus, a trade-off has to be made between reducing R-factor around to 1 and keeping P-factor higher than 0.70.

In this study, nine parameters were selected to calibrate the SWAT model. These parameters were reported as the sensitive parameters of the SWAT model by Arnold et al. [44] and were also suggested by Abbaspour et al. [52]. Moreover, a sensitivity analysis for the selected parameters was already conducted in a previous study by Cuceloglu et al. [43]. The model parameter sensitivity analysis was performed with the SWAT model and the same datasets for the region. More detailed information about the sensitivity analysis can be found in the given study [43]. Explanations of the parameters and related files, as well as their initial ranges for both models are given in

Table 3. Selected Soil and Water Assessment Tool (SWAT) parameters for calibration and their initial range (Term “r__” was used for the relative changes in the parameter according to the given interval, “()” was used for all layers of soil and extension of the parameter states the file name of the SWAT files. mgt: SWAT management files, gw: SWAT groundwater files, sol: SWAT soil files, hru: SWAT hydrological response unit files.). (CFSR: Climate Forecasting System Reanalysis, MGM: Turkish State Meteorological Service)

SWAT Parameters	Definition of Parameters	Initial Range	Fitted Value (CFSR)	Fitted Value (MGM)
r__CN2.mgt	SCS runoff curve number for moisture condition II	−0.5 to 0.5	−0.2489	−0.3322
r__GWQMN.gw	Threshold depth of water in shallow aquifer for return flow (mm)	−0.5 to 0.5	−0.1302	−0.2235
r__GW_REVAP.gw	Groundwater revap. Coefficient	−0.5 to 0.5	−0.4906	−0.3593
r__SOL_AWC().sol	Soil available water storage capacity (mm H2O/mm soil)	−0.5 to 0.5	−0.4114	−0.3843
r__REVAPMN.gw	Threshold depth of water in the shallow aquifer for ‘‘revap’’ (mm)	−0.5 to 0.5	0.3177	0.3906
r__ESCO.hru	Soil evaporation compensation factor	−0.2 to 0.2	0.0804	−0.0920
r__ALPHA_BF.gw	Base flow alpha factor (days)	−0.5 to 0.5	−0.0572	−0.3177
r__SOL_K().sol	Soil conductivity (mm h−1)	−0.5 to 0.5	0.4385	0.3322
r__SOL_BD().sol	Soil bulk density (g cm−3)	−0.5 to 0.5	0.3031	0.2031

. Relative changes given in the initial ranges in the SWAT-CUP procedure [52] were applied for all parameters. Since different parameter combinations are created with SUFI-2 by using Latin hypercube based on a number of simulation and parameter, in this study, a different number of simulations (i.e., 240,360,480 and 600) were conducted for during the model calibration procedure. Slightly better model performance was obtained for the MGM model with 360 simulations rather than 480 simulations. The primary objective of this study is to compare the effect of different climate data. Thus, equifinality of model parameters discussed by Beven [60] was also considered during the model parameterization in the calibration process. Hereby, 480 simulations were conducted that resulted in similar parameter values (see Table 3) for both models built with different climate datasets.

Local climate stations in the Melen Basin are located in the lower altitudes. Figure 2 shows the hypsometric curve of the Melen Basin, together with local climate stations elevation. Duzce province climate station (no: 17012) is located in the middle of the catchment at an elevation of 120 m (Figure 2). Thus, to consider orographic effects on precipitation in the mountainous areas of the Melen Basin, elevation band application was used in this study.

Five elevation bands have been applied in the Melen Watershed. This method modifies the regional precipitation by weighting the elevation difference between the band of the rain gauge and the other bands by using the following formulas:

$${\mathrm{R}}_{\mathrm{band}}={\mathrm{R}}_{\mathrm{day}}+\left({\mathrm{EL}}_{\mathrm{band}}-{\mathrm{EL}}_{\mathrm{gauge}}\right)·\frac{\mathrm{plaps}}{{\mathrm{days}}_{\mathrm{pcp},\mathrm{yr}}},{ \mathrm{R}}_{\mathrm{day}}>0.01$$

(5)

$${\mathrm{R}}_{\mathrm{band}}={\displaystyle \sum }_{\mathrm{bnd}=1}^{\mathrm{b}}{\mathrm{R}}_{\mathrm{band}}·{\mathrm{fr}}_{\mathrm{bnd}}$$

(6)

where R_band is the precipitation in the elevation band (mm), R_day is the precipitation measured at the rain gauge (mm), EL_band is the mean elevation at the elevation band (m), EL_gauge is the elevation at the recording gauge (m), plaps is the precipitation lapse rate (mm/km) and days_pcp,yris the average number of days of precipitation in the subbasin in a year, fr_bnd is the fraction of the subbasin area within the elevation band and b is the total number of elevation bands in the subbasin. More information and detailed application of elevation bands can be found from the related literature [29,41].

Figure 3 summarizes and shows the main workflow of the study. The model setup was conducted by using two different climate data (CFSR and local dataset) on ArcSWAT 10.3. In the following, the elevation band was applied to the local climate dataset and then model results were evaluated both statistically and visually to show the influence of elevation band application on the results. Finally, model calibration and validation were carried out with the most sensitive SWAT parameters based on the comprehensive study in the region by Cuceloglu et al. [43]. Moreover, model performances were evaluated for each climate dataset at each gauge stations and the water budget of the basin was given according to two different climate data.

3. Results

3.1. Comparison of Climate Datasets

Pair-wise comparisons of temperature and precipitation from 1995 to 2013 were made for CFSR and local climate datasets. These values were obtained by a weighted average of three local and six CFSR climate stations depending on the related subbasins through the closest station approach of the SWAT model. The monthly temporal distributions of temperature for both climate datasets showed fairly similar patterns for the region (Figure 4). The monthly temperature given by CFSR is 1.7–3.0 °C cooler than that provided by the MGM. The highest differences in temperature are shown between April and September.

The temporal monthly precipitation input data from CFSR and MGM have similar trends in the region (Figure 5). Large and significant differences in precipitation are shown in wet seasons (December to April). Monthly precipitation difference between CFSR and MGM ranges from 10% to 44%. The CFSR dataset produces higher amounts of precipitation during the year. According to the model simulation period in this study, MGM long-term mean annual precipitation is given as 683mm y⁻¹ and 1011 mm y⁻¹ by MGM and CFSR, respectively.

To compare the spatial distribution of the mean annual precipitation over the Melen Watershed is presented in Figure 6. While there is an agreement for temporal patterns, the spatial distribution of precipitation is quite different for both datasets. The CFSR dataset is able to produce higher rainfall over the mountainous regions. The precipitation by CFSR data ranged from 930 to 1267 mm in highlands, whereas the MGM data ranged from 549 to 929 mm. The MGM data shows 1163 mm at the outlet of the basin. The primary reason for this distribution is, the outlet of the catchment is nearby the Akcakoca station (Figure 1). As the SWAT model uses climate data from the gauge station, which is the closest to the centroid of each subbasin, precipitation value of subbasins has been obtained from the Akcakoca station. This climate station is located in the coastal zone of the Black Sea, which receives the highest amount of rainfall.

3.2. Model Performance with Raw Climate Inputs

Prior to the calibration, the performance of the SWAT model with each climate dataset (CFSR and local dataset) was compared with the monitored discharges at six gauge stations located at different altitudes. For the visual evaluation, monthly hydrographs of raw SWAT simulations (simulations before calibration) were plotted over the full period (Figure 7). For the simulations at the gauge station at the highest altitude (D13A032) both datasets resulted in poor performance. The hydrograph for the monthly simulations using the CFSR dataset showed quite reasonable agreements between the simulated and the observed streamflow at the gauge stations (D13A059) located closer to the watershed outlet (Figure 7).

In contrast, the model simulation using the local climate dataset (MGM) showed a major underestimation for the streamflow at all gauge stations. This result can be explained by the difference of around 320 mm rainfall between the two datasets. Despite the major underestimation of the streamflow (PBIAS ≥ 60%) using the MGM data, better results were obtained with this data at the gauge station closer to the outlet (in low altitudes), indicated by R² values ranging from 0.61 to 0.65. These R² values indicate that MGM data set has a good representation of the seasonal dynamics of the precipitation in the catchment. On the other hand, high PBIAS value showed that the MGM dataset significantly underestimates the amount of rainfall over the region compared to CFSR data. In the mountainous region of the watershed, the simulation quality decreased, achieving R² values ranging from 0.21 to 0.35. The systematic underestimation of the streamflow by using this dataset is most likely due to the poor representation of the spatial variability of precipitation patterns in the mountainous region. Better correlations of model results with the MGM data to observed discharge data can be achieved by using the elevation band features of the SWAT model.

3.3. Model Performance with Elevation Band

In order to reflect orographic effects on precipitation in the mountainous area, five elevation bands were used. Since the local climate observations cannot represent the climatic conditions accurately, CFSR data was used to find the precipitation lapse for the catchment. The lapse rate for annual precipitation was determined by plotting the mean annual precipitation vs. station elevation of the CFSR climate dataset. Precipitation lapse rate of 644 mm/km was obtained by the slope of the curve (Figure 8).

The obtained precipitation lapse rate was used to improve the simulation performance SWAT simulations by the MGM datasets. The CFSR was produced by coupled atmospheric, oceanic, and surface-modeling component so that this reanalysis data already contains an elevation gradient for the meteorological parameters [15]. Thus, we did not apply precipitation lapse adjustment to the SWAT simulations conducted by CFSR data.

Figure 9 shows both flow duration curves of uncalibrated SWAT simulations and the improvement resulting from the application of elevation bands for the MGM climate dataset. The application of the elevation band has positive impacts on the streamflow simulations for the entire watershed. Despite the considerable improvement at the gauge stations (i.e., D13A05 and E13A040) located in the lowland, the application of the elevation band did not improve the quality of the simulations at D13A033. The monthly time series of model results for after the implementation of elevation bands using the local climate data are shown in Figure 10. Model results with the elevation band application yielded with even better correlation than model results with the CFSR data. Substantial improvements on peak and the base flows can be seen for different gauges (Figure 10).

To assess the improvement in the model performances by using precipitation lapse rate, three objective metrics were evaluated for SWAT simulations that were carried out by using local climate datasets. The NSE and PBIAS values (

Table 4. Model performance improvement on uncalibrated SWAT simulations by the application of elevation bands (including full-time period for each gauge stations) to the local climate datasets. R² Coefficient of determination NSE: Nash–Sutcliffe efficiency, PBIAS: Percent bias.

Gauge Stations	Elevation (m. asl)	First Simulations			After Elevation Band (644 mm/km)
		R2	NSE	PBIAS	R2	NSE	PBIAS
D13A059	10	0.66	−0.05	61.7	0.78	0.77	0.9
E13A040	23	0.62	−0.10	65.2	0.70	0.70	7.6
E13A002	115	0.62	−0.01	61.2	0.70	0.66	−9.6
D13A038	276	0.25	−0.88	81.7	0.63	0.49	25.7
D13A033	276	0.32	−0.37	81.7	0.47	0.02	64.8
D13A032	873	0.23	−0.22	68	0.23	−0.07	−38.7

) at the discharges station in low altitudes such as D13A059, E13A040, and E13A002 (see Figure 1) indicated an improvement of the simulation performance. These stations are located closer to the watershed outlet. In the mountainous region of the watershed, the application of elevation bands yielded good results for the gauge stations D13A038 and D13A033 (except for the D13A032,). For instance, R² of the D13A038 increased from 0.25 to 0.63, whereas percent bias decreased to 25.7 from 81.7 (Table 4). According to suggested ranges by Moriasi et al. [61] the model performances at several gauges are quite satisfactory even before the calibration procedure.

Evaluation of the watershed model performances can be varied according to the processes (streamflow, sediment, nutrients, etc.), time scale (hourly, daily, monthly, etc.), the objective function (R², bR², NSE, PBIAS, etc.), available data and the objective of the study. No comprehensive guidance and the universally certain values are available in the literature to facilitate model evaluation in terms of the accuracy of the simulated data compared to the measured data [61]. However, Krause et al. [62], Moriasi et al. [61], and Abbaspour et al. [52] suggested ranges of values and performance ratings for the recommended model evaluation statistics and these works have been commonly accepted, and these studies have been highly cited in recent years. In addition, different approaches and the number of classes might be used to compare results depending on the goal of the study. Li et al. used [63] Taylor diagram [64] to visualize the statistical relationship of the simulated monthly streamflow using the baseline model structure. Kamali et al. [53] defined three different classes to compare R² values of the different model configurations.

In this study, to demonstrate the model performances both numerically and visually, the heat map was plotted. Model performances were classified into three classes based on the literature (Krause et al. [62], Moriasi et al. [61] and Abbaspour et al. [52]) for the simple visualization of the heat map. The green color was used to represent high performance, yellow color was used for average performance and red for poor performance (Figure 11). In this representation, the results for objective functions were given both for the calibration and validation periods by ascending order of elevation of gauge stations.

After the calibration, the overall simulation results of SWAT model predictions were quite satisfactory for both datasets at the outlet of the Melen Watershed according to the suggested ranges by Moriasi et al. [61] and Abbaspour et al. [52].

There is a geographical trend regarding the accuracy of the model performance in the watershed from lowland to mountainous regions. By increasing altitudes in the watershed, poor model performances were observed. Objective metrics for all simulations (both datasets) nearby the outlet (located in lowlands) were quite satisfactory. The study investigated that with the increasing altitudes, model performances decrease.

To investigate the variability in the streamflow simulations, the coefficient variation (CV = σ/μ × 100) was calculated, where σ is the standard deviation and μ is the mean of the streamflow. CV was calculated for the simulation period in each gauge station. CV values for gauge stations range between 77% and 86% for the CFSR dataset, whereas it varies between 73–107% for the local climate datasets. To demonstrate the variability of model predictions, a boxplot of CV values of all gauge stations is plotted (Figure 12). We used exactly the same spatial data (land use, soil) and model setup (number of subbasin and hrus) therefore, the variability of the coefficient of variation was stemmed from both parameters and climate. Since we used the same parameter interval in the SUFI-2 algorithm, variability in our model result was mainly caused by climate datasets. After model calibration, streamflow predictions with the local climate datasets showed larger variability than the model driven by CFSR.

4. Discussion

One of the main objectives of this study was to evaluate the impacts of the two different climate datasets (CFSR and local observations) on hydrological modeling performance by using the SWAT model. Our research demonstrated that hydrological simulations with the CFSR dataset showed much better performance than the local climate datasets in the Melen Watershed. The underlying causes of this outcome are mainly based on the local climate data scarcity in the Melen Watershed. In addition, the local meteorological stations do not cover the mountainous area, and the majority of these stations are located outside of the model domain. Another limitation of our study was that the local data are regionalized to the entire model domain through the simple “closest station” approach of the SWAT model.

Our findings in this study contradict several studies presented in the literature. Similar to our research, these studies evaluated the performance of CFSR and the local climate observations on the hydrological simulations. These studies were conducted by Dile and Srinivasan [23] in Upper Blue Nile Basin, by Roth and Lemann [24] in the three catchments in the Ethiopian Highlands, Auerbach et al. [11] in Puerto Rico and Alemayehu et al. [25] in the Mara Basin (Kenya/Tanzania). In these study areas, the local climate observations were adequate to represent the climatic conditions in the catchments.

In contrast to the above cited literature, there are numerous studies showing that CFSR data exhibited better performance than the local climate datasets. By using CFSR data, Fuka et al. [26] achieved better simulation results in the Catskill Mountains, NY, the USA, and the Gumera Watershed in the Blue Nile River in Ethiopia. Another study in the Garonne River Watershed, France showed that the CFSR dataset performed better hydrological simulations than local climate observations [27]. Our results are in agreement with the findings of Fuka et al. [26] and Grusson et al. [27], which showed better performance with the CFSR dataset.

Studies in the literature also demonstrate that the accuracy of the CFSR over local datasets on hydrological simulations has heterogeneity. The performance of the hydrological predictions varies according to geographical conditions and spatial distributions of the locally observed data. Therefore, evaluating and using the global reanalysis dataset as an alternative model input is useful in local hydrological modeling studies.

According to our simulations, the application of the elevation band has positive impacts, particularly at the outlet of the watershed. Furthermore, streamflow simulations at the gauge station D13A038 located in higher elevation were significantly improved by using precipitation lapse rate whereas the performance of two other gauge stations (D13A032 and D13A032) located in high altitudes slightly improved. This may result from that the elevation difference with climate station and elevation gradient of the related subbasin. Similar conclusions were also stated by Grusson et al. [32] for the Garonne River Basin in southwest France. Another possible reason for the low performance of both climate datasets in mountainous regions could be poor simulations of snow melt processes in the upstream subbasins. Accurate representation of the snow melt process could improve discharge predictions in mountainous catchments [32,65].

Numerous studies have shown that the application of elevation band method is quite useful in hydrological simulations, especially in mountainous catchments [30,32,36]. Results of the elevation band application conducted in the mountainous Black Sea catchment in this study are in accordance with those of previous studies.

To the best of our knowledge, in this study, the CFSR dataset was used for the first time to estimate the precipitation lapse rate of a mountainous catchment. As we have few local observations and therefore cannot represent the orographic effect of precipitation over the Melen Watershed, we used CFSR gridded data to estimate the precipitation lapse rate. Fontaine et al. [29], Kang et al. [31] and Galvan et al. [41] determined precipitation lapse rate using the local climate station over their study areas. The majority of the previous studies estimated precipitation lapse rate within their calibration procedure [26,29,33,34,35,36,37,38,39]. The present study demonstrated how CFSR data can be useful to improve hydrological simulation in data-scarce mountainous regions.

Due to the data non-availability, our research in the Melen Watershed only focused on monthly time step performances of the given climate dataset. However, it may be important also to include daily time-step performance of model predictions. Further studies, including daily simulation could provide detailed information on such factors as snow melting processes in the spring and floods.

5. Conclusions

This research provided the first systematic analysis of CFSR and local weather data as a climate input for the SWAT model in mountainous Black Sea Catchment of Turkey. The model performance and uncertainties for both datasets were evaluated at multiple gauge stations at different altitudes. The use of both datasets resulted in poor performances at higher elevations based on the simulations with the uncalibrated model.

In general, the use of the CFSR data for the SWAT simulations outperformed the local climate dataset. Unacceptable performance of uncalibrated simulations using the local climate data indicates that spatial distribution of the stations is not suitable for representing the climatic conditions over the entire catchment. However, the attempts to improve model performance by using the elevation band application and parameter adjustments were successful. Based on this improvement and physical understanding, it appears that high-resolution climate data is extremely important, particularly in mountainous regions.

Today, numerous climate databases are available for modelers, including reanalysis and global circulation model outputs. Comparing different datasets is extremely important for evaluating the model performance (i.e., accuracy), addressing uncertainties, and for improving the reliability of model applications in practice. This is particularly important in data-scarce regions, such as the mountainous case study presented in this paper.

This research demonstrates that using elevation bands through the use of CFSR data can improve the model results, particularly for mountainous regions where available weather data does not accurately represent the orographic precipitation. It is also important to note that using global climate data such as CFSR may be an effective and straightforward option for data-scarce areas or regions where the local climate stations exhibit high spatial heterogeneity.