Next Article in Journal
Coupling Riverbank Filtration with Reverse Osmosis May Favor Short Distances between Wells and Riverbanks at RBF Sites on the River Danube in Hungary
Previous Article in Journal
Five-Year Experimental Study on Effectiveness and Sustainability of a Dry Drainage System for Controlling Soil Salinity

Water 2019, 11(1), 112;

Improving Alpine Summertime Streamflow Simulations by the Incorporation of Evapotranspiration Data
Center for Earth and Environmental Studies, Texas A&M International University, Laredo, TX 78045, USA
Author to whom correspondence should be addressed.
Received: 23 August 2018 / Accepted: 5 January 2019 / Published: 10 January 2019


Over the last decade, autocalibration routines have become commonplace in watershed modeling. This approach is most often used to simulate a streamflow at a basin’s outlet. In alpine settings, spring/early summer snowmelt is by far the dominant signal in this system. Therefore, there is great potential for a modeled watershed to underperform during other times of the year. This tendency has been noted in many prior studies. In this work, the Soil and Water Assessment Tool (SWAT) model was auto-calibrated with the SUFI-2 routine. A mountainous watershed from Idaho was examined (Upper North Fork). In this study, this basin was calibrated using three estimates of evapotranspiration (ET): Moderate Resolution Imagining Spectrometer (MODIS), Simplified Surface Energy Balance, and Global Land Evaporation: the Amsterdam Model. The MODIS product in particular, had the greatest utility in helping to constrain SWAT parameters that have a high sensitivity to ET. Streamflow simulations that utilize these ET parameter values have improved recessional and summertime streamflow performances during calibration (2007 to 2011) and validation (2012 to 2014) periods. Streamflow performance was monitored with standard objective metrics (Bias and Nash Sutcliffe coefficients) that quantified overall, recessional, and summertime peak flows. This approach yielded dramatic enhancements for all three observations. These results demonstrate the utility of this approach for improving watershed modeling fidelity outside the main snowmelt season.
evapotranspiration; SWAT; MODIS; GLEAM; SSEBop

1. Introduction

In snow-dominant areas, a strong seasonal signal in runoff can be attributed to spring-time streamflow. Therefore, hydrologic model auto-calibration routines have a high probability of tuning parameters to fit this annual peak. Consequently, parameters that are not directly connected to peak flow runoff but those that control recessional streamflow, baseflow, and even soil moisture have a great potential to be unrealistically set for the vast majority of the year [1]. Even calibration by multiple gauges does not offer a significant improvement from this situation [2]. Auto-calibration routines can yield a constellation of acceptable results that are based on objective metrics. This reality of modeling is inherently governed by the principle of equifinality [3]. The issue of equifinality can become significant for distributed and semi-distributed models such as the Soil and Water Assessment Tool (SWAT). In SWAT, there are a large number of parameters that cannot be constrained by direct traditional observation (i.e., streamflow). Within the universe of possible model realizations, one should select a subset of physically realistic parameter values.
In recent years, there has been a proliferation of remote sensing products that provide an additional form of information that can be used to help calibrate parameters that are not directly related to streamflow. Earlier studies have calibrated the SWAT model with other fluxes, such as evapotranspiration (ET, [4,5]), leaf area index [6,7], and soil moisture [8]. Immerzeel and Droogers [4] produced SWAT-simulated ET results which were validated with streamflow data, whereas [5] followed the procedures that are outlined in this paper, that leverage ET observations to directly improve streamflow simulations. The approach to using ‘soft data’ not directly related to streamflow monitoring has been previously advocated [9,10,11]. Another approach uses a clustering method to conduct separate calibrations for high- and low-flow regimes [12,13]. In either case, the result is the delineation of a parameter set that more realistically represents the ambient conditions within the watershed.
Parameter selection must be governed by a fundamental understanding of the hydrological processes that are present within a system. While SWAT can represent snow melt peak streamflow reasonably well, based on previous studies [14], the model has difficulty with representing baseflow present during fall to winter, as well as peak flow outside the main snow melt season [1].
A characteristic pattern of soil moisture was noted in alpine regions during the warm season [15,16]. While streamflow monitoring is affected by this pattern, examining soil moisture and evapotranspiration data can elucidate the specific details of the hydrologic response at a watershed scale. This pattern is divided into four distinct phases that are evident within a time series of soil moisture. (1) The snow meltout phase—a period in which soil moisture values increase. This period supports increased streamflow. (2) Post-snow melt phase—elevated soil moisture is retained for a short period during late spring and early summer as the still cool conditions do not support high losses through evapotranspiration. During this period, streamflow reaches its seasonal peak. (3) Initial drying phase—where the depletion of soil moisture is noted at the surface (5 cm), but not at greater depth within the root zone. During this period, the difference between the shallow (20 cm) and deeper (50 cm) root zones becomes progressively greater. The surface layer can be temporarily recharged by summertime rainfall events producing variability in the time series. Deeper within the root zone soil moisture behaves more consistently. In general, this is a period of recession in streamflow unless affected by precipitation events. (4) Final drying phase—during this period, most of the surface soil moisture has been depleted and recessional baseflow is provided by soil moisture stored within the deeper root zone.
This paper examines an alpine watershed from Idaho (USA), and leverages the results from three remote sensing platforms to constrain evaporation, to improve warm season simulations of hydrologic response with the SWAT model. The SWAT model has exhibited variable performance in this geographical setting outside of the spring-time snow melt season [1,17,18,19]. By incorporating ancillary data sets streamflow performance markedly. This is of particular importance in areas that can receive enhanced orographic summertime precipitation, which in the past has led to lethal streamflow flooding in alpine areas such as the Front Range of Colorado [20,21]. Consequently, developing the ability for a hydrologic model to improve the representation of alpine hydrologic processes has the potential to save lives.

2. Materials and Methods

2.1. Watersheds Examined

The examined watershed is located in the southern panhandle of Idaho (Figure 1). Nestled along the western side of the Bitterroot Mountains, this watershed is immediately west of the North American continental divide. Climatically, this region has a greater maritime influence during the winter, with summer characterized by having a greater continental climate with occasional convective activity. In terms of temperature, wintertime temperatures are generally sub-freezing, whereas brief summers have warmer conditions (high temperatures around 28 °C) with an over 20 °C diurnal range. Abundant precipitation is noted during the winter with lesser amounts during the summer. In the Upper North Fork, basin total annual precipitation is approximately 1400 mm/year. During the wintertime months, the majority of this precipitation is in the form of snow. Spring-time snow melt dominates the hydrology in this region particularly during the late spring period when streamflow reaches its annual maximum. However, summer-time convective activity can produce flash flooding. In addition, winter-time flooding can be caused by anomalously warm temperatures or breaches in ice dams that are difficult to simulate.
The Upper North Fork watershed has steep slopes that range from a minimum elevation of 506 m near the outlet to peaks that exceed 2000 m in height. The streamflow gauge is located near the Canyon Ranger Station (station number 13340600). This basin spans 3356 km2. The Upper Fork Clearwater River forms an irregular trellis network, and this river’s outlet is in the northwestern corner. Soils are dominated by silty loam material with an area classified as rubble land, consisting of sandy loam in the southeast corner of the basin (Figure 2a). The percentage of soil families/textures are given in Table 1. This basin is dominated by both evergreen forest (FRSE—75%) and rangeland-brush (RNGB—24%) with 1% miscellaneous land use (Figure 2b).

2.2. Datasets Used

A 30 m Digital Elevation Model (DEM; USDA Geospatial Data Gateway) was used in the watershed delineation process that divided the watersheds into sub-basins. The DEM raster file from the National Elevation Dataset was saved in ArcGRID format [22,23], and it was projected to a Universal Transverse Mercator (UTM) Zone 11. Land cover data from the National Land Cover Dataset 2006 was utilized. Files were in TIFF format with a 30 m resolution [24]. Finally, soil data was obtained from the U.S. General Soil Map (STATSGO2) in ESRI Shapefile format, from the USGS Geospatial Data Gateway [25].
Several sources provided the hydrometeorological data used in this study. United States Geological Survey (USGS) streamflow data [26] was downloaded from the USGS National Water Information System. Daily minimum and maximum temperature data (2002 to 2014) were obtained from the climate data developed by the PRISM Climate Group, based at Oregon State University [27]. Three measurements were downloaded: precipitation, minimum temperature, and maximum temperature. The three datasets are available in BIL format, with a spatial resolution of 4 km.
Three evapotranspiration (ET) products were used in this study. All ET data products were aggregated to a sub-basin average within each watershed. The MOD16A2 Version 5 (MODIS16A2v5) product is monthly with a 0.5 km spatial resolution. This product is available in HDF format from the [28]. This product is based on the Penman–Monteith equation, which includes inputs of daily meteorological reanalysis data along with MODIS remotely sensed data products, such as vegetation property dynamics, albedo, and land cover.
The second ET product was the Simplified Surface Energy Balance (SSEBop). This monthly actual ET dataset is available across CONUS, and was obtained from the USGS Geo Data Portal [29,30]. This approach is based on using remotely sensed thermal retrievals, combined with assimilated meteorological observations (PRISM). It also incorporates MODIS observations for surface albedo, NDVI, and land surface temperatures. With the elevation constrained by the Shuttle Radar Topographic Mission these observations are assimilated into the Surface Energy Balance System (SEBS) model, which evaluates the energy balance terms for each pixel at the limiting conditions for the dry and saturated states. Therefore, the calculated actual ET is constrained within these two limits. This product has a GeoTiff format, with a 0.009 degrees spatial resolution.
The Global Land Evaporation: the Amsterdam Model (GLEAM) is the third ET used [31,32]. This product was derived from a number of measurements, including Clouds and the Earth’s Radiant Energy System net radiation, Atmospheric Infrared Sounder air temperature, Water Cycle Multi-Mission Observation soil moisture, Land Parameter Retrieval Model vegetation optical depth, and GlobSnow snow water equivalents. Version 3.1a (b) which is available with 0.25 degrees spatial resolution in netCDF format was used. GLEAM is a daily product, but in this study, it was aggregated to a monthly time step.

2.3. SWAT Model Description

Water balance computations form the basis for SWAT. This is a semi-distributed hydrological model described by [33,34]. This model’s main advantage is the comprehensive model structure in which all major hydrologic processes are simulated (i.e., surface runoff, subsurface runoff, evapotranspiration, infiltration, interception, snow accumulation/melting).
The subdivision of the watershed during the basin delineation process forms the sub-basins, with a total of 69 defined for the Upper North Fork watershed. Daily hydrologic fluxes within each sub-basin were estimated (surface runoff, subsurface runoff, evapotranspiration, infiltration, interception, etc.). In each sub-basin, the fundamental computational unit is the hydrologic response unit (HRU). In Upper North Fork, a total of 552 HRUs were defined. A unique combination of land use, soil, and slope within a sub-basin defines an HRU. Note that a 10% threshold was used in the definition process to limit the influence of minor soil types in this analysis. SWAT calculates a water balance that is based on the area-weighted average of the curve number (CN2) values present within a sub-basin. The SWAT model lacks the characteristics of a distributed model and the spatial location of the HRUs present in each sub-basin is not a consideration. The model also allows for the adjustment of the CN2 value, based on antecedent moisture conditions (dry, normal, or wet). The excessive surface runoff that is generated, based on sub-basin computations, are routed into overland flow. The flow that intersects with a channel is routed to adjacent sub-basins downstream, and flows into the watershed outlet. The variable storage method was used as the channel routing method [35].
SWAT also calculates daily potential evapotranspiration values through several approaches, and the Penman–Monteith method was chosen in this study. This method used the observed temperatures and values for radiation, relative humidity, wind velocity, and atmospheric pressure that were calculated by the SWAT weather generator. This method is commonly accepted as a more physically realistic approach to measure evapotranspiration, in contrast to the simpler Priestley Taylor and Hargreaves methods. In addition, the Penman–Monteith method forms the basis for the MODIS16A2v5 and SSEBop ET products examined in this study.
The SWAT model has 23 major parameters that influence surface hydrology in areas where snowmelt can strongly influence runoff, based on previous studies in alpine regions and the SWAT User Manual ([1,15,36], Table 2). SOL_AWC, SOL_K, and SOL_BD directly affect soil moisture. Potential and actual evaporation are controlled by five parameters (ESCO, EPCO, CANMX, GW_REVAP, REVAPMN). Snowmelt runoff is regulated by SFTMP, SMTMP, SMFMX, SMFMN, and TIMP. The other ten parameters control streamflow by either adjusting baseflow (ALAPHA_BF, GWQMN, GW_DELAY) or surface quickflow (CN2, SURLAG, OV_N, CH_N1, CH_K1, CH_N2, CH_K2).

2.4. SWAT Model Set-Up and Evaluation

A two year warm-up period initialized the models in both watersheds. Longer warm-up periods had no impact on the simulated streamflow performance. The streamflow simulation was supported by the 64-bit version of the SWAT Model (2012; revision 627), run at a daily time step. Evapotranspiration simulations were conducted at the level of the sub-basin, and at a monthly time-step. The channel routing method selected was the daily rain/CN/daily route rainfall routing option with Variable Storage. For the Upper North Fork watershed, the calibration period was 2007 to 2011, which was used for both streamflow and evapotranspiration model runs. The validation period for streamflow spanned 2012 to 2014.
This study used two primary objective criteria to quantify overall flow performance, which are standard in the hydrologic community, including the mass balance error (MBE) and the Nash–Sutcliffe efficiency (NS). The definitions of MBE and NS are given below:
MBE = i = 1 n ( Q sim , i Q obs , i ) i = 1 n Q obs , i
NS = i = 1 n ( Q sim , i Q obs , i ) i = 1 n ( Q sim , i Q obs , a )
where, Qobs,a is the average observed value, which includes streamflow (Q) or evapotranspiration (ET). Qsim,i and Qobs,i are the simulated and observed values at the ith observation, respectively and n is the number of observations. Three other observations were also used for evaluation, and they include: baseflow ratio, recessional flow performance, and summertime (July to September) peak flow performance. These last two observations were evaluated by using both MBE and NS metrics. To determine the quality of a simulation, the criterion of [37] was used. This paper outlines the basis for specifying MBE and NS values that constitute perfect, very good, good, satisfactory, and unsatisfactory simulations (Table 3).

2.5. SUFI-2 Autocalibration Routine

SUFI-2 has become in recent years an increasingly popular stand-alone algorithm used to develop optimized SWAT parameters [38]. SUFI-2 seeks the optimum parameter set to match simulations against an observed dataset using a specified objective criterion, while also minimizing parameter uncertainty. User-selected parameter ranges ensure that the sampling (based on the Latin Hypercube method) reflects only a physically realistic range within the parameter space. This autocalibration method’s greatest virtue is the relatively smallest number of runs that are needed to estimate uncertainty. This algorithm rapidly converges to identify the optimal solution [39,40,41]. In this study, the NS value was selected as the objective parameter that are used for calibration.
The SUFI-2 module in SWAT-CUP simulated within sub-basins for ET, and at the watershed outlet for streamflow. The sub-basins were organized based on dominant soil type (Table 1) supporting eight simulations per ET product. A ninth simulation was added that grouped sub-basins together that did not have a soil type that clearly dominated. These simulations were completed to obtain the maximum possible range of values for ET sensitive parameters. Since only two significant land use types exist in this basin, we opted to focus on soils to obtain the broadest spectrum of possible ET parameter values. The overall area-weighted basin average parameter values derived from the suite of nine ET simulations (Table 4) was used to constrain a second and third series of streamflow simulations. Each SUFI-2 scenario involved 1000 simulations. After model execution, SUFI-2 outputs an optimal parameter set, as well as an indication of parameter sensitivity. Parameters with p-values < 0.02 were highly sensitive (black bold in Table 1).

2.6. Scenarios Executed

The modeling procedure is an iterative process that focuses on constraining the parameter space by using physically realistic parameter values during the calibration period. Three separate sets of streamflow simulations were executed. The initial streamflow simulation is followed by an ET model run. Parameter values obtained from the ET modeling were used to support two additional series of streamflow, as described below.
The Base Q scenario involves setting the CN2 and ALPHA_BF values based on ancillary soft information. This model series was used only for a comparison. Reasonable potential ranges for CN2 were obtained by examining land cover and soil within the watersheds. A range of ±25% was used around the basin-wide average to set the range for this parameter (Upper North Fork, 35.0 to 58.1). However, the minimum value for CN2 was set at 35.0, which is a reasonable lower limit for this parameter. The ALAPHA_BF was set within a plus or minus of a factor of two of the ALAPHA_BF value determined by the baseflow program [42].
The next series of scenarios involved using ET simulations to constrain the following 12 parameters that can have a potential impact on ET: CN2, ALPHA_BF, SOL_AWC, SOL_BD, SOL_K, GW_DELAY, GWQMN, ESCO, EPCO, CANMX, GW_REVAP, and REVAPMN. To further constrain CANMX analysis calculation of interception based on land cover proportions (see Watersheds Examined section), conifer estimates of the annual fraction of evergreen forest interception [43] and previous CANMX estimates (3.56) from moderately dense rangeland settings [44] were compiled. For the Upper North Fork basin, the optimal ET values for CANMX yielded an annual fraction of precipitation lost to interception at 29%. Note that this value is close to the 25% average interception loss recorded in 18 studies of canopy interception summarized in [38]. Based on this supporting analysis, the ET parameters derived were physically realistic. Note that only sub-basins that exceed 1% of the overall basin were included the subsequent analysis.
The best parameter values (basin-wide average) from the five most consistently sensitive parameters (Table 4) were used to support a second series of streamflow models (Best Q MODIS, Best Q SSEBop). Since the GLEAM ET product yielded unsatisfactory ET results (Table 3), this product was not used to support the second phase of streamflow simulations. Only simulations that reached at least satisfactory performance thresholds (Table 5) were considered for further analysis. In Table 5, ‘n’ indicates the number of simulations that reached the specified performance threshold.
For Upper North Fork, the best modeling results were derived from the Best Q MODIS series (Table 5). All parameter values for the eight Best Q MODIS model runs that yielded at least a satisfactory performance were used to support a third and final streamflow scenario (Optimized Q) in which all parameter values were fully constrained based on the Best Q MODIS simulation (Table 5).
Finally, parameter values for the acceptable streamflow simulations for each calibration scenario were applied for the validation period. Only two simulations yielded satisfactory performances for the Base Q scenario, and only one for the Optimized Q model run during the validation period.

3. Results

3.1. ET

In the Upper North Fork basin, the MODIS16A2v5 and SSEBop yielded satisfactory results at the basin scale, unlike the GLEAM 3.1a product (Table 4). All but two sub-basins (58 and 61) yielded at least satisfactory NS values (Figure 1). These sub-basins poorly performed probably because of the presence of significant rubble land in these sub-basins. The sandy loam soil that dominates there sub-basins diverges from the silty loam soils that dominate most of the watershed (Figure 2a). The MODIS16A2v5 and SSEBop products did a reasonable job of capturing the seasonality of ET across most of this watershed. All products tended to overestimated summertime ET to varying degrees in many sub-basins. Both the MODIS13A2v5 and GLEAM 3.1a products overestimate both winter ET, but GLEAM 3.1a to a greater extent. SSEBop provided a better match with the simulated wintertime ET.

3.2. Streamflow

Streamflow results are presented in Table 5 and Figure 3 for the Upper North Fork watershed. The Base Q scenario involved using physically realistic parameter value ranges for CN2 and ALPHA_BF without any constraints on the other 21 parameters (Figure 3a). The relatively small number of acceptable Base Q simulations (n = 11; Table 5) is due to the rigorous criteria that are applied, involving three observations (overall flow, summertime peak flow, and recessional flow). For a simulation to be considered acceptable, it must exceed the specified thresholds for all three observations. This greatly limits the number of acceptable simulations that can be obtained from even an ensemble of 1000 model runs.
The next phase involved the inclusion of parameter values derived from ET model runs to two streamflow scenarios (Best Q MODIS, Best Q SSEBop). Using the rigorous three-observation criterion, eight satisfactory model runs for the Best Q MODIS scenario were found. The Best Q SSEBop yielded only two satisfactory model runs (Table 5). Better results were obtained for the Best Q MODIS series, so that parameters from this series were used to constrain the Optimized Q scenario. This scenario yielded two very good simulations based on overall flow that were better than any previous scenarios, supporting the utility of this approach. It is especially noteworthy that the final Optimized Q scenario had very good performance for not just overall flow, but also summertime peak flow and recessional flow. In addition, the 32 good and 232 satisfactory model runs that were yielded, vastly exceeded the 11 noted in the Base Q series. This result demonstrates the utility of this approach to markedly boost streamflow performance.
Figure 3 shows hydrographs for Upper North Fork Base Q (Figure 3a) and Optimized Q scenarios (Figure 3b). The Optimized Q scenario obviously more tightly matches with the observed data. This observation is supported by an analysis of summertime peak flow and recessional flow. For the Base Q scenario summertime peak flow, NS values ranged from (0.66 to 0.94; Table 5), whereas for the Optimized Q, a much tighter range was noted (0.80 to 0.89; Table 5). The same trends were noted for recessional flow (Base Q NS = 0.51 to 0.73; Optimized Q NS = 0.77 to 0.82; Table 5). Despite these improvements, not all runoff events outside the main snow melt season were captured, such as the anomalous winter-time high-flow events (Figure 3b). Finally, the drop in performance during the validation period is mostly attributable to a degraded ability to match summer peaks but not overall performance, which is comparable to the results from the calibration period.

4. Discussion and Conclusions

The three ET products provided a satisfactory to unsatisfactory performance when compared to SWAT-simulated ET. Generally, MODIS16A2v5 and SSEBop outperformed GLEAM 3.1a. This could be due to the course spatial resolution of GLEAM, compared with the other products. The effects of averaging remotely sensed observations within an area with complex terrain can obviously lead to spurious results. The ET results from the MODIS16A2v5 product had the most utility in limiting the parameter space for subsequent streamflow model runs. The proposed approach avoids the pitfall of adjusting ET parameters based on streamflow, to compensate for other processes. Therefore, the high streamflow objective metric derived during the calibration period did not come at the expense of an unrealistic depiction of other processes. Alemayehu et al. [45] suggested that to maintain a correct water balance CANMX and GWQMN were maximized, and ESCO and ALPHA_BF should be minimized, which translates into better flow performance in an examination of a data-scarce East African catchment. In this study, CANMX and ALPHA_BF were constrained by soft data (see Materials and Methods section). The relatively high ESCO values (0.74 to 0.96) for the superior MODIS16A2v5 product supports the supposition, that there was a relatively low evaporative demand from lower levels of the soil. These results are consistent with those of [16], who noted that evaporative drying during the summer is focused in the top soil levels. As the warm season progresses, the difference between the surface and the deeper root zone layers increases, implying that limited evaporative demand is contributed from the deeper soil levels. The high sensitivity of all three soil parameters simulated implies that there is a connection between actual evaporation and soil water. Water in soils can support directly recessional flow by lateral flow and by contributing water to the deeper subsurface. The parameters that mediate the transfer of water to aquifers below the soils (GW_Delay and GWQMN) had no sensitivity to ET, which is not surprising. These results imply that lateral flow might provide an important contribution to recessional flow in this system. While evaporation obviously has an insignificant impact on groundwater this flux can modulate soil moisture, thus influencing the contribution of lateral flow to the overall seasonal recession of flow. A final supporting observation is that reasonable fraction of baseflow was noted in the range of 0.627 to 0.772 [46], which was observed for all streamflow scenarios in the Upper North Fork basin (Table 5).
The main utility of this approach is the improvement in performance for not only overall flow, but also for recessional and summer peak flows that are typically not tuned for, within a standard autocalibration routine. This is best highlighted by a comparison of the Base Q and the final Optimized Q scenarios. In the Upper North Fork basin, there was a dramatic boost in performance from satisfactory to very good, based on the examination of overall, recessional, and summertime peak flows for the calibration period. This difference in performance is illustrated Figure 3. The Base Q simulations exhibit a wider spread for recessional flow, and a greater tendency to overestimate peak flow outside the main snow melt season (see arrows, Figure 3a). Conversely, the final Optimized Q model runs provided a much better match for both recessional and non-snow melt peak flows (see arrows, Figure 3b).
By better constraining actual evapotranspiration, more robust water supply forecasts can be derived. In the past, in alpine settings, water resources outside the spring snowmelt season have been poorly predicted from the SWAT model [1]. The proposed approach could be coupled with new methods for accounting for snow packs, producing even more robust streamflow simulations [47,48]. Also, this approach has a potential utility beyond the alpine setting, where the hydrology has a distorting signal; for example, with watersheds in which there are streamflow performance issues resulting from excessive irrigation [49] or large reservoir releases [50].
The virtue of the proposed approach is that it can provide an independent constraint on the land surface conditions that produce runoff. As such, the potential exists to provide a way to optimize streamflow performance using a multi-process approach, avoiding the problem of over-tuning based on peak annual flows. Given the proliferation of remote sensing products, the applications of these datasets has been already applied to the SWAT model for not only evapotranspiration (GLEAM, [5]; MODIS, [51,52,53]) but also for soil moisture (Soil Moisture Ocean Salinity, [54]; Soil Moisture Active Passive, [55]), and total terrestrial water (Gravity Recovery and Climate Experiment, [56]). The hydrologic community has the capability to derive independent estimates of parameters for most hydrological fluxes. The community needs to better embrace the fusion of multiple observational datasets to support more robust model calibration.

Author Contributions

K.J.T. obtained the funding, conceived and designed the experiment, processed and analyzed the data, and wrote the paper. M.E.B. developed visualizations and reviewed and edited the paper.


We acknowledge the support of the National Science Foundation Geoscience Equipment (Award Number 1636769), and of NASA Climate Indicators and Data Products for National Climate Assessments program (Award Number NNX16AH30G).


The assistance of research assistants Arturo Diaz and Roberto Torres is greatly appreciated.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.


  1. Ahl, R.S.; Woods, S.W.; Zuuring, H.R. Hydrologic calibration and validation of SWAT on a dominated Rocky Mountain watershed, Montana, U.S.A. J. Am. Water Resour. Assoc. 2008, 44, 1411–1430. [Google Scholar] [CrossRef]
  2. Chiang, L.-C.; Yuan, Y.; Mehaffey, M.; Jackson, M.; Chaubey, I. Assessing SWAT’s performance in the Kaskaskia River watershed as influenced by the number of calibration stations used. Hydrol. Process. 2014, 28, 676–687. [Google Scholar] [CrossRef]
  3. Beven, K.; Freer, J. Equifinality, data assimilation, and uncertainty estimation in mechanistic modeling of complex environmental systems using the GLUE methodology. J. Hydrol. 2001, 249, 11–29. [Google Scholar] [CrossRef]
  4. Immerzeel, W.W.; Droogers, P. Calibration of a distributed hydrological model based on satellite evapotranspiration. J. Hydrol. 2008, 349, 411–424. [Google Scholar] [CrossRef]
  5. Tobin, K.J.; Bennett, M.E. Constraining SWAT calibration with remotely sensed evapotranspiration data. J. Am. Water Resour. Assoc. 2017, 53, 593–604. [Google Scholar] [CrossRef]
  6. Chen, Y.; Marek, G.W.; Marek, T.H.; Brauer, D.K.; Srinivasan, R. Assessing the Efficacy of the SWAT Auto-Irrigation Function to Simulate Irrigation, Evapotranspiration, and Crop Response to Management Strategies of the Texas High Plains. Water 2017, 9, 509. [Google Scholar] [CrossRef]
  7. Feng, Q.; Chaubey, I.; Cibin, R.; Engel, B.; Sudheer, K.P.; Volenec, J. Simulating Establishment Periods of Switchgrass and Miscanthus in the Soil and Water Assessment Tool (SWAT). Trans. ASABE 2017, 60, 1621–1632. [Google Scholar] [CrossRef]
  8. Rajib, M.A.; Merwade, V. Improving soil moisture accounting and streamflow prediction in SWAT by incorporating a modified time-dependent Curve Number method. Hydrol. Process. 2016, 30, 603–624. [Google Scholar]
  9. Seibert, J.; McDonnell, J.J. On the dialog between experimentalist and modeler in catchment hydrology: Use of soft data for multicriteria model calibration. Water Resour. Res. 2002, 38, WR000978. [Google Scholar] [CrossRef]
  10. Arnold, J.G.; Youssef, M.A.; Yen, H.; White, M.J.; Sheshukov, A.Y.; Sadeghi, A.M.; Moriasi, D.N.; Steiner, J.L.; Amatya, D.M.; Skaggs, R.W.; et al. Hydrological processes and model representation: Impact of soft data on calibration. Trans. ASABE 2015, 58, 1637–1660. [Google Scholar]
  11. Winsemius, H.C.; Schaefli, B.; Montanari, A.; Savenije, H.H.G. On the calibration of hydrological models in ungauged basins: A framework for integrating hard and soft hydrological information. Water Resour. Res. 2009, 45, W12422.1–W12422.15. [Google Scholar] [CrossRef]
  12. Kang, H.; Moon, J.; Shin, Y.; Ryu, J.; Kum, D.H.; Jang, C.; Choi, J.; Kong, D.S.; Lim, K.J. Modification of SWAT autocalibration for accurate flow estimation at all flow regimes. Paddy Water Environ. 2016, 14, 499–508. [Google Scholar] [CrossRef]
  13. Guse, B.; Reusser, D.E.; Fohrer, N. How to improve the representation of hydrologic processes in SWAT for a lowland catchment—Temporal analysis of parameter sensitivity and model performance. Hydrol. Process. 2014, 28, 2651–2670. [Google Scholar] [CrossRef]
  14. Omani, N.; Srinivasan, R.; Karthikeyan, R.; Smith, P.K. Hydrological modeling of highly glacierized basins (Andes, Alps, and Central Asia). Water 2017, 9, 111. [Google Scholar] [CrossRef]
  15. McNamara, J.; Chandler, D.; Seyfried, M.; Achet, S. Soil moisture states, lateral flow, and streamflow generation in a semi-arid, snowmelt-driven catchment. Hydrol. Process. 2005, 19, 4023–4038. [Google Scholar] [CrossRef]
  16. Clayton, J.A. Summer soil moisture loss at Utah SNOTEL sites and streamflow recession at nearby gauges: Variability in runoff generation and the potential for flow forecasting. Hydrol. Process. 2016, 30, 119–134. [Google Scholar] [CrossRef]
  17. Rahman, K.; Maringanti, C.; Beniston, M.; Widmer, F.; Abbaspour, K.; Lehmann, A. Streamflow modeling in a highly managed mountainous glacier watershed using SWAT: The Upper Rhone River Watershed case in Switzerland. Water Resour. Manag. 2013, 27, 323–339. [Google Scholar] [CrossRef]
  18. Knowles, J.F.; Harpold, A.A.; Cowie, R.; Zeliff, M.; Barnard, H.R.; Burns, S.P.; Blanken, P.D.; Morse, J.F.; Williams, M.W. The relative contributions of alpine and subalpine ecosystems to the water balance of a mountainous, headwater catchment. Hydrol. Process. 2015, 29, 4794–4808. [Google Scholar] [CrossRef]
  19. Yin, Z.; Feng, Q.; Zou, S.; Yang, L. Assessing variation in water balance components in mountainous inland river basin experiencing climate change. Water 2016, 8, 472. [Google Scholar] [CrossRef]
  20. Maddox, R.A.; Hoxit, L.R.; Chappell, C.F.; Caracena, F. Comparison of meteorological aspects of the Big Thompson and Rapid City flash floods. Mon. Weather Rev. 1978, 106, 375–389. [Google Scholar] [CrossRef]
  21. Petersen, W.A.; Carey, L.D.; Rutledge, S.A.; Knievel, J.C.; Doesken, N.J.; Johnson, R.H.; McKee, T.B.; Haar, T.V.; Weaver, J.F. Mesoscale and radar observations of the Fort Collins flash flood of 28 July 1997. Bull. Am. Meteorol. Soc. 1999, 80, 197–216. [Google Scholar] [CrossRef]
  22. Gesch, D.; Oimoen, M.; Greenlee, S.; Nelson, C.; Steuck, M.; Tyler, D. The National Elevation Dataset. Photogramm. Eng. Remote Sens. 2002, 68, 5–32. [Google Scholar]
  23. Gesch, D.B. The National Elevation Dataset. In Digital Elevation Model Technologies and Applications: The DEM Users Manual, 2nd ed.; Maune, D., Ed.; American Society for Photogrammetry and Remote Sensing: Bethesda, MD, USA, 2007; pp. 99–118. [Google Scholar]
  24. Fry, J.; Xian, G.; Jin, S.; Dewitz, J.; Homer, C.; Yang, L.; Barnes, C.; Herold, N.; Wickham, J. Completion of the 2006 National Land Cover Database for the conterminous United States. Photogramm. Eng. Remote Sens. 2011, 77, 858–864. [Google Scholar]
  25. Soil Survey Staff. Natural Resources Conservation Service, Web Soil Survey. 2016. Available online: (accessed on 1 July 2016). [Google Scholar]
  26. U.S. Geological Survey. National Water Information System Data Available on the World Wide Web (USGS Water Data for the Nation). Available online: (accessed on 1 July 2016).
  27. PRISM Climate Group, Oregon State University. Available online: (accessed on 1 July 2016).
  28. Didan, K. MOD16A2: MODIS/Terra Net Evapotranspiration 8-Day L4 Global 500 m SIN Grid V006 Was Retrieved from the online Data Pool, Courtesy of the NASA Land Processes Distributed Active Archive Center (LP DAAC), USGS/Earth Resources Observation and Science (EROS) Center, Sioux Falls, South Dakota. Available online: (accessed on 1 July 2016).
  29. Savoca, M.E.; Senay, G.B.; Maupin, M.A.; Kenny, J.F.; Perry, C.A. Actual Evapotranspiration Modeling Using the Operational Simplified Surface Energy Balance (SSEBop) Approach; U.S. Geological Survey Scientific Investigations Report 2013-5126; U.S. Geological Survey: Reston, VA, USA, 2013; 16p. Available online: (accessed on 1 July 2016).
  30. Senay, G.B.; Bohms, S.; Singh, R.K.; Gowda, P.H.; Velpuri, N.M.; Alemu, H.; Verdin, V.P. Operational Evapotranspiration Mapping Using Remote Sensing and Weather Datasets: A New Parameterization for the SSEB Approach. J. Am. Water Resour. Assoc. 2013, 49, 577–591. [Google Scholar] [CrossRef]
  31. Miralles, D.G.; Holmes, T.R.H.; De Jeu, R.A.M.; Gash, J.H.; Meesters, A.G.C.; Dolman, A.J. Global land-surface evaporation estimated from satellite-based observations. Hydrol. Earth Syst. Sci. 2011, 15, 453–469. [Google Scholar] [CrossRef][Green Version]
  32. Martens, B.; Miralles, D.G.; Lievens, H.; van der Schalie, R.; de Jeu, R.A.M.; Fernández-Prieto, D.; Beck, H.E.; Dorigo, W.A.; Verhoest, N.E.C. GLEAM v3: Satellite-based land evaporation and root-zone soil moisture. Geosci. Model Dev. 2017, 10, 1903–1925. [Google Scholar] [CrossRef]
  33. Arnold, J.G.; Fohrer, N. SWAT2000: Current capabilities and research opportunities In applied watershed modeling. Hydrol. Process. 2005, 19, 563–572. [Google Scholar] [CrossRef]
  34. Gassman, P.W.; Reyes, M.R.; Green, C.; Arnold, J.G. The Soil and Water Assessment Tool: Historical development, applications, and future research directions. Trans. ASABE 2007, 50, 1211–1250. [Google Scholar] [CrossRef]
  35. Williams, J.R. Flood routing with variable travel time or variable storage coefficients. Trans. ASABE 1969, 12, 100–103. [Google Scholar] [CrossRef]
  36. Neitsch, S.L.; Arnold, J.G.; Kiniry, J.R.; Srinivasan, R.; Williams, J.R. Soil and Water Assessment Tool User’s Manual; Texas Water Resource Institute Report TR-192; Texas Water Resource Institute: College Station, TX, USA, 2002. [Google Scholar]
  37. Moriasi, D.N.; Starks, P.J.; Steiner, J.L.; Guzman, J.A.; Allen, P.B.; Naney, J.W. Upper Washita River Experimental Watersheds: Physiography data. J. Environ. Qual. 2014, 43, 1298–1309. [Google Scholar] [CrossRef]
  38. Abbaspour, K.C.; Yang, J.; Maximov, L.; Siber, R.; Bogner, K. Modeling hydrology and water quality in the pre-alpine/alpine Thur watershed using SWAT. J. Hydrol. 2007, 333, 413–430. [Google Scholar] [CrossRef]
  39. Yang, J.; Reichert, P.; Abbaspour, K.C.; Xia, J.; Yang, H. Comparing uncertainty analysis techniques for SWAT application to the Chaoche Basin in China. J. Hydrol. 2008, 358, 1–23. [Google Scholar] [CrossRef]
  40. Wu, H.; Chen, B. Evaluating uncertainty estimates in distributed hydrological modeling for the Wenjing River watershed in China by GLUE, SUFI-2, and Para Sol methods. Ecol. Eng. 2015, 76, 110–121. [Google Scholar] [CrossRef]
  41. Uniyal, B.; Jha, M.K.; Verma, A.K. Parameter identification and uncertainty analysis for simulating streamflow in a river basin of Eastern India. Hydrol. Process. 2015, 29, 3744–3766. [Google Scholar] [CrossRef]
  42. Arnold, J.G.; Allen, P.M. Automated methods for estimating baseflow and groundwater recharge from stream flow records. J. Am. Water Resour. Assoc. 1999, 35, 411–424. [Google Scholar] [CrossRef]
  43. Miralles, D.G.; Gash, J.H.; Holmes, T.R.H.; de Jeu, R.A.M.; Dolman, A.J. Global canopy interception from satellite observations. J. Geophys. Res. 2010, 115, D16122. [Google Scholar] [CrossRef]
  44. Afinowicz, J.D.; Munster, C.L.; Wilcox, B.P. Modeling effects of brush management on the rangeland water budget. Edwards Plateau, Texas. J. Am. Water Resour. Assoc. 2005, 41, 181–193. [Google Scholar] [CrossRef]
  45. Alemayehu, T.; van Griensven, A.; Bauwens, W. Evaluating CFSR and WATCH data as input to SWAT for the estimation of the potential evapotranspiration in a data-scarce Eastern-African Catchment. J. Hydrol. Eng. 2015, 21, 05015028. [Google Scholar] [CrossRef]
  46. Wolock, D.M. Base-Flow Index Grid for the Conterminous United States; U.S. Geological Survey Open-File Report 03-263; U.S. Geological Survey: Reston, VA, USA, 2003.
  47. Zhang, X.; Srinivasan, R.; Debele, B.; Hao, F. Runoff simulation of the headwaters of the Yellow River using the SWAT model with three snowmelt algorithms. J. Am. Water Resour. Assoc. 2008, 44, 48–61. [Google Scholar] [CrossRef]
  48. Debele, B.; Srinivasan, R.; Gosain, A.K. Comparison of process-based and temperature-index snowmelt modeling in SWAT. Water Resour. Manag. 2010, 24, 1065–1088. [Google Scholar] [CrossRef]
  49. Marek, G.; Gowda, P.; Evett, S.; Srinivasan, R. Calibration and validation of the SWAT model for predicting daily ET over irrigated crops in the Texas High Plains using lysimetric data. Trans. ASABE 2016, 59, 611–622. [Google Scholar] [CrossRef]
  50. Faramarzi, M.; Srinivasan, R.; Iravani, M.; Bladon, K.D.; Abbaspour, K.; Zehnder, A.J.B.; Goss, G.G. Setting up a hydrological model of Alberta: Data discrimination analyses prior to calibration. Environ. Model. Softw. 2015, 74, 48–65. [Google Scholar] [CrossRef]
  51. Abiodun, O.O.; Guan, H.; Post, V.E.A.; Batelaan, O. Comparison of MODIS and SWAT evapotranspiration over a complex terrain at different spatial scales. Hydrol. Earth Syst. Sci. 2018, 22, 2775–2794. [Google Scholar] [CrossRef][Green Version]
  52. Parajuli, P.B.; Jayakody, P.; Ouyang, Y. Evaluation of Using Remote Sensing Evapotranspiration Data in SWAT. Water Resour. Manag. 2018, 32, 985–996. [Google Scholar] [CrossRef]
  53. Wambura, F.J.; Dietrich, O.; Lischeid, G. Improving a distributed hydrological model using evapotranspiration-related boundary conditions as additional constraints in a data-scarce river basin. Hydrol. Process. 2018, 32, 759–775. [Google Scholar] [CrossRef]
  54. Patil, A.; Ramsankaran, R.A.A.J. Improving streamflow simulations and forecasting performance of SWAT model by assimilating remotely sensed soil moisture observations. J. Hydrol. 2017, 555, 683–696. [Google Scholar] [CrossRef]
  55. Zhang, Y.; Hou, J.; Gu, J.; Huang, C.; Li, X. SWAT-Based hydrological data assimilation system (SWAT-HDAS): Description and case application to river basin-scale hydrological predictions. J. Adv. Model. Earth Syst. 2017, 9, 2863–2882. [Google Scholar] [CrossRef]
  56. Qiao, L.; Herrmann, R.B.; Pan, Z. Parameter Uncertainty Reduction for SWAT Using Grace, Streamflow, and Groundwater Table Data for Lower Missouri River Basin. J. Am. Water Resour. Assoc. 2013, 49, 343–358. [Google Scholar] [CrossRef]
Figure 1. Locality map showing the position of the Upper North Fork basins in Idaho, USA. The larger map shows Nash Sufficliffe values within this watersheds sub-basin, based on comparison of simulated ET with the MODIS13A2v5 ET product (described below).
Figure 1. Locality map showing the position of the Upper North Fork basins in Idaho, USA. The larger map shows Nash Sufficliffe values within this watersheds sub-basin, based on comparison of simulated ET with the MODIS13A2v5 ET product (described below).
Water 11 00112 g001
Figure 2. (a) Soil types and (b) land use types within the Upper North Fork basin.
Figure 2. (a) Soil types and (b) land use types within the Upper North Fork basin.
Water 11 00112 g002
Figure 3. Streamflow hydrographs for the calibration period. (a) Base Q scenario versus observed streamflow. (b) Optimized Q scenario versus observed streamflow. Arrows 1 point to recessional flow and arrows 2, peak flows outside the main snow melt season.
Figure 3. Streamflow hydrographs for the calibration period. (a) Base Q scenario versus observed streamflow. (b) Optimized Q scenario versus observed streamflow. Arrows 1 point to recessional flow and arrows 2, peak flows outside the main snow melt season.
Water 11 00112 g003
Table 1. Soil name, texture, and areal percentages within the watershed.
Table 1. Soil name, texture, and areal percentages within the watershed.
Soil NameMukeyTextureAreal Percentage
Vay662090Silty loam17.33
Prospectors Variant661986Silty loam31.30
Vay662085Silty loam10.89
Buckhouse Family661824Silty loam12.89
Nakama Family662086Silty loam10.10
Honey jones661952Silty loam6.97
Narnett662001Silty loam5.94
Rubble Land (Latour-Honey Jones-Ahrs)661997Sandy loam4.51
Table 2. Streamflow parameters for the Upper North Fork Basin. R indicates change by Multiplication and A, addition. All other changes are by replacement or absolute adjustments. Numbers in bold indicate a highly sensitive parameter (p < 0.02).
Table 2. Streamflow parameters for the Upper North Fork Basin. R indicates change by Multiplication and A, addition. All other changes are by replacement or absolute adjustments. Numbers in bold indicate a highly sensitive parameter (p < 0.02).
ParametersInitial ParametersBase Q ParametersFinal Parameters
CN2 (Initial SCS Runoff Curve Number for Moisture Condition II)35.0 to 58.136.5 to 56.551.85 to 51.89
CH_N2 (Manning’s “n” Value for the Main Channel)0 to 0.30.826 to 0.28580.1360 to 0.1799
CH_K2 (Hydraulic Conductivity of the Main Channel Alluvium, mm/h)0 to 50036.7 to 183.261.99 to 63.15
CH_N1 (R; Manning’s “n” Value for the Tributary Channel)0 to 0.30.0283 to 0.27490.0467 to 0.0822
CH_K1 (R; Hydraulic Conductivity the Tributary Alluvium mm/h)0 to 30026.5 to 261.195.9 to 119.5
OV_N (Manning’s “n” Value for Overland Flow)0.01 to 0.60.0304 to 0.58320.0756 to 0.0908
SURLAG (Surface Runoff Lag Coefficient)1 to 342.40 to 26.7517.07 to 26.28
ALPHA_BF (Baseflow Alpha Factor)0.0385 to 0.10750.0612 to 0.10510.0777 to 0.0952
GW_DELAY (A; Groundwater Delay Time, days)30 to 450108.7 to 413.7150.9 to 213.6
GW_REVAP (A; Groundwater “Reevap” Coefficient)0.02 to 0.20.0275 to 0.19610.0986 to 0.1448
GWQMN (A; Threshold Depth of Shallow Aquifer Water Required for Return Flow to Occur, mm)0 to 5000762 to 35721354 to 3321
ESCO (Soil Evaporation Compensation Factor)0 to 10.1345 to 0.76050.8227 to 0.964
REVAPMN (A; Depth to Shallow Aquifer for “Reevap” or Percolation to Deep Aquifer, mm)0 to 50042.2 to 483.251.4 to 251.4
CANMX (A; Maximum Canopy Storage, mm)0 to 10033.95 to 86.951.738 to 1.931
EPCO (Plant Uptake Compensation Factor)0 to 10.0315 to 0.97450.6120 to 0.6196
SOL_AWC (R; Available Water Capacity of the Soil Layer (mm H2O/mm soil)−0.2 to 0.4−0.1697 to 0.36310.2879 to 0.3852
SOL_K (R; Saturated Hydraulic Conductivity, mm/h)−0.8 to 0.8−0.6680 to 0.74800.2810 to 0.5638
SOL_BD (R; Moisture Bulk Density, g/cm3)−0.5 to 0.6−0.4159 to 0.5687−0.2916 to 0.3192
SFTMP (Snowfall temperature, °C)−5 to 5−0.835 to 4.8951.118 to 1.358
SMTMP (Snow melt base temperature, °C)−5 to 5−3.205 to 4.8953.613 to 4.407
SMFMX (Melt factor for snow on June 21, mm H2O/°C-day)0 to 100.705 to 9.6052.754 to 5.402
SMFMN (Melt factor for snow December 21, mm H2O/°C-day)0 to 100.755 to 7.9250.916 to 2.181
TIMP (Snow pack temperature lag factor, °C)0.01 to 10.0362 to 0.95000.2089 to 0.3036
Table 3. Performance metrics used to evaluate model performance (modified from Moriasi et al., 2007).
Table 3. Performance metrics used to evaluate model performance (modified from Moriasi et al., 2007).
ThresholdNSAbsolute MBE
Very Good0.7510%
Table 4. Consistently sensitive evapotranspiration (ET) parameter value ranges, Nash–Sutcliffe efficiency (NS) performance, and bias from the Upper North Fork basin.
Table 4. Consistently sensitive evapotranspiration (ET) parameter value ranges, Nash–Sutcliffe efficiency (NS) performance, and bias from the Upper North Fork basin.
MODIS16A20.7435 to 0.95850.9789 to 2.3200.2719 to 0.3985−0.7688 to 0.7656−0.3773 to 0.32440.684219.1%
SSEBop0.4505 to 0.78851.146 to 2.2500.3565 to 0.3913−0.7912 to 0.3448−0.4521 to 0.57860.613820.7%
GLEAM 3.1a0.1525 to 0.56051.070 to 2.2690.2557 to 0.3943−0.7832 to −0.6568−0.4951 to 0.11650.263738.0%
Table 5. Upper North Fork streamflow. (a) Calibration and (b) validation results.
Table 5. Upper North Fork streamflow. (a) Calibration and (b) validation results.
a. CalibrationBase Q (n = 11)Best Q (MODIS; n = 8)Best Q (SSEBop; n = 2)Optimized Q (n = 2)
Overall- NS0.51 to 0.730.51 to 0.690.53 to 0.660.77 to 0.82
Overall- Bias5.5 to 24.4%0.2 to 17.7%5.5 to 16.7%3.3 to 6.1%
Baseflow Ratio0.627 to 0.7720.716 to 0.7660.736 to 0.7660.739 to 0.762
Recessional- NS0.54 to 0.770.56 to 0.720.51 to 0.620.77 to 0.86
Recessional- Bias1.1 to 24.2%1.9 to 23.9%3.5 to 13.1%7.0 to 8.7%
Summer Peak- NS0.66 to 0.940.67 to 0.910.69 to 0.780.80 to 0.89
Summer Peak- Bias−24.3 to 19.1%−22.8 to 24.7%−22.6 to −23.9%−5.1 to 2.4%
Performance ThresholdSatisfactorySatisfactorySatisfactoryVery Good
b. ValidationBase Q (n = 11)Best Q (MODIS; n = 8)Best Q (SSEBop; n = 2)Optimized Q (n = 2)
Overall- NS0.36 to 0.770.42 to 0.720.58 to 0.750.78 to 0.79
Overall- Bias9.0 to 27.9%6.1 to 21.2%7.5 to 22.2%8.8 to 10.6%
Baseflow Ratio0.608 to 0.7700.708 to 0.7610.739 to 0.7580.737 to 0.758
Recessional- NS0.33 to 0.790.59 to 0.750.60 to 0.780.80 to 0.81
Recessional- Bias5.9 to 30.3%8.0 to 23.6%6.6 to 16.1%12.1 to 14.7%
Summer Peak- NS−4.25 to 0.60−0.82 to 0.44−0.06 to 0.440.47 to 0.57
Summer Peak- Bias−98.7 to 27.2%−34.6 to 23.1%−28.6 to −30.3%−2.2 to 1.8%
Performance ThresholdUnsatisfactory to SatisfactoryUnsatisfactoryUnsatisfactoryUnsatisfactory to Satisfactory

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (
Back to TopTop