Agricultural in Asian monsoon region feeds about 60% of the world’s population by using 30% of the arable land, and by approximately 40 years after World War II, most Asia countries had achieved self-sufficiency in rice . The rainfall in an Asian monsoon region is concentrated on crop-growing season especially summer, so most of the nonpoint source pollution from crop land occurred during summer season. In paddy rice fields, dikes and forced drain alter the hydrological characteristics, as well as the effects from excessive nutrient supply and fertilization to ponded water quality . These characteristics of paddy rice fields make simulation of pollutant load and water movement more cumbersome.
Many researchers have tried to modify existing models or develop a new model to simulate the water budget, pollutant behavior in paddy rice fields. Most of them are field scale models such as PADDIMOD , GLEAMS-PADDY , RICEWQ , and the nitrogen balance model . Some researchers developed watershed scale models to simulate water and pollutant loading from mixed watershed including paddy rice field. Takeuchi et al.  developed a cell-based distributed hydro-environmental watershed model for water budget analysis with three-zone cell profiling which consists of a surface water zone with the modified tank model, a surface soil zone with the soil moisture model, and a groundwater zone with the unconfined shallow groundwater model. Soil and Water Assessment Tool (SWAT) was modified and has been applied for simulation in paddy rice fields for the watershed scale [8,9,10]. Jeon et al.  developed the HSPF-Paddy model to evaluate the effect of water and nutrient loading from paddy rice fields within complex watershed on the river environment by modifying Hydrological Simulation Program-Fortran (HSPF), and evaluated the developed model for the watershed scale as well as the field scale model with high model efficiency.
The important steps in modeling are calibration and validation processes. Calibration involves minimization of the difference between observed and simulated values by adjusting model parameters , while validation involves the use of the second set of independent information to ensure the calibration accuracy . Although newer versions of models continue to be released, much of recent research activity in model community is focused on automatic calibration by optimization techniques. The major advantage of optimization technique is to get better results without consuming significant efforts and time. Soil and Water Assessment Tool (SWAT) is currently linked with optimization technique, and has been used for automatic calibration [14,15,16]. Model-Independent Parameter Estimation and Uncertainty Analysis (PEST) is a nonlinear parameter estimation package, and widely used for automatic calibration tool of HSPF model [17,18,19,20].
The application of the model, developed to simulate a paddy rice field, to the observed data has been low due to the limited access for the model for the model to the public. In addition, there are a few study cases on characterizing water budget component of paddy rice field by comparing that of other land use types. In this study, HSPF-Paddy linked with PEST was calibrated and validated during the period between 1985 and 2001. The results were utilized to analyze the water budget components by land use type, and to characterize the water budget in paddy rice fields under an Asian monsoon climate compared with that under a Mediterranean climate.
2. Materials and Methods
2.1. Overview of HSPF-Paddy
The input and output time-series data such as meteorological data are stored in Watershed Data Management Utility (WDMUtil). Dike height and irrigation was added to the HSPF-Paddy model as WDM time-series. Surface runoff occurred when the water depth in paddy rice fields is greater than the dike height. The infiltration rate can be applied separately during growing and non-growing seasons using Special Action block in the HSPF-Paddy input file.
2.2. Overview of PEST
Model-Independent Parameters Estimator (PEST) is the software package for parameter estimation of complex environmental computer models. The time series processor (TSPROC), which is the surface utility of PEST, was developed to automatically calibrate HSPF. The TSPROC can manage the WDM time-series of observed and simulated values. Figure 2 shows the interrelation between the HSPF, WDM time-series, and TSPROC. HSPF reads meteorological data in WDM file, run, and store the output in WDM file. The TSPROC reads simulated and observed values from WDM file, parameter ranges, and generates PEST input file (*.pst). The PEST generates random numbers for calibration parameters, run HSPF, and calculates objective function.
The PEST employs the Gauss-Marquardt-Levenberg method which requires a continuous relationship between model parameters and model output. This method helps find the optimized parameter in fewer model runs than any other parameter estimation methods. The PEST minimizes a single weighted least-squares objective function, Φ:
2.3. Study Area
The study area is the Bochung watershed, Korea, where Bochung river crosses the watershed (Figure 3). The total area is 553.56 km2 and average elevation and slope of watershed are 263.93 m and 32.09%, respectively. The length of total streams and main stream are 1689.70 and 68.05 km, respectively. The last stream order is 7th and total number of 1st stream order is 2592. The watershed is a traditional rural area. The major land use type is forest (65%) and the urban area accounts for about 3% of the total area (Table 1). The cropped area occupies 28%, including 16% of paddy rice fields.
2.4. Modeling Approach
The land cover data which identifies sixteen possible land use types and digital elevation model (DEM) with 30 m cell size were achieved from Environmental Geographic Information System as GIS file . Hourly meteorological data, which contains air temperature, dew point temperature, cloud cover, wind speed, precipitation, and solar radiation, were obtained from Korea Meteorological Administration (KMA). Evapotranspiration and evaporation were calculated by the WDM Utility. Daily stream flow data at the mouth of watershed were collected from the Water Management Information System (WAMIS) .
Pre-processors of HSPF-Paddy including watershed segmentation, land cover overlay, and input file generation were performed using Better Assessment Science Integrating Point and Nonpoint Source (BASINS). Using DEM and BASINS Delination Utility, the Bochung watershed was divided into 21 sub-watersheds automatically. Using land cover and BASINS Land Use Definition Utility, a land use division was performed for each sub-watershed. The pervious area of the urban was set 10% of total urban area. Jeon et al.  studied regionalized CN parameters at Nakdong River Basin, Korea using eight watersheds, and reported that the hydrologic characteristics of the urban area in Korea represented 90% of the impervious area.
BASINS technical note 6 documented by USEPA guides researchers in estimating hydrology and hydraulic parameters for HSPF . The document provides “typical” and “possible” ranges for the HSPF parameters based on experiences with HSPF over the past two decades on watershed across the USA and abroad to be realistic parameters and reflect conditions on the watershed. The ranges of calibrated parameters are listed in Table 2 by referring BASINS technical note 6 . The calibration period is for 10-year (1992~2001), and the validation period is for 7 years (1985~1991). The PEST program searched optimized parameters within the range for each land use type.
|LZSN||Lower zone nominal soil moisture storage||5–38 cm|
|INFILT||Related to infiltration capacity of the soil||0.25–25 mm/h|
|KVARY||Groundwater recession flow parameter||0–7.62 cm–1|
|AGWRC||Groundwater recession rate||0.833–0.999 day–1|
|DEEPFR||Fraction of groundwater inflow to deep recharge||0.0–0.5|
|BASETP||Fraction of remaining ET from baseflow||0.0–0.05|
|CEPSC||Interception storage capacity||0.25–10.1 mm|
|UZSN||Upper zone nominal soil moisture storage||0.1–5.0 cm|
|NSUR||Manning’s n for overland flow||0.05–0.5|
|INTFW||Interflow inflow parameter||1.0–10.0|
|IRC||Interflow recession parameter||0.3–0.85|
|LZETP||Lower zone ET parameter||0.1–0.9|
The average values of parameters varied with the land cover, such as LZSN, INFILT, CEPSC, UZSN, NSUR, and LZETP, were calculated separately according to the land use type referred by BASINS technical note . Once the average values were determined for each land use type, maximum and minimum values are generated with the consideration of possible ranges shown in Table 2. The PEST program searched optimized values for calibration parameters within maximum and minimum values, and the ranges are shown in Table 3.
|Land use||LZSN (cm)||INFILT (mm/h)||CEPSC (mm)||UZSN (cm)||NSUR (none)||LZETP (none)|
Note: * apply at pervious area of urban.
Jeon et al.  evaluated calibration methods of stream flow using HSPF coupled with PEST program, and proposed the calibration based on monthly flow is more effective than that on daily flow for the yearly basis water budget analysis. Therefore, objective function for automatic calibration was performed by monthly stream flow in this study. The Nash-Sutcliff Efficiency (NSE)  has been widely used as a “goodness-of-fit” statistics indicator to access the goodness of model to simulate the flow . Ahmadi et al.  evaluate various statistical indices and concluded that root mean square (RMS) error, residual mass (RM) coefficient, and Nash-Sutcliff (NS) coefficient were better for the model calibration. In this study, RMS, NS, R2, and NS coefficients were used for calibration performance and these are as follows:
3. Results and Discussion
3.1. Calibration and Validation
Graphical comparison of yearly runoff between simulated and observed values is displayed in Figure 4. The mean annual runoffs, simulated and observed, are 654.5 and 719.2 mm for calibration and 685.6 and 700.5 mm for validation, respectively (Table 4). Donigian  listed general calibration/validation tolerances or target and the ranges should be applied to annual mean values as shown in Table 5. The mean annual volumes are within the 10% difference for the calibration and validation results, showing a very good agreement.
The graphical comparison of simulated and observed monthly flow and statistical analysis are depicted in Figure 5 and Figure 6, respectively, and listed in Table 6. The hydrology results reflected the observed data well (Figure 5). The monthly NS values for calibration and validation are greater than 0.80 and monthly residual mass (RM) coefficient are consistently less than 0.10. Donigian  provided value ranges for coefficient of determination (R2) for assessing model performance for both daily and monthly flows. As the criteria of model performance and Table 5, the HSPF-Paddy model performances illustrated very good agreement (R2 > 0.85 and RM < 0.1) for monthly calibration and validation. Overall, the hydrology results show a very good agreement based on annual and monthly comparisons by using statistical and graphical analysis, and can be used for water budget analysis.
|Calibration/validation||Observed (mm)||Simulated (mm)||Percent Mean Error (%)|
3.2. Optimized Parameters
Optimized HSPF parameters for each land use type are listed in Table 7. Vegetated land cover which includes paddy, crop, and forest areas can retain rainfall with high CEPSCS, UZSN, and LZSN values compared to non-vegetated area. Most of parameters are optimized by PEST within the possible range recommended by USEPA . The BASETP and INTFW values are optimized as upper and lower limit values, respectively. However, DEEPFR was not optimized within the possible range, showing 0.80, as the percent mean error for annual mean values couldn’t meet the criteria (Table 3) when PEST searched optimal DEEPFR value within the possible range. Some parameters of paddy rice field are outlier. The values for INFILT and NSUR are optimized under the lower limit values, indicating that, although the dike of paddy rice field can effectively reduce surface runoff by retaining rainfall, peak flow is higher and time for peak is shorter than other land use types once the surface runoff occurred.
3.3. Comparison of Water Budget Components
The results of water budget analysis during 1985–2001 are shown in Table 8. The influx for all land use types except paddy rice field is 1305 mm, contributed solely by rainfall, while that for paddy rice field is about 2695 mm with the addition of 1390 mm of irrigation. Total runoffs are ranged from 424 to 1697 mm, and are significantly influenced by land use types. Whereas runoff values for the pervious areas, such as crop, forest, pasture, and barren, show similar levels, the value for paddy rice field is higher than that for the urban area. This resulted from the low infiltration rate and manning’s coefficient despite the large irrigated volume.
|Total||1123 (86%)||424 (32%)||1697 (63%)||463 (35%)||536 (41%)||515 (39%)|
|Inactive GW||23 (2%)||267 (20%)||272 (10%)||206 (16%)||218 (17%)||578 (44%)|
|Total||159 (12%)||608 (47%)||736 (27%)||635 (49%)||550 (42%)||208 (16%)|
The rates of each runoff component are shown in Figure 7. Most of the runoff is generated from the surface except the barren area. The surface runoff rate of the urban area is about 98% due to the large impervious surface area. The rate of surface runoff from the vegetation area such as crop, paddy, forest, and pasture ranged from 58% to 74%, and the highest rate of 74% is from paddy rice field. The interflow runoff from the urban area is lowest accounting only 2% of total runoff, and that from the vegetation area ranged from 24% to 31% of total runoff. The highest rate of interflow runoff is from barren areas showing about 47% of total runoff. The runoff from base flow is not significant compared to other runoff sources. The rate of base flow runoff from the vegetation area is less than 11% and that from barren is highest with 22% of total runoff. Inflow to deep ground water from the urban area is just 23 mm and those from vegetated area are ranged from 206 to 272 mm. Inflow to deep ground water from barren is about 578 mm.
Hydrologic characteristics of barren are somewhat different with other pervious area and are summarized as less surface runoff and more interflow and base flow runoff. Those results are obviously influenced by the hydrologic soil condition as shown in Table 9. Soil can be divided into four groups according to water transmitting soil layers with the lowest saturated hydraulic conductivity. Hydrologic soil groups A, B, C, and D are characterized by low, moderately low, moderately high, and high runoff potential, respectively . Most of hydrologic soil group of pervious area is between B and C group however the area rate of hydrologic soil A of barren area is about 60% at the Bochung watershed. The calibrated infiltration rate of the barren area is highest with 11.68 mm/h, the high infiltration rate might reduce surface runoff increase interflow and base flow runoff.
|Land use||Calibrated infiltration rate (mm/h)||Hydrologic soil group (HSG)|
The rates of each evapotranspiration component are shown in Figure 8. The evapotranspiration is significantly influenced by land use type and ranged from 159 to 736 mm. The high impervious surface of the urban area makes to less evapotransrate compared with other land use type. The amounts of evapotranspiration from vegetated area are more than 500 mm/yr and those from paddy rice field are 736 mm/yr showing the highest value among the land use type compared. More than 50% of total evapotranspiration from vegetated area except paddy rice field occurred from lower zone storage within soil, while about 60% of total evaporanspiration from paddy rice fields are generated from the upper zone resulting from ponded water remained during growing season (Figure 8).
The water budget analysis of paddy rice fields in various countries was shown in Table 10. In Asian monsoon regions including Taiwan , Korea , and in this study, around 50% of total input water came from precipitation. More than 90% of total input water was feed from irrigation and precipitation was less than 10% in Mediterranean regions including Spain  and Greece  because most precipitation is concentrated during the winter~spring season. Most of the input water in paddy rice fields was lost by runoff in an Asian monsoon area whereas deep percolation was observed in a Mediterranean climate area. Overall, much of the water entered paddy rice fields by precipitation and was lost by runoff in paddy rice fields in an Asian monsoon region.
|Component||Mediterranean climate region||Asian monsoon region|
|Irrigation||1874 (93%)||3647 (94%)||624 (45%)||695 (34%)||1390 (52%)|
|Precipitation||150 (7%)||249 (6%)||756 (55%)||1355 (66%)||1305 (48%)|
|Evapotranspotation||731 (36%)||930 (24%)||503 (36%)||663 (31%)||736 (27%)|
|Runoff||372 (18%)||1118 (29%)||541 (39%)||1352 (63%)||1652 (61%)|
|Deep percolation||830 (41%)||1867 (48%)||336 (24%)||128 (6%)||317 (12%)|
The calibration results represented in this study at one particular site may have more uncertainties ; thus, further verification at various sites and water budget components may be necessary.
The HSPF-Paddy model coupled with PEST was applied to the Bochung watershed in Korea to analyze the characteristics of water budget components in paddy rice field under an Asian monsoon climate during 1985–2001. Water movement within the watershed was well simulated by HSPF-Paddy showing high model efficiency, and simulation results met the criteria of calibration and validation with the acceptable error level. The hydrological characteristics of paddy rice fields were somewhat different to those of other vegetated areas. Runoff from paddy rice fields was much higher than that from other land use types compared due to the low infiltration rate and manning’s coefficient as well as the irrigation, implying that most runoff is generated from the surface zone in paddy rice fields. Another characteristic of paddy rice fields is a significant amount of evapotranspiration compared with other vegetated areas resulting from the ponded water maintained during the growing season. The characteristics of climate influenced the water budget in paddy rice fileds. In an Asian monsoon area, much precipitation entered the paddy rice field and significant water was lost by runoff because most precipitation is concentrated on the summer season in contrast to a Mediterranean climate area.
This subject is supported by Korea Ministry of Environment as “Global Top Project” (Project No.: GT-11-B-02-004-1).
Young-Jin Kim wrote the paper and applied HSPF-Paddy linked with Model-Independent Parameters Estimator (PEST) program. Hae-Do Kim collaborated on calibration. Ji-Hong Jeon reviewed the model application and directed this research.
Conflicts of Interests
The authors declare no conflict of interest.
- Takase, K. Japan, Monsoon Asia and the global agricultural revolution in the 21st century. Paddy Water Environ. 2003, 1, 13–18. [Google Scholar] [CrossRef]
- Yoon, G.G.; Ham, J.-H.; Jeon, J.-H. Mass balance analysis in Korean paddy rice culture. Paddy Water Environ. 2003, 1, 99–106. [Google Scholar] [CrossRef]
- Jeon, J.-H.; Yoon, C.G.; Han, J.-H.; Jung, K.-W. Model development for surface drainage loading estimates from paddy rice fields. Paddy Water Environ. 2005, 3, 93–101. [Google Scholar] [CrossRef]
- Chung, S.O.; Kim, H.S.; Kim, J.S. Model development for nutrient loading from paddy rice fields. Agric. Water Manag. 2003, 62, 1–17. [Google Scholar] [CrossRef]
- Williams, M.W.; Ritter, A.M.; Cheplick, M.J.; Zdinak, C.E. RICEWQ: Pesticide runoff model for rice crops. In User Manual and Program Documentation; Version 1.6.1; Waterborne Environmental, Inc.: Leesburg, VA, USA, 1999. [Google Scholar]
- Antonopoulos, V.Z. Modeling of water and nitrogen balance in the ponded water of rice fields. Paddy Water Environ. 2008, 6, 387–395. [Google Scholar] [CrossRef]
- Takeuchi, J.; Kawachi, T.; Unami, K.; Meada, S.; Izumi, T. A distributed hydro-environmental watershed model with three-zoned cell. Paddy Water Environ. 2009, 7, 33–43. [Google Scholar] [CrossRef]
- Sakaguchi, A.; Eguchi, S.; Kato, T.; Kasuya, M.; Ono, K.; Miyata, A.; Tase, N. Development and evaluation of paddy module for improving hydrological simulation in SWAT. Agric. Water Manag. 2014, 137, 116–122. [Google Scholar] [CrossRef]
- Jung, C.-G.; Park, J.-Y.; Kim, S.-J.; Park, G.-A. The SRI (system of rice intensification) water management evaluation by SWAPP (SWAT-APEX Program) modeling in an agricultural watershed of South Korea. Paddy Water Environ. 2014, 12, 251–261. [Google Scholar] [CrossRef]
- Xie, X.; Cui, Y. Development and test of SWAT for modeling hydrological processes in irrigation districts with paddy rice. J. Hydrol. 2011, 1–2, 61–71. [Google Scholar] [CrossRef]
- Jeon, J.-H.; Yoon, C.G.; Donigian, A.S.; Jung, K.-W. Development of the HSPF-Paddy model to estimate watershed pollutant loads in paddy farming regions. Agric. Water Manag. 2007, 90, 75–86. [Google Scholar] [CrossRef]
- Jewell, T.K.; Nunno, T.J.; Adrian, D.D. Methodology for calibrating stormwater model. J. Environ. Eng. Div. 1978, 104, 485–501. [Google Scholar]
- U.S. Environmental Protection Agency (USEPA). Compendium of Tool for Watershed Assessment and TMDL Development; EPA841-B-97-006; USEPA, Office of Water: Washington, DC, USA, 1997. [Google Scholar]
- Shawul, A.A.; Alamirew, T.; Dinka, M.O. Calibration and validation of SWAT model and estimation of water balance components of Shaya mountainous watershed, Southeastern Ethiopia. Hydrol. Earth Syst. Sci. Discuss. 2013, 10, 13955–13978. [Google Scholar] [CrossRef]
- Wu, Y.; Liu, S.; Li, Z.; Dahal, D.; Young, C.J.; Schmidt, G.L.; Liu, J.; Davis, B.; Sohl, T.L.; Wemer, J.M.; et al. Development of a generic auto-calibration package for regional ecological modeling and application in the Central Plains of the United States. Ecol. Inform. 2014, 19, 35–46. [Google Scholar] [CrossRef]
- Wu, Y.; Liu, S. Improvement of the R-SWAT-FME framework to support multiple variables and multi-objective functions. Sci. Total Environ. 2014, 466–467, 455–466. [Google Scholar] [CrossRef]
- Kim, J.; Ryu, J. A threshold of basin discretization level for HSPF simulation with NEXRAD input. J. Hydrol. Eng. 2013. [Google Scholar] [CrossRef]
- Diaz-Ramirez, J.N.; Johnson, B.E.; McAnally, W.H.; Martin, J.L.; Alarcon, V.J. Estimation and propagation of parameter uncertainty in lumped hydrological models: A case study of HSPF model applied to Luxapallila Creek Watershed in Southeast USA. J. Hydrogeol. Hydrol. Eng. 2013. [Google Scholar] [CrossRef]
- Kim, S.M.; Benham, B.L.; Brannan, K.M.; Zeckoski, R.W.; Doherty, J. Comparison of hydrological calibration of HSPF using automatic and manual method. Water Resour. Res. 2007, 43. [Google Scholar] [CrossRef]
- Jeon, J.-H.; Lee, S.-B. Automatic calibration for noncontinuous observed data using HSPF-PEST. J. KoreanSoc. Agric. Eng. 2012, 54, 111–119. (In Korean). [Google Scholar]
- Environmental Geographic Information System. Available online: http://egis.me.go.kr/main.do (accessed on 13 May 2014).
- Water Management Information System. Available online: http://www.wamis.go.kr/ (accessed on 13 May 2014). (In Korean).
- Jeon, J.-H.; Choi, D.H.; Kim, J.J.; Kim, T.-D. Regionalization of CN parameters for Nakdong Basin using SCE-UA algorithm. J. Korean Soc. Water Qual. 2009, 25, 245–255. (In Korean). [Google Scholar]
- U.S. Environmental Protection Agency (USEPA). BASINS Technical Note 6: Estimating Hydrology and Hydraulic Parameters for HSPF; EPA-823-R00-012; USEPA, Office of Water: Washington, DC, USA, 2000. [Google Scholar]
- Jeon, J.-H.; Choi, D.; Kim, J.-J.; Kim, T.-D. Evaluating calibration methods of stream flow for water quality management. J. Korean Soc. Agric. Eng. 2009, 25, 432–440. (In Korean). [Google Scholar]
- Nash, J.E.; Sutcliffe, J.V. River flow forecasting through conceptual model, Part I: A discussion of principles. J. Hydrol. 1970, 10, 282–290. [Google Scholar] [CrossRef]
- Shrestha, N.K.; Shakti, P.C.; Gurung, P. Calibration and validation of SWAT model for Low Lying Watersheds: A case study on the Kliene Nete Watershed, Belgium. Hydrol. Nepal. J. Water Energy Environ. 2010, 6, 47–51. [Google Scholar]
- Ahmadi, S.H.; Amin, S.; Keshavarzi, A.R.; Mirzamostafa, N. Simulating watershed outlet sediment concentration using the ANSWERS model by applying two sediment transport capacity equation. Biosyst. Eng. 2006, 94, 615–626. [Google Scholar] [CrossRef]
- Donigian, A.S., Jr. Slide #L19–22; Washington, DC, USA, 10–14 January 2000, U.S. EPA Office of Water, Office of Science and Technology: Washington, DC, USA, 2000.
- Donigian, A.S. Watershed Model Calibration and Validation: The HSPF Experience, National TMDL Science and Policy Specialty Conference 2002; Water Environment Federation: Phoenix, AZ, USA, 2000. [Google Scholar]
- United States Department of Agriculture, Soil Conservation Service (USDA-SCS). National Engineering Handbook Chapter: Chapter 7: Hydrologic Soil Groups; USDA-SCS: Washington, DC, USA, 2004. [Google Scholar]
- Chien, C.-P.; Fang, W.-T. Modeling irrigation return flow for the return flow reuse system in paddy fields. Paddy Water Environ. 2012, 10, 187–196. [Google Scholar] [CrossRef]
- Song, J.-H.; Kang, M.-S.; Song, I.-H.; Jang, J.-R. Comparing farming methods in pollutant runoff loads from paddy fields using the CREAMS-PADDY model. Korean J. Environ. Agric. 2012, 31, 318–327. [Google Scholar] [CrossRef]
- Playán, E.; Pérez-Coveta, O.; Martínez-Cob, A.; Herrero, J.; García-Navarro, P.; Latorre, B.; Brufau, P.; Garcés, J. Overland water and salt flows in a set of rice paddies. Agric. Water Manag. 2008, 95, 645–658. [Google Scholar] [CrossRef]
- Wang, S.; Huang, J.; Li, J.; Rivera, A.; McKenney, D.W.; Sheffield, J. Assessment of water budget for sixteen large drainage basins in Canada. J. Hydrol. 2014, 512, 1–15. [Google Scholar] [CrossRef]
© 2014 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/3.0/).