# Hindcasting and Forecasting Total Suspended Sediment Concentrations Using a NARX Neural Network

## Abstract

**:**

^{2}. Land cover within the catchment consists mainly of native and exotic vegetation, eroded soil, and urban areas. Input data consisting of precipitation and stream flow time-series were fed to a NARX network for forecasting daily suspended sediments (SST) concentrations for years 2013–2014, and hindcasting for years 2008–2010. Training of the network was performed with daily SST, precipitation, and flow data from years 2012 and 2013. The resulting NARX net consisted of an open-loop, 12-node hidden layer, 100 iterations, using Bayesian regularization backpropagation. Hindcasting of daily and monthly SST concentrations for years 2008 through 2010 was successful. Daily SST concentrations for years 2013 and 2014 were forecasted successfully for baseflow conditions (R

^{2}= 0.73, NS = 0.71, and Kling-Gupta efficiency index (K-G) = 0.84). Forecasting daily SST concentrations for year 2014 was within acceptable statistical fit and error margins (R

^{2}= 0.53, NS = 0.47, K-G = 0.60, d = 0.82). Forecasting of monthly maximum SST concentrations for the two-year period (2013 and 2014) was also successful (R

^{2}= 0.69, NS = 0.60, K-G = 0.54, d = 0.84).

## 1. Introduction

## 2. Materials and Methods

#### 2.1. Study Area

^{2}[12]. Land cover within the catchment consist mainly of native and exotic vegetation, eroded soil, and urban areas. Soils in the Francia Creek catchment consist of weathered sedimentary rock (sand, clay, and silt) with a thin organic top layer [20]. The instability of these materials makes the watershed side slopes susceptible to mass removal processes, mainly because the sedimentary rock expands and mobilizes when rainfall increases [21]. Hillslopes are steeper than 40% and they are prone to erodible processes and mudslides during the rainy season. Water flow in the creek can be strong enough to suspend particles in the water column and transport them downstream. To avoid the sediment entering the stormwater sewer network, a sand trap was built at the lowest point of the catchment area (Figure 2). The sand trap accumulates up to 1500 m

^{3}of sediment annually. The accumulated sediment is cleaned up once a year and disposed in a sanitary landfill. Figure 2A shows the upstream side of the sand trap soon after it was cleaned up of debris. Figure 2B shows the sand trap almost full of debris and sediments after a series of strong precipitation events.

^{3}/s to 23 m

^{3}/s. It should be noted that those peaks are preceded by lower intensity precipitation events in days previous to the peak events. Figure 3 also shows measured and estimated SST concentrations corresponding to days when there was absence of rainfall and Francia Creek only carries baseflow (water seeping into the stream from the vadose zone even in the absence of rain). Concentrations ranging between 200 mg/L to 350 mg/L correspond to baseflows ranging between 3 L/s to 5 L/s. Baseflow is continuous throughout the year.

^{3}/s and SST concentrations of 4770 mg/L. The day after (11 July 2006), the rain continued to fall accumulating 53 mm of precipitation at the end of the day, generating flows of up to 9.90 m

^{3}/s and SST concentrations of up to 4320 mg/L. On 12 July 2006, it did not rain, but mudflow (generated in Francia Creek watershed) produced injuries to five people and damaged several cars in areas close to the watershed exit. In total, twenty people had to be evacuated. Two weeks later (24 July 2006), a precipitation event of almost 160 mm, that produced flows of up to 18 m3/s and SST concentrations of 5920 mg/L, did not produce consequences. The difference in this latter event was that precipitation intensity in days before and after the event was much milder than the recorded peak precipitation of 160 mm. Therefore, the antecedent conditions (rainfall and soil status) in days previous to the peak precipitation event, as well as rainfall and flow occurring in the following day, strongly influenced the generation of the mudflow incident.

#### 2.2. Forecasting or Predicting SST Time-Series

#### 2.3. Selection of Model Architecture

#### 2.4. Determination of Model Structure

#### 2.5. Input Data Selection and Data Splitting

#### 2.6. Model Calibration and Validation

^{2}); Nash–Sutcliffe coefficient (NS), Kling-Gupta efficiency index (K-G), and Willmott’s index of agreement (d). The root-mean-squared-error to standard deviation ratio (RSR) was calculated to quantify statistical errors, and percent bias (PBIAS) was computed to determine the bias of simulated data with respect to observed data [25]. Table 2 summarizes the statistical indicators used.

^{2}> 0.5, and NS > 0.50. However, the statistical indicators’ ranges need to be modified appropriately for shorter simulation time-steps. It is recommended to extend the acceptability ranges corresponding to monthly simulations by 20% to apply them to daily simulations. [36]. Nevertheless, the literature shows that the modeling community reports much wider and flexible ranges when assessing the quality of simulated data, which reflects the degree of difficulty on modeling and simulating SST concentrations. The following NS ranges for daily sediment concentration simulations are reported in Moriasi et al. [36]: −2.5 < NS < 0.11 for calibration and −3.51 < NS < 0.23 for validation. The following ranges were found by Ang and Oeurng [26]: NS > 0.38, PBIAS < 5.1%, and RSR < 0.79, for calibration. For validation: NSE > −6.61, PBIAS < −78.38%, and RSR < 2.67 are reported [37]. Kling-Gupta efficiencies within the range 0.5 < K-G < 1.0 are usually considered acceptable ([38].

^{2}, and K-G) were set up so that the prediction capabilities of the NARX model are better than satisfactory. The upper limit for the estimators of fit is 1.0, corresponding to perfect fit between simulated and observed data. On the other hand, correlation coefficient values greater than 0.7 are considered acceptable (R > 0.7); therefore, the acceptability range for the coefficient of determination would be: R

^{2}> 0.5. Moreover, Knoben et al. [38] (after performing a random sampling experiment) report that K-G = 0.5 correspond (in average) to NS = 0.5. Based on these considerations, Table 2 shows the statistical indicators acceptability ranges and formulae used in this research.

- Monthly time-step simulations: RSR < 0.7, −15% < PBIAS < 15%, R
^{2}> 0.6, NS > 0.6, and K-G > 0.6. - Daily time-step simulations: RSR < 0.79, −18% < PBIAS < 18%, R
^{2}> 0.5, NS > 0.5, and K-G > 0.5. - In cases in which simulations for SST produced unbalanced performance ratings, the Willmott’s index of agreement (d) was calculated, and if most statistical indicators were on the acceptable ranges, the simulation was rated “satisfactory”. Since in this research, d is used as a deciding qualifier of statistical fit, its threshold values of acceptability are more stringent than those of R
^{2}and NS indicators–for monthly simulations d > 0.78 and for daily simulations d > 0.65. - Ranges for PBIAS and RSR were assigned after [36].

## 3. Results

#### 3.1. NARX Training

#### 3.2. Calibration Results

^{2}, and K-G).

#### 3.3. Validation Results

#### 3.3.1. Hindcasting SST Concentrations

^{2}= 0.82, K-G = 0.83). Error indices (PBIAS and RSR) were also optimal—percent bias (PBIAS) was smaller than 14% and RSR (root mean square error normalized by the standard deviation of observed concentrations) was smaller than 0.52. Therefore, the neural network is able to hindcast concentration values occurring in Francia Creek under baseflow conditions (i.e., absence of rain).

^{2}= 0.64, NS = 0.55, K-G = 0.72). However, since NS is slightly lower than the required minimum value (Table 6), the Willmott’s index of agreement is calculated to further ascertain on the statistical fit of the simulated SST concentrations. The resulting index of agreement is d = 0.86 which confirms the good quality of the monthly SST mean concentrations forecasting.

^{2}= 0.69, NS = 0.69, and K-G = 0.71) categorize the NARX net output as a good estimation of monthly maximum SST concentrations. Therefore, the neural network performance in hindcasting was successful.

#### 3.3.2. Forecasting SST Concentrations

^{2}= 0.43 is not satisfactory. To settle this apparent contradiction, the Willmott’s index of agreement, d, was calculated. The computation of this statistical indicator of fit produced a value of d = 0.79, indicating good fit between simulated and observed SST concentrations. If simulation for one output variable produces unbalanced performance ratings (as in this case), then, the overall performance should be described conservatively as satisfactory [25].

^{2}, and NS) indicators are within the acceptable ranges. The statistical fit of simulated SST concentrations to observed data is further reinforced by the calculation of the Willmott’s index of agreement—d = 0.84 (indicating good statistical fit). Therefore, the NARX model is able to forecast the monthly maximum SST concentrations for years 2013 and 2014.

^{2}and NS present values lower than the acceptable threshold. Indicators of statistical error are also in disagreement—PBIAS = 3.41 reveals low levels of bias, while RSR = 0.89 is higher than the acceptable value (0.79). As in previous cases, the Willmott’s index of agreement (d) was calculated for having more insight into the quality of simulated values. The resulting value is d = 0.68. Therefore, the daily forecasting of SST concentrations for years 2013 and 2014 is satisfactory.

^{2}=0.53, NS = 0.47, K-G = 0.60). Moreover, the additional calculation of Willmott’s index of agreement yielded d = 0.82. Therefore, the statistical indicators for year 2014 reveal that forecasting for that year alone was of acceptable quality.

^{2}= 0.73, NS = 0.71, K-G = 0.84). Likewise, indicators of statistical error are well below the acceptable maximum values—PBIAS = 3.6% < 15%, RSR = 0.54 < 0.79. Moreover, the NARX net is able to capture daily SST concentration peaks occurring during the period of analysis. Therefore, the forecasting of daily SST concentrations occurring under baseflow conditions is of very good quality.

## 4. Discussion

^{2}= 0.73, NS = 0.71, and K-G = 0.84, which indicate very good quality prediction levels of daily SST concentrations under baseflow conditions. Indicators of statistical error were below the acceptable maximum values—PBIAS = 3.6% < 15%, RSR = 0.54 < 0.79, which demonstrates that the simulated daily SST data had low residual variation and low bias, characteristics of accurate forecasting.

^{2}= 0.53, NS = 0.47, K-G = 0.60, Willmott’s index of agreement d = 0.82. Therefore, the statistical indicators of fit were within acceptable margins. The forecasted SST concentrations underperformed in statistical error (RSR = 0.91 > 0.79) but overperformed bias indicators (PBIAS = −15.86% is within the range: −18% < PBIAS < 18%).

^{2}= 0.28, NS = 0.21, Willmott’s index of agreement, d = 0.68. Indicators of statistical error were PBIAS = 3.41 (low bias), RSR = 0.89 > 0.79. Therefore, the daily forecasting of SST concentrations for years 2013 and 2014 was satisfactory. However, monthly maximum SST concentrations for the two-year period (2013 and 2014) were well captured by the NARX network: RSR = 0.62 > 0.6, PBIAS = 18.87% < 20%, R

^{2}= 0.69 > 0.6, NS = 0.60 > 0.6, K-G = 0.54 > 0.5, d = 0.84 > 0.8.

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Conflicts of Interest

## References

- Zhang, X.; Chen, X.; Liu, M.; Xu, Z.; Wei, H. Coupled changes in soil organic carbon fractions and microbial community composition in urban and suburban forests. Sci. Rep.
**2020**, 10, 1–12. [Google Scholar] [CrossRef] - Parlak, M.; Everest, T.; Ruis, S.J.; Blanco, H. Impact of urbanization on soil loss: A case study from sod production. Environ. Monit. Assess.
**2020**, 192, 1–12. [Google Scholar] [CrossRef] [PubMed] - Alarcon, V.J.; Sassenrath, G.F. Nitrate, Total Ammonia, and Total Suspended Sediments Modeling for the Mobile River Watershed. Int. J. Agric. Environ. Inf. Syst.
**2017**, 8, 20–31. [Google Scholar] [CrossRef] - Forecasting Pollutant Concentration in River to Protect Drinking Water Production. Available online: https://www.scopus.com/inward/record.uri?eid=2-s2.0-85093908096&partnerID=40&md5=5aacaeb5c86709e4ea5569591815ccc0 (accessed on 15 November 2020).
- Tsai, L.-Y.; Chen, C.-F.; Fan, C.-H.; Lin, J.-Y. Using the HSPF and SWMM Models in a High Pervious Watershed and Estimating Their Parameter Sensitivity. Water
**2017**, 9, 780. [Google Scholar] [CrossRef] [Green Version] - Alarcon, V.J.; Johnson, D.; McAnally, W.H.; Van Der Zwaag, J.; Irby, D.; Cartwright, J. Nested Hydrodynamic Modeling of a Coastal River Applying Dynamic-Coupling. Water Resour. Manag.
**2014**, 28, 3227–3240. [Google Scholar] [CrossRef] - Alarcon, V.J. The Role of Boundary Conditions in Water Quality Modeling; Lecture Notes in Computer Science; Springer: Berlin, Germany, 2014; pp. 721–733. [Google Scholar] [CrossRef]
- Yang, W.; Chen, H.; Xu, C.-Y.; Huo, R.; Chen, J.; Guo, S. Temporal and spatial transferabilities of hydrological models under different climates and underlying surface conditions. J. Hydrol.
**2020**, 591, 125276. [Google Scholar] [CrossRef] - Ji, L.; Wu, T.; Xie, Y.; Huang, G.; Sun, L. A novel two-stage fuzzy stochastic model for water supply management from a water-energy nexus perspective. J. Clean. Prod.
**2020**, 277, 123386. [Google Scholar] [CrossRef] - Dikshit, A.; Pradhan, B.; Alamri, A.M. Long Lead Time Drought Forecasting Using Lagged Climate Variables and a Stacked Long Short-term Memory Model. Sci. Total Environ.
**2021**, 755, 142638. [Google Scholar] [CrossRef] - Gormaz-Cuevas, D.; Riffo-Rivas, J.; Montastruc, L.; Brüning-González, M.; Díaz-Alvarado, F.A. A multi-objective optimization model to plan city-scale water systems with economic and environmental objectives: A case study in santiago, Chile. J. Clean. Prod.
**2021**, 279, 123737. [Google Scholar] [CrossRef] - Alarcon, V.J.; Magrini, C. Scenarios of Sediment Transport Management in Francia Creek; Lecture Notes in Computer Science, 10962; LNCS: Valparaiso, Chile, 2018; pp. 205–218. [Google Scholar] [CrossRef]
- Aguas Andinas. Aguas Andinas Detalla Plan de Inversiones. 2017. Available online: https://www.aguasandinas.cl/noticias/historico-noticias/aguas-andinas-detalla-plan-de-inversiones (accessed on 15 November 2020).
- Rajaee, T.; Jafari, H. Two decades on the artificial intelligence models advancement for modeling river sediment concentration: State-of-the-art. J. Hydrol.
**2020**, 588, 125011. [Google Scholar] [CrossRef] - Afan, H.A.; El-Shafie, A.; Mohtar, W.H.M.W.; Yaseen, Z.M. Past, present and prospect of an Artificial Intelligence (AI) based model for sediment transport prediction. J. Hydrol.
**2016**, 541, 902–913. [Google Scholar] [CrossRef] - Afan, H.A.; El-Shafie, A.; Yaseen, Z.M.; Hameed, M.M.; Mohtar, W.H.M.W.; Hussain, A. ANN Based Sediment Prediction Model Utilizing Different Input Scenarios. Water Resour. Manag.
**2014**, 29, 1231–1245. [Google Scholar] [CrossRef] - Ghose, D.K.; Samantaray, S. Modelling sediment concentration using back propagation neural network and regression coupled with genetic algorithm. Procedia Comput. Sci.
**2018**, 125, 85–92. [Google Scholar] [CrossRef] - Alp, M.; Cigizoglu, H. Suspended sediment load simulation by two artificial neural network methods using hydrometeorological data. Environ. Model. Softw.
**2007**, 22, 2–13. [Google Scholar] [CrossRef] - Wu, W.; Dandy, G.C.; Maier, H.R. Protocol for developing ANN models and its application to the assessment of the quality of the ANN model development process in drinking water quality modelling. Environ. Model. Softw.
**2014**, 54, 108–127. [Google Scholar] [CrossRef] - SERNAGEOMIN. Efectos del Incendio del 12 al 16 de abril de 201, Comuna de Valparaiso, Region de Valparaiso. Ministerio de Mineria, Santiago 2015. Available online: http://sitiohistorico.sernageomin.cl/pdf/volcanes/3-Efectos-del-incendio-del-12-al-16-de-abril-de-2014-en-la-comuna-de-Valparaiso.pdf (accessed on 15 November 2020).
- Universidad Andrés Bello. Geomorfología Dinámica del Cajón del Maipo. 2015. Available online: https://www.geoaprendo.com/2014/12/informe-geomorfologia-dinamica-del.html. (accessed on 15 November 2020).
- Xu, C.Y.; Xiong, L.; Singh, V.P. Black-Box Hydrological Models. In Handbook of Hydrometeorological Ensemble Forecasting; Duan, Q., Pappen-berger, F., Thielen, J., Wood, A., Cloke, H., Schaake, J., Eds.; Springer: Berlin/Heidelberg, Germany, 2017. [Google Scholar]
- Nacar, S.; Hınıs, M.A.; Kankal, M. Forecasting Daily Streamflow Discharges Using Various Neural Network Models and Training Algorithms. KSCE J. Civ. Eng.
**2017**, 22, 3676–3685. [Google Scholar] [CrossRef] - Dariane, A.B.; Azimi, S. Streamflow forecasting by combining neural networks and fuzzy models using advanced methods of input variable selection. J. Hydroinformatics
**2017**, 20, 520–532. [Google Scholar] [CrossRef] [Green Version] - Daliakopoulos, I.N.; Tsanis, I. Comparison of an artificial neural network and a conceptual rainfall–runoff model in the simulation of ephemeral streamflow. Hydrol. Sci. J.
**2016**, 61, 2763–2774. [Google Scholar] [CrossRef] - Besaw, L.E.; Rizzo, D.M.; Bierman, P.; Hackett, W.R. Advances in ungauged streamflow prediction using artificial neural networks. J. Hydrol.
**2010**, 386, 27–37. [Google Scholar] [CrossRef] - Halecki, W.; Kruk, E.; Ryczek, M. Estimations of nitrate nitrogen, total phosphorus flux and suspended sediment concentration (SSC) as indicators of surface-erosion processes using an ANN (Artificial Neural Network) based on geomorphological parameters in mountainous catchments. Ecol. Indic.
**2018**, 91, 461–469. [Google Scholar] [CrossRef] - Meral, R. Analyses of turbidity and acoustic backscatter signal with artificial neural network for estimation of suspended sediment concentration. Appl. Ecol. Environ. Res.
**2018**, 16, 697–708. [Google Scholar] [CrossRef] - Sari, V.; Castro, N.M.D.R.; Pedrollo, O.C. Estimate of Suspended Sediment Concentration from Monitored Data of Turbidity and Water Level Using Artificial Neural Networks. Water Resour. Manag.
**2017**, 31, 4909–4923. [Google Scholar] [CrossRef] - Bhattacharya, B.; Van Kessel, T.; Solomatine, D.P. Spatio-temporal prediction of suspended sediment concentration in the coastal zone using an artificial neural network and a numerical model. J. Hydroinformatics
**2012**, 14, 574–584. [Google Scholar] [CrossRef] [Green Version] - Boussaada, Z.; Curea, O.; Remaci, A.; Camblong, H.; Mrabet, N.B. A Nonlinear Autoregressive Exogenous (NARX) Neural Network Model for the Prediction of the Daily Direct Solar Radiation. Energies
**2018**, 11, 620. [Google Scholar] [CrossRef] [Green Version] - Buitrago, J.; Asfour, S. Short-Term Forecasting of Electric Loads Using Nonlinear Autoregressive Artificial Neural Networks with Exogenous Vector Inputs. Energies
**2017**, 10, 40. [Google Scholar] [CrossRef] [Green Version] - Navratil, O.; Esteves, M.; Legout, C.; Gratiot, N.; Nemery, J.; Willmore, S.; Grangeon, T. Global uncertainty analysis of suspended sediment monitoring using turbidimeter in a small mountainous river catchment. J. Hydrol.
**2011**, 398, 246–259. [Google Scholar] [CrossRef] - Burden, F.; Winkler, D. Bayesian Regularization of Neural Networks. In Artificial Neural Networks; Livingstone, D.J., Ed.; Methods in Molecular Biology; Humana Press: Totowa, NJ, USA, 2008; Volume 458. [Google Scholar] [CrossRef]
- Mathworks. Divide Data for Optimal Neural Network Training (r2020b). 2020. Available online: https://www.mathworks.com/help/deeplearning/ug/divide-data-for-optimal-neural-network-training.html;jsessionid=0322bf67b3599c9de6ef0bf5e7cc (accessed on 20 December 2020).
- Moriasi, D.N.; Arnold, J.G.; Van Liew, M.W.; Bingner, R.L.; Harmel, R.D.; Veith, T.L. Model Evaluation Guidelines for Systematic Quantification of Accuracy in Watershed Simulations. Trans. ASABE
**2007**, 50, 885–900. [Google Scholar] [CrossRef] - Ang, R.; Oeurng, C. Simulating streamflow in an ungauged catchment of Tonlesap Lake Basin in Cambodia using Soil and Water Assessment Tool (SWAT) model. Water Sci.
**2018**, 32, 89–101. [Google Scholar] [CrossRef] [Green Version] - Knoben, W.J.M.; Freer, J.E.; Woods, R.A. Technical note: Inherent benchmark or not? Comparing Nash–Sutcliffe and Kling–Gupta efficiency scores. Hydrol. Earth Syst. Sci.
**2019**, 23, 4323–4331. [Google Scholar] [CrossRef] [Green Version] - Djebou, D.C.S. Assessment of sediment inflow to a reservoir using the SWAT model under undammed conditions: A case study for the Somerville reservoir, Texas, USA. Int. Soil Water Conserv. Res.
**2018**, 6, 222–229. [Google Scholar] [CrossRef]

**Figure 1.**Study area. Francia Creek watershed (covering 3.24 km

^{2}) is located in Valparaiso (Chile). Dry weather, with few, but intense, precipitation events in June–August, characterize the region. Native and non-native vegetation are predominant in the watershed. Slopes are greater than 40%.

**Figure 2.**Sand trap at Francia Creek watershed exit. (

**A**) Sand trap after clean-up of upstream side. (

**B**) Mudflow filling up the upstream side of the sand trap after a series of heavy rain events.

**Figure 3.**Historical data in Francia Creek watershed. Daily time-series of suspended sediments concentrations (SST), precipitation, and stream flow are shown (1 January 2005–31 December 2014). Sediment concentration peaks occur soon after intense and consecutive precipitation events.

**Figure 4.**Time-series of SST, precipitation, and stream flow for dates 4 July 2006 through 26 July 2006. A mudflow event occurred on 12 July 2020 after two consecutive rain events.

**Figure 5.**Conceptual description of the open-loop neural network used in this research. The future value of total suspended sediments,$SS{T}^{N+i}$, is predicted from observed present and past values of precipitation (${P}^{N-k}$ ), and stream flow (${Q}^{N-k}$ ), and the observed past values of total suspended sediments ($SS{T}^{N-r}$ ).

**Figure 6.**NARX training results. Daily SST concentrations and flow data for years 2011 and 2012 were used for training.

**Figure 7.**Hindcasting daily baseflow SST concentrations using the NARX net. The neural network successfully reproduced daily SST concentrations observed in daily baseflow in years 2008, 2009 and 2010.

**Figure 8.**Hindcasting using the NARX net. The estimations of the neural network are shown to be successful for monthly mean and maximum SST concentrations.

**Figure 9.**Forecasting of monthly mean and maximum for years 2013 and 2014. The neural network forecast of monthly maxima is shown to be successful while monthly means forecasting is less effective but acceptable.

**Figure 10.**Forecasting of daily SST concentrations for years 2013 and 2014. The statistical indicators for the NARX net forecast show that prediction of daily SST concentrations for 2013 and 2014 is limited. However, if 2014 is analyzed separately, the daily SST concentrations forecast is successful.

**Figure 11.**Forecasting daily SST concentrations for daily baseflow during years 2013 and 2014. The NARX net successfully predicts daily concentrations as reflected by the statistical indicators of fit, error and bias.

Model Run | Number of Years Used for Training | Number of Years Used for Forecasting | ||||
---|---|---|---|---|---|---|

Precipitation | Flow | SST | Precipitation | Flow | Predicted SST | |

Initial | 2005–2012 | 2005–2012 | 2005–2012 | 2005–2012 | 2005–2012 | 2013–2014 |

Final | 2011–2012 | 2011–2012 | 2011–2012 | 2011–2012 | 2011–2012 | 2013–2014 |

**Table 2.**Formulae for calculation of indicators of statistical fit, error, and bias. The table includes acceptability ranges for daily-step simulations.

Indicators of Fit | Formulae | Range |
---|---|---|

Root-mean-squared-error to standard deviation ratio, $RSR=\frac{RMSE}{STDE{V}_{Obs}}$ | $\frac{\sqrt{{{\displaystyle \sum}}_{i=1}^{n}\left({Y}_{i}^{Obs}-{Y}_{i}^{Sim}\right){.}^{2}}}{\sqrt{{{\displaystyle \sum}}_{i=1}^{n}\left({Y}_{i}^{Obs}-{Y}_{i}^{Mean}\right){.}^{2}}}$ | RSR < 0.79 |

Percent bias, PBIAS | $\frac{{{\displaystyle \sum}}_{i=1}^{n}\left({Y}_{i}^{Obs}-{Y}_{i}^{Sim}\right)\ast 100}{{{\displaystyle \sum}}_{i=1}^{n}\left({Y}_{i}^{Obs}\right).}$ | −18% < PBIAS < 18% |

Correlation coefficient, R | $\sqrt{\frac{{{\displaystyle \sum}}_{i=1}^{n}\left({Y}_{i}^{Sim}-{Y}_{i}^{Mean}\right){.}^{2}}{{{\displaystyle \sum}}_{i=1}^{n}\left({Y}_{i}^{Obs}-{Y}_{i}^{Mean}\right){.}^{2}}}$ | R > 0.71 |

Coefficient of determination | ${R}^{2}$ | ${R}^{2}$> 0.50 |

Nash–Sutcliffe efficiency, NS | $1-\frac{{{\displaystyle \sum}}_{i=1}^{n}\left({Y}_{i}^{Obs}-{Y}_{i}^{Sim}\right){.}^{2}}{{{\displaystyle \sum}}_{i=1}^{n}\left({Y}_{i}^{Obs}-{Y}_{i}^{Mean}\right){.}^{2}}$ | NS > 0.50 |

Kling-Gupta efficiency, K-G | $1-\sqrt{{\left(\frac{{Y}_{Sim}^{Mean}}{{Y}_{Obs}^{Mean}}-1\right)}^{2}+{\left(\frac{STDE{V}_{Sim}}{STDE{V}_{Obs}}-1\right)}^{2}+{\left(R-1\right)}^{2}}$ | K-G > 0.50 |

Willmott’s index of agreement, d | $1-\frac{{{\displaystyle \sum}}_{i=1}^{n}\left({Y}_{i}^{Obs}-{Y}_{i}^{Sim}\right){.}^{2}}{{{\displaystyle \sum}}_{i=1}^{n}\left(\left|{Y}_{i}^{Sim}-{Y}_{Obs}^{Mean}\right|+\left|{Y}_{i}^{Obs}-{Y}_{Obs}^{Mean}\right|\right){.}^{2}}$ | d > 0.65 |

${Y}_{i}^{Obs}=\mathrm{Observed}\mathrm{SST}\mathrm{concentration}$ ${Y}_{i}^{Sim}=$ Simulated SST concentration ${Y}_{Obs}^{Mean}=\mathrm{Mean}\mathrm{of}\mathrm{observed}\mathrm{SST}\mathrm{concentration}$ ${Y}_{Sim}^{Mean}=\mathrm{Mean}\mathrm{of}\mathrm{simulated}\mathrm{SST}\mathrm{concentration}$, $n=$ Total number of daily SST concentrations |

Non-Linear Autoregressive Exogenous Neural Network (NARX) Feature | Adopted Value |
---|---|

Number of layers | 3 (input, hidden, output) |

Number of nodes in hidden layer | 12 |

Architecture | Open loop |

Number of iterations | 100 |

Input delays | 1:30 |

Feedback delays | 1:3 |

Train ratio | 70/100 |

Value ratio | 15/100 |

Test ratio | 15/100 |

Training algorithm | Bayesian regularization |

Comparison Period | Indicator Ranges for Daily SST Simulation | ||||
---|---|---|---|---|---|

RSR < 0.79 | −18% < PBIAS < 18% | R^{2} > 0.5 | NS >0.5 | K-G > 0.5 | |

2011 | 0.26 | −0.28% | 0.94 | 0.93 | 0.94 |

2012 | 0.32 | 9.99% | 0.94 | 0.90 | 0.88 |

Comparison Period | Indicator Ranges for Daily SST Simulation | ||||
---|---|---|---|---|---|

RSR < 0.79 | −18% < PBIAS < 18% | R^{2} > 0.5 | NS > 0.5 | K-G > 0.5 | |

2008–2010 | 0.51 | −13.40% | 0.82 | 0.74 | 0.83 |

Monthly SST for 2008–2010 | Indicator Ranges for Monthly SST Simulation | ||||
---|---|---|---|---|---|

RSR < 0.70 | −15% < PBIAS < 15% | R^{2} > 0.6 | NS > 0.6 | K-G > 0.6 | |

Mean | 0.74 | −15.24% | 0.64 | 0.55 | 0.72 |

Maximum | 0.55 | 1.52% | 0.69 | 0.69 | 0.71 |

Monthly SST for 2013–2014 | Indicator Ranges for Monthly SST Simulation | ||||
---|---|---|---|---|---|

RSR < 0.70 | −15% < PBIAS < 15% | R^{2} > 0.6 | NS > 0.6 | K-G > 0.6 | |

Mean | 0.79 | 3.29% | 0.43 | 0.34 | 0.65 |

Maximum | 0.62 | 18.87% | 0.69 | 0.60 | 0.54 |

Comparison Period | Indicator Ranges for Daily SST Simulation | ||||
---|---|---|---|---|---|

RSR < 0.79 | −18% < PBIAS < 18% | R^{2} > 0.5 | NS > 0.5 | K-G > 0.5 | |

2013–2014 | 0.89 | 3.41% | 0.28 | 0.21 | 0.50 |

2014 | 0.91 | −15.86% | 0.53 | 0.47 | 0.60 |

Baseflow | 0.54 | 3.6% | 0.73 | 0.71 | 0.84 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Alarcon, V.J.
Hindcasting and Forecasting Total Suspended Sediment Concentrations Using a NARX Neural Network. *Sustainability* **2021**, *13*, 363.
https://doi.org/10.3390/su13010363

**AMA Style**

Alarcon VJ.
Hindcasting and Forecasting Total Suspended Sediment Concentrations Using a NARX Neural Network. *Sustainability*. 2021; 13(1):363.
https://doi.org/10.3390/su13010363

**Chicago/Turabian Style**

Alarcon, Vladimir J.
2021. "Hindcasting and Forecasting Total Suspended Sediment Concentrations Using a NARX Neural Network" *Sustainability* 13, no. 1: 363.
https://doi.org/10.3390/su13010363