Performance Evaluation of Gradient Descent Optimizers in Estuarine Turbidity Estimation with Multilayer Perceptron and Sentinel-2 Imagery

Abstract

Accurate monitoring of estuarine turbidity patterns is important for maintaining aquatic ecological balance and devising informed estuarine management strategies. This study aimed to enhance the prediction of estuarine turbidity patterns by enhancing the performance of the multilayer perceptron (MLP) network through the introduction of stochastic gradient descent (SGD) and momentum gradient descent (MGD). To achieve this, Sentinel-2 multispectral imagery was used as the base on which spectral radiance properties of estuarine waters were analyzed against field-measured turbidity data. In this case, blue, green, red, red edge, near-infrared and shortwave spectral bands were selected for empirical relationship establishment and model development. Inverse distance weighting (IDW) spatial interpolation was employed to produce raster-based turbidity data of the study area based on field-measured data. The IDW image was subsequently binarized using the bi-level thresholding technique to produce a Boolean image. Prior to empirical model development, the selected spectral bands were calibrated to turbidity using multilayer perceptron neural network trained with the sigmoid activation function with stochastic gradient descent (SGD) optimizer and then with sigmoid activation function with momentum gradient descent optimizer. The Boolean image produced from IDW interpolation was used as the base on which the sigmoid activation function calibrated image pixels to turbidity. Empirical models were developed using selected uncalibrated and calibrated spectral bands. The results from all the selected models generally revealed a stronger relationship of the red spectral channel with measured turbidity than with other selected spectral bands. Among these models, the MLP trained with MGD produced a coefficient of determination (r²) value of 0.92 on the red spectral band, followed by the MLP with MGD on the green spectral band and SGD on the red spectral band, with r² values of 0.75 and 0.72, respectively. The relative error of mean (REM) and r² results revealed accurate turbidity prediction by the sigmoid with MGD compared to other models. Overall, this study demonstrated the prospect of deploying ensemble techniques on Sentinel-2 multispectral bands in spatially constructing missing estuarine turbidity data.

1. Introduction

Turbidity is considered a significant indicator of estuarine water quality [1], and its assessment is important for estuarine water management and aquatic ecosystem conservation. Turbidity, measured in nephelometric turbidity units (NTU), is defined as the amount of light scattered or blocked by particles suspended in water [2]. It is typically composed of nutrients, bacteria, algae, dissolved colored organic compounds, and other organic and inorganic matter [3]. High concentration of turbidity is known to inhibit the depth at which light can reach [4]. In the Great Fish Estuary, turbidity has become a common phenomenon due to the water exchange between the ocean and the river. Exacerbating this situation are various anthropogenic activities, such as agricultural practices, the removal of riparian vegetation, and raw sewage from Cradock Town, South Africa, discharged into the Great Fish River without being treated. This has subsequently compromised the water quality and ecological functioning of the estuary. Therefore, knowledge regarding spatial distribution and plume dynamics of turbidity in these environments is critical for effective estuarine resource management [5]. Data on turbidity are conventionally obtained through field-based sampling techniques, which are labor-intensive, time-consuming, and costly. Moreover, field-based data are acquired from individual sampled locations, posing questions about missing data at unsampled locations. To this end, spatial interpolation approaches have been extensively deployed to account for missing data at unsampled locations [6,7]. Kriging and inverse distance weighting (IDW) techniques are commonly used to generate turbidity data based on the linear weighting of surrounding measured values [8,9]. However, there is no single technique which alone is capable of accurately predicting dynamic variables such as estuarine turbidity.

Advances in remote sensing technology offer a convenient and spatially explicit opportunity to enhance estuarine turbidity monitoring [10]. Multispectral sensors have consistently and efficiently been deployed to track estuarine turbidity through analysis of spectral radiance properties of estuarine waters [11], guided by the notion that turbid waters tend to inherit optical properties from suspended particles. These sensors are still considered vital tools for turbidity mapping in many developing countries due to the free access to their data. Therefore, satellite multispectral data from sensors such as Sentinel-2 [12,13], Landsat generations [14,15], Aster [16], and MODIS [17] have been widely used to monitor turbidity. Among these images, Sentinel-2 imagery is recommended for turbidity estimation due to its higher spatial resolution of 10 m. As such, individual spectral bands and indices generated from the combination of spectral bands, such as the Normalize Difference Turbidity Index (NDTI) and the turbidity index [1], have been used to predict turbidity patterns. However, characterizing estuarine turbidity with these sensors is challenging because estuarine waters are dynamic in nature, and they require sensors with high signal-to-noise ratios [12]. Moreover, individual spectral bands and spectral indices serve only as proxies for turbidity, as they do not provide the actual level of turbidity concentration. As such, these spectral bands and indices, in conjunction with field-measured turbidity data, are used to predict turbidity concentrations through either empirical or analytical approaches [18]. While analytical methods rely on radiative transfer equations to derive the inherent optical properties of dynamic waters [19], empirical methods optically retrieve waters by establishing relationships between water reflectance and field-measured waters. However, accurate prediction of estuarine turbidity patterns using either analytical or empirical approaches remains a challenging task, due to the dynamic nature of estuarine waters [20]. Empirical approaches require large data samples with good spread [21] and are valid for the areas they were established in. On the other hand, analytical approaches are prone to atmospheric correction errors when applied to satellite imagery [19]. As such, a more sophisticated, data-driven, and self-adaptive approach is required to exploit spectral information from satellite imagery [22].

Evidently, the applications of artificial neural networks (ANNs) to address limitations associated with analytical and/or empirical models have extensively been acknowledged [23,24]. The multilayer perceptron neural network (MLP), in particular, has demonstrated the ability to learn dynamic estuarine processes [25,26], taking advantage of its backpropagation architecture, which facilitates back-forward learning [27]. Trained with a sigmoid activation function, the MLP can learn dynamic, non-linear, complex processes such as estuarine turbidity [28]. From a multispectral image perspective, the outcome of the MLP learning process is the probability of image pixels belonging to turbidity, instead of pixels directly becoming members of the turbidity class. Therefore, all pixels in the image are reconfigured to represent turbidity only. Several studies have recommended the application of the MLP in turbidity prediction [29,30,31]. However, it is worth noting that any traditional deep learning approach alone does not have the ability to learn rapidly changing patterns [32], such as estuarine turbidity. The MLP, in particular, is symmetrical and does not perform well with highly biased distributions [29]. The performance of the MLP is influenced by its learning rate, which determines the values of weights and biases that reduce the prediction error with training data [33]. Moreover, the influence of initial weights in the MLP has not been effectively addressed, as they play a major role in determining its performance [34,35]. The backpropagation algorithm has multiple drawbacks that affect convergence in the MLP learning process, such as terminating the learning process in local optima. This has led to the MLP becoming more reliant on the initial values of hyperparameters [36].

If parameters such as network topology, initial weights, biases, learning rate value, and activation function are not appropriately selected, it can result in a slow convergence rate, network error, or failure when calibrating image pixels to turbidity when training the MLP using the sigmoid cross-entropy function [37,38]. As such, the MLP approach is not suitable for learning dynamic processes in unknown environments alone [39]. Several performance optimizers were proposed to enhance MLP performance by accurately determining the initial weights and these include Kaiming initialization [40], Xavier (Glorot) initialization [41], orthogonal initialization [42] and gradient descent (GD) optimizers [40]. However, the GD algorithm is considered one of the most reliable optimization algorithms [37]. As such, several GD approaches have been proposed to perform error backpropagation processing, i.e., optimizing the learning rate while minimizing the error between the output value computed by the model and the anticipated value [21]. The primary focus in gradient descent is to minimize the error function by adjusting the weights in the network [35]. These GD approaches include stochastic gradient descent (SGD) [43], momentum gradient descent (MGD) [44], Fractional Order Gradient Descent [45], Damped Newton Stochastic Gradient Descent [46], Nesterov accelerated gradient (NAG) [47] and Adaptive Gradient Algorithm (AdaGrad) [48], among others. When specifically used with momentum gradient descent, the MLP learning rate is increased by setting the momentum factor to 0.9, weight decay to 10⁻⁴, and the initial learning rate to 0.1 [49]. In this case, the role of the momentum factor in gradient descent is to accelerate the convergence rate of the MLP optimization algorithm [50]. Furthermore, Lee et al. [51] noted that the significant influence of the gradient approaches in enhancing model performance in water quality prediction is frequently not evaluated. Through the literature search, studies that sought to monitor turbidity patterns based on remote sensing approaches have mainly utilized semi-analytic or empirical models based on the relationship between spectral radiance properties of water and field-measured turbidity [52,53,54,55,56]. As such, we noted that studies on enhancing MLP performance through the introduction of SGD and MGD for optimized prediction of estuarine turbidity patterns are lacking. Remote sensing reflectance to field turbidity [57]. In this study, we evaluated two commonly used GD approaches, viz., SGD and MGD, in improving MLP performance for predicting estuarine turbidity patterns. Prior to performance evaluation of SGD and MGD, we evaluated the efficacy of linear regression on uncalibrated spectral bands of Sentinel-2 imagery.

2. Materials and Methods

2.1. Study Area

The Great Fish Estuary is situated in the Ndlambe Local Municipality of the Sarah Baartman District Municipality of the Eastern Cape Province in South Africa. The study area is located between 33°28′13.16″ S; 27°04′55.76″ E and 33°30′22.4″ S; 27°07′58.21″ E absolute locations. It serves as the base for ecosystem functioning of the coastal environment of the Eastern Cape Province of South Africa. This estuary is also used for fishing by the local communities. The estuary is situated within the elevation range of 0 m above sea level (a.s.l.) to 16.16 m a.s.l. Figure 1 shows the location of the study area, with the locations for sampled data highlighted. The Great Fish Estuary in South Africa is no exception regarding water contamination. The principal alarming source of water pollution in this estuary is a large volume of raw, untreated wastewater from the Cradock dysfunctional water treatment plant, which is discharged into the Great Fish River.

2.2. Methods

The sequence of methods and techniques deployed to attain the main objective of this study is presented in Figure 2.

2.3. Data Acquisition

2.3.1. Satellite Data

The Sentinel-2 imagery, acquired with the multispectral instrument (MSI) and having the ID: T35HND_20220630T074619, was downloaded from the European Space Agency (ESA) Sentinel Scientific Data Hub (https://scihub.copernicus.eu/), accessed on 13 August 2022. This satellite imagery was preferred over other cost-free sensors, such as Landsat, due to its reasonably higher spatial resolution of 10 m in its four visible-NIR bands, 20 m in three bands in the red edge and two bands in shortwave infrared (SWIR) [58], 60 m in its coastal aerosol band, SWIR-cirrus band, and water vapor band, and a 12-bit radiometric resolution. The image used for this study was acquired on 30 June 2022 at 07:46:19 South African local time, with the date and time selected based on cloudless conditions. The empirical relationship between turbidity and spectral reflectance in the blue [59], green [52], red [60], red edge [61], NIR [62], and SWIR [63] spectral channels justified the selection of these spectral channels from Sentinel-2 imagery for model training. Coastal aerosol, water vapor, and SWIR-cirrus spectral channels were omitted in this study due to their lack of association with turbidity.

Table 1. Characteristics of Sentinel-2 spectral bands selected for this study.

Spectral Band	Wavelength Center (nm)	Spatial Resolution (m)
Blue	490	10
Green	560	10
Red	665	10
Red edge 1	705	20
Red edge 2	740	20
Red edge 3	783	20
Near-infrared	842	10
Red edge 4	865	20
Shortwave infrared 1	1610	20
Shortwave infrared 2	2190	20

provides detailed characteristics of the Sentinel-2 spectral bands used in this study.

2.3.2. Field Data Collection

A total of 40 surveyed point samples out of 42 surveyed samples were considered for turbidity modeling in this study. A simple random sampling technique was employed to select the sites where measurements were carried out. Upon the successful identification of the survey sites, the absolute location of each sample point was recorded using the Garmin eTrex 22× Handheld GPS^®, with 3 m precision. The field survey was conducted on 30 June 2022 to coincide with the estuarine turbidity data collection period and the Sentinel-2 satellite overpass. The field data were collected from 8h00 to 11h00 local time. The collection of field data during this time was to ensure it was as close to the MSI scanning time as possible. A canoe was used to access sites of deep water. At random pattern, turbidity data were measured near the water’s surface (i.e., at most 0.05 m) using the HANNA HI9829 portable logging multi-parameter meter. The GPS points were recorded in Microsoft Excel and saved as a “.csv” file. This file was imported into the TerrSet software package and converted to a vector GIS point layer. The acquired dataset was partitioned into two sets; i.e., 28 points (70%) of the dataset served in model training, and 12 points (30%) in model validation. However, all the surveyed field sample points were used in the spatial interpolation of turbidity.

2.4. Pre-Processing of Sentinel-2 Imagery

The acquired Sentinel-2 imagery was received already geo-rectified to the Universal Transverse Mercator (UTM) coordinate system, based on the World Geodetic System of 1984 datum (WGS-84), 36° South grid reference. This georeferencing system facilitated the successful position adjustment of the Sentinel-2 imagery in line with its ground position. However, the downloaded image scene extended beyond the boundary of the estuary under investigation. As such, only the portion of the imagery falling within the estuarine boundary was extracted in ArcMap 10.8 GIS software package, using the estuarine boundary shapefile as a mask feature. The Sentinel-2 imagery used in this study was received in Level 1C, having already been radiometrically corrected, with its radiance (L) values ranging from 0 to 1. The Sentinel-2 sensor achieved this by employing geographical locations for imagery referencing [64]. Prior to model development and validation, outliers were removed from the field datasets, to minimize bias in the prediction of turbidity. The outliers in this study were two turbidity records whose values were extremely high, such that their inclusion would move the mean value closer to the highest turbidity value when they were not included.

2.5. Estuarine Water Body Demarcation

Prior to the extraction of pixel values from selected spectral bands, the estuarine water body requires to be masked to ensure that the experimental site contained only water pixels [65]. The estuarine water body was demarcated on satellite imagery by employing the Modified Normalized Difference Water Index (MNDWI) based on Equation (1):

$$MNDWI={\displaystyle \frac{{\rho }_G+{\rho }_{SWI{R}_1}}{{\rho }_G+{\rho }_{SWI{R}_1}}}$$

(1)

where $\rho$ is the radiance value of the specified spectral channel; G is the green channel; and $SWI{R}_1$ is the first shortwave-infrared channel of the Sentinel-2 sensor.

The selection of this spectral index was based on its ability to suppress reflectance from built-up features by deploying the shortwave infrared band instead of the near-infrared, commonly used in the normalized difference water index (NDWI) [66].

2.6. Analyzing Spatial Variability in Water Radiance Values across Selected Spectral Bands

A total of 300 pixels were randomly collected to obtain the spectral radiance sample that served as input for model training. Prior to model training, Levene’s k-comparison of variance statistical test was used to determine spatial variability in the spectral radiance properties of estuarine waters across selected spectral channels of Sentinel-2 imagery. Levene’s test was computed using Equation (2), adopted from Pare’ et al. [67]:

$$W={\displaystyle \frac{\left[\left(N-k\right){\sum }_{i=1}^k{N}_i{\left({\overline{Z}}_i.-\overline{Z}..\right)}^2\right]}{\left[\left(k-1\right){\sum }_{i=1}^k{\sum }_{j=1}^{N_i}{\left(Z_{ij}-{\overline{Z}}_i.\right)}^2\right]}}$$

(2)

where $Z_{ij}=|Y_{ij}-{Ȳ}_i|$, with Y_ij the value of Y for the j-th observation of the i-th subgroup; Ȳ_i is the mean of the i-th subgroup; ${\overline{Z}}_i$ is the mean of the group $Z_{ij}$; and $\overline{Z}$ is the overall mean of the $Z_{ij}$.

Subsequently, the mean radiance values for selected spectral channels were used to plot the spectral reflectance behavior of estuarine waters.

2.7. Spatial Pattern Analysis of Turbidity

Spatial variability in the field-measured turbidity was established using the Bayesian one-sample t-test. Under a supposition that sample $X\left(n\right)=\{X_1, X_2, \dots , X_n\}$ of size n is randomly distributed, and each data point is sampled from a location in estuarine waters, the Bayesian one-sample t-test computed the pattern by applying Equation (3), adopted from Emmert-Streib and Dehmer [68]:

$$\bar{X}~N\left(\mu ,{\displaystyle \frac{\sigma }{\sqrt{n}}}\right)$$

(3)

where $\{X_1, \cdots , X_n\}$ are random samples with μ and σ², which follow a normal distribution.

Subsequently, the turbidity pattern was constructed using IDW interpolation on the field-measured data. This technique functions based on the notion that “values closer together tend to be more similar than those further apart” [69]. To spatially construct missing values, the algorithm utilizes two or more of the closest measured values [70]. As such, a weighted sum was applied across measured values to determine the value of the interpolated region, based on Euclidean distance by employing Equation (4), adopted from Chin et al. [69]:

$$Z_0={\displaystyle \frac{{\sum }_{i=1}^S{Z}_i\frac{1}{d_i^k} }{{\sum }_{i=1}^S\frac{1}{d_i^k}}}$$

(4)

where $Z_0$ is the estimated value at the unsampled location; $Z_i$ is the measured value; d_i is the distance between the estimated and observed locations; s is the number of measured sampling points within the neighborhood; and k is the parameter containing the exponential value, which defines the degree to which the value is reduced as the distance increases.

2.8. Binarization of IDW Image and Thresholding of Turbidity Level

The raster-based turbidity image acquired through IDW was used as a training mask in the MLP model development. The training sites were demarcated using the bi-level thresholding technique, such that:

$$T_{site}=\left\{\begin{array}{c}1 if Turbidity\ge T\\ 0, otherwise \end{array}\right.$$

(5)

where T_site is the training site; and T is the threshold, equal to or above which the model was trained. This was set to 50.0 NTU according to Kopsachilis et al. [71]. The motive behind training the model on a binarized IDW map was due to the fact that the interpolated turbidity contained the actual field-measured turbidity.

2.9. Model Training

A point shapefile containing the locations and values of turbidity was overlaid on each selected spectral band and NDTI in the ArcGIS 10.8 GIS platform. The radiance and index values were extracted from the selected spectral bands and NDTI, respectively, using the “Extract Multi Values To Point” module. The extracted spectral bands and spectral index were used in model training. The following models were tested for their efficacy in spatially constructing missing turbidity data.

2.9.1. Linear Regression

The relationship between the spectral radiance values and field-measured turbidity was established using linear regression analysis, based on Equation (6), adopted from Hossain et al. [14]:

$$y=mx+b$$

(6)

where Y is the dependent variable; m is the slope; x is the explanatory variable; and b is the intercept of Y.

The nature and degree of the relationship were evaluated by interpreting the regression coefficient values and least-squared regression (r²).

2.9.2. Training MLP with Sigmoid Activation Function and SGD

Prior to the data split into training and validation sets, the data were pre-randomized to minimize selection bias associated with training and validation data. During the training process, each sample (pixel values extracted from each selected spectral band of Sentinel-2 imagery) was fed into the respective neurons of the input layer of the MLP algorithm. Each receiving neuron in the input layer then weighted signals from all the neurons to which it is connected in the preceding layer. The weight of the signals received by a single neuron was determined by employing Equation (7), adopted from Fierro et al. [72]:

$$ne{t}_j=\sum _{i-1}^m{x_{i}w}_{ij}+o_j$$

(7)

where $Ne{t}_j$ is the total weight of the j-th neuron for the input samples received from the preceding layer with n neurons; $x_i$ is the input value to the i-th node of the input layer; and $w_{ij}$ is the weight between the i-th node of the input layer and node j of the hidden layer.

Assuming $w_m$ represents the value after iteration m of a weight w, which can be either a hidden neuron weight w_ij or an output neuron weight $w_{jk}$, $w_m$ was determined using Equation (8), according to Shimodaira [73]:

$$w_m=w_{m-1}+∆w_m$$

(8)

where ${\Delta }_{wm}$ is the change in the weight $w$ at the end of iteration m and is obtained by employing Equation (9):

$$∆w_m=\epsilon {\times d}_m$$

(9)

where ε is the parameter controlling the proportion by which the weights are changed. However, d_m is obtained by applying Equation (10), adopted from Shimodaira [73]:

$$d_m={\sum }_{n=1}^N{\left({\displaystyle \frac{\partial E}{\partial w_m}}\right)}_n$$

(10)

where E is the cost error; and $w_m$ is the parameter at the recurrent step.

O_j is the output from neuron j, expressed using Equation (11):

$$o_j=f\left(ne{t}_j\right)$$

(11)

where

$Ne{t}_j$ is provided in Equation (7).

The sigmoid activation function was used to account for variability over the sampled spectral radiance denoting turbidity by employing a non-linear transformation to optimize turbidity prediction using Equation (12), adopted from Platt [74]:

$$s\left(x\right)={\displaystyle \frac{1}{1+e^{-x}}}$$

(12)

where $s\left(x\right)$ denotes the sigmoid activation function, and e ≈ 2.71 is the base of the natural logarithm.

Stochastic gradient descent was introduced into the sigmoid activation function, to optimize MLP performance by minimizing the loss function. The SGD method improved the learning rate by randomly selecting samples based on Equation (13), according to Kaddoura [75]:

$$\nabla \mathrm{J}{\left(\mathsf{\theta }\right)}_{\mathrm{i}}={\displaystyle \frac{1}{\mathrm{N}}}\left({\mathrm{y}}_{\mathrm{i}}-{\mathsf{\theta }}^{\mathrm{T}}{\mathrm{X}}_{\mathrm{i}}\right){\mathrm{X}}_{\mathrm{i}}$$

(13)

where

$\nabla J\left(\theta \right)$ is the gradient descent;

$i$ denotes the individual row of the dataset;

$N$ denotes the number of datasets in the i-th row;

where $i$ is each row of the data set.

The model was continuously trained to compute the gradient and update parameters until the error was less than a threshold, or the number of iterations reached a threshold. The model was trained in the TerrSet Geospatial Monitoring and Modeling software platform.

Table 2. Learning parameters for MLP-based sigmoid calibration function.

MLP with Sigmoid Activation Function and SGD for Predicting Turbidity from Sentinel-2 Spectral Data
		Blue	Green	Red	NIR
Input specifications	Independent variables	Radiance values of all bands	Radiance values of all bands	Radiance values of all bands	Radiance values of all bands
	Dependent variable	Radiance values of blue wavelength	Radiance values of green wavelength	Radiance values of red wavelength	Radiance values of NIR wavelength
	Max. training sample used	300	300	300	300
	Max. testing sample used	300	300	300	300
Network topology	No. of input layer nodes	4	4	4	4
	No. of output layer nodes	1	1	1	1
	No. of hidden layers	2	2	2	2
	Layer 1 nodes	5	5	5	5
Training parameters	Training type	Automatic	Automatic	Automatic	Automatic
	Learning rate type	Dynamic	Dynamic	Dynamic	Dynamic
	Learning rate start	0.01	0.01	0.01	0.01
	Learning rate end	0.001	0.001	0.001	0.001
	Momentum factor	0.9	0.9	0.9	0.9
	Sigmoid constant	1.0	1.0	1.0	1.0
Stopping criteria	Iteration	10,000	10,000	10,000	10,000
	Actual training pixels	490	497	474	483
	Actual testing pixels	491	497	475	483
	Learning rate	0.001	0.001	0.001	0.001
	RMS	0.01	0.01	0.01	0.01
	Training RMS	0.0322	0.034	0.035	0.019
	Testing RMS	0.0344	0.037	0.033	0.02
	R Sqr.	0.942	0.987	0.992	0.979

provides a summary of learning parameters for the MLP with a sigmoid activation function.

As such, the sigmoid function for each dependent variable was set to saturate at 1.0, to allow the activation function to become increasingly more linear in character.

2.9.3. Training MLP with Sigmoid Activation Function and MGD

The MLP was also trained with the sigmoid activation function (explained in Section 2.9.2) and momentum gradient descent. In this case, the training epochs were also set to 500, at a learning rate of 10⁻⁵. Momentum gradient descent was introduced into the MLP algorithm according to Chen et al. [43], such that:

$$∆{\omega }_{ij}=\eta \times \left({\displaystyle \frac{\partial E}{\partial {\omega }_{ij}}}\right)$$

(14)

where

${\omega }_{ij}$ is the weight increment along the i-th and j-th nodes;

η is the learning rate;

${\displaystyle \frac{\partial E}{\partial {\omega }_{ij}}}$ is the weight gradient, according to Chen et al. [43], such that:

$$∆{\omega }_{ij}\left(t\right)=\eta \times \left({\displaystyle \frac{\partial E}{\partial {\omega }_{ij}}}\right)+\left(\gamma \times ∆{\omega }_{ij}^{t-1}\right)$$

(15)

where

$\gamma$ is the momentum factor; ${∆}_{\omega }$ is the weight along the gradient; t is the time at which the weight is incremented; and $∆{\omega }_{ij}^{t-1}$ is the weight increment at the previous iteration.

2.10. Evaluation of the Model Performance

A total of 12 samples (30%) of the total turbidity data reserved for performance evaluation were saved as .csv and converted to shapefiles. The created point shapefile was then superimposed on the turbidity images predicted by the LR, MLP with SGD, and MLP with MGD algorithms. The pixel values were then extracted to the point shapefile, with a view to determine the nature and degree of their association with the predicted turbidity. The performance of each utilized model was evaluated by employing least-squared regression (r²) using Equation (16), adopted from Yan et al. [76], and the relative error of mean (REM) using Equation (17), adopted from Ndou et al. [77]:

$$R^2=1-{\displaystyle \frac{\left[{\sum }_{i=1}^n{\left(x_i-x_i\right)}^2\right]}{\left[{\sum }_{i=1}^n{\left(y_i-{\overline{y}}_i\right)}^2\right]}}$$

(16)

where $y_i$ denotes the observed turbidity; $y_i$ denotes the mean of the observed turbidity;

$$REM={\displaystyle \frac{\left[T_{measured}-T_{predicted}\right]}{T_{measured}}}\times 100\%$$

(17)

where $T_{measured}$ is the measured value of turbidity, and $T_{predicted}$ is the predicted value of turbidity.

3. Results

3.1. Spectral Radiance Patterns of Estuarine Turbid Waters

Spectral radiance patterns of estuarine waters were analyzed in this study. In the absence of turbidity in the estuary, water radiance properties are expected to be the same across all parts of the estuary. Figure 3 shows radiance values of estuarine waters across different spectral channels of Sentinel-2 imagery.

Generally, the radiance properties of estuarine waters were noted to vary from one spectral channel to another. From Figure 3, the minimum radiance values of the (a) blue, (b) green, (c) red, (d) red edge 1, (e) red edge 2, (f) red edge 3, (g) NIR, (h) red edge 4, (i) SWIR 1, and (j) SWIR 2 spectral bands were noted to be 0.19, 0.16, 0.13, 0.04, 0.11, 0.11, 0.1, 0.05, 0.0, and 0.0, respectively. Moreover, the maximum radiance values of the same spectral bands were noted to be 0.54, 0.53, 0.53, 0.56, 0.61, 0.61, 0.73, 0.74, 0.5, and 0.38, respectively. From randomly sampled pixels, Levene’s k-comparison of variance statistics for the spectral radiance properties of estuarine waters is presented in

Table 3. Descriptive statistics of radiance values sampled from selected Sentinel-2 bands.

	N	DF	Min	Max	μ	σ	SE Mean	Coeff. Var	p-Value
Blue	300	299	0.212	0.264	0.225	0.006	0.001	0.03	<0.001
Green	300	299	0.202	0.283	0.227	0.009	0.001	0.04
Red	300	299	0.169	0.323	0.228	0.016	0.001	0.07
Red edge	300	299	0.079	0.221	0.144	0.02	0.001	0.14
Red edge	300	299	0.134	0.314	0.186	0.026	0.002	0.14
Red edge	300	299	0.132	0.315	0.186	0.028	0.002	0.15
NIR	300	299	0.132	0.315	0.186	0.028	0.002	0.15
Narrow NIR	300	299	0.095	0.305	0.138	0.032	0.002	0.24
SWIR-1	300	299	0.013	0.244	0.034	0.032	0.002	0.94
SWIR-2	300	299	0.016	0.09	0.021	0.008	0.001	0.41

.

At N of 300, DF of 299, and 0.05 significance alpha (α), the mean radiance properties of waters were found to be 0.225, 0.227, 0.228, 0.144, 0.186, 0.186, 0.186, 0.138, 0.034, and 0.021 for blue, green, red, red edge 1, red edge 2, red edge 3, NIR, red edge 4, SWIR 1, and SWIR 2, respectively. The p-value obtained from Levene’s k-comparison of variance statistics was noted to be below 0.001, which is less than 0.05 significance alpha. By implication, the radiance properties of waters varied across the spectral wavelengths of the Sentinel-2 sensor.

3.2. Mean Radiance Values of Various Sentinel-2 Spectral Channels

Upon the successful analysis of descriptive statistics of water radiance properties, the mean spectral radiance profile for estuarine waters was plotted (Figure 4).

From the mean radiance properties graph, it was apparent that estuarine waters had higher reflectance in the red spectral channel and lower reflectance in the SWIR 2 spectral channel (Table 2).

3.3. Spatial Patterns of Turbidity across the Surveyed Sites

Under a supposition that there is no difference in the measured turbidity levels across the surveyed sites in the estuary, the computed descriptive statistics are presented in

Table 4. Detailed description of the surveyed turbidity.

Statistical Results	Turbidity
Count	40
μ	43.31
σ	32.34
SE Mean	8.84
t-test	5.12
p-Value (2-sided)	<0.001
UC (2-sided, 95%)	63.67
LC (2-sided, 95%)	26.89

.

At N of 32, DF of 31, and 0.05 significance alpha (α), the mean value, computed from field-based data, for turbidity was noted to be 43.635, with a standard deviation, standard error, t-value, and p-value of 41.279, 8.607, 5.070, and less than 0.001, respectively. As the computed p-value was noted to be less than 0.05 α, the assumption that there are no significant variations in the distribution of turbidity across the surveyed sites does not hold, implying heterogeneity in turbidity concentration.

3.4. Relationship Establishment between Sentinel-2 Spectral Radiance Values and Turbidity Concentration

3.4.1. Linear Regression Model on Uncalibrated Spectral Bands

Linear regression analysis was performed to establish the relationship between measured turbidity data and the spectral radiance properties of estuarine waters. Figure 5 presents the nature and degree of the association between turbidity and the radiance properties of waters.

From Figure 5, the linear regression results between water turbidity and uncalibrated spectral radiance data revealed r² values of 0.33, 0.31, 0.69, 0.02, 0.53, 0.39, 0.04, 0.34, 0.15, and 0.45 for the blue, green, red, red edge 1, red edge 2, red edge 3, NIR, red edge 4, SWIR 1, and SWIR 2, respectively. As such, the red band exhibited a stronger relationship with turbidity concentration than the other spectral bands.

3.4.2. MLP Trained with Sigmoid Calibration Function

Demarcation of Training Sites

For the artificial neural network method to learn patterns from remotely sensed images, the training site must be demarcated in the image. In this study, the training site was demarcated by binarizing the IDW image. Figure 6 shows the raster image produced by interpolating field-measured turbidity using the IDW interpolation technique and the Boolean image derived from the IDW image.

Linear Regression Model on Spectral Bands Calibrated with Stochastic Gradient Descent

The results obtained from the linear regression model performed on spectral bands calibrated with the sigmoid activation function and stochastic gradient descent are provided in Figure 7.

From Figure 7, the linear regression results between water turbidity and spectral radiance calibrated with the sigmoid function and SGD revealed r² values of 0.0003, 0.51, 0.72, 0.56, 0.63, 0.56, 0.55, 0.33, 0.45, and 0.38 for blue, green, red, red edge 1, red edge 2, red edge 3, NIR, red edge 4, SWIR 1, and SWIR 2, respectively. As such, the red band calibrated with the sigmoid function and SGD also exhibited a stronger relationship with turbidity concentration than other spectral bands.

Linear Regression Model on Spectral Bands Calibrated with Momentum Gradient Descent

Linear regression analysis results for the relationship between turbidity and spectral bands calibrated with the sigmoid function and MGD are presented in Figure 8.

From Figure 8, the linear regression results between water turbidity and spectral radiance calibrated with the sigmoid function and MGD revealed r² values of 0.38, 0.76, 0.92, 0.31, 0.69, 0.56, 0.65, 0.6, 0.58, and 0.49 for blue, green, red, red edge 1, red edge 2, red edge 3, NIR, red edge 4, SWIR 1, and SWIR 2, respectively. As such, green and red bands calibrated with the sigmoid function and MGD exhibited a stronger relationship with turbidity concentration than other spectral bands.

3.4.3. Spatial Modeling of Turbidity Concentration

After successfully establishing the relationship between turbidity concentration levels and spectral radiance properties, regression equations for spectral bands that exhibited a strong relationship with turbidity concentration were deployed to model spatial patterns in turbidity levels Figure 9 shows the spatial configuration of turbidity concentration.

From Figure 9, turbidity concentration levels were noted to range from 4.31 NTU to 167.73 NTU, 1.17 NTU to 161.97 NTU, 5.26 NTU to 146.45 NTU, and 3.26 NTU to 113.26 NTU, as modeled with linear regression on the red band, linear regression on the red band calibrated with the sigmoid function and SGD, linear regression on the green band calibrated with the sigmoid function and MGD, and linear regression on the red band calibrated with the sigmoid function and MGD, respectively.

3.4.4. Performance Evaluation of the Models Deployed

R-Squared Results

Figure 10 shows the performance evaluation of the models deployed to estimate spatial patterns in turbidity concentration in the study area.

Based on Figure 10, the established empirical models provided reasonably accurate spatial predictions of turbidity in the study area. However, the linear regression model generated from the red spectral band calibrated with the sigmoid function and MGD showed higher accuracy, with an r² value of 7.2, than other deployed models (Figure 10d).

3.5. REM Results

Table 5. REM results for model performance evaluation.

REM Results
Model	$\mathit{M}\mathit{e}\mathit{a}\mathit{n} {\mathit{T}}_{\mathit{m}\mathit{e}\mathit{a}\mathit{s}\mathit{u}\mathit{r}\mathit{e}\mathit{d}}$	$\mathit{M}\mathit{e}\mathit{a}\mathit{n} {\mathit{T}}_{\mathit{p}\mathit{r}\mathit{e}\mathit{d}\mathit{i}\mathit{c}\mathit{t}\mathit{e}\mathit{d}}$	REM
LR on uncalibrated red band	60.42	85.21	−41.03
MLP on red band calibrated with sigmoid and SGD	60.42	88.48	−46.44
MLP on green band calibrated with sigmoid and MGD	60.42	72.69	−20.31
MLP on red band calibrated with sigmoid and MGD	60.42	66.06	−9.34

presents the REM results for the performance evaluation of the models deployed in this study.

Generally, all the deployed models were noted to have overestimated turbidity concentration in the study area. However, turbidity values predicted by the sigmoid function and MGD on the red band showed smaller deviation from the measured turbidity data compared to other deployed models. These findings support the model performance evaluation results obtained by the R-squared model in this study.

4. Discussion

Monitoring spatial patterns in estuarine turbidity is crucial for effectively managing estuarine resources and functionality. However, field-based methods alone do not present a practical approach for full comprehension of spatial patterns in turbidity [61]. As such, acquiring the spatial distribution of water turbidity using remote sensing techniques is deemed a time-efficient, cost-effective, and practical approach, since it presents the overall state of turbidity levels in a water body [1]. This study aimed to evaluate the performance of two gradient descent algorithms in optimizing estuarine turbidity prediction based on MLP and remotely sensed data. Li et al. [12] noted that, even though various statistical and machine learning regression methods have been shown to be capable of predicting turbidity patterns, it is important to evaluate their effectiveness in a comparative manner. For this purpose, Sentinel-2 imagery was used as the base from which spectral radiance properties of turbidity were obtained. The selection of this satellite imagery was informed by its reasonably higher spatial resolution of 10 m and spectral bands sensitive to turbidity. Ma et al. [55] noted that Sentinel-2 imagery has presented a promising base for assessing water turbidity. Other studies have also acknowledged the importance of Sentinel-2 imagery in turbidity monitoring [22,78,79]. However, it must be worth noting that the accuracy of the remote sensing approach alone in studying dynamic processes such as estuarine water turbidity remains questionable. This is because several optically active water constituents that do not have any association may have the same spectral reflectance properties. For this reason, several studies have noted the need to advance existing models for optimizing turbidity estimation using remotely sensed data [10,80,81]. To this end, the integration of remote sensing data and several machine learning algorithms has been recommended as a way to advance turbidity estimation [22,82,83].

Machine learning-based regression approaches have become common techniques in turbidity mapping due to their ability to minimize the impact of data outliers [84,85]. However, when these models are employed without incorporating an appropriate activation function, the envisaged results may not be achieved. As such, deep learning architectures may end up operating in a manner similar to traditional linear regression models. For this reason, non-linear activation functions are preferred in deep neural network architectures [86]. In this study, the MLP algorithm was deployed due to its reliance on the sigmoidal activation function, which has the ability to handle linearly inseparable data [86] during its backpropagation learning. The performance of the sigmoid activation function in training MLP is determined by the desired conditions for convergence speed and error minimization. Although the MLP approach has shown the ability to overcome these issues, the interactions between water constituents and solar radiation are quite complex and present a challenge for this technique. This approach not only exhibits limitations when faced with changing or incremental data regimes but also suffers from catastrophic forgetting [32], as a result of rapidly changing patterns. Yan et al. [87] suggest using the gradient technique to address the limitations of the traditional MLP approach. In this case, the gradient descent is computed using backpropagation on data at different iteration stages [88]. However, minimizing a cost function in a high-dimensional space is intricate for optimizers such as classical gradient descent (GD) [89]. In this study, SGD and MGD optimizers were introduced into the MLP algorithm to discover model parameters that conform to the best predicted actual output. The SGD optimizer demonstrated the ability to overcome limitations associated with classical GD. As the SGD approaches a local optimum, the parameters may oscillate back and forth around the optimum position [90]. From the SGD perspective, global convergence analysis remains crucial in determining the necessity of local analyses, complexity analyses, or saddle point analyses [91]. It is widely recognized that incorporating momentum into an optimization algorithm is necessary for effectively training a large-scale deep network [92]. Including a momentum term m in traditional stochastic gradient descent can speed up learning in relevant directions and minimize oscillations during training by decelerating in lateral dimensions with inconsistent gradients [89]. Sun et al. [46] noted that the MGD optimizer shows no sensitivity to deep learning architectures and hyperparameters. MGD operates by modifying the SGD to speed up convergence and minimize oscillation [92], thereby minimizing noise from stochastic gradients in the early stages [93]. Based on empirical evidence, Sutskever et al. [94] noted that architectures trained with momentum have better performance compared to those trained without it.

Selecting suitable spectral channels for spatial modeling and predicting water turbidity remains a challenge [95]. This is especially true when there is a presence of other non-turbidity materials with inherent optical properties in the water. The results from the current study revealed a stronger relationship between the observed turbidity and the red spectral channel. The MLP trained with MGD produced an r² value of 0.92, followed by the MLP trained with MGD on the green spectral band (r² value of 0.75), the MLP trained with SGD on the red band (r² value of 0.72), and lastly by linear regression on the red band (r² value of 0.69). The coefficient of determination and relative error of mean (REM) results revealed accurate turbidity predictions by sigmoid with MGD compared to other models. Previous investigations also noted a strong correlation between spectral reflectance in the red wavelength and turbidity [96,97]. Chen et al. [96] also noted the near-infrared spectral band to be suitable for empirical models establishment for turbidity retrieval. The MLP trained with MGD also revealed a strong relationship between the green spectral band and turbidity. These findings align with those of Ndou [52], who also noted a strong relationship between this spectral band and turbidity. Moreover, Potes et al. [98] also found the green spectral band to be ideal for turbidity retrieval. The current study also noted a poor relationship between turbidity and the blue, red-edge, NIR, and SWIR spectral channels. Whereas the reflectance properties of water drastically increase from the red to red edge, and to the NIR spectral channel [5], the current study noted an insignificant relationship between turbidity and the red edge and NIR spectral bands. Choubey [99] also noted the weak relationship between the infrared spectral region and turbidity and suggested that this could be due to the high backscattering of solar radiation in the visible spectral wavelengths. It must be noted that heterogeneity in turbidity data may substantially affect the performance of any model. When modeling turbidity, it is also worth noting that other dissolved materials and phytoplankton organisms may compound the desired turbidity results. Moreover, the models deployed in this study were based on a limited dataset. It could be intriguing to investigate the accuracy levels of these approaches with larger training and testing datasets. Furthermore, it must also be noted that the turbidity data used in this study were acquired only from the estuary. Therefore, it is recommended that future studies include control points in the deep sea, with a view to verifying the variations in turbidity levels between the sea and estuary.

5. Conclusions

The current study demonstrated the prospect of employing ensemble techniques on multispectral imagery, coupled with field-measured data, to construct missing turbidity data in dynamic estuarine waters. The study generated empirical models based on uncalibrated spectral bands, bands calibrated with the sigmoid activation function with SGD, and spectral bands calibrated with the sigmoid activation function with MGD. Our results show that the accuracy of predictions varies greatly based on the deployed approach. The results of the study revealed significant relationships between the uncalibrated red spectral band, the red spectral band calibrated with both SGD and MGD, and the green spectral band calibrated with MGD. However, as the results of this study present the regression, MLP algorithm with SGD, and MLP with MGD optimizers, other performance optimizers must be considered in future studies for their efficacy evaluation in predicting estuarine turbidity patterns. Overall, this study demonstrated the prospect of deploying ensemble techniques on Sentinel-2 multispectral bands to spatially construct missing estuarine turbidity data. Moreover, we recommend these techniques in conjunction with uncertainty compensation approaches, particularly in estuarine waters that exhibit heterogeneous turbidity patterns.