1. Introduction
Harmful algal blooms (HABs) have emerged as a significant environmental concern in coastal and estuarine waters due to their ability to rapidly degrade water quality and disrupt aquatic ecosystems [
1]. HAB events are typically associated with excessive phytoplankton growth, which reduces water clarity, limits light penetration to submerged vegetation, and contributes to oxygen depletion during bloom decay [
2]. Certain bloom-forming species may also produce toxins, posing risks to marine organisms, fisheries, and public health through contaminated seafood [
3]. Because bloom dynamics can evolve rapidly and exhibit strong spatial variability, reliability and continuous monitoring are essential for early detection and timely water-quality management. Chlorophyll-a (Chl-a), a photosynthetic pigment present in phytoplankton, is widely used as an indicator of algal biomass and HAB-related activity [
4]. Elevated Chl-a concentrations often signal intensified phytoplankton presence and can help identify areas experiencing active or developing blooms. In estuarine systems such as the Chesapeake Bay—where nutrient inputs, freshwater inflow, and hydrodynamic processes vary substantially across space—Chl-a measurements provide an effective means of tracking bloom patterns at regional scales [
5]. Continuous monitoring of Chl-a therefore plays a critical role in understanding bloom development and supporting HAB-related water-quality assessment.
In situ observations are commonly used to monitor HAB conditions by directly measuring Chl-a concentrations at discrete sampling locations [
6]. These measurements, collected through water sampling or in-water sensors, are considered highly reliable and often serve as validation references in scientific studies [
7]. However, in situ monitoring is inherently limited in spatial and temporal coverage. Observations represent specific points rather than continuous surfaces, leaving large areas unmeasured between stations. Field sampling also requires substantial logistical effort and resources, restricting measurement frequency and limiting the ability to capture rapidly evolving bloom dynamics. Because HAB events may intensify within days and vary considerably across short distances [
8], reliance solely on periodic field observations is insufficient for comprehensive spatial–temporal characterization. Satellite remote sensing offers a complementary and scalable approach for HAB monitoring by providing repeated, synoptic coverage over extensive coastal regions [
9]. Satellite missions such as Moderate Resolution Imaging Spectroradiometer (MODIS) and Visible Infrared Imaging Radiometer Suite (VIIRS) have been widely used for large-scale ocean and coastal observation [
10]. Satellite-derived Chl-a products enable detection of spatial gradients across nearshore waters, tributary-influenced zones, and offshore regions where algal conditions can change rapidly [
11]. Nevertheless, satellite monitoring in estuarine environments faces challenges due to atmospheric interference and optically complex waters that can affect retrieval accuracy [
12]. Cloud cover remains one of the most persistent limitations, frequently obscuring the surface and creating missing values in daily Chl-a products [
13]. These data gaps reduce temporal continuity and limit the reliability of bloom monitoring.
Improving the completeness of satellite-derived Chl-a datasets through gap-filling is therefore essential for enhancing operational monitoring capacity [
14]. Machine-learning, deep-learning, and convolutional autoencoder approaches provide a flexible framework for estimating missing values by learning spatial and temporal patterns from available observations [
14]. By leveraging historical data and contextual features, these models can reconstruct cloud-obscured regions while preserving environmental variability. The primary objective of this study is to identify an effective supervised learning framework for Chl-a gap reconstruction that improves satellite-based HAB monitoring and supports more reliable assessment of bloom dynamics in estuarine systems.
The remainder of this paper is organized as follows. Section Related Work reviews related research to establish methodological background and defines the research objectives.
Section 2 describes the study area and provides geographic and environmental context.
Section 3 presents the datasets used in this work, and
Section 4 details the methodological framework adopted for Chl-a gap reconstruction and evaluation.
Section 5 presents the results of the model performance and reconstruction analysis.
Section 6 discusses the findings. Finally,
Section 7 concludes the study and outlines key insights and potential future work.
Related Work
Extensive research has investigated gap-filling techniques to reconstruct missing pixels in satellite-derived environmental datasets, with the goal of improving the continuity and overall quality of remote-sensing products rather than estimating a single Chl-a value [
15]. A broad range of approaches has been evaluated, including temporal interpolation, inverse-distance weighting (IDW), ordinary kriging, spatiotemporal kriging, empirical orthogonal function (EOF)-based reconstruction, unsupervised learning, and supervised machine-learning models. Interpolation-based methods, including temporal interpolation and Inverse Distance Weighting (IDW), are computationally efficient and straightforward to implement, making them useful baseline approaches. However, these methods assume smooth spatial or temporal variation and often produce over-smoothed estimates when data gaps are large or irregular [
16]. Kriging-based methods incorporate spatial autocorrelation and can provide more statistically robust predictions under moderate missing-data conditions. However, they rely on assumptions about spatial structure and can become computationally expensive for large-scale satellite datasets [
17]. EOF- and DINEOF-based approaches are effective in capturing dominant spatiotemporal variability and are generally more robust under higher levels of missing data. Nevertheless, they may smooth localized variability and fail to adequately represent rapid, nonlinear bloom dynamics.
Comparative assessments indicate that no single method consistently outperforms others across all regions [
15]. Traditional approaches such as ordinary kriging, spatiotemporal kriging, and Data-Interpolating Empirical Orthogonal Functions (DINEOF) often demonstrate stable regional performance. However, tree-based machine-learning methods such as random forest show advantages for pixel-level prediction due to their ability to capture complex nonlinear relationships [
15]. Building on these foundations, subsequent studies expanded gap-filling efforts to the global scale by merging observations from multiple satellite sensors and applying DINEOF to generate daily, gap-free ocean-color products [
15]. In parallel, spatial–spectral machine-learning frameworks were developed to incorporate multispectral and neighborhood information, achieving improved reconstruction accuracy compared to traditional interpolation techniques [
18]. Although not always designed specifically for Chl-a, these studies established important methodological principles for handling spatial continuity, nonlinear variability, and large missing-data fractions.
DINEOF has been widely adopted for satellite-derived Chl-a reconstruction under cloud contamination due to its ability to extract dominant spatiotemporal patterns from incomplete datasets. Regional applications have demonstrated improved spatial coherence compared to spatiotemporal kriging, particularly under moderate to high missing-data conditions [
19], although some studies rely primarily on satellite-based validation. Enhanced variants, such as concentration-stratified DINEOF, have been shown to reduce reconstruction error and improve computational efficiency while preserving spatial detail, albeit with stricter data filtering requirements [
20]. At broader scales, multi-sensor DINEOF implementations improve spatial coverage and reduce data gaps relative to single-sensor products [
21]. Time-series and climatology-driven approaches using long-term satellite data have also demonstrated improved performance compared to simple interpolation techniques [
22]; however, their reliability declines under extensive or continuous missing conditions. Compared with interpolation-based methods, DINEOF provides more robust reconstruction under heavy cloud cover by leveraging spatiotemporal correlations. However, it tends to smooth localized variability and may not adequately capture rapid or nonlinear bloom dynamics, particularly in optically complex coastal and estuarine environments [
23]. In contrast, machine-learning approaches are better suited to preserving local variability and nonlinear patterns but require sufficient training data and carefully designed features. These differences highlight the trade-off between stability and flexibility among reconstruction methods, emphasizing the need for systematic evaluation under consistent experimental conditions.
More recently, machine-learning and deep-learning methods have been applied to Chl-a reconstruction by learning complex spatial–temporal patterns. Convolutional neural network (CNN) frameworks have generated daily global gap-filled Chl-a products by integrating multiple satellite datasets and environmental predictors [
24], though performance may degrade when training data are limited, or overly complex input structures are used. Sequence-based models, such as Long Short-Term Memory (LSTM) networks and hybrid convolutional–LSTM architectures, have been developed to capture temporal dependencies and seasonal variability [
25]. However, their performance may degrade in rapidly evolving coastal environments. In parallel, supervised machine-learning models—particularly tree-based methods such as Random Forest (RF), Extra Trees (ET), and XGBoost—have demonstrated strong performance for Chl-a reconstruction by effectively modeling nonlinear relationships among spatial, temporal, and contextual features [
26,
27]. These approaches are relatively efficient, require less complex model design compared to deep-learning frameworks, and can perform well even with limited training data. However, their performance depends on the quality of feature engineering and may require explicit incorporation of temporal information to maintain continuity across time. Recent studies have also explored representation-learning and convolutional autoencoder approaches, including variational autoencoders and Data Interpolating Convolutional Autoencoder (DINCAE)-based frameworks, to capture nonlinear variability in satellite-derived datasets [
28]. Collectively, these developments highlight the growing potential of data-driven methods for improving Chl-a reconstruction, while also emphasizing the need to balance model complexity, data requirements, and interpretability.
Quantitative improvements reported in prior studies underscore the potential of advanced reconstruction strategies. For example, concentration-stratified DINEOF reduced RMSE by 0.0281 mg/m
3 in regional applications [
22], while multi-sensor merging improved representation of high-Chl-a features by approximately 10–20% in productive coastal zones [
19]. Cloud-filling approaches have reduced regional mean estimation errors by 50–80% under certain conditions [
19]. Despite these advances, most studies focus on improving a single method or generating large-scale products, with limited emphasis on systematic regional comparison under a unified modeling and evaluation framework. In particular, structured hyperparameter optimization and explicit uncertainty quantification remain underexplored in dynamic estuarine systems.
While previous studies have explored individual gap-filling techniques, systematic comparisons of multiple reconstruction approaches under a unified and realistic experimental framework remain limited, particularly for optically complex estuarine systems such as the Chesapeake Bay. Accordingly, there remains a need for a consistent, application-oriented assessment in which different reconstruction methods are evaluated under identical data, preprocessing, and validation protocols. Such an approach enables transparent identification of model strengths and limitations and supports the selection of reliable methods for operational HAB monitoring. In response to these gaps, this study pursues the following objectives:
To apply different gap-filling approaches to reconstruct missing Chl-a values in cloud-affected satellite imagery using a consistent dataset and spatial resolution.
To systematically evaluate and compare a classical interpolation baseline (Inverse Distance Weighting), an EOF-based reconstruction method (DINEOF), supervised machine-learning models (K-Nearest Neighbors, Random Forest, Extra Trees, and XGBoost), a sequence-based deep-learning model (Long Short-Term Memory) and a convolutional autoencoder-based model (Temporal DINCAE) under identical training and testing conditions.
To assess predictive uncertainty in reconstructed Chl-a values by constructing prediction intervals and evaluating their calibration and reliability.
By conducting a structured comparative evaluation within a unified satellite-based framework, this research provides a reproducible foundation for scalable and operational Chl-a gap reconstruction in complex estuarine environments.
4. Methodology
Figure 2 illustrates the methodological framework developed for Chl-a gap reconstruction in the Chesapeake Bay using spatial interpolation, supervised machine-learning, and deep-learning approaches. The workflow begins with the acquisition of daily Level-3 Chl-a products derived from the Sentinel-3 OLCI, accessed through the NOAA CoastWatch ERDDAP interface to ensure reproducible and standardized data retrieval [
35]. To enable consistent data acquisition, structured API queries were used to request chlorophyll-a data over a fixed spatial bounding box corresponding to the Chesapeake Bay region and a defined temporal range spanning 2023–2025. Each query returns data in NetCDF format with consistent spatial resolution and metadata, enabling automated retrieval of daily observations. The downloaded daily datasets are systematically organized and stacked into a single multi-temporal NetCDF cube, where time is explicitly represented as a dimension. This stacking procedure preserves the spatiotemporal continuity of satellite observations and allows the dataset to be handled as a structured data cube rather than isolated images. The cube is subsequently clipped using a predefined Chesapeake Bay boundary shapefile to restrict analysis to the study region and eliminate irrelevant land pixels. This spatial filtering step reduces computational complexity and ensures that the modeling process focuses only on water bodies. This standardized and spatially constrained structure provides a consistent input foundation for subsequent preprocessing and modeling tasks.
Following data structuring, the workflow transitions to the data-driven modeling component of the framework [
39]. The stacked and clipped data cube is transformed into a tabular dataset, where each valid pixel observation at a given time step is represented as an individual record. Predictor variables include spatial coordinates (latitude and longitude), temporal indicators such as day of year (encoded using sine and cosine transformations to capture seasonal patterns), and neighborhood-based statistical features that represent local spatial variability in chlorophyll concentration. These neighborhood features, including mean and standard deviation computed within a moving window, are derived using only available valid pixels to avoid introducing information from missing values. To further incorporate temporal dependencies, lag-based features (lag-1, lag-2, and lag-3) are constructed for each pixel location, enabling the models to capture short-term temporal continuity in Chl-a dynamics [
40]. All predictor variables are standardized to ensure numerical stability and consistency across models. A set of gap-filling approaches—including spatial interpolation, EOF-based reconstruction, machine-learning, deep-learning, and convolutional autoencoder methods—is implemented within a unified experimental framework [
41]. The spatial interpolation baseline consists of Inverse Distance Weighting (IDW), while Data Interpolating Empirical Orthogonal Functions (DINEOF) is used as the EOF-based reconstruction method. The machine-learning models include K-Nearest Neighbors (KNN), Random Forest (RF), Extra Trees (ET), and Extreme Gradient Boosting (XGBoost). In addition, a Long Short-Term Memory (LSTM) model is employed to explicitly capture temporal dependencies, while a Temporal Data Interpolating Convolutional Autoencoder (Temporal DINCAE) framework is implemented to learn spatial–temporal reconstruction patterns using convolutional encoder–decoder architecture. All machine-learning, deep-learning, and convolutional autoencoder models are trained using consistent input features and standardized preprocessing procedures to ensure a fair comparison. In contrast, the IDW baseline does not require model training and relies solely on spatial distance relationships between observed and missing pixels. This distinction ensures that the comparative evaluation reflects differences in modeling capability rather than inconsistencies in feature representation [
16]. This design allows the framework to evaluate different learning methods for obtaining nonlinear spatial–temporal relationships in environmental data. By maintaining consistency in preprocessing and feature engineering, this design ensures that performance differences reflect algorithmic capability rather than preprocessing bias.
In the final stage, trained models are applied to predict chlorophyll concentrations at locations where observations are missing due to cloud cover or retrieval limitations. Machine-learning, deep-learning and convolutional autoencoder models generate predictions by learning spatial and temporal relationships from the data, whereas the IDW baseline reconstructs values using spatial distance-based interpolation and DINEOF reconstructs missing observations through EOF-based spatiotemporal decomposition. The predicted values are merged with the observed data to produce a spatially continuous chlorophyll dataset that maintains the original coordinate reference system and resolution. Model performance is quantitatively assessed using statistical evaluation metrics such as Root Mean Square Error (RMSE), Mean Absolute Error (MAE), prediction bias and the coefficient of determination (R
2) to measure reconstruction accuracy and goodness of fit. These metrics provide objective evidence of model reliability and predictive strength. Overall, the architecture presented in
Figure 2 provides a systematic, reproducible, and technically rigorous workflow that integrates satellite data acquisition, feature engineering, supervised learning, evaluation, and geospatial deployment into a unified modeling framework.
4.1. Inverse Distance Weighting (IDW)
Inverse Distance Weighting (IDW) was implemented as a spatial interpolation method to estimate missing Chl-a values based on nearby observations. The method assumes that spatial proximity corresponds to similarity, assigning greater weight to observations closer to the prediction location. Unlike machine-learning approaches, IDW does not require model training and relies directly on spatial relationships between known and unknown locations [
42]. The prediction formulation is given in Equation (1).
denotes the predicted chlorophyll-a value at location
,
denotes the observed value at the
neighboring point, and
is the spatial distance between the prediction location and the
observation. The parameter
controls the distance-decay rate, with larger values assigning greater weight to nearer neighbors.
IDW assumes smooth spatial variation and performs best in regions with strong spatial continuity. However, it does not account for temporal dynamics or nonlinear relationships, which may limit its effectiveness under large or irregular data gaps [
16]. In this study, IDW is employed as a baseline method and evaluated under controlled artificial missingness scenarios to provide a benchmark for comparison with data-driven models.
4.2. Data Interpolating Empirical Orthogonal Functions (DINEOF)
Data Interpolating Empirical Orthogonal Functions (DINEOF) was applied as an EOF-based method for reconstructing missing chlorophyll-a (Chl-a) observations covered by cloud [
43]. The method estimates missing values by identifying dominant spatial–temporal variability patterns within the dataset through iterative EOF decomposition [
22]. The reconstruction is given in Equation (2).
where
represents the missing Chl-a data matrix,
and
correspond to the spatial and temporal EOF components, and
contains the singular values associated with the selected
modes. Missing observations are iteratively reconstructed until the solution stabilizes.
DINEOF can effectively recover broad spatial–temporal structures under high missing-data rate, although localized variability and rapidly changing bloom patterns may become smoothed [
44]. In this study, DINEOF was evaluated under controlled artificial missingness conditions as an EOF-based benchmark for comparison with supervised machine-learning, deep-learning, and convolutional autoencoder reconstruction models.
4.3. Supervised Machine Learning Models
The supervised modeling framework was designed to approximate the nonlinear relationship between spatial–temporal predictors and Chl-a concentration using historical satellite observations. Four regression algorithms were implemented to evaluate alternative learning mechanisms for gap reconstruction.
4.3.1. K-Nearest Neighbors (KNN)
K-Nearest Neighbors (KNN) regression was implemented as a non-parametric estimator operating within a multidimensional predictor space. For each prediction instance, Euclidean distances were computed across latitude, longitude, day-of-year, and neighborhood-based statistical features to identify the k most similar training samples. The predicted Chl-a value was calculated as a distance-weighted average of these neighbors [
45]. Equation (3) represents the prediction formulation used in the KNN model.
represents the set of the nearest training samples for a given input location . The term denotes the observed Chl-a value of the neighbor, while represents the weight assigned to that neighbor based on its distance from the prediction point. Neighbors that are closer in the predictor space receive higher weights and therefore contribute more strongly to the final prediction, while more distant observations have a smaller influence.
KNN assumes spatial smoothness and seasonal consistency under comparable environmental conditions. Because it does not impose an explicit functional relationship between predictors and the target variable, its predictive performance depends strongly on the density of training samples and the coverage of the feature space [
46]. While this approach works well in environments exhibiting spatial coherence, such as the Chesapeake Bay, it remains sensitive to feature scaling and to the choice of the parameter
. In this study, the model was trained using spatial coordinates, temporal information, and neighborhood-based statistical predictors, and the value of
was experimentally adjusted to obtain stable prediction performance. KNN, therefore, serves as a baseline model for comparison with the ensemble-based approaches evaluated in this study.
4.3.2. Random Forest (RF)
Random Forest (RF) regression was employed as a bagging-based ensemble method to capture nonlinear interactions among predictors [
47]. RF constructs multiple decision trees using bootstrap-resampled training subsets while randomly selecting a subset of predictors at each split. Each tree partitions the feature space independently, and predictions are averaged across trees to reduce variance. Equation (4) describes how the final prediction is obtained by averaging the outputs from all trees in the ensemble.
represents the total number of decision trees in the ensemble, and denotes the prediction generated by the tree for input sample The final predicted Chl-a value is obtained by averaging the outputs from all trees, which reduces prediction variance and improves model stability.
This ensemble aggregation allows RF to model complex, hierarchical relationships without assuming linearity [
48]. In coastal systems, where chlorophyll dynamics are influenced by localized bloom intensification, seasonal transitions, and spatial gradients, RF effectively captures structured variability. The random selection of predictors during tree construction reduces correlation among trees and helps mitigate overfitting [
49]. In this study, the RF model was trained using the same spatial and temporal predictor variables used across all models, and key parameters such as the number of trees and maximum tree depth were experimentally adjusted to achieve stable prediction performance [
50].
4.3.3. Extra Trees (Extremely Randomized Trees)
Extra Trees extends the RF framework by introducing additional stochasticity during split selection. Instead of optimizing split thresholds based on impurity reduction, candidate thresholds are selected randomly. Trees are typically constructed using the full training dataset, while randomness in feature selection and split thresholds increases ensemble diversity [
51]. Equation (5) shows how the final prediction is obtained by averaging the outputs from multiple randomized trees.
represents the total number of trees in the ensemble, and denotes the prediction produced by the tree built using randomly selected split thresholds. The final predicted value is obtained by averaging the predictions from all trees, which helps reduce variance and improve model stability.
This enhanced randomization reduces tree correlation and can lower model variance, improving generalization stability. For satellite-derived environmental data—often characterized by noise and spatial discontinuities—such ensemble diversity is advantageous [
52]. Extra Trees demonstrated strong capability in representing nonlinear gradients and seasonal variability while maintaining computational efficiency, making it particularly suitable for large-scale gap-filling tasks [
15].
4.3.4. Extreme Gradient Boosting (XGBoost)
Extreme Gradient Boosting (XGBoost) was implemented as a boosting-based regression framework to iteratively minimize prediction error. Unlike bagging methods, which build trees independently, XGBoost constructs trees sequentially, with each new tree trained to correct residual errors from the existing ensemble. This gradient-based optimization allows incremental refinement of predictions [
53]. Equation (6) describes the prediction method used in the XGBoost model.
represents the prediction produced by the regression tree, and denotes the total number of trees in the boosting sequence. The final predicted value is obtained by summing the contributions of all trees, where each new tree focuses on reducing the residual errors from previous trees.
Regularization terms, learning-rate control, and tree-depth constraints were incorporated to balance model complexity and generalization. The boosting mechanism places greater emphasis on difficult-to-predict instances, such as abrupt seasonal transitions or localized bloom intensification events. By sequentially minimizing residuals, XGBoost captures subtle nonlinear dependencies among spatial coordinates, seasonal indicators, and contextual features. XGBoost provides an adaptive learning approach that improves bias reduction and predictive accuracy for chlorophyll gap reconstruction in cloud-affected satellite imagery [
54].
4.4. Long Short-Term Memory (LSTM)
Long Short-Term Memory (LSTM) networks were implemented as a deep-learning approach to model temporal dependencies in Chl-a dynamics. Unlike traditional machine-learning models that treat observations independently, LSTM processes sequential data and captures temporal dependencies across multiple time steps. This capability is particularly important for environmental time series, where chlorophyll concentrations exhibit persistence, seasonal cycles, and short-term variability [
55]. The prediction formulation is given in Equation (7).
denotes the input feature vector at time step t, while and represent the hidden and cell states from the previous time step, respectively. The LSTM captures temporal dependencies by modeling the influence of past observations on current chlorophyll-a values. The predicted output is obtained by applying a linear transformation with weights and bias to the learned temporal representation.
In this study, input sequences were constructed using fixed-length temporal windows for each spatial pixel, incorporating spatial, seasonal, and lag-based features at each time step. This formulation enables the model to jointly capture spatial context and temporal evolution [
55]. Although LSTM effectively models temporal continuity and dynamic variability, its performance depends on the availability of sufficiently long sequential data and appropriate parameter tuning [
56].
4.5. Temporal Data Interpolating Convolutional Autoencoder (Temporal DINCAE)
The Temporal Data Interpolating Convolutional Autoencoder (Temporal DINCAE) was implemented as a convolutional autoencoder-based reconstruction model for estimating missing Chl-a observations from incomplete satellite imagery. The model combines convolutional encoder–decoder architecture with temporal and spatial feature representation to learn nonlinear reconstruction patterns directly from the data [
57]. The prediction formulation is given in Equation (8).
where
represents the multi-channel input feature tensor,
denotes the convolutional autoencoder parameterized by learnable weights
,
is the reconstructed Chl-a output. The model was trained using masked reconstruction loss to estimate missing observations while preserving spatial–temporal continuity.
Temporal DINCAE enables the reconstruction of complex nonlinear Chl-a variability by integrating spatial structure, temporal dependencies, and seasonal dynamics within a unified framework [
58]. In this study, the model was evaluated as a convolutional autoencoder-based approach for comparison with EOF-based, supervised machine-learning, and deep-learning reconstruction methods.
4.6. Spatiotemporal Feature Engineering and Data Preparation
Prior to model development, the Chl-a dataset was transformed from a multi-dimensional raster format into a tabular structure compatible with regression-based modeling approach. Each valid pixel–time pair was treated as an independent observation, enabling the modeling of spatial–temporal relationships. Geographic coordinates (latitude and longitude) were retained as continuous predictors to preserve large-scale spatial gradients across the study domain. Seasonal dynamics were encoded using day-of-year (DOY), which was further encoded using sine and cosine transformations to represent cyclical temporal patterns. These features enable the models to capture recurring intra-annual bloom dynamics while avoiding artificial discontinuities at the start and end of the year. Together, the spatial and temporal predictors represent the primary drivers of chlorophyll variability in estuarine systems, including spatial gradients associated with riverine inputs, seasonal phytoplankton cycles, and local spatial autocorrelation [
59].
To incorporate short-range spatial context, neighborhood-derived statistics were calculated for each pixel, including local mean and standard deviation values from adjacent grid cells using a fixed 3 × 3 moving window centered on the target pixel. These contextual features enhance the representation of spatial continuity and reduce sensitivity to isolated noise [
60]. To ensure reproducibility and prevent data leakage, neighborhood statistics were computed exclusively from valid (non-missing) chlorophyll-a values within the window, without any prior imputation or interpolation. Pixels with missing neighboring values were excluded from the computation, and observations with insufficient valid data were removed during preprocessing. Observations containing undefined predictor variables were removed to prevent numerical instability during training. Feature matrices were constructed with consistent column ordering, and redundant or duplicated attributes were eliminated to maintain structural integrity. For the Temporal DINCAE framework, the engineered variables were additionally organized into an 18-channel spatial–temporal input representation consisting of temporal lag variables, observation masks, current chlorophyll-a observations, spatial coordinates, seasonal sine and cosine features, neighborhood-based statistics, and temporal trend variables. This multi-channel formulation enabled the model to jointly learn spatial, temporal, and seasonal reconstruction patterns from incomplete observations.
To improve numerical stability and regression performance, the response variable was transformed using log(1 + Chl-a), reducing skewness associated with episodic bloom peaks and stabilizing variance across concentration ranges. This transformation facilitates smoother decision boundaries for tree-based, distance-based, deep-learning, and convolutional autoencoder models. Predictor variables were inspected to ensure consistent units and appropriate scaling, particularly for distance-sensitive algorithms such as KNN. Missing Chl-a values were excluded from the training dataset to avoid bias in parameter estimation. The DINEOF framework used iterative EOF decomposition of dominant spatial–temporal variability patterns for reconstructing missing observations. Data from 2023–2024 were used for model training, while data from 2025 were reserved for testing. This temporal split avoids spatiotemporal leakage and enables a more realistic assessment of model performance on unseen future observations. Feature and target arrays were formatted for compatibility with scikit-learn and gradient boosting implementations. All preprocessing steps were applied uniformly across KNN, Random Forest, Extra Trees, XGBoost, LSTM, and Temporal DINCAE to ensure methodological consistency. This structured preprocessing pipeline establishes a stable and statistically rigorous foundation for EOF-based, supervised machine-learning, deep-learning, and convolutional autoencoder reconstruction of satellite-derived Chl-a data.
4.7. Model Training
Model training was conducted within a temporally consistent framework, with Chl-a observations from 2023–2024 used for training and data from 2025 reserved as an independent test set. This temporal separation ensures evaluation on unseen future observations, thereby preventing spatiotemporal leakage and enabling a more realistic assessment of model generalization. The response variable was modeled using a log-transformed formulation, log(1 + Chl-a), to stabilize variance and mitigate the influence of extreme bloom events. The DINEOF framework reconstructed missing observations using iterative EOF decomposition of dominant spatial–temporal variability patterns through repeated singular-value decomposition until convergence was achieved.
Machine-learning models, including K-Nearest Neighbors (KNN), Random Forest (RF), Extra Trees (ET), and XGBoost, were implemented within a unified experimental framework. For the K-Nearest Neighbors model, the optimal number of neighbors was selected to balance local sensitivity and smoothing in feature space, with distance-based weighting applied to emphasize closer observations. Random Forest and Extra Trees were trained as ensemble tree-based models to capture nonlinear interactions among predictors. Hyperparameters such as the number of estimators and maximum tree depth were tuned to provide adequate model capacity while controlling variance. Bootstrap sampling and randomized feature selection were applied to enhance generalization stability. XGBoost was trained using gradient-based boosting to sequentially minimize residual errors. Learning rate, tree depth, subsampling, and regularization parameters were configured to control model complexity and ensure stable convergence. Fixed random seeds were applied across ensemble-based methods to ensure reproducibility. Computational implementation leveraged optimized machine-learning libraries to efficiently handle the large spatiotemporal dataset.
In parallel, a Long Short-Term Memory (LSTM) network was developed to capture temporal dependencies in Chl-a dynamics. Sequential input data were constructed for each spatial pixel using fixed-length temporal windows, enabling the model to learn temporal evolution patterns across multiple time steps. To maintain sequence continuity, missing chlorophyll-a values were first estimated using an XGBoost-based gap-filling approach, thereby ensuring complete input sequences. Unlike traditional models that treat observations independently, LSTM processes ordered data and leverages historical information through its internal memory structure. Spatial, seasonal, and lag-based features were incorporated at each time step to jointly represent spatial context and temporal variability. This design enables the model to capture dynamic processes such as seasonal transitions and short-term fluctuations. By relying exclusively on past observations, the sequence formulation avoids temporal leakage. However, model performance depends on the availability of sufficient sequential data and appropriate parameter tuning for stable generalization.
Feature ordering and dimensional consistency were strictly maintained throughout training to avoid structural mismatches during deployment. In addition, the Temporal DINCAE framework was implemented as a convolutional autoencoder-based reconstruction model using multi-channel spatial–temporal input tensors. The model incorporated spatial coordinates, seasonal indicators, neighborhood-based statistics, temporal lag features, and observation masks within a convolutional encoder–decoder architecture to learn nonlinear reconstruction patterns from incomplete chlorophyll-a observations. Iterative reconstruction was applied during inference to progressively estimate missing regions while preserving spatial–temporal continuity. The IDW method, which does not require model training, was applied independently during the reconstruction stage by identifying neighboring valid pixels, computing distance-based weights, and estimating missing values through weighted spatial averaging. Model fitting for machine learning and deep learning approaches was monitored to ensure numerical stability and prevent data leakage from missing-value placeholders. Finalized model objects were serialized and stored for subsequent gap reconstruction. This structured training protocol ensures that the models effectively capture nonlinear spatial–temporal relationships while maintaining computational robustness and reproducibility.
4.8. Model Testing
Model performance was assessed using the temporally independent test dataset from 2025, which was strictly excluded from model training and hyperparameter tuning. This separation ensured an unbiased evaluation of predictive generalization. The testing samples retained the identical predictor structure used during training, including latitude, longitude, day-of-year, and neighborhood-based contextual statistics. All preprocessing steps, including feature ordering and transformation, were applied consistently to prevent structural discrepancies between training and testing phases. Predictions were generated in the log-transformed space and subsequently back-transformed to the original Chl-a concentration scale using the inverse log(1 + x) transformation. Only observations with valid measured Chl-a values were included in the evaluation to allow direct comparison between predicted and observed concentrations. Spatial and temporal alignment was preserved throughout the testing process to maintain consistency with the original data structure. For the IDW baseline, performance was evaluated separately on the same test dataset to ensure a consistent reference for comparison with data-driven models.
4.9. Hyperparameter Tuning
Hyperparameter tuning was performed to optimize model performance and improve generalization for Chl-a gap reconstruction. A temporally separated tuning strategy was adopted to eliminate spatiotemporal leakage. Data from 2023 were used for model fitting, data from 2024 for validation and parameter selection, and data from 2025 as an independent test set. After selecting the optimal parameters, the models were retrained using the combined 2023–2024 dataset and evaluated on the 2025 test set. A structured search strategy (
Table 1) was implemented to systematically explore the parameter space for each model, and the final selected hyperparameters are summarized in
Table 2. For the DINEOF framework, tuning focused on the number of EOF modes and convergence iterations used during iterative reconstruction. The selected configuration utilized 10 EOF modes and iterative convergence criteria that provided stable reconstruction performance under controlled missingness conditions. For the KNN model, the number of neighbors
was optimized, as it determines how many nearby observations are used for prediction. The influence of each neighbor, represented through the weighting term
, was controlled by tuning the distance metric parameter
and the weighting scheme, which affects how strongly closer samples contribute to the final estimate. The final configuration selected was k = 5, with distance-based weighting and
p = 2, providing a balance between local sensitivity and smoothing.
For the ensemble-based models, tuning focused on parameters that control the structure and complexity of the decision trees. In Random Forest and Extra Trees, the number of trees was optimized, as it determines how many individual models are combined in the final prediction. Additional parameters such as maximum tree depth, minimum samples required for splitting, and minimum samples per leaf were tuned to regulate how each tree function partitions the feature space. These parameters influence the ability of the models to capture nonlinear spatial and seasonal patterns while maintaining stability and avoiding overfitting. For the XGBoost model, tuning focused on parameters controlling the sequential boosting process. The number of boosting iterations was optimized to determine how many trees contribute to the final prediction. Parameters such as maximum tree depth and subsampling ratio were adjusted to control model complexity and improve robustness. The learning rate η, although not explicitly shown in the model equation, was also tuned to regulate the contribution of each tree during error correction. The final configuration consisted of 500 estimators, a learning rate of 0.05, a maximum depth of 6, and a subsampling ratio of 0.8, resulting in strong predictive performance and stable convergence.
For the LSTM model, hyperparameter tuning focused on parameters governing temporal representation and training stability. The sequence length, hidden layer size, number of layers, dropout rate, optimizer, and learning rate were systematically adjusted to effectively capture temporal dependencies. The final configuration—sequence length of 24, hidden size of 384, two LSTM layers, dropout rate of 0.25, AdamW optimizer, and learning rate of 0.0003—resulted in stable training and improved generalization. For the Temporal DINCAE framework, hyperparameter tuning focused on convolutional architecture depth, dropout rate, batch size, masking fraction, and learning rate. The final configuration utilized an 18-channel input representation, a three-level convolutional encoder–decoder architecture, base filter size of 64, dropout rate of 0.10, batch size of 16, and learning rate of 0.0001, resulting in stable convergence and strong reconstruction performance.
Model selection was primarily guided by the R2 evaluated on the validation subset, with RMSE used as a complementary metric to monitor absolute error magnitude. Overfitting was assessed by comparing training and validation performance to avoid overly complex configurations. The final tuned models demonstrated consistent validation performance and stable generalization behavior. This systematic optimization process ensured that reported performance gains reflect genuine improvements in model structure rather than arbitrary parameter choices, thereby enhancing the reliability and robustness of Chl-a reconstruction.
4.10. Model Evaluation
Model performance was evaluated using four standard regression metrics to ensure a clear and consistent assessment of prediction accuracy and error behavior. These metrics include the R2, RMSE, MAE, and prediction bias. RMSE quantifies the overall magnitude of prediction errors and is more sensitive to larger deviations, while MAE provides a complementary measure of the average absolute differences between predicted and observed values. R2 evaluates the explanatory strength of the model by measuring the proportion of variance captured in unseen data. Prediction bias was also computed to identify any systematic tendency to overestimate or underestimate Chl-a concentrations. All metrics were computed on the independent testing dataset to ensure that the evaluation reflects true model performance on unseen data. These metrics are applied to the predicted values obtained from the model formulations described in Equations (9) and (10).
The R
2, defined in Equation (9), measures how well the predicted values match the observed Chl-a concentrations [
61]. It represents the proportion of variance explained by the model, where a higher value indicates better predictive performance.
In this equation, represents the observed Chl-a value, represents the predicted value obtained from the model, is the mean of observed values, and is the total number of samples. These variables are consistently used across all evaluation metrics.
RMSE, shown in Equation (10), measures the overall magnitude of prediction errors and gives greater weight to larger errors, making it useful for identifying significant deviations [
61].
MAE, defined in Equation (11), calculates the average absolute difference between predicted and observed values, providing a simple and interpretable measure of typical prediction error [
50].
Prediction bias, given in Equation (12), evaluates whether the model consistently overestimates or underestimates Chl-a values [
62]. A value close to zero indicates that the model does not exhibit systematic deviation.
Collectively, these metrics provide a comprehensive evaluation framework by capturing explained variance (R2), overall error magnitude (RMSE), average deviation (MAE), and systematic bias (Bias), ensuring a reliable and consistent comparison of all models used in this study.
5. Results
5.1. Comparative Model Performance
The predictive performance of the evaluated models was assessed using a temporally independent test dataset from 2025, ensuring an unbiased evaluation of model generalization. While the IDW baseline provides simple distance-based interpolation, it lacks the capacity to capture complex spatiotemporal variability under high missing-data conditions. Similarly, the EOF-based DINEOF framework effectively reconstructs dominant large-scale spatial–temporal patterns but tends to smooth localized variability and rapidly changing bloom structures. The results show a clear relationship between model structure and predictive performance. The distance-based KNN approach provides stable local interpolation but shows limited capability in capturing broader nonlinear spatial–seasonal interactions. Random Forest improves performance by modeling nonlinear relationships through ensemble averaging, enhancing robustness under heterogeneous chlorophyll conditions. Extra Trees further increases ensemble diversity by introducing randomized split thresholds, which improves generalization stability by reducing tree correlation. In contrast, the boosting-based XGBoost framework enables sequential learning of residuals, allowing more effective representation of complex environmental patterns. The LSTM model further extends this capability by explicitly incorporating temporal dependencies, enabling improved representation of seasonal evolution and short-term chlorophyll variability. The convolutional autoencoder-based Temporal DINCAE framework additionally captures nonlinear spatial–temporal reconstruction patterns through multi-channel feature integration, enabling effective reconstruction of missing chlorophyll-a observations under extensive cloud-cover conditions.
To quantify these differences, model performance was evaluated using RMSE, MAE, and R2 on the temporally independent 2025 test dataset. XGBoost achieved the highest predictive accuracy, with R2 = 0.86, RMSE = 9.61 mg m−3, and MAE = 5.30 mg m−3, indicating strong explanatory power and low prediction error. Extra Trees and Random Forest showed comparable performance, with Extra Trees yielding R2 = 0.85, RMSE = 10.06 mg m−3, and MAE = 5.55 mg m−3, and Random Forest producing R2 = 0.85, RMSE = 10.07 mg m−3, and MAE = 5.55 mg m−3. KNN demonstrated relatively lower performance, with R2 = 0.80, RMSE = 11.52 mg m−3, and MAE = 7.11 mg m−3, reflecting higher prediction errors and reduced ability to capture complex variability. The LSTM model achieved the lowest prediction errors, with RMSE = 5.87 mg m−3, MAE = 2.16 mg m−3, and R2 = 0.83, demonstrating a strong ability to capture temporal dynamics while maintaining competitive explanatory performance. The Temporal DINCAE model achieved competitive reconstruction performance, with R2 = 0.84, RMSE = 11.15 mg m−3, and MAE = 6.38 mg m−3, demonstrating the capability of convolutional autoencoder-based reconstruction for capturing nonlinear spatial–temporal variability. The EOF-based DINEOF framework produced moderate reconstruction performance, with R2 = 0.64, RMSE = 13.76 mg m−3, and MAE = 7.62 mg m−3, indicating its effectiveness in reconstructing dominant large-scale spatial–temporal structures despite high missing-data conditions. In contrast, the IDW baseline exhibited the weakest performance (R2 = 0.61, RMSE = 13.91 mg m−3, MAE = 6.89 mg m−3), underscoring the limitations of purely distance-based interpolation in representing complex spatiotemporal variability.
These quantitative results are summarized in
Figure 3, which compares model performance across RMSE, MAE, and R
2. While all models capture the primary variability in Chl-a concentrations, clear differences in predictive accuracy are evident. The IDW baseline exhibits the highest error values, reflecting its limited ability to represent complex spatiotemporal variability. The EOF-based DINEOF framework improves reconstruction capability relative to IDW by capturing dominant spatial–temporal patterns, although its performance remains lower than the data-driven reconstruction models. Ensemble-based approaches provide improved accuracy and stability, with XGBoost consistently achieving the best overall performance. Extra Trees and Random Forest also perform strongly, yielding comparable results. In contrast, the higher error values observed for KNN suggest sensitivity to local data distribution and reduced generalization capability. The LSTM model achieves the lowest RMSE and MAE values, underscoring its effectiveness in capturing temporal dynamics; however, its R
2 remains slightly lower than that of XGBoost, indicating a modest reduction in variance explanation. The Temporal DINCAE framework achieved competitive performance, with reconstruction accuracy higher than KNN, DINEOF, and IDW, while remaining slightly lower than the ensemble-based models.
To further understand model behavior in detail, the distribution of residual prediction errors was examined. This analysis provides a clearer view of prediction consistency and systematic deviation across varying Chl-a conditions. As shown in
Figure 4, the residuals for all models are centered close to zero, indicating that there is no strong systematic overestimation or underestimation in the predictions. However, a slight tendency toward underestimation is observed across all models, as indicated by the negative bias values. KNN shows a bias of −1.53, Random Forest −1.32, Extra Trees −1.32, and XGBoost −1.14, indicating that predictions are, on average, slightly lower than the observed concentrations. In contrast, the LSTM model exhibits a substantially lower bias of −0.26, suggesting improved calibration and reduced systematic error relative to the other approaches. Despite this, the bias remains small in magnitude, confirming that all models maintain stable and well-calibrated predictions.
Differences in residual spread provide further insight into model stability and consistency. XGBoost exhibits a narrower distribution of residuals, indicating more consistent performance across the dataset. Random Forest and Extra Trees show moderate dispersion, reflecting balanced behavior with reduced variance due to ensemble averaging. In contrast, KNN displays a wider spread of residuals, suggesting greater variability in prediction error and reduced stability, particularly under heterogeneous environmental conditions. The LSTM model exhibits the most compact residual distribution, indicating improved prediction consistency and reduced error variability, attributable to its ability to capture temporal dependencies.
5.2. Time Series and Spatial Distribution
The time series behavior of Chl-a concentrations was analyzed using monthly averages for the 2025 test period. The monthly comparison (
Figure 5) demonstrates that all models successfully reproduce the progression of Chl-a, including the rapid increase from winter baseline levels (~20 mg m
−3) to a peak during late summer (~40 mg m
−3), followed by a sharp decline toward the end of the year. Predicted monthly means closely track observed values across all models, indicating that the dominant bloom cycle relevant for HAB monitoring is preserved after gap reconstruction. Among the models, XGBoost and Extra Trees show the closest agreement with observed values across most months, with minimal deviations during both peak and transition periods. In contrast, KNN exhibits larger deviations, particularly during periods of rapid change, highlighting its limited capacity to capture broader temporal variability. Despite these differences, all models preserve the timing of key seasonal transitions, including the onset, peak, and decline phases of chlorophyll concentration. Overall, the results demonstrate that the gap-filling framework effectively preserves temporal dynamics while improving data continuity. The ability of the models to accurately reproduce seasonal variability indicates that the reconstructed datasets retain essential environmental signals necessary for reliable monitoring of chlorophyll dynamics and HAB-related processes in the Chesapeake Bay.
To evaluate spatial consistency, the reconstructed Chl-a distributions were examined for selected cloud-affected days using the XGBoost model. As shown in
Figure 6, the observed Chl-a (left) and XGBoost-based reconstructed Chl-a (right) are presented for three representative dates: 4 February 2025, 14 February 2025, and 21 February 2025, with corresponding missing data rates of 11.97%, 6.73%, and 39.62%, respectively. The reconstructed maps demonstrate that the XGBoost model effectively fills missing regions while maintaining realistic spatial patterns of chlorophyll concentration. The distribution of high- and low-concentration areas remains consistent with observed conditions, indicating that important spatial features are well retained after reconstruction. Notably, even for the higher missing rate case (39.62%), the model is able to reconstruct spatial patterns with reasonable accuracy, indicating robustness under data-sparse conditions. A slight underestimation is observed in areas of high chlorophyll concentration, particularly in the upper and mid-Bay regions shown in
Figure 6a,c, where peak values appear slightly reduced in the reconstructed maps compared to the observed data. This effect is more noticeable on 21 February 2025 (
Figure 6c), which also corresponds to the highest missing data rate, and is likely due to ensemble averaging and the variance-stabilizing transformation applied during training. These results support the reliability of ensemble-based methods for operational HAB monitoring in cloud-affected coastal environments.
5.3. Evaluation Under Controlled Missing Data Conditions
To evaluate whether model performance reflects realistic gap-filling conditions under cloud-induced data loss, a controlled masking experiment was conducted using the 2025 test dataset. A representative daily Chl-a scene (14 February 2025) was selected based on maximum spatial coverage of valid pixels. Artificial missingness was introduced by randomly masking originally valid pixels at levels of 50% to 90%, ensuring that the evaluation focuses on recoverable observations rather than pre-existing gaps. For each missingness level, spatial neighborhood features were recomputed after masking to prevent information leakage. The trained XGBoost model was then applied exclusively to the masked pixels, and performance metrics (RMSE, MAE, and R2) were computed only over these withheld locations.
Results indicate a systematic degradation in reconstruction performance with increasing missingness. R
2 decreases from approximately 0.705 at 50% missingness to 0.563 at 90%, while RMSE and MAE increase correspondingly. Spatial reconstructions (
Figure 7) demonstrate that large-scale chlorophyll-a patterns are preserved even under high missing rates; however, errors increase in regions with strong spatial gradients and localized variability. This experiment provides a rigorous, application-oriented evaluation of the proposed framework. By simulating realistic cloud-induced data gaps, the analysis extends beyond conventional validation and demonstrates the robustness of the model under operational conditions. Based on the performance trends (
Figure 8), the model maintains stable predictive skill up to approximately 70–80% missingness, beyond which reliability declines for fine-scale reconstruction, although large-scale spatial structure remains preserved.
5.4. Uncertainty Quantification
Uncertainty was quantified for both Extra Trees (ET) and XGBoost to evaluate the reliability of model predictions. For the ET model, prediction intervals were derived by leveraging variability among individual tree predictions rather than relying solely on the ensemble mean [
63]. For each test sample, predictions from all trees were extracted in log-transformed space, and the ensemble mean together with the 5th and 95th percentiles was used to construct a 90% prediction interval (PI90). These bounds were then back-transformed to the original Chl-a concentration scale using the inverse log(1 + x) transformation. The resulting interval width (PI
upper − PI
lower) serves as a pixel-level measure of predictive uncertainty. The ET model achieved a PI90 coverage of approximately 0.843, with a mean interval width of 11.64 mg m
−3 and a median width of 6.61 mg m
−3, indicating relatively narrow intervals with moderate coverage. In contrast, uncertainty for the XGBoost model was estimated using a residual-based approach, where the 90th percentile of absolute training residuals was used to construct symmetric prediction intervals. This approach enables uncertainty estimation for boosting-based models that do not inherently provide per-tree variability. The XGBoost model achieved a PI90 coverage of approximately 0.905, closely matching the nominal 90% level and indicating well-calibrated uncertainty. However, the larger mean interval width (27.26 mg m
−3) reflects a more conservative uncertainty estimate relative to ET. The comparison between the two models highlights an important trade-off between interval width and reliability. The ET model produces narrower intervals but underestimates uncertainty, as indicated by its lower coverage. In contrast, XGBoost provides wider but better-calibrated intervals, ensuring that a higher proportion of true observations fall within the predicted bounds. Monthly aggregation of uncertainty is illustrated in
Figure 9. The blue solid line represents the monthly mean observed Chl-a for the held-out testing subset, while the orange dashed line shows the corresponding monthly mean predictions. The shaded region denotes the monthly mean 90% prediction interval. Temporal widening of the band indicates periods of elevated uncertainty, typically associated with rapid seasonal transitions. When observed values fall outside the shaded region, the model underestimates uncertainty during those intervals.
Overall, these results indicate that XGBoost provides more reliable and well-calibrated uncertainty estimates, whereas ET produces tighter but less conservative bounds.