Explainable Machine Learning for Streamflow Forecasting: Application to the Bosna River Basin

Abstract

As one of the key water systems in Bosnia and Herzegovina, the Bosna River Basin plays a vital role in sustaining agricultural production, industrial development, and water supply for municipalities. Accurate streamflow forecasting is fundamental to optimising water resource planning. This study explores streamflow forecasting using long-term data (1961–2020) from five meteorological stations and one hydrological station distributed across various sections of the basin. For precise streamflow forecasting, the study employs several machine-learning models: Random Forest, LSTM, and XGBoost. Model performance is evaluated using widely used metrics, including mean absolute error, root mean square error, Nash–Sutcliffe efficiency (NSE), and Kling–Gupta efficiency (KGE). Among the tested models, Random Forest proved to be the most accurate for streamflow forecasting, confirming its effectiveness in capturing the complex dynamics of hydrological processes. During the testing phase, the Random Forest model achieved an NSE of 0.591 and a KGE of 0.591, demonstrating good generalisation and reliable predictions. The results demonstrate the strength of Random Forest in capturing nonlinear hydrological patterns and supporting reliable streamflow forecasting for national water management. Moreover, as a novel approach, explainable AI was applied using SHAP analysis to go beyond the regular predictions of the models, thereby providing a deeper understanding of the model’s performance described by the magnitude and direction of influence of each problem feature.

1. Introduction

Reliable streamflow predictions are fundamental to strengthening resilience and sustainability in water resource management [1]. Management practices commonly involve assessments that incorporate hydrological extremes analysis, reservoir operation, public water use, hydraulic infrastructure planning at the national scale, and agricultural systems [2,3]. At the same time, pronounced shifts in the hydrological cycle, driven by anthropogenic climate change forcing, continue to raise concerns, as projected changes in hydrological regimes suggest an overall decline in annual precipitation, coupled with an elevated likelihood of droughts and amplification of rapid extreme events [4]. Accurate streamflow prediction is a demanding task owing to the complex, nonlinear behavior of hydrological systems, shaped by precipitation patterns, temperature variations, land use, human interventions, and a lack of empirical data [5,6].

Significant efforts have been devoted to developing and implementing approaches to enhance the reliability and forecast precision of streamflow predictions, which can generally be grouped into two categories. The first category comprises models intended to simulate the rainfall–runoff response grounded in physical principles. Although they rely on simplified premises, they require substantial data inputs [7]. The second category comprises data-driven models, including statistical and ML methodologies, which rely on historical observations rather than physical process information, enabling empirical use and straightforward implementation [7,8]. Over the past decade, machine learning (ML) models have established a significant role in hydrological modelling, especially in streamflow forecasting, where they tend to outperform traditional physically based approaches in forecast precision [9].

The use of ML techniques for streamflow forecasting has become increasingly common among scientists over the past decade, primarily because of their capability to capture nonlinear dynamics and temporal variability in hydrological processes and interacting factors without requiring explicit representation of underlying physical processes [9,10]. A variety of models have been utilised in streamflow prediction research, with Artificial Neural Networks (ANNs) [11,12,13], Long short-term memory (LSTM) [14,15,16], Deep Neural Networks (DNNs) [17,18], Support Vector Regression (SVR) [19,20], Random Forest (RF) [21,22], Light Gradient Boosting Machine (LGBM) [23,24], and eXtreme Gradient Boosting (XGBoost) [25,26,27] being among the most commonly employed techniques. On the other hand, several studies have successfully applied alternative HBV- and SWAT-type models based on physical laws of hydrology, simulating water movement step-by-step using explicit equations for monthly streamflow prediction. These models have been shown to be particularly useful for simulating streamflow in various basins [24,28,29,30,31,32].

This research examines the use of RF, XGBoost, and LSTM networks for streamflow forecasting at the Doboj hydrological stations in the Bosna River Basin (BRB) in Bosnia and Herzegovina (BH). These techniques have proven to be remarkably scalable, robust, and highly efficient in solving various problems [33], not only in hydrological forecasting, as already mentioned, but also in complementary fields such as finance [34], remote sensing [35], sentiment analysis [36], and even solar energy forecasting [37], making them an attractive choice for predictive modelling.

To the best of our knowledge, this study represents the first application of machine learning (ML) methods for streamflow forecasting in Bosnia and Herzegovina. The primary objective was to address a critical gap in the existing literature by demonstrating the potential of ML techniques in the water sciences through a case study of the Bosna River Basin (BRB), thereby moving beyond the predominantly descriptive analyses of streamflow previously conducted in the country. An additional objective was to provide a foundation for understanding basin-specific hydrological processes and water management challenges within the BRB, with particular emphasis on accurate and interpretable forecasting. To this end, explainable artificial intelligence (XAI) tools were employed [38] to relate model predictions to fundamental hydrological drivers and their temporal lags. This approach offers new insights into the relative influence of individual features, facilitating high-quality predictions while enhancing model transparency and hydrological interpretability.

A comprehensive evaluation of multiple ML frameworks across the BRB is expected to inform future research directions and facilitate the effective integration of machine learning into water resources management practices worldwide.

2. Materials and Methods

2.1. Study Area

The Bosna River Basin (BRB) is the largest and one of the principal basins of Bosnia and Herzegovina (BH), with its entire area situated within the national territory and extending across the central, eastern and northern parts of the country. The BRB encompasses primary urban and industrial centers, such as Sarajevo, Zenica, Tuzla, and Doboj (Figure 1), which together host more than 50% of the national population, thereby establishing the basin as an economic hotspot. The basin occupies an area of 10,457 km2, covering ~20% of the overall area of BH, whereas the length of the river and mean elevation of the basin are 275 km and 640 m, respectively [39]. The river has its source at the foot of Mt Igman at 500 m a.s.l., emerging as a substantial karst spring. The upper section of the BRB features a well-developed drainage network, comprising several major and numerous smaller torrential streams. It extends from the river source to the city of Zenica. Extending from Zenica to Doboj, the middle section of the BRB is supplied by several major tributaries, including the Usora, Krivaja, and Spreča Rivers. The lower northern part of the Bosna River Basin extends from the town of Doboj to the confluence of the Bosna with the Sava River and lacks major tributaries. The Bosna River, along with its main affluents, is characterised by the Posavina subtype of the pluvial–nival water regime and exhibits peak streamflow in April and March and minimum streamflow in August and September [38,40]. Climatic patterns in the Bosna River basin differ from mountainous (Dfc) conditions in the south to continental (Dfb) and moderately continental (Cfb) types in the central and northern segments of the basin [41]. Mean annual air temperatures range from 9.5 to 12 °C in the northern, lower section of the basin [41]. In the middle section, mean annual air temperatures range between 7 and 10 °C, whereas in the highest, upper section they decrease to as low as 1.2 °C [41]. Mean annual precipitation generally decreases towards the north. The highest precipitation amounts are recorded in the upper section, where some areas receive more than 2000 mm annually, while lower valley areas receive around 900 mm. In the middle section of the basin, precipitation ranges from 900 to 1200 mm, whereas the northern areas experience the lowest values, between 800 and 900 mm [41].

2.2. Dataset Description

The dataset consists of monthly aggregated observations spanning the period from 1961 to 2020. Each record corresponds to a single month. Meteorological predictors include the mean monthly precipitation (P) and mean monthly air temperature (T), observed at five meteorological stations located within the Bosna River basin: Bjelašnica, Sarajevo, Zenica, Tuzla, and Doboj. The target variable for prediction is the mean monthly streamflow (Q), measured at the most downstream hydrological station on the Bosna River (Doboj station). All variables were aggregated to a monthly resolution to ensure temporal consistency and long-term coverage. In addition, we found a few missing data points for the Zenica station over a 6-month period in 2017. These were properly filled in by applying the Climatological Monthly Mean Filling across all historical data for that station. All hydroclimatic data were provided by the Republic Hydrometeorological Institute of the Republic of Srpska.

2.3. Compared Models

In this study, three data-driven modelling approaches were evaluated for predicting mean monthly streamflow in the Bosna River: Random Forest (RF), Extreme Gradient Boosting (XGBoost), and Long Short-Term Memory (LSTM) neural networks. Random Forest is an ensemble learning method based on bootstrap aggregation of decision trees [42]. Owing to its robustness to noise, ability to capture nonlinear relationships, and reduced sensitivity to multicollinearity, RF has been widely adopted in hydrological modelling and is commonly used as a strong baseline [43,44]. XGBoost is a gradient boosting framework that sequentially constructs decision trees, with each new tree aiming to correct the errors of its predecessors [45,46]. Compared with RF, XGBoost provides enhanced control over model complexity through regularisation and learning-rate parameters and has demonstrated strong performance in hydro-climatic prediction tasks [33,47]. LSTM is a class of recurrent neural networks designed to capture long-term temporal dependencies in sequential data [15,48]. It is included in this study to assess whether deep learning models can extract additional predictive skill from long-term hydro-climatic time series compared with tree-based ensemble methods, as demonstrated in recent rainfall–runoff and streamflow forecasting studies [49,50].

Together, these models represent complementary modelling paradigms: bagging-based ensembles (RF), boosting-based ensembles (XGBoost), and sequence-based deep learning (LSTM).

All experiments were conducted using the Python programming language v3.11. The implementation relies on widely used open-source libraries to ensure reproducibility and transparency. The XGBoost model was implemented using the XGBoost library (XGBRegressor). Random Forest models were implemented using the scikit-learn library (RandomForestRegressor). The LSTM model was developed using the TensorFlow framework with the Keras high-level API. Hyperparameter tuning and model selection were performed using GridSearchCV from scikit-learn combined with TimeSeriesSplit to ensure temporally consistent cross-validation.

2.4. Feature Engineering and Lagged Variables

To account for hydrological memory and delayed catchment responses, lagged predictor variables were incorporated into the modelling framework, generating them independently for each station and time step (t). Let (Pt), (Tt), and (Qt) denote precipitation, air temperature, and streamflow at the time step, respectively. The predictor set comprised lagged precipitation terms at one-, two-, and three-month intervals (Pt-1, Pt-2, Pt-3), lagged temperature terms at one- and two-month intervals (Tt-1, Tt-2), and an autoregressive streamflow term representing previous-month discharge (Qt-1). The autoregressive streamflow term captures short-term persistence in river discharge and is widely used in hydrological forecasting to enhance predictive skill. Missing values introduced by lagged feature construction were removed using a complete-case approach, excluding them from the training dataset prior to model calibration.

Prior to model training, all datasets were merged into a single time-indexed table, where each row corresponds to a monthly observation and columns represent hydro-meteorological variables from all stations. Missing values in temperature data for the Zenica station (July–December 2017) were filled using climatological monthly means computed across all available years. No additional missing values were present after merging.

In the case of LSTM, note that input data are structured as three-dimensional tensors of shape (N, 1, F), where N is the number of samples and F is the number of input features. A single timestep was used, meaning that temporal dependencies were not modeled through sequential input windows but instead explicitly encoded via lagged features: lagged precipitation (Pt-1, Pt-2, Pt-3), lagged temperature (Tt-1, Tt-2), and lagged discharge (Qt-1) were included for each station. This approach ensures consistency with the feature representation used in tree-based models while still allowing nonlinear modeling through the LSTM architecture. Note that more lagged precipitation and temperature features were included in the experimentation; however, they did not provide any significant advantage in the overall results over those presented in the paper.

No explicit normalization or scaling of input variables was applied either in the training or the test phase. While normalization is often recommended for neural networks such as LSTM, the model was trained directly on raw values to maintain consistency with tree-based models and preserve the physical interpretability of hydrological variables.

2.5. Performance Metrics

Model performance was evaluated using hydrologically relevant efficiency metrics. Besides Minimum Absolute Error (MAE) and the Root Mean Square Error (RMSE), Nash–Sutcliffe and Kling–Gupta measures were determined. The Nash–Sutcliffe Efficiency (NSE) [15,51,52] measures the predictive skill of a model relative to the mean of observed streamflow values and is defined as:

$$NSE=1-{\displaystyle \frac{{\displaystyle {\sum }_{i=1}^n}{(Q_i-{\widehat{Q}}_i)}^2}{{\displaystyle {\sum }_{i=1}^n}{(Q_i-{\overline{Q}}_i)}^2}}$$

(1)

where $Q_i$ and ${\widehat{Q}}_i$ denote observed and predicted streamflow values, respectively, and ${\overline{Q}}_i$ is the mean of observed streamflows.

The Kling–Gupta Efficiency (KGE) [15,51,52] decomposes model performance into correlation, bias, and variability components:

$$\mathrm{K}\mathrm{G}\mathrm{E}=1-\sqrt{\{{\left(\mathrm{r}-1\right)}^2+{\left(\mathrm{\beta }-1\right)}^2+{\left(\mathrm{\gamma }-1\right)}^2\}}$$

(2)

where $\mathrm{r}$ is the Pearson correlation coefficient, $\mathrm{\beta }$ is the bias ratio, and $\mathrm{\gamma }$ is the variability ratio between predicted and observed streamflows.

NSE was used as the primary optimization criterion during model tuning, while KGE was employed for complementary evaluation.

For the sake of completeness, the formula for MAE is defined as:

$$MAE={\displaystyle \frac{1}{n}}{\sum }_{i=1}^n\left|Q_i-{\widehat{Q}}_i\right|$$

(3)

where n is the total number of observations (i.e., data points), and ∣⋅∣ is the absolute-value operator. Lower MAE values indicate better model performance.

Root Mean Square Error (RMSE): The RMSE also quantifies prediction error but places greater emphasis on larger errors because the differences are squared before averaging. It is defined as

$$RMSE=\sqrt{{\displaystyle \frac{1}{n}}{\sum }_{i=1}^n{\left(Q_i-{\widehat{Q}}_i\right)}^2}$$

(4)

where all symbols retain the same meanings as in the MAE equation, and the square-root operation returns the metric to the original units of the data. As with MAE, smaller RMSE values reflect superior predictive accuracy [53,54,55].

2.6. Training, Validation, and Temporal Splitting

To respect the temporal structure of the data and avoid information leakage, a time-based splitting strategy was adopted. The dataset was divided into training (1961–2010) and testing (2011–2020) periods.

During the training period, rolling-origin cross-validation was performed using a time-series split with 5 folds. In this setup, validation sets always occur chronologically after the corresponding training subsets.

2.7. Hyperparameter Tuning of XGBoost, RF, and LSTM

The hyperparameters of all three models were optimized using a grid search strategy coupled with time-series cross-validation applied to the training dataset, ensuring that temporal dependencies were preserved during model calibration. For the XGBoost model, the tuning process explored combinations of the number of trees, maximum tree depth, learning rate, subsampling ratio, and column subsampling ratio. For each predefined hyperparameter configuration, model performance was evaluated using the NSE across five cross-validation folds, and the configuration yielding the highest mean NSE was selected as optimal. An analogous procedure was employed for the Random Forest and Long Short-Term Memory models, with hyperparameter spaces tailored to their respective architectures. The optimal hyperparameters identified through this procedure were as follows: for RF, a maximum tree depth of 30, square-root feature sampling, minimum samples per leaf of 1, minimum samples for splitting of 2, and 2000 trees; for XGBoost, a column subsampling ratio of 0.8, learning rate of 0.05, maximum tree depth of 4, 600 estimators, and a subsampling ratio of 0.7; and for LSTM, 64 hidden units, a learning rate of 0.001, a batch size of 16, and 200 training epochs.

2.8. One-Step-Ahead Forecasting and Bias Correction

Despite achieving reasonable predictive performance, all evaluated models exhibited systematic bias in the magnitude of predicted discharge values, particularly in the form of consistent overestimation or underestimation. This behavior is commonly observed in data-driven hydrological models, which may capture temporal variability well but fail to reproduce absolute magnitudes accurately. To address this issue, a linear scaling bias correction was applied as a post-processing step.

Using the optimized model, one-step-ahead forecasts were generated for the testing period. Systematic bias was reduced by applying a linear scaling correction. A linear regression model was fitted between predicted and observed streamflow values on the training set:

$${\mathrm{Q}}_{\left\{\mathrm{o}\mathrm{b}\mathrm{s}\right\}}=\mathrm{a}\cdot {\mathrm{Q}}_{\left\{\mathrm{p}\mathrm{r}\mathrm{e}\mathrm{d}\right\}}+\mathrm{b}$$

(5)

where $a$ and $b$ are regression coefficients estimated from the training data.

The derived correction was subsequently applied to test-period predictions, preserving temporal dynamics while improving distributional consistency. The complete modelling workflow is presented in Figure 2.

3. Results

The performance metrics in

Table 1. Performance comparison of the evaluated models on training and testing data. RMSE and MAE are expressed in streamflow units, while NSE and KGE are dimensionless.

Model	Dataset	RMSE	MAE	NSE	KGE
XGBoost	Train	2.920	2.208	0.999	0.993
	Test	64.680	48.029	0.589	0.589
RF	Train	20.769	14.888	0.964	0.884
	Test	64.500	47.169	0.591	0.591
LSTM	Train	81.581	57.432	0.443	0.382
	Test	75.086	54.763	0.445	0.489

reveal notable differences in both fitting capacity and generalisation ability across the evaluated models. XGBoost achieves near-perfect accuracy on the training dataset (NSE = 0.999, KGE = 0.993), indicating a strong ability to capture nonlinear relationships between hydro-meteorological predictors and streamflows. However, this excellent training performance does not fully carry over to the testing period, where NSE and KGE drop to approximately 0.59 (Table 1). The substantial gap between training and testing accuracy suggests that XGBoost may partially overfit the historical training data, despite the use of time-series cross-validation and regularisation parameters.

Random Forest shows slightly lower training accuracy (NSE = 0.964, KGE = 0.884) than XGBoost but achieves nearly identical performance on the testing dataset (NSE = 0.591, KGE = 0.591) (Table 1). This suggests a more balanced bias–variance trade-off and indicates that Random Forest generalises marginally better to unseen data. The similarity in test performance between Random Forest and XGBoost implies that, for the given feature set and temporal resolution, increased model complexity does not provide substantial gains in predictive skill.

LSTM performs worst among the evaluated approaches. Training accuracy is moderate (NSE = 0.443), and testing performance remains limited (NSE = 0.445), suggesting the model struggles to learn streamflow dynamics (Table 1). Several factors may contribute to this outcome. First, the dataset consists of monthly-aggregated observations, which may not provide sufficient temporal resolution for LSTM models to exploit long-range dependencies. Second, the relatively small number of training samples compared to the high parameterisation of LSTM networks may hinder stable learning. Finally, the use of a single-step input structure with engineered lag features may reduce the advantage of sequence-based deep learning compared to tree-based methods. In particular, its relatively limited performance could be attributed to the chosen input representation, where temporal dependencies are captured through lagged features rather than sequential input windows. As a result, the model does not fully exploit the temporal memory capabilities typically associated with recurrent neural networks. Future work will investigate multi-timestep input sequences to better capture temporal dependencies and fully leverage the capabilities of recurrent neural networks.

Overall, the results show that tree-based ensemble models outperform deep learning approaches for monthly streamflow prediction in the Bosna River, given the available data and feature configuration. While XGBoost achieves the highest fitting accuracy, Random Forest offers comparable test performance with greater robustness. The observed performance gap between training and testing across all models highlights the challenge of non-stationarity in long-term hydro-climatic time series, emphasising the importance of careful validation strategies and model selection. These findings suggest that, for long-term monthly streamflow prediction in data-limited settings, well-regularised ensemble methods combined with physically motivated lag features can provide reliable predictive performance without the complexity of deep learning architectures (see Figure 3).

3.1. Comparison to a Rainfall–Runoff Model

The simplified conceptual rainfall–runoff model (HBV-lite) was implemented as a baseline physically interpretable benchmark against the machine learning approaches (RF, XGBoost, and LSTM). The HBV-lite model is a simplified conceptual rainfall–runoff representation inspired by the HBV framework.

Unlike fully distributed hydrological models, the proposed version uses only precipitation and temperature as driving variables, aggregated from multiple stations into basin-average time series. The model consists of a single soil moisture storage reservoir, where incoming precipitation increases soil water content while temperature-based evapotranspiration reduces it. A nonlinear transformation is then applied to represent runoff generation as a function of relative soil moisture, controlled by a small set of interpretable parameters governing storage capacity, nonlinearity, and runoff responsiveness. The resulting runoff is taken as the simulated discharge. This simplified structure retains the core hydrological concept of storage–release dynamics while avoiding the need for detailed spatial information such as soil type, land use, or topography.

The obtained results indicate a significantly lower predictive performance of the HBV-lite model compared to the data-driven approaches, with a training NSE of −0.539 and a testing NSE of 0.154.

These negative training NSE values suggest that the model performs worse than a simple mean-flow predictor, indicating that the simplified conceptual structure is insufficient to capture the complexity of the rainfall–runoff relationship in the Bosna River basin using the available input data.

3.2. Feature Importance and Explainable Artificial Intelligence

While predictive accuracy is essential, understanding the contribution of individual hydro-meteorological predictors is equally important for decision support and scientific insight. To this end, an Explainable Artificial Intelligence (XAI) analysis was conducted to assess the relative importance of input features and their influence on streamflow prediction. Given the heterogeneous nature of the evaluated models, different yet conceptually consistent feature importance approaches were employed. For the tree-based models (XGBoost and RF), feature importance was derived from model-based importance measures, while for the LSTM model a permutation-based approach was adopted to ensure comparability across model classes.

3.3. Permutation-Based Feature Importance

Permutation importance quantifies the contribution of a feature by measuring the decrease in model performance when the feature’s values are randomly permuted. Let f denote a trained model and $M(·)$ a performance metric, here the NSE. The importance of feature x_j is computed as:

$$I_j= M\left(f(X)\right)-M\left(f(X_{\left\{\pi \left(j\right)\right\}})\right)$$

(6)

where X$\pi \left(j\right)$ denotes the input matrix with feature x_j permuted across samples.

A larger decrease in NSE indicates a more influential feature. This model-agnostic approach enables consistent interpretation across both tree-based and neural network models and directly reflects the impact of each feature on predictive skill. For more about explainable AI and its applications, see [56].

3.4. SHAP-Based Explainability

In addition to permutation-based feature importance, SHAP (Shapley Additive exPlanations) analysis was employed to provide a more detailed and theoretically grounded interpretation of model predictions. While permutation importance quantifies the impact of features on model performance, it does not capture the direction or context-dependent influence of individual predictors. SHAP is based on cooperative game theory and decomposes the prediction of a model into contributions from individual features. For a given instance x, the model output f(x) can be expressed as:

$$f\left(x\right)={\phi }^0+{\sum }_{j=1}^p{\phi }_j$$

(7)

where φ₀ represents the average model output and φ_j denotes the contribution of feature j = 1, …, p to the prediction. Each contribution is computed as a weighted average over all possible subsets of features:

$${phi}_j=\sum _{S\subseteq F\setminus j}{\displaystyle \frac{\left(\left|\left(S\right)\right|!\ast \left(p-\left|\left(S\right)\right|-1\right)!\right)}{p!}}\left[f\left(S\cup j\right)-f\left(S\right)\right]$$

(8)

This formulation ensures desirable properties such as consistency and additivity.

In this study, SHAP analysis was applied to the XGBoost and Random Forest models using TreeSHAP, which allows efficient and exact computation of Shapley values for tree-based models. The analysis provides both:

• Global interpretability, through ranking features based on mean absolute SHAP values.
• Local interpretability, by explaining individual predictions and identifying how each feature increases or decreases predicted discharge.

Unlike permutation importance, which only measures performance degradation, SHAP values provide both the magnitude and direction of feature influence, as well as insight into nonlinear interactions between predictors.

3.5. Dominant Predictors of Streamflows

The analysis focused on the five most influential features for each model, ranked by their contribution to NSE reduction. Results indicate that lagged precipitation variables consistently dominate model predictions, particularly precipitation from the preceding one to three months. This finding is hydrologically meaningful, reflecting basin-scale storage effects and delayed runoff generation. The autoregressive streamflow term (Qt-1) also emerges as a key predictor, highlighting the strong persistence of monthly river flow and confirming the importance of short-term memory in streamflow dynamics. Temperature-related features show a similarly important influence, especially T_Bjelašnica, which likely captures seasonal snowmelt and evapotranspiration effects, particularly for upstream stations such as Bjelašnica. Notably, feature importance rankings remain broadly consistent across XGBoost and Random Forest, especially for the two most influential features—T_Bjelašnica and the autoregressive Q_lag1—suggesting stable hydro-climatic controls on streamflow. In contrast, the LSTM model displays weaker and less structured importance patterns, consistent with its overall lower predictive performance. The agreement between data-driven feature importance and established hydrological understanding provides confidence in the physical plausibility of the models. The dominance of lagged precipitation and streamflow terms in both reasonably efficient models aligns with conceptual rainfall–runoff processes, while the secondary role of temperature in Bjelašnica reflects its indirect influence at monthly time scales. From an operational perspective, these results demonstrate that explainable machine learning techniques can support transparent and trustworthy streamflow forecasting. By identifying the most influential predictors, the proposed framework enables improved model diagnostics and enhanced interpretability for water resource management and decision-making. Overall, the XAI analysis confirms that the tree-based ensemble models not only achieve higher predictive skill but also produce feature importance patterns consistent with hydrological theory, reinforcing their suitability for long-term monthly streamflow prediction in the BRB. Figure 4 illustrates the relative importance and directional influence of the dominant predictors for each model, highlighting the consistent dominance of lagged precipitation and autoregressive streamflow terms across tree-based ensemble models.

4. Discussion

The comparative assessment of XGBoost, Random Forest, and LSTM shows that tree-based ensemble methods provide the most accurate and robust one-step-ahead monthly streamflow forecasts for the Bosna River at Doboj, given the available data and model configuration. Both XGBoost and Random Forest achieve similar generalisation skill during the testing period (NSE ≈ 0.59; KGE ≈ 0.59; Table 1), while the LSTM clearly underperforms (NSE ≈ 0.45). In hydrological terms, these NSE values indicate moderate predictive capability at the monthly scale and are comparable to those reported in other data-driven streamflow forecasting studies using limited predictor sets and single-site applications [50,57]. The close agreement between observed and predicted flows for the two ensemble models in Figure 3a–b, particularly in reproducing intra-annual variability and the seasonal regime, further confirms that the dominant hydro-climatic signal is well captured, although residual errors remain evident during high-flow and low-flow extremes.

The near-perfect fit of XGBoost on the training data (NSE = 0.999; KGE = 0.993), compared with its markedly lower performance during the testing period, reveals a pronounced tendency towards overfitting, even with regularisation and time-series cross-validation. In contrast, Random Forest shows lower training skill (NSE = 0.964; KGE = 0.884) but almost identical test performance to XGBoost, suggesting a more favourable bias–variance trade-off. This pattern aligns with broader experience in hydro-climatic prediction, where highly flexible boosting algorithms can over-specialise in historical idiosyncrasies when trained on relatively short or non-stationary records [24,31,32,58,59]. The long modelling horizon increases the likelihood that climatic and anthropogenic changes have altered runoff generation processes, thereby limiting the transferability of complex models tuned on earlier decades to more recent conditions [60]. The similar out-of-sample performance of XGBoost and Random Forest indicates that, for this monthly forecasting problem and given the chosen feature set, increasing model complexity beyond bagging-based ensembles yields negligible gains in predictive skill.

In contrast, the LSTM model struggles to learn streamflow dynamics effectively (training NSE = 0.443; testing NSE = 0.445), and its predictions show greater dispersion and weaker reproduction of peaks and troughs in Figure 3c. This appears to contradict recent studies in which LSTM networks substantially outperform traditional models in daily rainfall–runoff and streamflow forecasting when large, information-rich datasets are available [10,49,61]. Several factors likely contribute to this discrepancy. First, using monthly aggregated data strongly smooths short-term dynamics and reduces the need to model long-range temporal dependencies, which are the main strength of recurrent architectures [62]. Second, the number of effective training samples (~600 months) is small relative to the parameterisation of even a moderately sized LSTM, increasing the risk of under-training or convergence to sub-optimal solutions [50]. Third, the input representation already encodes temporal information via lagged predictors and an autoregressive Q term, which partly diminishes the comparative advantage of sequence-based learning. Together, these constraints imply that, in data-limited, monthly-resolution settings, tree-based ensembles remain a more reliable choice than deep learning, a conclusion aligned with recent critical assessments of ML model selection in hydrology [57,63,64].

The explainable AI analysis provides an important consistency check between data-driven learning and hydrological process understanding. Permutation-based feature importance (Figure 5) shows that lagged precipitation is the dominant predictor class across the better-performing models, particularly rainfall in the preceding one to three months [65,66]. This is hydrologically plausible for a medium-sized basin with significant storage in soils, shallow groundwater, and the river network, where monthly flows reflect the integration of recent precipitation rather than only contemporaneous inputs [10,62]. The strong role of the autoregressive streamflow term (Qlag1) corroborates the pronounced persistence of monthly discharge, representing baseflow and slow-release components of catchment storage. The importance of temperature, especially at the high-elevation Bjelašnica station, indicates that snow accumulation, melt processes and temperature-dependent evapotranspiration modulate the seasonal hydrograph, consistent with established conceptual models and with ML-based analyses in other snow-affected basins [10,50]. The similarity in feature rankings between XGBoost and RF suggests that the inferred hydro-climatic controls are robust across ensemble algorithms.

In contrast, the LSTM shows weaker and less structured importance patterns, reflecting its poorer predictive performance and suggesting that it did not converge on a clear hydrological representation of the system. This finding highlights the value of XAI tools not only for post hoc interpretation but also for model diagnostics and selection: models that do not produce physically interpretable feature importance patterns, even when achieving modest skill, should be treated with caution in operational contexts [63]. In this study, the alignment between hydrological expectations and XAI-based rankings for the ensemble models supports their use as transparent decision-support tools.

From an operational perspective, an NSE of around 0.6 at monthly resolution is sufficient for various water management applications, including seasonal allocation planning, reservoir rule-curve support, and drought early warning, particularly in data-scarce contexts. For watershed-scale models, monthly streamflow performance is often judged satisfactory when NSE ≥ 0.50 [67,68,69,70]. However, inspection of Figure 3 shows that both XGBoost and Random Forest tend to underestimate extremely high flows and occasionally misrepresent sustained low-flow periods, a behaviour frequently documented for data-driven models trained on error metrics dominated by moderate flows [61]. Improving the representation of hydrological extremes in the Bosna River Basin may require either multi-objective training focused on high- and low-flow performance, incorporation of additional predictors such as snow indices, soil moisture, or large-scale climate indices, or hybrid approaches that blend data-driven models with process-based constraints [50,63].

Several structural limitations of this study warrant emphasis. The analysis is limited to a single downstream gauging station, which prevents explicit consideration of spatial heterogeneity within the basin. The predictor set includes only precipitation and air temperature from five stations, along with antecedent discharge, without remote sensing or land-use information that could better capture the non-stationary impacts of human activities. The use of monthly aggregation, although motivated by data availability and long-term coverage, obscures sub-monthly flood processes and may limit applicability to short-lead flood forecasting. Additionally, the results indicate that the models are primarily suited for capturing general discharge dynamics and seasonal variability, rather than precise prediction of extreme events.

Addressing these limitations with higher-temporal-resolution data, multi-site modelling, expanded predictor sets, and experimentation with hybrid or physics-informed machine learning architectures is a promising direction for future work and will be essential for fully leveraging machine learning to support climate-resilient water management in the BRB and similar regions. This includes direct comparisons with more powerful conceptual rainfall–runoff models to further assess the strengths and limitations of data-driven methods.

5. Conclusions

This study presents the first application of machine learning techniques for streamflow forecasting in Bosnia and Herzegovina, specifically evaluating Random Forest (RF), XGBoost, and Long Short-Term Memory (LSTM) models for mean monthly predictions at the Doboj station in the Bosna River Basin (BRB). The findings highlight the effectiveness of tree-based ensemble methods over deep learning approaches in data-limited, monthly-resolution settings, with XGBoost and RF achieving NSE ≈ 0.59 on the test data, outperforming LSTM’s NSE of 0.445. This performance gap demonstrates the robustness of tree-based models to non-stationarity and multicollinearity, whereas LSTMs struggle with aggregated sequences that lack fine-grained temporal details. Bias correction further improved predictions, aligning them with observed hydrographs and reducing systematic underestimation of peaks, as shown in Figure 3. XAI permutation analysis (Figure 5) identified lagged precipitation and autoregressive streamflow as the main drivers, with upstream temperature influencing seasonal dynamics, providing interpretable insights into the BRB’s pluvial–nival regime and karstic hydrology.

These outcomes address a critical gap in the literature by moving from descriptive streamflow studies in the region to advanced and transparent forecasting. The models’ ability to generalise, despite training-testing discrepancies, informs resilient water management amid climate-driven changes such as declining precipitation and increased extremes. By identifying key hydrological controls, this framework supports targeted interventions, such as reservoir optimisation in urban–industrial centres (e.g., Sarajevo, Zenica). Broader implications extend to global water resources, demonstrating the potential of machine learning in ungauged or sparsely monitored basins, where traditional physical models require extensive data. Future research should integrate multiple sources (e.g., satellite-derived snow cover) and hybrid physics–machine learning architectures to improve accuracy and reduce uncertainty, fostering sustainable practices worldwide. Ultimately, this work establishes a foundation for interpretable AI in hydrology, promoting equitable decision-making in vulnerable regions such as the BRB. Concerning potential drawbacks of the approach, the absence of explicit seasonality indicators (e.g., month-of-year encoding) and snow-related variables may limit the ability of the models to accurately reproduce extreme events and regime transitions. Future work will focus on extending the feature space to include such hydrologically meaningful predictors and assessing their impact on predictive performance.