Next Article in Journal
A Case Study of Drought during Summer 2022: A Large-Scale Analyzed Comparison of Dry and Moist Summers in the Midwest USA
Next Article in Special Issue
Hindcast Insights from Storm Surge Forecasting of Super Typhoon Saola (2309) in Hong Kong with the Sea, Lake and Overland Surges from Hurricanes Model
Previous Article in Journal
Long-Term Tropospheric Ozone Data Analysis 1997–2019 at Giordan Lighthouse, Gozo, Malta
Previous Article in Special Issue
Research on Typhoon Precipitation Prediction over Hainan Island Based on Dynamical–Statistical–Analog Technology
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Hydrological Drought Prediction Based on Hybrid Extreme Learning Machine: Wadi Mina Basin Case Study, Algeria

1
Laboratory of Water and Environment, Faculty of Nature and Life Sciences, Hassiba Benbouali University of Chlef, Chlef 02180, Algeria
2
Algeria Georessources, Environment and Natural Risks Laboratory, University of Oran 2 Mohamed Ben Ahmed, Oran 31000, Algeria
3
Department of Civil Engineering, Erzincan Binali Yıldırım University, Erzincan 24002, Turkey
4
Research Institute of Engineering and Technology, Hanyang University, Ansan 15588, Republic of Korea
5
Department of Civil Engineering & Technology, Qurtuba University of Science and Information Technology, Dera Ismail Khan 29050, Pakistan
6
Construction and Project Management Research Institute, Housing and Building National Research Centre, Giza 12311, Egypt
7
Department of Civil Engineering, Engineering Faculty, Siirt University, Siirt 23119, Turkey
8
Department of Earth and Space Sciences, Southern University of Science and Technology, Shenzhen 518055, China
*
Author to whom correspondence should be addressed.
Atmosphere 2023, 14(9), 1447; https://doi.org/10.3390/atmos14091447
Submission received: 30 July 2023 / Revised: 8 September 2023 / Accepted: 13 September 2023 / Published: 17 September 2023

Abstract

:
Drought is one of the most severe climatic calamities, affecting many aspects of the environment and human existence. Effective planning and decision making in disaster-prone areas require accurate and reliable drought predictions globally. The selection of an effective forecasting model is still challenging due to the lack of information on model performance, even though data-driven models have been widely employed to anticipate droughts. Therefore, this study investigated the application of simple extreme learning machine (ELM) and wavelet-based ELM (W-ELM) algorithms in drought forecasting. Standardized runoff index was used to model hydrological drought at different timescales (1-, 3-, 6-, 9-, and 12-month) at five Wadi Mina Basin (Algeria) hydrological stations. A partial autocorrelation function was adopted to select lagged input combinations for drought prediction. The results suggested that both algorithms predict hydrological drought well. Still, the performance of W-ELM remained superior at most of the hydrological stations with an average coefficient of determination = 0.74, root mean square error = 0.36, and mean absolute error = 0.43. It was also observed that the performance of the models in predicting drought at the 12-month timescale was higher than at the 1-month timescale. The proposed hybrid approach combined ELM’s fast-learning ability and discrete wavelet transform’s ability to decompose into different frequency bands, producing promising outputs in hydrological droughts. The findings indicated that the W-ELM model can be used for reliable drought predictions in Algeria.

1. Introduction

A lack of precipitation that interferes with human, crop, and animal life cycles is called a drought. Water resources, desertification, soil degradation, and food security are all severely affected by drought [1]. Extreme drought occurrences were recently predicted to occur frequently and intensely [2]. The primary difference between drought and other natural occurrences is the slow onset of drought. Drought is routinely predicted using several meteorological conditions and drought indicators on a monthly or seasonal timescale [3]. Yet, it continues to be one of the least understood natural occurrences due to its unpredictability, high complexity, and nonlinearity [4,5]. Therefore, it is crucial to create prediction systems to give early warning of the need for mitigation and adaptation measures because of the associated serious social and environmental implications [6].
According to how long they last, droughts are divided into four categories: hydrologic, meteorological, socioeconomic, and agricultural droughts [7,8]. It is necessary to define the characteristics of a hydrologic drought, such as a lack of groundwater supplies or streamflow deficiencies, and variables for a meteorological drought, such as transpiration, evaporation, and precipitation. Water storage resilience and inflow–demand reliability are also required for socioeconomic drought. Lastly, defining evaporation stress and soil moisture deficiency is necessary for agricultural drought. Governments, stakeholders, academics, and the general public pay particular attention to hydrological drought because of its disastrous effects on groundwater and surface water supplies. Hence, hydrological drought monitoring, forecasting, and mitigation are essential for protecting water resources and long-term economic viability.
Several drought indices, such as the standardized runoff index (SRI), standardized precipitation index (SPI), standardized precipitation evapotranspiration index (SPEI), standardized hydrological drought index (SHDI), soil moisture drought index (SMDI), surface water supply index (SWSI), and Palmer drought severity index (PDSI), can be used to monitor hydrological and meteorological drought [9]. These indicators have been widely used because of their simplicity and adaptability for predicting, measuring, and monitoring drought throughout different timeframes [10,11,12]. Furthermore, Ali et al. [13] introduced a composite drought index designed to encompass meteorological drought (SPI), agricultural drought (sETI and SSI), and hydrological drought (SRI).
Traditional time series and machine learning models differ in data analysis speed and flexibility [14]. For instance, Dikshit et al. [15] predicted drought in New South Wales, Australia using an artificial neural network (ANN) and support vector regression (SVR) forecasting models. The results revealed that ANN (coefficient of determination: R2 = 0.86) outperformed SVR (R2 = 0.75) in anticipating temporal drought trends. Sattar et al. [16] evaluated the hydrological and meteorological droughts using the Markov Bayesian classifier model. The model was 36 to 76% accurate in predicting meteorological drought, whereas it was 33 to 70% accurate in predicting hydrological drought.
Using optimized ANN models, Nabipour et al. [17] predicted short-term hydrological droughts using biogeography-based optimization, particle swarm optimization (PSO), salp swarm algorithm (SSA), and grasshopper optimization algorithms. The hybrid ANN model outperformed the conventional ANN when combined with the PSO algorithm. R2 = 0.68 and root-mean-square error (RMSE) = 0.58 for SHDI1, R2 = 0.81 and RMSE = 0.45 for SHDI3, and R2 = 0.82 and RMSE = 0.40 for SHDI6 were the results of the best models. Considering humidity, temperature, and precipitation, Jehanzaib et al. [12] examined six machine learning methods for hydrological drought forecasting in the Han River basin, South Korea. The decision tree (DT) technique outperformed in all watersheds, with an average Nash–Sutcliffe efficiency (NSE) of 0.87, RMSE of 0.34, mean absolute error (MAE) of 0.26, and R2 of 0.89. The DT method exhibited the most accurate assessment criteria and computation time results.
Algeria, although the biggest African nation, has few water supplies, with almost 40% of the population experiencing water scarcity. The country is mostly arid and dry, with only a few inches of rain each year. In the last few decades, prolonged drought has become a typical occurrence that negatively impacts the agricultural output in Algeria [18,19]. Achite et al. [20] tested several machine learning methods for predicting hydrological drought in the Wadi Ouahrane basin, Algeria. Several evaluation metrics were used to assess the adaptive neuro-fuzzy inference system, ANN, DT, and support vector machine (SVM) models. The SVM model outperformed the others, with an average MAE of 0.19, R2 of 0.90, RMSE of 0.28, and NSE of 0.86. In order to anticipate SPI indices in the Algerian Wadi Ouahrane basin for different timeframes, Achite et al. [21] used the Bayesian averaging model with several ANNs. The models comprised optimized ANN models using PSO, SSA, water strider algorithm, and sine cosine algorithm. The optimization algorithms enhanced the accuracy of ANN models for drought forecasts.
In the existing literature, several models have been developed for predicting and monitoring drought, encompassing a wide range of temporal and spatial scales. While these models have significantly contributed to our understanding of drought dynamics, there remains a crucial gap that our study seeks to address.
The primary gap in the literature pertains to the integration of extreme learning machine (ELM) and discrete wavelet transform (DWT) techniques for estimating multiscale SRI-based hydrological drought values. While previous studies have explored various machine learning and statistical approaches for drought prediction, the combination of ELM and DWT in the context of drought forecasting has not been extensively investigated. This novel hybrid approach holds the potential to provide more accurate and timely predictions of hydrological droughts, especially at multiple temporal scales.
Furthermore, our study extends beyond the mere application of ELM and DWT by incorporating partial autocorrelation function (PACF) graphs to optimize the selection of input combinations for the model. This addition adds another layer of sophistication to our approach, allowing us to identify the most effective input variables for drought prediction.

2. Study Area Description

The Wadi Mina basin in northwest Algeria was selected as the study area (Figure 1). It lies between 34°41′57″ N and 35°35′27″ N and between 00°22′59″ E to 01°09′02″ E. Its altitude varies from 164 to 1327 m, with an area of 4900 km2. It has complex and rugged topography and a continental climate with substantial temperature variations. The Wadi Mina, the main and last tributary on the left bank of the Wadi Chelif, runs 135 km from south to north [22]. It is limited to the northeast by the Ouancharis mountains, to the northwest by the Bani Chougrane mountains, to the west by the Saida mountains, to the southeast by the Frenda mountains, and in the south by the high plateau.
For the ground vegetation cover, scrubs account for 32% and forests and cereal crops account for 35.8%. The average yearly temperature ranges between 16 and 19.5 degrees Celsius. The yearly precipitation averages 500 to 250 mm, most falling between November and March [22]. This study obtained monthly runoff data for five hydrometric stations from 1974 to 2009 (Table 1 and Figure 1). Specifically, we obtained this dataset from the Agence National des Ressources Hydriques (ANRH), the National Water Resources Agency, and we duly acknowledged their contribution to our paper.
Data were examined for homogeneity using the double mass curve, linear regression, and Mann–Whitney test procedures to ensure quality. The technique found a few inhomogeneities, and the irregular data were corrected using data from nearby dependable stations [23].
Runoff data from these five stations were analyzed statistically to evaluate runoff variability in the study area (Table 2). These preliminary statistical analyses included measures of central tendency (mean and median), dispersion (standard deviation SD and coefficient of variation CV), and distribution (skewness Cs and kurtosis Ck).
The climatology of the study area is typically Mediterranean continental. The analysis of the monthly temperatures (T) over 34 years (1977/2010) of the Matemore station (ID: 506) showed that July and August are the hottest months of the year, with average temperatures of 26.7 and 26.8 °C, while January records low temperatures of up to 3 °C. The inter-annual average temperature is 17 °C. For the relative humidity (RH), the monthly values over the same period showed that February has the highest value at 94%, while the lowest value is recorded in July at 30% relative humidity. For the wind speed (Ws), April has the highest average wind speed at 3 m/s, while November has the lowest at 2.1 m/s. The monthly variability of the insolation showed that the maximum sunshine duration is observed in July with 331.1 h, while the minimum value is recorded in the winter period in December with a value of 173.8 h. Furthermore, the calculation of corrected potential evapotranspiration showed a maximum value of 168.3 mm and a minimum of 16.6 mm. Summer is the most dominant period of the year, due to the rise in temperature at this time of year (Figure 2).
The establishment of the ombrothermic diagram using meteorological data from the Matemore station enabled us to distinguish two periods; a dry period from May to October, characterized by high temperatures and minimal rainfall, and a wet period between November and April, due to the abundance of rainfall during this period (Figure 3).

3. Material and Methods

3.1. SRI

The SRI was calculated using the accumulative likelihood of monthly runoff [23].
g x = 1 β α Γ α x α 1 e x β
where β and α stand for scale and shape components, respectively, while x represents cumulated precipitation. The gamma function Γ ( α ) was determined with Equation (2).
Γ a = 0 y a 1 e y d y
The scale and shape parameters were determined using the precipitation time series as per Equation (3).
α = 1 4 A 1 + 1 + 4 A 3 ,     A = l n x ¯ l n   x i   n ,     β = x _ α
where x ¯ and x i refer to the average and total precipitation, respectively. Meanwhile, n is the number of data points. Equations (4) and (5) were used to illustrate the cumulative probability.
G x = 0 x g x d x = 1 β a Γ a 0 x x a 1 e x β d x
H x = q + 1 q G x
where q is the probability of zero precipitation. The SPI was determined using Equation (6)
S P I = { t c 0 + c 1 t + c 2 t 2 1 + d 1 t + d 2 t 2 + d 3 t 3 , 0 < H x 0.5 + t c 0 + c 1 t + c 2 t 2 1 + d 1 t + d 2 t 2 + d 3 t 3 , 0.5 < H x 1.0
where t was defined by Equation (7)
t = l n 1 H x 2 .   0 < H x 0.5   l n 1 1 H x 2 .   0.5 < H x 1.0  
where c 0 , c 1 , c 2 , d 1 , d 2 , and d 3 are values of coefficients as follows
c 0 = 2.515517 .   c 1 = 0.802853 .   c 2 = 0.010328
d 1 = 1.432788   d 2 = 0.189269   d 3 = 0.001308
SPI can be used to explore various classifications and projected probabilities of dry and wet spells for the time period under consideration (see Table 3) [24]. Similarly, for SRI, the hydrometric data were fitted to a log-normal probability distribution, and accumulative probabilities were then converted to a standard normal variate using Equations (5)–(7).

3.2. Extreme Learning Machine

A newly proposed machine learning algorithm for state-of-the-art single-layer feed-forward neural networks (SLFNs), namely ELM, was introduced by Huang et al. [25]. It is commonly applied for solving forecasting problems in various fields and is currently at the center of attention among climate change and engineering applications [26]. Furthermore, the studies based on ELM demonstrated that ELM models are useful and practical according to vector classification schemes or conventional neural networks.
ELM is simple to operate, and no parameters must be set other than the architecture of the predefined network. Hence, several challenges encountered with gradient-based algorithms (learning epochs and rate, local minima, etc.) are overcome. Furthermore, it has also been established that it is a faster algorithm in comparison to other traditional learning algorithms (i.e., SVM, back-propagation (BP), etc.) [27].
The ELM approach enables the majority of training to be performed in seconds or at least minutes for large complex practices that cannot be carried out with conventional neural network models [26,28]. Moreover, the model performs similarly to the generalization to singular value decomposition (SVD), BP, and SVM algorithms for data categorization and forecasting. For this reason, it is accepted as the ideal calculation algorithm for predicting atmospheric and meteorological variables (air temperature, solar energy, rainfall, etc.) [28,29,30].
ELM model theory shows that the standard SLFNs with M hidden nodes and an activation g(.) function are as follows for N arbitrary input samples (xk, yk) ∈ Rn × Rn
i = 1 M β i g x k , ; c i , a i = y k           k = 1,2 ,   3 ,   . , N
in which wiR is the input weight vector linking the input node and ith hidden node, ciR is bias randomly designed for the ith hidden node, βi is the weight vector linking the ith hidden node to the output node, and g (xk; ci, wi) is the output of the ith hidden node related to the input sample xk. Each input is randomly designed to the hidden nodes for ELM. Then, Equation (9) is rewritten as
H   β = Y
in which
H = g ( x 1 , ; c 1 , , w 1 )     g ( x 1 , ; c M , , w M )   g ( x N , ; c 1 , , w 1 )     g ( x N , ; c M , , w M )   N × M
H   β = β 1 T ,   β 2 T , ,   β L T m × M T
H   β = t 1 T ,   t 2 T , ,   t L T m × M T
β = H + Y
where Y is the output, H+ is the Moore–Penrose hidden layer matrix inversed, hidden biases and the input weights are randomly selected, and the output weights are obtained analytically.
ELM differs from the traditional neural network methodology in that all feed-forward network parameters (input weights and hidden layer biases) do not need to be set at the initialization. Huang [31] demonstrated that SLFNs with randomly selected input weights efficiently recognize different training samples with minimized error. Once hidden layer biases and the input weights are selected randomly, SLFNs can be considered linear. The output weights can then be specified analytically through a simple normalized inverse operation of the hidden layer output matrices connecting the hidden layer of this linear system to the output layer. Such a simplified approach enables many times faster ELM than conventional feed-forward learning algorithms. ELM’s speed, efficiency, generalization ability, and flexibility make it a valuable tool in many machine learning tasks. For one, ELM has a speedy training process compared to traditional machine learning models. ELM significantly speeds up training by randomly assigning input weights and thresholds. This feature is advantageous in large datasets or time-sensitive applications. Furthermore, ELM requires fewer computational resources than other models. ELM’s simple structure allows for efficient training and forecasting processes. Additionally, ELM reduces the risk of getting stuck to local minimums during training. Plus, ELM can apply knowledge gained from training data to unprecedented data. Finally, ELM can keep the risk of overfitting at a lower level than other models [26,29,30].

3.3. Wavelet Transform

This is an alternative to Fourier transform. It is a signal process that decomposes time series into various subsignals at various frequencies and gives the desired properties [32,33]. It is a practical mathematical function that gives a time-frequency description of a signal analyzed in the time domain [34]. A wavelet function Ψ(t) is a small wave that distinguishes distinct frequencies [35]. It comprises a wavelet transform scale, a data decomposition level, and a mother wavelet. The mother wavelet Ψ(t) is computed in Equation (14).
+ Ψ t d t = 0
The wavelet transform (WT) detects the connection between the wavelet function and the signal. It is performed at various scales (a) and localized around time (b). Consequently, the contour map, including the wavelet coefficient, is obtained. It is dependent on two scales [36]. Equation (15) can be applied in the DWT [37].
Ψ m , n t b a = a o m 2 Ψ * ( t n b o a o m a o m )
where m represents the wavelet expansion coefficient, n represents the coefficient of the wavelet translation, α represents the step of precision expansion, and bo represents the location parameter DWT for a discrete time series xi, in which xi happens at discrete time i. This takes the following form
W m , n = 2 m / 2 i = 0 N 1 x i Ψ ( 2 m i n )
In DWT studies, it is important to specify the appropriate decomposition level and mother wavelet [38,39]. Several mother wavelet types are available.
The decomposition level number is computed using Equation (17). This value depends on signal length [40].
L = i n t [ N ]
where L is the level of the decomposition and N is the number of runs. The present study set hybrid wavelet–EML models based on discrete WT and EML models. The input variables were subdivided using db10 mother wavelets.
This study aimed to model the multiscale SRI values by using past SRI values as inputs to ELM and WT-ELM algorithms and finally established a prediction model. According to the correlation analysis, each selected input variable was decomposed into various levels of subcomponents and presented to the WT-ELM model. The effect of the db10 mother wavelet on drought prediction was evaluated based on various statistical indicators. Thus, the goal was to improve the drought prediction model.

3.4. Choosing the Approach of the Input Parameters

The best input and output variables should be chosen because they significantly impact modeling success, particularly for nonlinear hydrological processes. For instance, N is the total number of observations, while k is the Yt series’ delay value and denotes the series average. The PACF is expressed in Equation (18) for kth delays. The lag number is expressed as a function of the time periods or lag number in the PACF. The lag number represents the time window in which the autocorrelation between a variable and its lagged values is calculated.
P A C F k , k = A C F j = 1 k 1 P A C F k 1 , j A C F k 1 1 j = 1 k 1 P A C F k 1 , j A C F k 1
5 percent significance level of PACF was determined by Equation (19), values calculated for kth delays were evaluated following upper and lower critical limits.
C I u p p e r / l o w e r = ± 1.96 N

4. Results

This study used SRI to model hydrological drought at different timescales. The predictions of hydrological drought were conducted at 1-, 3-, 6-, 9-, and 12-month timescales. The PACF analysis was performed to select model input combinations to predict hydrological drought. PACF develops negative and positive relationships between lagged SRIs and output SRI at the 95% confidence interval. The lags with PACF values lower or higher than the confidence levels were used as input variables to predict SRI at different timescales (1, 3, 6, 9, 12 months). The results of PACF analysis at all the stations are shown in Figure 4, Figure 5, Figure 6, Figure 7 and Figure 8. The lagged inputs to predict SRI at 1-, 3-, 6-, 9-, and 12-month timescale for all the stations are shown in Table 4.
Wavelet transformation analysis was performed and decomposition levels were chosen according to Int(log (410)) = 3. SRI values were divided into three levels of subcomponents with db10 wavelet. Due to space limitations, the results of wavelet decomposition levels at just Station 1 are presented in Figure 9.
Finally, ELM and wavelet-based ELM were used for hydrological drought prediction at different SRI levels at all stations. Both techniques’ performance was compared based on three performance indicators: R2, MSE, and MAE. The comparative results of both methods are shown in Table 5. The performance of W-ELM in HS1 and HS4 at all the selected timescales remained superior. The drought prediction performance of W-ELM increased as the SRI timescale increased because when we move from a smaller timescale to a bigger timescale, the SRI time series becomes smoother and uncertainty decreases. In HS2, HS3, and HS5, the performance of W-ELM remained superior while predicting drought at 1- and 3-month timescales, whereas at some stations such as HS2 and HS5, the performance of ELM was better for predicting SRI 9. Overall, the results showed that wavelet-based ELM outperformed ELM in hydrological drought prediction at most stations’ timescales.
The comparison between ELM and W-ELM was further extended by creating scatter plots at all the stations for all timescales. It is very clear from Figure 10, Figure 11, Figure 12, Figure 13 and Figure 14 that the drought prediction performance of W-ELM was better than ELM. Uncertainty in drought prediction was high at SRI 1 and performance was lower. At the same time, the uncertainty decreased as we moved to a longer timescale, and model prediction performance also increased. These findings agree with the previous study by Achite et al. [20], which suggested that the performance of models increases with the SRI timescale.
Figure 15 shows radar plots to compare ELM with W-ELM based on MSE. In the case of W-ELM, the value of MSE was lower at almost all the stations’ timescales. Based on these comprehensive results, it is recommended that W-ELM be used for drought prediction with higher accuracy than conventional ELM at different timescales. These results also agree with the previous study [41], which suggested that wavelet-based models improve drought prediction efficiency.

5. Discussion

In this study, two well-known machine learning techniques, the ELM algorithm and W-ELM, were used to predict hydrological drought in the Wadi Mina basin, Algeria. The results of drought predictions were also compared with previous studies by Achite et al. [20] and Katipoğlu [41] in the same region. Achite et al. [20] obtained the results with DT, ANFIS, ANN, and SVM to analyze hydrological drought. They obtained R2 values for DT (0.85), ANFIS (0.80), ANN (0.90), and SVM (0.95). Katipoğlu [41] applied combined wavelet models to forecast evaporation. The author stated that sym and dmey wavelet models give better results than db4 wavelet in forecasting evapotranspiration. In this study, the R2 value was 0.877 for the W-ELM model for HS4 using SRI 12. This value is higher than those obtained from the study of Achite et al. [20] for DT and ANFIS. Rajesh and Prakash [27] noted that The ELM model is faster than other traditional learning algorithms (i.e., SVM, BP, etc.). The studies of Şahin [29], Şahin et al. [30] and Deo and Şahin [28] showed that the ELM model has similar performance in the generalization to SVD, BP, and SVM algorithms for data categorization and forecasting. For this reason, it is accepted as the ideal calculation algorithm for predicting atmospheric and meteorological variables (air temperature, solar energy, rainfall, etc.).
Deo and Sahin [28] found that the ELM model gives better results than the ANN model in predicting drought at a monthly timescale. They also observed that the ELM model can be used in agriculture, water environments, and infrastructure practices. Therefore, this research explored the potential of ELM and W-ELM models in drought prediction at various timescales. The SRI was employed to calculate hydrological drought at different timescales (1, 3, 6, 9, and 12 months). PACF analysis was conducted to select model input combinations for hydrological drought prediction. PACF develops negative and positive relationships between lagged SRIs and output SRI at the 95% confidence interval. The lags with PACF values lower or higher than confidence levels were used as input variables to predict SRI at different timescales (1, 3, 6, 9, 12 months). The performance of W-ELM in HS1 and HS4 at all the selected timescales remained superior. The drought prediction performance of W-ELM improved as the SRI timescale increased because when moving from a smaller timescale to a larger timescale, the SRI time series becomes smoother and the uncertainty decreases. The performance of W-ELM remained superior for HS2, HS3, and HS5 while predicting drought at 1- and 3-month timescales, whereas at some stations such as HS2 and HS5, the performance of ELM was better for predicting SRI 9. In general, results prove that wavelet-based ELM outperformed ELM in hydrological drought prediction at most of the timescales for all the stations. Moreover, the drought prediction performance of W-ELM was better than that of ELM.

6. Conclusions

This study combined the ELM algorithm with DWT to predict SRI-based hydrological droughts in the Wadi Mina basin, Algeria. For this purpose, past drought indices values were decomposed into three detail components and an approximation component with db10 wavelet and presented to the machine learning model. In addition, the PACF of SRI values was used as feature selection for selecting input combinations of the drought prediction model. As a result of the analysis, it was revealed that the DWT technique significantly improved the predictive power of the ELM algorithm in drought prediction. In addition, DWT multiscale SRI values were divided into sub-bands, helping to comprehensively model fluctuations and submodes in the data and increasing SRI estimation accuracy. In addition, when compared to statistical indicators, the highest accuracy in estimating SRI values on the 1-, 3-, 6-, 9-, and 12-month timescales was obtained at the HS2 station, and the R2 values were 0.768, 0.804, 0.862, 0.865, and 0.870, respectively.
This study is limited to historical measurements. Trends in drought will be further extended to coming periods based on the expected outputs of global climate models to make an educated choice for sustainable watershed planning and management and to optimize the rules of operation of available water resources. Furthermore, the findings’ validity will be improved if future studies consider hydrological, agricultural, and socioeconomic droughts.

Author Contributions

Conceptualization, M.A. and O.M.K.; methodology, M.A., O.M.K. and M.J.; software, O.M.K. and M.J.; validation, M.A., N.E., M.J. and V.K.; formal analysis, N.E., V.K. and S.A.; investigation, M.A., N.E. and M.J.; data curation, O.M.K.; writing—original draft preparation, M.A., O.M.K., M.J. and N.E.; writing—review and editing, M.A., O.M.K., N.E., M.J., V.K. and S.A.; visualization, M.A., O.M.K. and S.A.; supervision, M.J. and N.E. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Data will be available upon request.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Danandeh Mehr, A.; Rikhtehgar Ghiasi, A.; Yaseen, Z.M.; Sorman, A.U.; Abualigah, Ü.L. A novel intelligent deep learning predictive model for meteorological drought forecasting. J. Ambient Intell. Humaniz. Comput. 2022, 4, 85–97. [Google Scholar] [CrossRef]
  2. Khan, N.; Sachindra, D.A.; Shahid, S.; Ahmed, K.; Shiru, M.S.; Nawaz, N. Prediction of droughts over Pakistan using machine learning algorithms. Adv. Water Resour. 2020, 139, 103562. [Google Scholar] [CrossRef]
  3. Hao, Z.; Singh, V.P.; Xia, Y. Seasonal drought prediction: Advances, challenges, and future prospects. Rev. Geophys. 2018, 56, 108–141. [Google Scholar] [CrossRef]
  4. Durbach, I.; Merven, B.; McCall, B. Expert elicitation of autocorrelated time series with application to e3 (energy-environment-economic) forecasting models. Environ. Model. Softw. 2017, 88, 93–105. [Google Scholar] [CrossRef]
  5. Adnan, R.M.; Mostafa, R.R.; Islam, A.R.M.T.; Kisi, O.; Kuriqi, A.; Heddam, S. Estimating reference evapotranspiration using hybrid adaptive fuzzy inferencing coupled with heuristic algorithms. Comput. Electron. Agric. 2021, 191, 106541. [Google Scholar] [CrossRef]
  6. Azizi, E.; Tavakoli, M.; Karimi, H.; Faramarzi, M. Evaluating the efficiency of the neural network to other methods in predicting drought in arid and semi-arid regions of western Iran. Arab. J. Geosci. 2019, 12, 489. [Google Scholar] [CrossRef]
  7. Heim, R.R., Jr. A review of twentieth-century drought indices used in the United States. Bull. Am. Meteorol. Soc. 2002, 83, 1149–1166. [Google Scholar] [CrossRef]
  8. Ahmadalipour, A.; Moradkhani, H.; Demirel, M.C. A comparative assessment of projected meteorological and hydrological droughts: Elucidating the role of temperature. J. Hydrol. 2017, 553, 785–797. [Google Scholar] [CrossRef]
  9. Mishra, A.K.; Singh, V.P. Drought modeling—A review. J. Hydrol. 2011, 403, 157–175. [Google Scholar] [CrossRef]
  10. Jain, V.K.; Pandey, R.P.; Jain, M.K.; Byun, H.R. Comparison of drought indices for appraisal of drought characteristics in the Ken River Basin. Weather Clim. Extrem. 2015, 8, 1–11. [Google Scholar] [CrossRef]
  11. Kim, T.W.; Jehanzaib, M. Drought risk analysis, forecasting and assessment under climate change. Water 2020, 12, 1862. [Google Scholar] [CrossRef]
  12. Jehanzaib, M.; Bilal Idrees, M.; Kim, D.; Kim, T.W. Comprehensive evaluation of machine learning techniques for hydrological drought forecasting. J. Irrig. Drain. Eng. 2021, 147, 04021022. [Google Scholar] [CrossRef]
  13. Ali, M.; Ghaith, M.; Wagdy, A.; Helmi, A.M. Development of a new multivariate composite drought index for the Blue Nile River Basin. Water 2022, 14, 886. [Google Scholar] [CrossRef]
  14. Liang, G.; Panahi, F.; Ahmed, A.N.; Ehteram, M.; Band, S.S.; Elshafie, A. Predicting municipal solid waste using a coupled artificial neural network with archimedes optimisation algorithm and socioeconomic components. J. Clean. Prod. 2021, 315, 128039. [Google Scholar] [CrossRef]
  15. Dikshit, A.; Pradhan, B.; Alamri, A.M. Temporal hydrological drought index forecasting for New South Wales, Australia using machine learning approaches. Atmosphere 2020, 11, 585. [Google Scholar] [CrossRef]
  16. Sattar, M.N.; Jehanzaib, M.; Kim, J.E.; Kwon, H.H.; Kim, T.W. Application of the hidden Markov Bayesian classifier and propagation concept for probabilistic assessment of meteorological and hydrological droughts in South Korea. Atmosphere 2020, 11, 1000. [Google Scholar] [CrossRef]
  17. Nabipour, N.; Dehghani, M.; Mosavi, A.; Shamshirband, S. Short-term hydrological drought forecasting based on different nature-inspired optimization algorithms hybridized with artificial neural networks. IEEE Access 2020, 8, 15210–15222. [Google Scholar] [CrossRef]
  18. Habibi, B.; Meddi, M.; Torfs, P.J.; Remaoun, M.; Van Lanen, H.A. Characterisation and prediction of meteorological drought using stochastic models in the semi-arid Chéliff–Zahrez basin (Algeria). J. Hydrol. Reg. Stud. 2018, 16, 15–31. [Google Scholar] [CrossRef]
  19. Achite, M.; Wałęga, A.; Toubal, A.K.; Mansour, H.; Krakauer, N. Spatiotemporal characteristics and trends of meteorological droughts in the Wadi Mina Basin, Northwest Algeria. Water 2021, 13, 3103. [Google Scholar] [CrossRef]
  20. Achite, M.; Jehanzaib, M.; Elshaboury, N.; Kim, T.W. Evaluation of machine learning techniques for hydrological drought modeling: A case study of the Wadi Ouahrane Basin in Algeria. Water 2022, 14, 431. [Google Scholar] [CrossRef]
  21. Achite, M.; Banadkooki, F.B.; Ehteram, M.; Bouharira, A.; Ahmed, A.N.; Elshafie, A. Exploring Bayesian model averaging with multiple ANNs for meteorological drought forecasts. Stoch. Environ. Res. Risk Assess 2022, 36, 1835–1860. [Google Scholar] [CrossRef]
  22. Achite, M.; Touaibia, B. Sécheresse et gestion des ressources en eau dans le bassin versant de la Mina. In Proceedings of the International Conference on Water and Environment, Sidi Fredj, Algeria, 30–31 January 2007; Volume 30. [Google Scholar]
  23. Kumar, A.; Gosling, S.N.; Johnson, M.F.; Jones, M.D.; Zaherpour, J.; Kumar, R.; Leng, G.; Schmied, H.M.; Kupzig, J.; Breuer, L.; et al. Multi-model evaluation of catchment-and global-scale hydrological model simulations of drought characteristics across eight large river catchments. Adv. Water Resour. 2022, 165, 104212. [Google Scholar] [CrossRef]
  24. McKee, T.B.; Doesken, N.J.; Kleist, J. The relationship of drought frequency and duration to time scales. In Proceedings of the 8th Conference on Applied Climatology, Anaheim, CA, USA, 17–22 January 1993; Volume 17, pp. 179–183. [Google Scholar]
  25. Huang, G.B.; Zhu, Q.Y.; Siew, C.K. Extreme learning machine: Theory and applications. Neurocomputing 2006, 70, 489–501. [Google Scholar] [CrossRef]
  26. Acharya, N.; Shrivastava, N.A.; Panigrahi, B.; Mohanty, U. Development of an artificial neural network based multi-model ensemble to estimate the northeast monsoon rainfall over south peninsular India: An application of extreme learning machine. Clim. Dyn. 2013, 43, 1303–1310. [Google Scholar] [CrossRef]
  27. Rajesh, R.; Prakash, J.S. Extreme learning machines—A review and state of-the-art. Int. J. Wisdom Based Comput. 2011, 1, 35–49. [Google Scholar]
  28. Deo, R.C.; Şahin, M. Application of the extreme learning machine algorithm for the prediction of monthly effective drought index in eastern Australia. Atmos. Res. 2015, 153, 512–525. [Google Scholar] [CrossRef]
  29. Şahin, M. Modelling of air temperature using remote sensing and artificial neural network in Turkey. Adv. Space Res. 2012, 50, 973–985. [Google Scholar] [CrossRef]
  30. Şahin, M.; Kaya, Y.; Uyar, M.; Yıldırım, S. Application of extreme learning machine for estimating solar radiation from satellite data. Int. J. Energy Res. 2014, 38, 205–212. [Google Scholar] [CrossRef]
  31. Huang, G.B. Learning capability and storage capacity of two-hidden layer feedforward networks. IEEE Trans. Neural. Netw. 2003, 14, 274–281. [Google Scholar] [CrossRef]
  32. Maheswaran, R.; Khosa, R. Comparative study of different wavelets for hydrologic forecasting. Comput. Geosci. 2012, 46, 284–295. [Google Scholar] [CrossRef]
  33. Sang, Y.F. A review on the applications of Wavelet transform in hydrology time series analysis. Atmos. Res. 2013, 122, 8–15. [Google Scholar] [CrossRef]
  34. Daubechies, I. The wavelet transform, time-frequency localization and signal analysis. IEEE Trans. Inf. Theory 1990, 36, 961–1005. [Google Scholar] [CrossRef]
  35. Schneiders, M.G.E. Wavelets in Control Engineering. Master’s Thesis, Eindhoven University of Technology, Eindhoven, The Netherlands, 2001; p. 38. [Google Scholar]
  36. Ahmadi, F.; Mehdizadeh, S.; Mohammadi, B. Development of bio-inspired-and wavelet-based hybrid models for reconnaissance drought index modeling. Water Resour. Manag. 2021, 35, 4127–4147. [Google Scholar] [CrossRef]
  37. Mallat, S.G. A theory for multi resolution signal decomposition: The wavelet representation. IEEE Trans. Pattern Anal. Mach. Intel. 1989, 11, 674–693. [Google Scholar] [CrossRef]
  38. Sun, Z.; Chang, C. Structural damage assessment based on wavelet packet transform. J. Struct. Eng. 2002, 128, 1354–1361. [Google Scholar] [CrossRef]
  39. Katipoğlu, O.M. Combining discrete wavelet decomposition with soft computing techniques to predict monthly evapotranspiration in semi-arid Hakkâri province, Türkiye. Environ. Sci. Pollut. Res. 2023, 30, 44043–44066. [Google Scholar] [CrossRef] [PubMed]
  40. Tiwari, M.K.; Chatterjee, C. Development of an accurate and reliable hourly flood forecasting model using Wavelet–bootstrap–ANN (WBANN) hybrid approach. J. Hydrol. 2010, 394, 458–470. [Google Scholar] [CrossRef]
  41. Katipoğlu, O.M. Prediction of streamflow drought index for short-term hydrological drought in the semi-arid Yesilirmak Basin using Wavelet transform and artificial intelligence techniques. Sustainability 2023, 15, 1109. [Google Scholar] [CrossRef]
Figure 1. Map of the study area along with located hydro-meteorological stations.
Figure 1. Map of the study area along with located hydro-meteorological stations.
Atmosphere 14 01447 g001
Figure 2. Variability of the meteorological data from the Matemore station (1977–2010).
Figure 2. Variability of the meteorological data from the Matemore station (1977–2010).
Atmosphere 14 01447 g002
Figure 3. Ombrothermic diagram of the study area (1977–2010).
Figure 3. Ombrothermic diagram of the study area (1977–2010).
Atmosphere 14 01447 g003
Figure 4. PACF plots of droughts at station HS1.
Figure 4. PACF plots of droughts at station HS1.
Atmosphere 14 01447 g004
Figure 5. PACF plots of droughts at station HS2.
Figure 5. PACF plots of droughts at station HS2.
Atmosphere 14 01447 g005
Figure 6. PACF plots of droughts at station HS3.
Figure 6. PACF plots of droughts at station HS3.
Atmosphere 14 01447 g006
Figure 7. PACF plots of droughts at station HS4.
Figure 7. PACF plots of droughts at station HS4.
Atmosphere 14 01447 g007
Figure 8. PACF plots of droughts at station HS5.
Figure 8. PACF plots of droughts at station HS5.
Atmosphere 14 01447 g008
Figure 9. SRI values’ subcomponents by discrete wavelet transform at the HS1 station.
Figure 9. SRI values’ subcomponents by discrete wavelet transform at the HS1 station.
Atmosphere 14 01447 g009
Figure 10. Scatter diagrams of SRI values at the HS1 station.
Figure 10. Scatter diagrams of SRI values at the HS1 station.
Atmosphere 14 01447 g010
Figure 11. Scatter diagrams of SRI values at the HS2 station.
Figure 11. Scatter diagrams of SRI values at the HS2 station.
Atmosphere 14 01447 g011
Figure 12. Scatter diagrams of SRI values at the HS3 station.
Figure 12. Scatter diagrams of SRI values at the HS3 station.
Atmosphere 14 01447 g012
Figure 13. Scatter diagrams of SRI values at the HS4 station.
Figure 13. Scatter diagrams of SRI values at the HS4 station.
Atmosphere 14 01447 g013
Figure 14. Scatter diagrams of SRI values at the HS5 station.
Figure 14. Scatter diagrams of SRI values at the HS5 station.
Atmosphere 14 01447 g014
Figure 15. Radar plot graphs of MSE values.
Figure 15. Radar plot graphs of MSE values.
Atmosphere 14 01447 g015
Table 1. Characteristics of gauging stations.
Table 1. Characteristics of gauging stations.
IDNameElevation (m)Basin Area (km2)LatitudeLongitude
H1013402Oued Abtal210412635°29′26.28″ N0°41′00.49″ E
H2013401Sidi Abdelkader Djillali24148035°28′46.05″ N0°35′19.99″ E
H3013302Ain Hammara285248035°23′50.09″ N0°40′33.19″ E
H4013001Kef Mehboula50268035°18′05.21″ N0°50′47.89″ E
H5013301Takhmaret634155335°06′20.08″ N0°38′46.54″ E
Table 2. Descriptive statistics of monthly and annual runoff in Wadi Mina basin (1974–2009).
Table 2. Descriptive statistics of monthly and annual runoff in Wadi Mina basin (1974–2009).
HS1JFMAMJJASONDAnn.
Min (m3/s)2.0512.5172.6771.8972.2450.9250.9120.8451.7803.4941.9751.5311.904
Max (m3/s)1.6302.6793.0802.5183.0531.4141.6541.6942.3625.0442.7001.6350.975
Mean (m3/s)2.0512.5172.6771.8972.2450.9250.9120.8451.7803.4941.9751.5311.904
SD (m3/s)1.6302.6793.0802.5183.0531.4141.6541.6942.3625.0442.7001.6350.975
Kurtosis−1.0001.6951.9096.4983.2900.8163.5865.5095.67612.9402.5542.690−1.312
Skewness0.4971.4471.4712.3011.8361.4462.0072.4042.2023.1301.8131.695−0.077
CV79.452106.427115.074132.752135.951152.785181.393200.400132.640144.354136.715106.77251.193
HS2JFMAMJJASONDAnn.
Min (m3/s)0.0000.0000.0000.0000.0000.0000.0000.0000.0000.0000.0000.0000.001
Max (m3/s)1.0941.0564.2000.6240.5990.4502.1120.2920.8561.6191.6280.7370.492
Mean (m3/s)0.2140.2230.3370.1120.0840.0560.1130.0270.0910.2560.2270.1670.159
SD (m3/s)0.2510.2680.7230.1580.1270.1100.4160.0700.1710.4140.3470.1820.130
Kurtosis4.6971.42124.4753.9107.1725.53018.08410.61311.1794.8717.4882.769−0.173
Skewness1.9411.3534.6282.0052.4542.4494.2713.3313.0602.2162.5821.6640.766
CV116.87119.96214.69141.31151.66197.68369.04257.85188.04161.58152.78109.0481.89
HS3JFMAMJJASONDAnn.
Min (m3/s)0.4080.1780.0960.0300.0060.0000.0000.0000.0170.0640.1220.3180.370
Max (m3/s)3.7005.4098.0583.2698.4172.7583.52711.17210.01019.4356.7132.8742.541
Mean (m3/s)1.2111.3251.4550.9390.9790.4080.2920.6471.3572.8561.4921.0721.169
SD (m3/s)0.8391.1951.6450.8221.8570.5470.6071.8692.0254.0981.6130.6520.539
Kurtosis2.6673.9617.0310.91713.1919.24824.15130.9089.7967.3982.9600.6340.492
Skewness1.7301.9562.4481.2343.6772.6764.6045.4292.9552.5341.9021.1450.829
CV69.32590.178113.04287.520189.731133.982207.598288.936149.268143.481108.14260.80046.093
HS4JFMAMJJASONDAnn.
Min (m3/s)0.0100.0080.0040.0000.0000.0000.0000.0000.0000.0000.0060.0060.022
Max (m3/s)3.1502.7502.6412.2003.3690.8000.2231.2191.9213.5002.6003.2001.194
Mean (m3/s)0.4470.4680.4650.2830.3050.1160.0300.0880.2920.5110.3980.3980.317
SD (m3/s)0.6320.6920.7060.4690.6070.1910.0460.2220.4950.7860.5090.6320.209
Kurtosis9.1084.2743.5048.92419.1405.6518.17320.4733.3055.3679.33711.5378.080
Skewness2.7502.2582.0942.8844.0632.4492.5384.3312.0622.2582.6543.1802.180
CV141.414147.886151.765165.434198.901164.933152.445253.629169.243153.735128.027158.84665.888
HS5JFMAMJJASONDAnn.
Min (m3/s)0.0520.0000.0000.0250.0070.0000.0000.0000.0000.0130.0430.0700.085
Max (m3/s)2.1192.7204.6462.69417.95414.46312.81010.7118.6989.6342.3752.1104.281
Mean (m3/s)0.4750.5270.5940.4431.2050.5830.4650.5001.1621.8770.4830.4080.727
SD (m3/s)0.4300.5700.8610.5913.3482.4032.1281.7952.0752.6260.5180.3890.827
Kurtosis7.6756.56514.5657.03218.84534.49435.17232.3017.5112.1545.82110.12511.321
Skewness2.6922.4643.5802.6334.1635.8275.9045.5872.7091.7732.3732.8763.192
CV90.675108.131145.009133.426277.900412.475457.964358.748178.650139.894107.16395.295113.758
Table 3. Categorization of SRI drought indices.
Table 3. Categorization of SRI drought indices.
SRI ValuesDrought CategoryProbability (%)
≥2.00Extremely wet2.3
1.50–1.99Very wet4.4
1.00–1.49Moderate wet9.2
−0.99–0.99Near normal68.2
−1.00–−1.49Moderately drought9.2
−1.50–−1.99Severely drought4.4
≤−2.00Extremely drought2.3
Table 4. Selected input combinations for estimation of multiscale SRI(t) values.
Table 4. Selected input combinations for estimation of multiscale SRI(t) values.
SRI1SRI3SRI6SRI9SRI12
HS1SRI1 (t-1)
SRI1 (t-2)
SRI1 (t-4)
SRI3 (t-1)
SRI3 (t-4)
SRI6 (t-1)
SRI6 (t-2)
SRI9 (t-1)
SRI9 (t-10)
SRI12 (t-1)
SRI12 (t-2)
SRI12 (t-3)
HS2SRI1 (t-1)
SRI1 (t-2)
SRI1 (t-4)
SRI3 (t-1)
SRI3 (t-4)
SRI6 (t-1)
SRI6 (t-2)
SRI9 (t-1)
SRI9 (t-2)
SRI12 (t-1)
SRI12 (t-2)
SRI12 (t-3)
HS3SRI1 (t-1)SRI3 (t-1)SRI6 (t-1)SRI9 (t-1)
SRI9 (t-10)
SRI12 (t-1)
SRI12 (t-2)
HS4SRI1 (t-1)SRI3 (t-1)
SRI3 (t-4)
SRI6 (t-1)
SRI6 (t-7)
SRI9 (t-1)
SRI9 (t-10)
SRI12 (t-1)
SRI12 (t-2)
HS5SRI1 (t-1)SRI3 (t-1)SRI6 (t-1)SRI9 (t-1)
SRI9 (t-7)
SRI12 (t-1)
Table 5. The results of the established SRI prediction model. The bold shows that these models performed well as compared to others and easy to identify.
Table 5. The results of the established SRI prediction model. The bold shows that these models performed well as compared to others and easy to identify.
HS1HS2HS3HS4HS5
ELMW-ELMELMW-ELMELMW-ELMELMW-ELMELMW-ELM
SRI 1R20.0830.7080.2230.7680.0550.5610.2400.5490.2510.537
MSE0.9960.3420.4490.1331.2880.5960.8540.4481.4120.666
MAE0.8010.4700.5000.2810.9190.6300.6850.5120.9280.615
SRI 3R20.5000.6100.5650.8040.5200.6830.5340.6640.4900.640
MSE0.6950.5210.3090.1260.6550.4430.6540.4240.7730.547
MAE0.5820.5050.4400.2800.5870.5090.5920.5040.6080.563
SR6R20.7020.8220.8130.8620.6780.6690.8350.8550.7150.782
MSE0.4340.3230.1620.1050.4430.4860.2500.2300.5400.494
MAE0.4040.4160.3030.2470.4090.4950.3500.3490.4360.502
SRI 9R20.7120.7940.8650.8110.6870.7410.8550.8610.7690.766
MSE0.4380.4670.1190.2190.3810.3840.2550.2550.3140.459
MAE0.4750.5010.2450.3810.3760.4490.3450.3430.3420.482
SRI 12R20.8170.8710.8700.7790.7380.7940.8690.8770.8250.855
MSE0.2530.2210.1010.6510.2730.2050.2850.2570.2710.228
MAE0.3090.3490.1910.6400.3050.3220.3070.3180.2930.318
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Achite, M.; Katipoğlu, O.M.; Jehanzaib, M.; Elshaboury, N.; Kartal, V.; Ali, S. Hydrological Drought Prediction Based on Hybrid Extreme Learning Machine: Wadi Mina Basin Case Study, Algeria. Atmosphere 2023, 14, 1447. https://doi.org/10.3390/atmos14091447

AMA Style

Achite M, Katipoğlu OM, Jehanzaib M, Elshaboury N, Kartal V, Ali S. Hydrological Drought Prediction Based on Hybrid Extreme Learning Machine: Wadi Mina Basin Case Study, Algeria. Atmosphere. 2023; 14(9):1447. https://doi.org/10.3390/atmos14091447

Chicago/Turabian Style

Achite, Mohammed, Okan Mert Katipoğlu, Muhammad Jehanzaib, Nehal Elshaboury, Veysi Kartal, and Shoaib Ali. 2023. "Hydrological Drought Prediction Based on Hybrid Extreme Learning Machine: Wadi Mina Basin Case Study, Algeria" Atmosphere 14, no. 9: 1447. https://doi.org/10.3390/atmos14091447

APA Style

Achite, M., Katipoğlu, O. M., Jehanzaib, M., Elshaboury, N., Kartal, V., & Ali, S. (2023). Hydrological Drought Prediction Based on Hybrid Extreme Learning Machine: Wadi Mina Basin Case Study, Algeria. Atmosphere, 14(9), 1447. https://doi.org/10.3390/atmos14091447

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop