A Dynamic Recurrent Neural Network for Predicting Higher Heating Value of Biomass

The higher heating value (HHV) is the main property showing the energy amount of biomass samples. Several linear correlations based on either the proximate or the ultimate analysis have already been proposed for predicting biomass HHV. Since the HHV relationship with the proximate and ultimate analyses is not linear, nonlinear models might be a better alternative. Accordingly, this study employed the Elman recurrent neural network (ENN) to anticipate the HHV of different biomass samples from both the ultimate and proximate compositional analyses as the model inputs. The number of hidden neurons and the training algorithm were determined in such a way that the ENN model showed the highest prediction and generalization accuracy. The single hidden layer ENN with only four nodes, trained by the Levenberg–Marquardt algorithm, was identified as the most accurate model. The proposed ENN exhibited reliable prediction and generalization performance for estimating 532 experimental HHVs with a low mean absolute error of 0.67 and a mean square error of 0.96. In addition, the proposed ENN model provides a ground to clearly understand the dependency of the HHV on the fixed carbon, volatile matter, ash, carbon, hydrogen, nitrogen, oxygen, and sulfur content of biomass feedstocks.


Introduction
Since fossil fuels have been used extensively for energy [1], concerns have grown over the uncertain future of energy supplies as well as the adverse effects on the environment caused by their direct combustion [2,3]. Consequently, environmental protection is gaining much attention through the use of alternative energy sources [4][5][6]. As a renewable alternative to fossil fuels, energy production from biomass has gained considerable attention due to its considerable environmental advantages [7]. The thermochemical conversion of biomass is one of the most widely studied biofuel conversion technologies, but a strong focus on biomass large-scale production can result in controversies.
Thus, the use of waste-oriented biomass for energy generation as a sustainable and environmentally-acceptable method of generating energy can only be supported by using agricultural residues, municipal solid wastes, animal manure, sewage, and food waste [8,9]. A significant variation in the chemical and structural compositions of biomaterials used as feedstocks in thermochemical conversion results in a profound difference in the amount of energy that they contain. The energy content of biomass is lower than that of In this work, the Elman neural network (ENN), as a dynamic predictive tool with a great degree of flexibility and outstanding simulation performance, is used for the first time to estimate the biomass HHV. The structure-tuned ENN model can precisely anticipate the effect of the proximate and ultimate composition terms on the HHV and help find the best biomass type with the highest energy value.

Topology Tuning the ENN Model
The optimization techniques are needed to find either the maximum or minimum value of an objective function [38,39]. This study employed two well-known optimization scenarios, i.e., the LM (Levenberg-Marquardt) [40] and SCG (Scaled Conjugate Gradient) algorithms, to adjust the ENN's adjustable parameters. The accuracy of the ENN models with different sizes trained by the SCG and LM algorithms has been displayed in Figures 1 and 2, respectively. This analysis approved that the ENN prediction accuracy is a function of the training scenario as well as the model size. Therefore, it was necessary to determine the most suitable ones by a comparative analysis.
help find the best biomass type with the highest energy value.

Topology Tuning the ENN Model
The optimization techniques are needed to find either the maximum or minimum value of an objective function [38,39]. This study employed two well-known optimization scenarios, i.e., the LM (Levenberg-Marquardt) [40] and SCG (Scaled Conjugate Gradient) algorithms, to adjust the ENN's adjustable parameters. The accuracy of the ENN models with different sizes trained by the SCG and LM algorithms has been displayed in Figures  1 and 2, respectively. This analysis approved that the ENN prediction accuracy is a function of the training scenario as well as the model size. Therefore, it was necessary to determine the most suitable ones by a comparative analysis.
Both Figures 1 and 2 show that the model's accuracy for predicting the training group increased by enlarging the ENN size (increasing the number of hidden neurons). On the other hand, the reliability of the ENN models for estimating the testing HHV samples decreased by enlarging the model size. Since the ENN model was necessary to estimate both the training and testing HHVs with acceptable accuracy, the best possible size of the ENN-SCG and ENN-LM models is also shown in Figures 1 and 2. The ENN-SCG models with the two and five hidden neurons showed enough accuracy for predicting the training and testing datasets, while the ENN-LM only needed four hidden nodes to accurately estimate these two datasets.  Both Figures 1 and 2 show that the model's accuracy for predicting the training group increased by enlarging the ENN size (increasing the number of hidden neurons). On the other hand, the reliability of the ENN models for estimating the testing HHV samples decreased by enlarging the model size. Since the ENN model was necessary to estimate both the training and testing HHVs with acceptable accuracy, the best possible size of the ENN-SCG and ENN-LM models is also shown in Figures 1 and 2. The ENN-SCG models with the two and five hidden neurons showed enough accuracy for predicting the training and testing datasets, while the ENN-LM only needed four hidden nodes to accurately estimate these two datasets. Table 1 applies five statistical indices to compare the selected ENN models' performance in the training and testing stages. This table also introduces the ENN models' accuracy for predicting the whole HHV database.  Table 1 applies five statistical indices to compare the selected ENN models' performance in the training and testing stages. This table also introduces the ENN models' accuracy for predicting the whole HHV database.
The observed numerical indices approved that the ENN-LM model had a better performance than either the ENN-SCG 1 or the ENN-SCG 2 models. The absolute average relative deviation percent (AARD%), mean absolute error (MAE), relative absolute error percent (RAE%), and mean squared error (MSE) values achieved by the ENN-LM in the training and testing stages were smaller than those obtained from the ENN-SCG models. In addition, the correlation coefficient (R) values of the ENN-LM were higher than those obtained by the trained ENN model with the SCG algorithm.
Hence, it can be claimed that the Levenberg-Marquardt had a better performance than the Scaled Conjugate gradient algorithm to accomplish the training phase of the Elman neural network. In addition, the LM algorithm provided the ENN model with a better generalization ability in the testing phase. Therefore, the ENN-LM model with only four hidden nodes ( Figure 3) was selected as the best tool for predicting the biomass HHV.   The observed numerical indices approved that the ENN-LM model had a better performance than either the ENN-SCG 1 or the ENN-SCG 2 models. The absolute average relative deviation percent (AARD%), mean absolute error (MAE), relative absolute error percent (RAE%), and mean squared error (MSE) values achieved by the ENN-LM in the training and testing stages were smaller than those obtained from the ENN-SCG models. In addition, the correlation coefficient (R) values of the ENN-LM were higher than those obtained by the trained ENN model with the SCG algorithm.
Hence, it can be claimed that the Levenberg-Marquardt had a better performance than the Scaled Conjugate gradient algorithm to accomplish the training phase of the Elman neural network. In addition, the LM algorithm provided the ENN model with a better generalization ability in the testing phase. Therefore, the ENN-LM model with only four hidden nodes ( Figure 3) was selected as the best tool for predicting the biomass HHV.
It should be mentioned that all the developed ENN models have the logarithm sigmoid and hyperbolic tangent activation functions in the output and hidden layers, respectively. These functions help the ENN model to understand the nonlinear behavior of the biomass HHV in different operating conditions. Moreover, the continuous and differentiable characteristics of these activation functions are essential to adjust the ENN parameters by the training algorithm.
It should be mentioned that all the developed ENN models have the logarithm sigmoid and hyperbolic tangent activation functions in the output and hidden layers, respectively. These functions help the ENN model to understand the nonlinear behavior of the biomass HHV in different operating conditions. Moreover, the continuous and differentiable characteristics of these activation functions are essential to adjust the ENN parameters by the training algorithm.

Performance Monitoring
This section relied on numerical and graphical investigations to evaluate how accurate the structure-tuned ENN-LM model was in predicting the biomass HHV.

Training Stage
The cross-plot showing the actual versus calculated biomass HHVs for the training stage is depicted in Figure 4. The visual inspection of this figure and the observed R = 0.88335 indicate that an acceptable agreement existed between the actual and predicted biomass HHVs. The actual as well as predicted values of the biomass HHV shown in Figure 5 approved that the ENN-LM model was accurate enough in the training stage. The

Performance Monitoring
This section relied on numerical and graphical investigations to evaluate how accurate the structure-tuned ENN-LM model was in predicting the biomass HHV.

Training Stage
The cross-plot showing the actual versus calculated biomass HHVs for the training stage is depicted in Figure 4. The visual inspection of this figure and the observed R = 0.88335 indicate that an acceptable agreement existed between the actual and predicted biomass HHVs. differentiable characteristics of these activation functions are essential to adjust the parameters by the training algorithm. .

Performance Monitoring
This section relied on numerical and graphical investigations to evaluate accurate the structure-tuned ENN-LM model was in predicting the biomass HHV.

Training Stage
The cross-plot showing the actual versus calculated biomass HHVs for the tr stage is depicted in Figure 4. The visual inspection of this figure and the observe 0.88335 indicate that an acceptable agreement existed between the actual and pre biomass HHVs.   The histogram of the residual error between the actual and the predicted values of the biomass HHV in the training step is shown in Figure 6. This figure approved that the training HHV samples had been predicted by the outstanding residual error ranging from −3.5 to 4 MJ/kg. Furthermore, the standard deviation and average values of these residual errors were very small, i.e., 0.942 and 0.071 MJ/kg, respectively.   The histogram of the residual error between the actual and the predicted values of the biomass HHV in the training step is shown in Figure 6. This figure approved that the training HHV samples had been predicted by the outstanding residual error ranging from −3.5 to 4 MJ/kg. Furthermore, the standard deviation and average values of these residual errors were very small, i.e., 0.942 and 0.071 MJ/kg, respectively.

Testing Stage
The predicted HHVs by the ENN-LM model versus their counterpart actual values in the testing stage has been illustrated in Figure 7. The acceptable value of the coefficient of determination, i.e., R = 0.82255 showed that our small-size ENN-LM model was able to generalize its learning to the testing HHVs. The observed deviation between the actual and the predicted HHVs may be associated with the wide range of the involved biomass samples and their compositions, uncertainty in the experimental data, and model errors.

Testing Stage
The predicted HHVs by the ENN-LM model versus their counterpart actual v in the testing stage has been illustrated in Figure 7. The acceptable value of the coeff of determination, i.e., R = 0.82255 showed that our small-size ENN-LM model was a generalize its learning to the testing HHVs. The observed deviation between the a and the predicted HHVs may be associated with the wide range of the involved bio samples and their compositions, uncertainty in the experimental data, and model er The actual HHVs and their related ENN-LM predictions in the testing stage been simultaneously depicted in Figure 8. It can be seen that the proposed ENN model reliably interpolated these highly scattered experimental HHVs. Moreove numerical values of the AARD = 3.94%, MAE = 0.73, and MSE = 1.03 approve reasonable agreement between the actual and the predicted HHVs of the bio feedstocks.
It can be simply seen that the constructed ENN-LM model underestimated fiv overestimated four HHV samples in the testing stage. Since none of these HHVs seen by the ENN-LM model before and they were highly scattered, this lev uncertainty is acceptable from the modeling perspective. Figure 9 indicates that the designed ENN-LM model predicted the testing sam of the biomass HHV with low residual errors ranging from −3 to 2.5 MJ/kg. The av and standard deviation of the testing stage residual errors were 0.274 and 1.00 M respectively. This figure also clarifies that a major part of the testing HHVs was estim by the residual error of ~0 MJ/kg. The actual HHVs and their related ENN-LM predictions in the testing stage have been simultaneously depicted in Figure 8. It can be seen that the proposed ENN-LM model reliably interpolated these highly scattered experimental HHVs. Moreover, the numerical values of the AARD = 3.94%, MAE = 0.73, and MSE = 1.03 approved the reasonable agreement between the actual and the predicted HHVs of the biomass feedstocks.
It can be simply seen that the constructed ENN-LM model underestimated five and overestimated four HHV samples in the testing stage. Since none of these HHVs were seen by the ENN-LM model before and they were highly scattered, this level of uncertainty is acceptable from the modeling perspective. Figure 9 indicates that the designed ENN-LM model predicted the testing samples of the biomass HHV with low residual errors ranging from −3 to 2.5 MJ/kg. The average and standard deviation of the testing stage residual errors were 0.274 and 1.00 MJ/kg, respectively. This figure also clarifies that a major part of the testing HHVs was estimated by the residual error of~0 MJ/kg.       Figure 10 introduces the violin graph of the overall experimental HHVs and ENN-LM predictions. The complete similarity between these two graphs is an indicator of the acceptable accuracy of the structure-tune ENN-LM model for predicting the HHV of a wide range of biomass feedstocks.

Overall Data
Int. J. Mol. Sci. 2023, 24, x FOR PEER REVIEW Moreover, Table 2, which reports the experimental and predicted values median and average HHVs, approves an excellent performance of the proposed LM model. It can be seen that a slight deviation existed between the actual and mo values.

Elman Neural Network
Based on Figure 11, the Elman neural network (also known as the recurrent network) is made up of the input, hidden, context, and output layers. Indeed, the E a multilayer perceptron neural network with a feedback connection between the h and input layers [42]. As Figure 11 shows, the context layer gets its inputs from the h layer's output. This internal feedback connection increases the network's ability to p dynamic systems. ENNs possess short-term memory capabilities and are widely u a means of managing either classification or approximation problems [42]. Moreover, Table 2, which reports the experimental and predicted values of the median and average HHVs, approves an excellent performance of the proposed ENN-LM model. It can be seen that a slight deviation existed between the actual and modeling values.

Elman Neural Network
Based on Figure 11, the Elman neural network (also known as the recurrent neural network) is made up of the input, hidden, context, and output layers. Indeed, the ENN is a multilayer perceptron neural network with a feedback connection between the hidden and input layers [42]. As Figure 11 shows, the context layer gets its inputs from the hidden layer's output. This internal feedback connection increases the network's ability to process dynamic systems. ENNs possess short-term memory capabilities and are widely used as a means of managing either classification or approximation problems [42].

Data Collection
A model was developed to estimate the HHV of biomass data using eight independent variables, including FC, VM, ash contents on a dry basis, C, O, H, S (ash-free distribution), and N as inputs. Based on the input data (X1 to X8), the modeling method seeks to find y that best fits the data as follows: ( ) 1  2  3  4  5  6  7  8 , , , , , , , y X X X X X X X X (1) where X1, X2, X3, X4, X5, X6, X7, and X8 refer to the model inputs which are FC, VM, Ash, C, H, O, N, and S content of a biomass sample, and y denotes HHV.
The modeling procedure used 532 biomass data sets alongside their corresponding HHVs for use in the simulations. Table 3 presents the dependent/independent variables and their statistical information. The researchers randomly took 452 out of 532 available data patterns to use in the training stage. In addition, the overtraining was monitored by the remaining 80 samples to use as test data. Consequently, 85% of the dataset was used for training, while 15% was chosen randomly for testing.

Data Collection
A model was developed to estimate the HHV of biomass data using eight independent variables, including FC, VM, ash contents on a dry basis, C, O, H, S (ash-free distribution), and N as inputs. Based on the input data (X 1 to X 8 ), the modeling method seeks to find y that best fits the data as follows: y(X 1 , X 2 , X 3 , X 4 , X 5 , X 6 , X 7 , X 8 ) where X 1 , X 2 , X 3 , X 4 , X 5 , X 6 , X 7 , and X 8 refer to the model inputs which are FC, VM, Ash, C, H, O, N, and S content of a biomass sample, and y denotes HHV. The modeling procedure used 532 biomass data sets alongside their corresponding HHVs for use in the simulations. Table 3 presents the dependent/independent variables and their statistical information. The researchers randomly took 452 out of 532 available data patterns to use in the training stage. In addition, the overtraining was monitored by the remaining 80 samples to use as test data. Consequently, 85% of the dataset was used for training, while 15% was chosen randomly for testing.

Accuracy Evaluation
With the use of statistical testing, the most promising ideas were whittled down to a select few. Metrics like residual error (Equation (2)), standard deviation (Equation (3)), mean absolute error (Equation (4)), relative absolute error (Equation (5)), mean square error (Equation (6)), coefficient of determination (Equation (7)), and average absolute relative deviation (Equation (8)) abbreviated by RE, SD, MAE, RAE%, MSE, R, and AARD% were computed to assess the quality of the developed models. Each of these variables is defined in a different way using the following equations [43,44]:

Conclusions
In this research, a systematic procedure was followed to construct the efficient Elman neural network model to anticipate the higher heating value of biomass feedstocks. The ENN topological features and its training algorithm are well-determined by a systematic procedure. The appropriate training algorithm and the optimum number of hidden neurons of the ENN have been determined by a combination of trial-and-error and sensitivity analysis. The proximate (fixed carbon, volatile matter, and ash) and the ultimate (carbon, oxygen, hydrogen, sulfur, and nitrogen) composition analyses of the biomass are the independent variables used to estimate the HHV. The results showed that the ENN with only four hidden nodes trained by the Levenberg-Marquardt algorithm should be introduced as the best tool for estimating the HHV of the biomass. This structure-tuned ENN predicted the HHV of 532 biomass samples with outstanding accuracy (i.e., MAE = 0.67, MSE = 0.96, and AARD = 3.63%). The perfect compatibility between the actual HHVs and their associated predicted values by the ENN was also approved by different graphical investigations, including cross-plot, violin graph, and residual error monitoring. Our reliable ENN model could be easily employed to choose a biomass feedstock with the highest HHV as a fuel source. Since the bomb calorimeter analysis is not always available, interested readers may conduct the HHV modeling by ignoring this type of information.