Ensemble Tree-Based Approach towards Flexural Strength Prediction of FRP Reinforced Concrete Beams

Due to rise in infrastructure development and demand for seawater and sea sand concrete, fiber-reinforced polymer (FRP) rebars are widely used in the construction industry. Flexural strength is an important component of reinforced concrete structural design. Therefore, this research focuses on estimating the flexural capacity of FRP-reinforced concrete beams using novel artificial intelligence (AI) decision tree (DT) and gradient boosting tree (GBT) approaches. For this purpose, six input parameters, namely the area of bottom flexural reinforcement, depth of the beam, width of the beam, concrete compressive strength, the elastic modulus of FRP rebar, and the tensile strength of rebar at failure, are considered to predict the moment bearing capacity of the beam under bending loads. The models were trained using 60% of the database and were validated first-hand on the remaining 40% database employing the correlation coefficient (R), error indices namely, mean absolute error, root mean square error (MAE, RMSE) and slope of the regression line between observed and predicted results. The developed models were further validated using sensitivity and parametric analysis. Both models revealed comparable performance; however, based on the comparison of the slope of the validation data (0.83 for GBT model against 0.75 for the DT model) and higher R for the validation phase in case of the GBT model in comparison to the DT, the GBT model can be considered more accurate and robust. The sensitivity analysis yielded depth of the beam as the most influential parameter in contributing flexural strength of the beam, followed by the area of flexural reinforcement. The developed GBT model surpasses the existing gene expression programming (GEP) model in terms of accuracy; however, the current American Concrete Institute (ACI) model equations are more reliable than AI models in predicting the flexural strength of FRP-reinforced concrete beams.


Introduction
Cement-based materials and concrete are widely used worldwide due to their low porosity and high mechanical strength, taking additional advantage of reinforced steel bars in reinforced concrete (RC) structures without modifying the cementitious properties of the matrix [1]. Various structural facilities such as dams, bridges and nuclear powerplants are constructed from RC due to its high durability and protective nature of concrete for the rebars [2]. Despite the good durability of RC structures, steel bars are susceptible to corrosion [3]; an electrochemical process involving cathodic reduction of oxygen and anodic dissolution of iron in the presence of concrete pore solution acting as an electrolyte [4]; consequently leading to the failure of RC structures [5]. The corrosion mechanism encompasses the infiltration of chloride ions in the porous concrete into the rebar surface under an aggressive environment, degrading the protective layer formed on the steel surface. After steel corrosion, cracks appear on the surface of concrete due to resulting precipitates and the generation of tensile stresses [6]. This leads to frequent maintenance of RC structures, drastically impacting the economy in general [7,8].
The rapid increase in the population of the world poses a huge demand for the development of infrastructure; thus, the production of concrete is considerably increased [9]. The use of fresh water and river sand poses a negative impact on drinking water and river ecosystems. Besides, the coastal infrastructure demands direct use of seawater and sea sand in the concrete to avoid wasting time and money [10]. Concrete made of seawater and sea sand is highly aggressive due to its high alkalinity and chloride ions [11]. The use of conventional steel reinforcement is hindered in such a situation. Fiber-reinforced polymers (FRP) bars offer suitable applications instead of steel reinforcement due to their corrosion resistance [12][13][14][15]. FRP rebars provide several advantages such as high strength to weight ratio, low density, ease of handling, and corrosion resistance; therefore they have been used as internal reinforcement in concrete structures [16,17].
Several types of FRPs rebars, namely carbon FRPs, glass FRPs, aramid FRPs, and basalt FRPs are used as an alternative to conventional steel. Over the past three decades, numerous experimental works have been undertaken to assess the durability of FRPs [18][19][20][21][22][23][24]. The durability of FRPs in normal concrete is very satisfactory; however, its durability in harsh alkaline concrete environments is still debatable [15]. The design of flexural members such as beams and columns are key parameters to structural integrity. American Concrete Institute (ACI) 440.1 R-15 recommends basic formulations for the design of beams on the basis of mechanics [25]. However, Murad et al. [26] found that the flexural strength achieved from FRP reinforced concrete beams has some deviations from experimental test results. Gene expression programming (GEP), a modern Artificial intelligence (AI) technique based on genetic algorithms, can accurately predict the flexural strength of FRPs.
In the modern era, Finite Element Analysis [27][28][29] and AI techniques are widely used for solutions of problems in civil engineering, owing to their high accuracy and efficiency [30][31][32][33][34][35][36][37][38]. Awoyera et al. [39] employed an Artificial Neural Network (ANN) and GEP for estimating compressive, split tensile and flexural strength of geo-polymer concrete. A reliable accuracy of low error indices and high correlation was achieved in predicting strength characteristics of geo-polymer concrete. Wang, X.-Y. [40] successfully estimated the flexural strength of pozzolana and limestone blended concrete deploying ANN and GEP. Ahmad et al. [41] investigated decision tree (DT), ANN and gradient boost (GB) to predict the compressive strength of concrete at high temperature. Aslam et al. [37] employed GEP for the compressive strength of concrete containing Rice Husk Ash (HRA). The strength prediction of FRP-reinforced concrete has been investigated by several researchers using advanced AI techniques. Congro et al. [42] developed an ANN model for estimating the flexural strength of fiber reinforced concrete under bending loads. Lee, S. and C. Lee [43] investigated the ANN model to estimate the shear strength of FRP flexural members without stirrups and found that the ANN model can more accurately estimate the shear strength of FRP reinforced concrete beams compared to existing equations. The application of various AI techniques for predicting the strength characterization of FRP-reinforced and retrofitted concrete can be found in previous research [44][45][46][47].
Decision tree (DT) and Gradient boosting tree (GBT) are modern operational techniques used for regression and classification problems [48]. These techniques develop a tree-like structure in decision making, splitting the data into root node, branch nodes and several lead nodes [49]. Kermain et al. [50] found the GEP model better in predicting dam air demand than GBT and Random Forest (RF). Similarly, Song et al. [51] also found GEP had better accuracy than ANN and DT in predicting compressive strength of fly ash containing concrete. On the other side, Kamari et al. [52] developed GEP, ANN and DT models for characterizing CO 2 -brine solution interfacial tension and found DT more promising compared to GEP and ANN. Ahmad et al. [41] found the GBT model to be more accurate model than the DT or ANN model while investigating the compressive strength of concrete at high temperatures. Similarly, Huat et al. [53] proved GBT a more promising techniques in estimating pile friction-bearing capacity than RF and DT models.
In conclusion, empirical equations are available for the calculation of the flexural capacity of FRP-reinforced beams. Murad et al. [26] investigated the capability of the GEP model and inferred that the developed model is capable of estimating the flexural capacity more accurately. After critically evaluating the results of the GEP model (Section 3.4), it was concluded that Murad et al. [26] based their evaluation on correlation coefficient (R) only; however, error analysis showed that the empirical model was better compared to the developed GEP model. Moreover, the authors opine that the superiority of the AI technique is not specific, however, strongly depends on the nature (non-linearity) of the problem. One AI technique may perform better for a particular problem, whereas; some other model may surpass it in accuracy for a different situation. In continuation of previous research by Murad et al. [26] for developing a GEP tree-based model for flexural strength of an FRP-reinforced concrete beam, it is desirable to evaluate the performance of other models such as DT and GBT in solving a similar problem. Therefore, this research concentrates on developing novel DT and GBT models for estimating the flexural capacity of FRP reinforced concrete beams and comparing the capabilities of empirical relations and AI solution in terms of correlations and error analysis.

Experimental Database
The experimental database was collected from various experimental studies listed in Table 1. It can be seen that flexural strength is considered a function of six input parameters, namely; width of beam (W), depth of beam (D), compressive strength of concrete (f c ), area of flexural reinforcement (A s ), Elastic modulus of FRP rebar (EM), and the tensile strength of rebar at failure (T f ) as expressed in Equation (1).
where, M is flexural capacity. The role of these input parameters in contributing towards bending capacity is evident from the well-known ACI formulations expressed as Equation (2) through (6). The distribution histogram and descriptive statistics of the variables used in the study are shown in Figure 1 and     A Decision tree (DT) belongs to a family of supervised machine learning algorithms. As its name suggests, DT is similar to a tree having any branches which represent a possible reaction or different outcomes to a problem. DTs have the ability to visualize all the probable outcomes to a problem under all circumstances and thus have great importance in decision making [75]. DTs provide a better interpretation of model outputs compared to "black-box" models such as neural nets [76]. Due to their simple and easy-to-understand analytics and their precision on multiple data forms, DTs have found many applications in various fields [77,78].
Each tree consists of nodes and branches in the DT model. Each node denotes the features in a classification category, while every subset describes a possible value that the node can take [79]. The instance space from each internal node is divided into two or multiple sub-spaces in accordance with a specific discrete function of input attributes values. At the model output, each leaf is assigned to a unique category that is representative of the most fitting target value. The values in the leaves of DTs are also known as weights in some literature. DTs are sequential models that rationally combine the outcome of simple tests where each one compares a nominal attribute against a set of possible values or a numeric attribute versus a threshold value [80]. The partitioning scheme in DTs is designed to determine the possible partition values along the midpoint of a set of consecutive unique responses for each feature (i.e., gene expression). Different scoring criteria as Gini index (Gini), Information Gain (InfoGain) are used to compare and evaluate each of the possible tree partition values (Equations (1)-(4)) [75].
where N (t) is the number of samples in node t; N j represents the number of samples belonging to the class j; N j (t) is the class j samples associated with node t. The terms q and p k denotes the features sub-space and portion of samples passed to kth sub-space, respectively.

Gradient Boosting Tree
The fundamental intuitive idea behind all boosting algorithms is to combine multiple weak learners that should result in classification and regression models with improved predictive performance compared to a single model [81]. In machine learning, the weak learner refers to a model that performs slightly better than a random chance. This concept, which is known as "strength of weak learnability", was originated from the introduction of Ababoost in the early 1990s [82]. For gradient boosting trees, such weak learners are shallow decision trees.
Three main elements are involved during the predictive modeling of gradient boosting trees, i.e., (i) a loss function to be optimized, (ii) a weak learner for making predictions, (iii) and an additive model for combining the weak learners in order to minimize the loss function. The adopted loss function is typically influenced by the type of problem or task being addressed. For regression problems, a squared loss is the conventional choice, whereas, for classification tasks, an exponential error is a plausible loss function [81]. However, there may be circumstances in where other loss functions such as binomial deviance, Huber loss or absolute error could be more appropriate. The weak learners (decision trees) are constructed in a greedy manner by choosing the best split points (based on nodes purity index such as Gini) to minimize the loss. When adding the trees, a stochastic gradient descent procedure is employed to optimize the loss. Predictions of each tree in the gradient boosting tree algorithm are added sequentially, and the overall prediction error decreases as the iteration progresses. It is important that weak learners remain weak despite having some skill. An efficient heuristic is one having constrained tree creation which can be undertaken using several constraints such as number of trees, tree depth, number of nodes or leaves, observations per split, and minimum improvement to loss. In addition to their structure, additional constraints may be imposed to improve their performance. The leaf weight values can be optimized using different regularization functions which are intended to overcome the "overfitting" problem by placing restrictions on model parameters [83]. In the context of GBT, this refers to controlling the iterations (T) or trees during the training process. Additive models of GBT with potentially all input variables (x) may be represented by the general expression shown in Equation (6), while the function for optimizing the number of iterations is shown in Equation (7).
where the term h(x; a t ) is often regarded as a simple function characterized by the multiplier β and set of parameters (a = {a1, a2, . . . }).The parameter has a retarding effect on the learning rate of the series, which means that for achieving better accuracy, the series has to be long enough to compensate for shrinkage.

Development of the Model and Hyper-Parameters Tuning
Two models, namely DT and GBT were created in a rapidminer environment following the methodology explained in Figure 2. Overall, the modelling encompasses basic processing, feature modelling, transform validation and scoring data, validation of the models alongside generating scores, weights and simulator, production of the model, and finally delivering the results.
Firstly, basic processing of the database was initiated. The database was subjected to pre-processing. A single row of all the inputs was created to be deployed into the model, attributes were converted into nominal or real values, attributes were labelled, the target variable was defined, and input variables were selected based on their correlations with the target variable. Subsequently, the data were partitioned into the training (60%) and validation datasets (40%). Some basic features were introduced, such as handling unknown values, replacing missing values, and remembering known and unknown values, which were then subjected to feature engineering and modelling. This includes operators, such as handling the columns with the text, automating feature engineering, optimization, crossvalidation operator for introducing the specific ML model, applying the ML model, and training the model based on the optimized hyperparameters. After this, the transformation of the training and scoring data (no known target value) to missing values was completed. The models were applied to the validation and scoring data to validate the models. A model simulator was created for future predictions of the new data. Finally, a production model was generated based on the training and validation sets using the same hyperparameters to deliver the results.
The hyperparameters of the DT and GBT were tuned using the trial and error method listed in Table 3. For the DT model, maximal depth was varied from 2 to 25, whereas the minimal error rate was obtained for 7. Similarly, for the GBT model, the number of trees, maximal depth and learning rate were varied from 30-150, 2-7, and 0.001-0.1, respectively. The optimum results were obtained for 90 trees, 2 maximal depth and a 0.1 learning rate.

Evaluation Criteria
The models were evaluated using correlation coefficient (R), mean absolute error (MAE), and root mean square error (RMSE) in accordance with the previous literature [84]. The mathematical equations for these statistical evaluation functions are shown as Equations (8)-(10), respectively.
where e i and m i are nth experimental and model results, respectively; e i and m i denote the average values of experimental and model results, respectively, and n is the number of samples in the data set.

Pearson's Linear Correlations
In order to evaluate the correlation among the variables, the data employed in the current study were investigated for linear Pearson's correlation. It can be seen that A s , and D have a strong positive correlation with the flexural capacity of beams. The basic ACI equation for flexural strength of beam also inhibits a similar trend in flexural capacity with an increase in A s and D. The width of the beam (W) and f c show comparatively moderate positive correlation with M. The remaining attributes EM and T f have minor linear correlations, suggesting the existence of non-linear correlations between the inputs and the target variable. The detailed coefficient matrix provided can be seen from Table 4 and Figure 3.

Prediction Performance of the Developed Models
This section concentrates on statistical evaluation of the developed models in terms of correlation coefficient, MAE, RMSE and the slope of regression line to assess the robustness, effectiveness, and comparative investigation of the developed DT and GBT models for estimating the flexural strength of FRP-reinforced concrete beams. For the development of a reliable and efficient AI model, the ratio between the number of observed records (i.e., 60% training and 40% validation data points which in this case are 80 and 54, respectively) and contributing input variable (six in the current study) must not be less than three and preferably more than 5 [85]. In this research study, for considered flexural strength of FRP-reinforced concrete beams, this ratio equals 13.33 in the training and 9 in the validation phase, which is far beyond the recommended limit, representing relatively more reliable development of AI models. The observed (experimental) and forecasted moments for FRP beams (Figure 4a) for the DT model ( Figure 4b) and for the GBT model in the training and validation stage are visualized along with the performance indicator (i.e., slope, R, MAE, and RMSE). The 45-degree standard regression line represents the ideal fitted line with a slope exactly equaling 1. The distribution of the plotted points must be nearer to the standard line, with a slope greater than 0.8, minimal error metrics (MAE and RMSE) and R > 0.8 for reliable performance and strongly correlated models [36,86]. The slope of the regression line for DT and GBT corresponds to 0.75 and 0.83, respectively. In the case of the GBT model, the slope is greater than 0.8, hence representing a more robust performance compared to the DT model. The values of R for the training data of DT and GBT models are 0.974 and 0.964, respectively. Similarly, the validation data also yielded comparable R values equaling 0.92 and 0.94 for DT and GBT models, respectively. The magnitude of R in both the training and validation suggests strongly correlated models; however, R in the case of GBT is greater in the validation phase for the GBT model. Moreover, the values of R in the validation phase are almost comparable to that of the training phase, hence ruling out the problems of overfitting in the developed models. It is noteworthy that a higher value of R is not the sole indicator to judge the robustness of the AI model. Therefore, several error indices such as MAE and RMSE were also considered in this study. Mainly, while training the model, the optimization process targets the minimization of MAE alongside higher correlation magnitudes. The values of MAE was observed as 10.7 MPa and 11.74 MPa for the training phase of DT and GBT models, respectively. The validation dataset yielded 10.31 and 11.25 MPa for DT and GBT models. Based on the statistical evaluation from the validation phase, the value of R and the slope of the regression line were greater in the GBT model. For the same data, the value of RMSE was smaller for the GBT model; hence, the GBT model can be considered more robust compared to DT model.
It is imperative to evaluate the overall performance of the models, examining the error between experimental records and model-predicted output [87,88]. It can be observed from Figure 5 that predictions followed the experimental results more closely for both the developed models. Figure 6 displays the error generated in the validation stage for DT and GBT models. It can be seen that for DT and GBT models, the maximum of the points were within ±10 MPa. The value of MAE for the test data in the case of DT model was 10.32 MPa, whereas, for the GBT model it was 11.25 MPa as shown in Figure 4. Therefore, the results from statistical viewpoint and relative error ( Figure 6) show that the error in the developed models was within 20% (10.32/62.4 = 16.5% and 11.25/62.4 = 18.02%), where 62.4 is average value of the flexural strength of the database used in this study. The trend line of the errors intersects the y-axis at zero, reflecting the minimal errors in the developed models. Thus, in addition to higher correlation and lower error statistics, the developed models can be effectively used to estimate the flexural strength of FRP-reinforced concrete beams that will aid designers and practitioners to avoid heavy testing and save money and time.

Second Level Validation of the Models (Parametric and Sensitivity Analysis)
For this purpose, a simulated data set was established such that the input variable was varied uniformly between its extremes, and the other input variables were kept constant at their average values, as shown in Table 5. The change in the target variable was plotted against the variable input to obtain the parametric influence of the particular variable. Similarly, a simulated dataset was employed for sensitivity analysis. The difference in the values of the target variable with respect to each input variable was normalized to obtain the relative percentage of each contributing variable. Parametric and sensitivity analyses were conducted using the GBT model due to its high accuracy in comparison to DT model.  Figure 7 presents the parametric study based on the GBT model. The increase in width from 130-381 mm yielded a change of almost 6 kN-m in the bending capacity. The change in the depth significantly affected the bending capacity, almost 120 kN-m, with the increase in depth from 152 to 550 mm. Bending Moment also increased with increasing fc , A s , EM and T f as shown in Figure 7. It can be observed that moment capacity significantly changed with the variation in the depth rather than the width of the specimen. The empirical equations and principles of mechanics also suggest similar variations in bending moment with respect to the change in the depth of the beam. Moreover, the increase was not significant with a rise in elastic modulus of FRP rebar. The sensitivity analysis showed that depth of the beam contributed 60% of bending capacity, followed by A s , T f , fc , EM and W, respectively, as shown in Figure 8. The parametric and sensitivity analysis results are in line with the principles of mechanics and empirical equations, suggesting the reliability of the developed model.

Comparison with Previously Developed Models and ACI
The accuracy of the developed model was compared with that of already available formulations for flexural strength calculation of FRP-reinforced concrete beams as per ACI 440-17 and previously developed AI models. The equations given as Equation (11) through (15) were employed to calculate the flexural capacity of FRP-reinforced concrete beams. The prediction made by Murad et al. [26] using GEP are reported in Figure 9 for comparison. The statistical evaluation of Figure 9 demonstrates that the GEP model had a higher correlation compared to ACI model of 0.977 and 0.974, respectively. The other indices were MAE (4, 11.91) and RMSE (15.23, 19.27) for the ACI and GEP models, respectively. In contrast to the correlation values, the magnitude of errors was higher in the case of the GEP model compared to the ACI formulation. For statistical evaluation, the magnitude of errors should be small alongside correlation coefficients for an accurate model. Hence, it is inferred that R should not be solely used as an evaluation index for the developed model. In the comparison of GEP and ACI models, ACI was more accurate because of comparable correlation and least error values. Figure 9 also shows that ACI follows the experimental values more closely compared to the GEP model. Comparing the developed GBT model with the existing models, it can be seen that the GBT model has an almost comparable R of 0.964 to that of the ACI and GEP model. The magnitude of MAE and RMSE were 11.25 and 16.36, respectively. The magnitude of errors was less than in the previously developed GEP model; however, the existing ACI formulations surpassed the accuracy of the developed GBT model.
ρ f is FRP reinforcement ratio, A f is the area of longitudinal flexural reinforcement (mm 2 ), b is width of the beam (mm), and d is the depth of the beam (mm).
β 1 is compressive stress block parameter, f c is the concrete compressive strength ρ b is a balanced reinforcement ratio. f f is tensile stress of FRP rebar at failure, E f is elastic modulus of longitudal FRP bars, ε cu is ultimate concrete strain = 0.003, f f u is ultimate tensile strength of FRP rabars c is the distance from extreme compression fiber to the neutral axis of the member (mm), and c b is the distance from extreme compression fiber to the neutral axis of the member at balanced strain condition (mm).

Conclusions
This article presents estimation of the bending capacity of FRP-reinforced concrete beams. Previously, Murad et al. [26] developed a GEP tree-based model for flexural capacity of concrete beams reinforced with FRP rebars. It was shown that the strength achieved from the existing ACI equations for flexural capacity deviated from the experimental results. Therefore, the GEP model was found to have better accuracy than ACI 440 in terms of correlations (R) only. Upon other statistical evaluations of their results, our research demonstrated a higher error matrix of GEP predictions compared to ACI equations. Therefore, this research presented modern regression AI techniques namely DT and GBT, for this purpose. The following major conclusions were drawn from this study.

1.
The value of the correlation coefficient (R) for DT and GBT models were significantly higher than 0.8 (0.974 and 0.964 for the training stage and 0.92 and 0.94 for the validation stage, respectively), reflecting a solid agreement of input attributes in predicting flexural strength. Error evaluation such as MAE (10.32 kN-m) showed lower values in the validation phase in the case of DT models, whereas lower RMSE (16.36 kN-m) in the GBT model was observed. The performance of both the models were comparable; however, based on the comparison of the slope of validation data recorded as 0.83 (more closer to 1) for GBT models against 0.75 for the DT model and higher R for the validation phase, the GBT model can be considered more accurate and robust.

2.
The parametric study revealed a similar trend of the target variables with the change in the input variables coherent with the literature, further validating the trained model. The sensitivity analysis revealed the depth of the beam as the most influential parameter contributing towards flexural strength.

3.
The currently developed GBT model surpassed the accuracy of the previously developed GEP model. Hence, the GBT model can effectively predict flexural strength; however, the existing ACI equations are more reliable than the current and previously developed AI models. While comparing the models, it was shown that R should not be used as a single parameter in assessing the performance of the AI models; rather, a few error indices, specifically the MAE should be included.

Data Availability Statement:
The data used in this research has been properly cited and reported in the main text.