Evaluation of the Strength of Slab-Column Connections with FRPs Using Machine Learning Algorithms

Slab-column connections with FRPs fail suddenly without warning. Machine learning (ML) models can model the behavior with high precision and reliability. Nineteen ML algorithms were examined and compared. The comparisons showed that the ensembled boosted tree model showed the best, most precise prediction with the highest coefficient of determination (R2) (0.98), the lowest Root Mean Square Error (RMSE) (44.12 kN), and the lowest Mean Absolute Error (MAE) (35.95 kN). The ensembled boosted model had an average of 0.99, a coefficient of variation of 12%, and a lower 95% of 0.97, respectively, in terms of the measured strength. Thus, it was found to be more accurate and consistent compared to all implemented machine learning models and selected traditional models. In addition, the significance of various parameters with respect to the predicted strength was identified, where the effective depth was the most significant by a factor of 0.9, and the concrete compressive strength was the lowest by a factor of 0.3.


Introduction
Vital infrastructures suffer from the risk of demolition due to a lack of maintenance, severe environmental conditions due to steel corrosion, and the sudden nature of slabcolumn connection failures. Thus, fiber-reinforced polymers (FRPs) are replacing steel to avoid problems due to steel corrosion because of their excellent properties, which include, but are not limited to, being non-corrosive in nature, having a high strength-to-weight ratio, and performing well under fatigue [1][2][3][4][5]. It is worth noting that, back in the 90's, FRPs were used for strengthening structures and continue to be valuable in this field [6,7]. In addition, most of the existing design models for slab-column connections lack a physical sense, which is due to their being empirical or semi-empirical [4]. On the other hand, machine learning (ML) models can model the behavior with a high level of precision and consistency [8][9][10].
Some studies have tackled slab-column connections with FRP reinforcements. In [24], Jumaa and Yousif examined three ML prediction models, Nonlinear Regression analysis (NLR), an Artificial Neural Network model (ANN), and GEP to predict the punching shear failure of FRPs. The models were trained on a dataset composed of 269 records. The results showed that the ANN model outperformed the other two models regarding prediction accuracy. In [20], two models were presented, one based on an ANN and one based on an SVM; both models were trained using a dataset composed of 82 records. In [25], Metwally employed a Levenberg-Marquardt Artificial Neural Network (LM-ANN) for the prediction of the punching shear strength of concrete slabs with various types of FRPs. His model was trained on a small dataset composed of 59 records. His method showed promising results when compared to the experimental results.
All previous ML models, used for the punching shear of FRP-reinforced slabs that were examined, worked with a relatively small dataset. In our study, a comprehensive study was performed for five main ML algorithms, and all of our models were trained with a relatively large dataset, composed of 189 records. The dataset of the five ML algorithms, used for the prediction of the punching shear of FRP slabs, was divided into three subsets: training, validation, and testing. In addition, the effect of all input parameters on prediction was examined, and all of our models were compared with each other with respect to model efficiency and prediction accuracy. Several design models for slab-column connection design models are shown in Table 1. Models were selected to represent wellknown simple design models, including, but not limited to, well-established design codes, guides, and recently developed models, with both empirical and semi-empirical designs. However, these models were developed using an old experimental database and thus lack the consistency needed for reliable strength prediction. In addition, these models vary in the variables considered and their patterns. Thus, there is a need for a machine learning model capable of accurately predicting strength and identifying the inter-relationships between variables. The selected models in Table 1 are being used for comparison with the proposed model as evidence of its accuracy and consistency.

Analysis of the Dataset
An extensive dataset of 189 records representing the test results of slab-column capacity with FRP slabs tested under punching shear was gathered from 37 research investigations, which will be referred to hereinafter as the dataset, as shown in Figure 1 and Table 2, where CFRP is carbon FRP, GFRP is glass FRP, N is the number of tested specimens, V is the punching shear failure load, E is Young's modulus, d is the effective depth, fc' is the concrete compressive strength, ρ is the flexure reinforcement ratio, b and c are the column dimensions, and A and B are the slab dimensions. In Figure 1, the dataset is close to normally distributed with respect to all variables. In Figure 1 and Table 2, the dataset covered a range with respect to all significant variables, including, but not limited to, the following:
The loading area dimensions vary from 25 to 635 mm.
FRP reinforcements have different diameters and configurations, where the diameter varies between 6 to 24 mm and the configurations are bars or grids. However, the flexure reinforcement ratio and Young's modulus were implemented to consider the influence of FRP material type, diameter, and configuration.

Machine Learning Methods
Before the implementation phase of the five ML models, dividing the dataset into two subsets, a training set with 80% of the dataset, with a handout validation of 15%, and a testing set with the remaining 20%, is recommended in order to determine the best-fit model. The testing dataset was not used in the training phase. The models were trained with five input variables: the column dimension C, the effective depth d, the concrete compressive strength f'c, the flexure reinforcement ratio ρ, and young's modulus E. The output of our models is the prediction of the slab-column strength of FRP V. The evaluation of the five models was performed on the testing set.

Linear Regression Model
This model is defined as a linear fit regression that interprets the relationship between the output and the influencing inputs. The key concept is to indicate the coefficient parameters' linearity. Different types of linear regression were examined, including normal, interaction, robust, and step-wise.

Regression Decision Tree
Decision trees subdivide data in a practical tree illustration using simple rules [67]. These rules are set through the decision tree, and the response prediction is performed in an iterative segmentation manner. The tree is composed of roots, leaves, and branches. The training dataset is arranged at the bottom of the tree. The training starts from the top-most roots of the tree. Afterwards, a conditional test is performed in order to draw a path along the tree branches for every node. Various judging conditions are applied to assess the testing at each node, such as the Mean Square Error (MSE). The output of the tree is the prediction that exists in the leaves of the tree, at the end of each path. Various types of regression trees are examined, including complex trees, medium trees, and simple trees. They all follow the same concept of prediction; however, they differ in the fixation of the minimum size of the leaf.

Ensemble Trees
The ensemble method was introduced in [68] as a group of separate, inadequate models that provide a powerful mathematical prediction. These types of trees can combine similar or non-similar prediction algorithms. There are two types of ensemble trees: bagged and boosted trees. The bagged trees create many models by implementing various bootstraps in a single tree and then merging them into single decision tree by computing the average between them. Boosted trees work as a two-step technique. In the first step, a subset of data is used to obtain a sequence of average working models; in the second step, the performance is boosted by joining the models with each other using a fixed cost function. The boosted algorithm relies on an iterative approach, which means that the parameters in the next step are updated using the residual computed from the previous step in order to optimize the objective function, which is defined as where θ is the trained parameters among the given data, J(θ) is the objective function, L(θ) is the training loss function, which is computed through the comparison between the predicted outputŷ i and the real output y i to evaluate the accurate prediction of the model, Ω(θ) is the regularization term, which is added to prevent model over-fitting by controlling the complexity of the algorithm, n and m are the number of predictions and trees, respectively, f k is the individual tree prediction function to evaluate the output in the functional space F of all regression trees, Υ and λ are the regularization parameters terms, also used in controlling the complexity of the boosted algorithm, T is the number of tree leaf nodes, and w j is the weight of the i th leaf node. The prediction resultŷ i of the boosted algorithm is computed from the prediction of each individual tree, defined asŷ where x i is the i th input variable. The boosted algorithm constructs a tree and splits a leaf node into two sub-tresses/ branches, left and right. Afterwards, the gain is computed at each leaf node for the determination of the best node. The optimal branch gain is selected once the gain after the splitting reaches its maximum value.
where G R and G L are the gain of the new right and left branches, respectively, and H R and H L are the original right and left branches, respectively. The boosted algorithm showed the best fit model for the prediction for FRP capacity with respect to the available dataset.

Support Vector Machine (SVM)
The concept of a SVM was first introduced in [69]. It is a method of implementing kernel functions for transforming data into a high dimensional feature space through a linear model. This model is utilized to remove any sophisticated nonlinear relationship. The main concept underlying the SVM is the linear regression function calculations, where the input data is mapped using a nonlinear function.
The training in the regression process can be defined as {(x 1 , y 1 ), (x 2 , y 2 ), . . . ., (x n , y n )}, where x i and y i are the input vector and predicted output value of the SVM models, respectively, and n is the size of the sample. The aim is to find the function f (x i ) that has the maximum derivation ε of the real output y i for all samples in the training set. f (x) is assumed to have a linear regression function that can be defined as where the vector w is required to be minimized to allow the function f (x) to be as flat as possible. This minimization is achieved by computing the norm Some points in the training data may not satisfy the constraint condition. In this case, a slack variable ϕ i will be introduced for such a sample to be able to deal with the constraint condition. This slack variable ϕ i will measure the derivation of the training points outside the ε supported points. Therefore, the SVM function will be computed to minimize as follows: where C > 0 is the regularization parameter and is responsible for the determination of the trade-off between the function f (x) and the calculated error. The final estimation function for to minimize:SVM is recalculated to be where σ i is the Lagrange multiplier, K(X, x i ) is the kernel function, and b is the bias term. Many SVM approaches have been examined, including linear, quadratic, cubic, fine, medium, and coarse SVMs. They all follow the same concept but have different kernel functions.

Gaussian Process Regression
Gaussian Regression (GPR) was introduced in [64]. It is a complex model that is capable of solving sophisticated ML problems. The power of such a model is its flexible, nonparametric models. GPR models are able to analysis the smoothness and noise parameters from the training data. The models find a stochastic process in which random variables are assumed to follow a Gaussian distribution. GPR models are non-parametric kernel-based probabilistic supervised learning models used for generalizing a complex and nonlinear function mapping hidden in the datasets used in training. The power of GPR methods is based on the use of kernel functions that improve efficiency when handling nonlinear data. GPR models provide a reliable response to the input data. These models assume that the output is computed as where ε is the noise representation of sample x i . For a given training set, the main goal is to predict the output value y * of a new input pattern. In order to be able to establish this goal, it is essential to establish three co-variance matrices as follows: where k(x 1 , x 1 ) is the co-variance function, which maps the relation between one output and the next. Many GPR methods have been examined, including the squared Gaussian process, the Marten 5/2 GPR, the exponential GPR, and the rational quadratic GPR.

Results and Discussion
To develop the used ML-based models, a grid search method with a 15-fold crossvalidation approach was used in the training phase to determine the optimal hyperparameters. In order to evaluate the effectiveness of our models, the following statistical measures were reported: • Correlation Coefficient (R 2 ), defined in Equation (14) Models were trained using an Intel core i5, 8GB RAM, using the MATLAB 2021a Machine Learning toolbox. In addition, the R 2 , RMSE, and MAE values were calculated for each model as shown in Table 3, which will be discussed in this section.

Linear Regression
The stepwise model had the highest R 2 and the lowest RMSE and MAE, with values of 0.95, 69.438, and 55.976, respectively, in training. However, the model produced the worst values in testing. The stepwise model reported the lowest R 2 and the highest RMSE and MAE, with values 0.64, 266.688, and 153.129, respectively, among other linear regression models. This could be because the testing set was not used in training and thus provided value ranges for the input patterns that were different from the values used in training. This means that this model was not able to produce accurate predictions.

Tree
Three different approaches were employed for solving the slab-column strength problem. The fine tree method performed best among the other tree methods in training and testing. It reported the highest R 2 and the lowest RMSE and MAE, with values of 0.94, 75.741, and 52.98, respectively, in training and 0.84, 85.3446, and 60.8346, respectively, in testing. It also consumed the most training time among the different tree approaches. The fine tree method could be used for prediction; however, it is not the optimal solution for our problem. The worst tree method in training was the coarse tree, while the medium tree reported the worst values in testing. This means that these tree methods may not be reliable enough for solving such a problem.

Support Vector Machine
Several SVM methods were examined for the prediction of the punching shear. The medium Gaussian SVM performed the best among the other tree methods in training and testing. It reported the highest R 2 and the lowest RMSE and MAE, with values of 0.96, 57.815, and 46.092, respectively, in training. I was not able to produce the same good results for testing, reporting 0.69, 236.587, and 109.372, respectively. This means that this method may not be able to produce accurate predictions for our problem.

Ensembled Trees
The boosted model reported the most optimal values for R 2 , RMSE, and MAE in both training and testing phases. It reported vales of 0.98, 44.12, and 39.95, respectively, in training and 0.97, 71.963, and 43.452, respectively, in testing. This means that this is the most optimal and powerful method that can capture all of the changes in the input parameter patterns and provide accurate predictions of the punching shear. The bagged methodology also gave good results in both training and testing, but it was not the most optimal and consumed more training time.

Gaussian Process Regression
GPR methods performed well in both training and testing phases. They provided a solution that was close to the optimum boosted ensembled tree solution but with double training time. The exponential GPR reported the highest R 2 and the lowest RMSE and MAE, with values of 0.96, 60.245, and 43.738, respectively, in training and 0.93, 67.839, and 43.738, respectively, in testing, compared with the other GPR methods. Figure 2 shows the predicted vs. actual values of the punching shear. This figure shows how well each model produces predictions for different response values. A perfect regression model has a predicted response equal to the actual response, where all of the points lie in the diagonal model, but the vertical distance from the line to any point is the error in the prediction of that point. Based on Figure 2, the best optimal solution of our problem was Model 1.14-the boosted ensembled tree model. Almost all points either have the smallest distance with respect to the vertical line or lie on the vertical line itself, i.e., a zero error between the predicted and real output values, which is the best case. Figure 2 is generic, but further specifications regarding model performance are found in Table 3, where the R 2 , RMSE, and MAE values are reported for all models. We also investigated the effect of each of the five inputs on the most optimal model, the boosted ensembled tree. The analysis showed that the effective depth d had the most important effect on the prediction of the slab-column strength, i.e., it has highest R 2 and the lowest RMSE and MAE, followed by column dimension C, while Youngs' modulus E had the least important effect, i.e., the lowest R 2 and the highest RMSE and MAE, as shown in Figure 3. The values are reported with respect to R 2 , MAE, and RMSE.
For the optimal method, the boosted tree in our case, Figure 4 shows how the error decreases as different combinations of hyper-parameters are evaluated. It also shows the model behaviour with the hyper-parameters that are optimized best. The best model converges in the 30th training iteration. It achieved the best performance in its 25th iteration.

Precision and Reliability of ML and Existing Models
In this section, the reliability and precision of the proposed mode will be compared to the most recent models, including, but not limited to, the Ju model developed in 2021 and the Hemzah developed in 2018. The precision and reliability of the capacity calculated using a model will be examined using the ratio between the experimentally observed capacity and that determined using that model (SF). Applying statistical measures on the SF calculated using the ML models and the existing ones could express the precision, reliability, and safety of the model. The closer the average of the SF is to unity, the more precise the used model is. The lower the coefficient of variation of the SF is, the more reliable and precise the model is. If the lower 95% confidence limit is close to unity and larger than 0.85, the model has acceptable safety. In addition, the SF can be plotted versus all effective parameters; thus, the variation in model safety can be examined. Moreover, the ideal pattern is plotted using a solid line, and a linear trendline for the SF is plotted using a dotted line. The inclination of the trendline is an indication of variation and scattering, while the sign of that inclination is indicative of an increase or decrease with the investigated parameter. Table 4 shows the statistical measures calculated for the existing models as well as the proposed model. It is clear that, overall, the ML model captured the behavior with a significantly lower RMSE, MAE, and coefficient of variation compared to all existing models, with values of 64.23, 37.97, and 12%, respectively. In addition, the proposed ML model is more precise, is more reliable, and is reasonably safe compared to the existing design codes in terms of the mean, R 2 , and the lower 95%, which is close to unity, with values of 0.96, 0.99, and 0.97, respectively. Excluding the ML model, the CSA, Hemzah, and Ju models are more reliable and accurate compared to the ACI and JSCE. Figure 5 shows the SF calculated using the JSCE, CSA, ACI, Hemzah, Ju, and ML models. In the figure, the inclinations of the trend line for the JSCE, CSA, ACI, Hemzah, and Ju models were −51 × 10 −4 , −20 × 10 −4 , −34 × 10 −4 , −15 × 10 −4 , −26 × 10 −4 , and −3 × 10 −4 , respectively. It is clear that the ML model has less scattering compared to the selected existing models.   Figure 6 shows the SF calculated using the JSCE, CSA, ACI, Hemzah, Ju, and ML models versus the effective depth. In the figure, the inclinations of the trend line for the JSCE, CSA, ACI, Hemzah, and Ju models was 0.8 ×10 −4 , −9 × 10 −4 , −9 × 10 −4 , −12 × 10 −4 , −6 × 10 −4 , and −0.4 × 10 −4 . For all models, the safety decreases with the depth increase, except in the case of the JSCE and CSA models. The ML has less scattering compared to the other models. The JSCE and Ju models have a small inclination, which might be due to the size effect factor.

Conclusions
Several machine learning models were developed and evaluated using an extensive experimental database of 189 slab-column connections with FRP reinforcements. Although concluding remarks are limited to the range of parameter values in the database, a problem that can be solved with the testing of more slabs, the following can be concluded: • A grid search with a 15-fold cross-validation was used to determine the optimal hyper-parameters of ML-based models during the training process. • The comparison to the experimental data showed that the five ML-based models with the input variables and optimal hyper-parameters are fully capable of predicting the punching shear strength of FRP-RC slabs. • The ensembled boosted model was found to be the most reliable and accurate model among all implemented machine learning models with the best accuracy: R 2 = 0.97, RMSE = 71.963 kN, and MAE = 43.452 kN for the testing dataset. In addition, the boosted model predicted the actual strength more precisely and reliably compared to the existing design models. It minimized the variability of the traditional models with respect to the effective variables. • For the most accurate model-the boosted ensemble-the effect of all input variables on the predicted Shear capacity was examined. Variables can be arranged from most to least influential as follows: 1. the effective depth; 2.
the longitudinal reinforcement modulus of elasticity; 5.
the concrete compressive strength.
• The proposed model has high accuracy and consistency and thus provides a reliable alternative to the existing strength models, which are inconsistent and have a high coefficient of variation. In addition, the interpretation results of the model reflect the importance and contribution of the parameters that influence the strength in the proposed model. Moreover, these findings confirm findings from concurrent research studies [70][71][72].