Prediction of Compressive Strength of Fly-Ash-Based Concrete Using Ensemble and Non-Ensemble Supervised Machine-Learning Approaches

Abstract

The utilization of waste material, such as fly ash, in the concrete industry will provide a valuable alternative solution for creating an eco-friendly environment. However, experimental work is time-consuming; employing soft machine learning techniques can accelerate the process of forecasting the strength properties of concrete. Ensemble machine learning modeling using Python Jupyter Notebook was employed in the forecasting of compressive strength (CS) of high-performance concrete. Multilayer perceptron neuron network (MLPNN) and decision tree (DT) were used as individual learning which then ensembled with bagging and boosting to provide strong correlations. Random forest (RF) and gradient boosting regression (GBR) were also used for prediction. A total of 471 data points with input parameters (e.g., cement, fine aggregate, coarse aggregate, superplasticizer, water, days, and fly ash), and an output parameter of compressive strength (CS), were retrieved to train and test the individual learners. Cross-validation with K-fold and statistical error (i.e., MAE, MSE, RMSE, and RMSLE) analysis was applied to check the accuracy of all models. All models showed the best correlation with an ensemble model rather than an individual one. DT with AdaBoost and random forest gave a strong correlation of R² = 0.89 with fewer errors. Cross-validation results revealed a good response with an error of less than 10 MPa. Thus, ensemble modeling not only trains the data by employing several weak learners but also produces a robust correlation that can then be used to model and predict the mechanical performance of concrete.

1. Introduction

Cement is an important material in the manufacture of concrete [1]. Its production in industry has a significant influence on global warming. Climate change and the release of gases during the manufacture of cement has a harmful effect on the Earth [2,3,4,5,6]. Nearly a billion tons of gases are emitted during its production due to the burning of natural resources and fossil fuels. Thus, apart from its benefits in the construction industry the manufacture of concrete has negative effects on the environment [7]. Efforts have been made to make new binders that will have substantial effects on the properties of concrete but have minimum effect on the environment [8]. One approach is to use waste-hazard material in making high-performance concrete (HPC). Its use in making an alternative binder not only minimizes disposal on the land but can produce significant benefits in making better, stronger concrete [9]. HPC has many advantages in terms of workability, durability, and strength as compared to traditional cement [10]. However, supplementary materials, such as silica fume, fly ash, and blast furnace slag, and chemical admixture are normally required to make HPC.

Concrete is classified as the most used material after water [11]. This is due to its several advantages in the domain of civil engineering [12]. However, making concrete requires deep study of the constituents used and often requires experimental work to achieve the target compressive strength. This creates uncertainties in making concrete as it is a mixture of fine aggregate, coarse aggregate, cement, admixtures and supplementary raw materials (SRMs) [13]. These constituents are thus randomly distributed in the concrete matrix. This uncertainty in making HPC can result in different strengths and consume much time to achieve a precise strength [14]. HPC is dependent on many factors, such as the size of particles, the water-cement ratio, and aggregate proportions. Generally, physical measures are carried out to obtain compressive strength. This is achieved by testing various cubes and cylinders with mixed design ratios in the laboratory [15]. However, the process of laboratory work to ensure strength is quite uneconomical and time-consuming. Thus, efforts have been made to predict the variables using regression and machine learning models [16,17,18,19]. This can not only reduce uncertainty in prediction but can ensure the required quantities in mix design to give strength. These supervised machine-learning algorithms identify robust relationships and predict the best model using a set of input variables.

Numerous algorithms, classified as supervised and unsupervised learning, have been utilized in the past to forecast the CS of concrete. Shahmansouri et al. [20] used gene expression programming (GEP) by utilizing waste material granulated blast furnace slag (GGBS) with 351 data points in the prediction of geopolymer concrete properties (GPC). The author used sodium hydroxide (NaOH), days, natural zeolite (NZ), silica fume (SF), and granulated blast-furnace slag (GBFS) as input variables, and compressive strength as the output variable. The results revealed that using a GEP model, the training set and validation set gave strong correlations of R² = 0.918 and 0.940, respectively. Awoyera et al. [21] utilized both (GEP) and artificial neuron network (ANN) machine learning algorithms in the prediction of GPC strength using FA, GGBS, SF, slump flow and several other input parameters to predict the outcomes of compressive, tensile and flexural strength. The results revealed that the GEP model excelled as it described the empirical relationships more effectively than ANN. Dantas et al. [22] and Duan et al. [23] used ANN applying it to the use of recycled aggregate and waste material from construction with 1178 data points in the forecasting of concrete properties. Nguyen et al. [24] used a hybrid algorithm using least square support vector machine (LSSVM) and particle swarm optimization (PSO) for the prediction of properties of fresh concrete with 142 data points. Zhang et al. [25] used such models for the prediction of sand concrete properties. Siddique et al. [26] used bottom ash (BA) in self-compacting concrete (SCC) and predicted its properties using the ANN technique with 80 mixes from literature studies and 31 samples from experimental work. Tran et al. [27] used the ANN model for the prediction of the strength of concrete-filled steel tube (CSFT) with ultra-high-strength concrete (768 data sets) and reported a strong correlation. Iqbal et al. [28] employed the GEP technique for prediction of green concrete properties utilizing waste foundry sand as a constituent with a 234-set database for compressive strength and 163 sets for splitting test. Congro et al. [29] used an ANN algorithm for the prediction of the flexural strength of fiber-reinforced concrete (FRC) with 400 data samples constructed from literature review. Machine learning is a useful tool in civil engineering and many other domains in the prediction of properties.

Table 1. Literature-based application of ML approaches.

Concrete Type	Properties	Techniques	References
Normal concrete	Compressive strength	Genetic programming	[30]
		ANN	[31]
High-performance concrete	Compressive strength	Random forest	[32]
		ANN	[33,34,35]
		M5P	[36]
		GEP	[37]
		Genetic weighted pyramid operation tree	[38]
Foamed cellular lightweight concrete	Compressive strength	ANN	[39]
Silica fume concrete	Compressive strength	Hybrid ANN	[40]
		Biogeography-based programming (BBP)	[41]
		ANN and ANFIS	[42]
Self-compacting concrete	Modulus of Elasticity	Biogeography-based programming (BBP)	[43]
Recycled aggregate concrete	Modulus of Elasticity	M5P	[44]
Concrete-filled steel tube	Compressive strength	GEP	[18]

illustrates the various machine learning algorithms used by researchers.

Two machine learning (ML) approaches were used for modeling and predictions for making a model. One was the standard technique, usually maintained on a separate model, while another was the ensemble approach, such as bagging, boosting, and AdaBoost [45]. The outcomes of research on these ensemble models has demonstrated that these newly developed models are more effective than an individual ML model [46]. The ensemble ML approaches have a different way of operating; these techniques normally initially train multiple weak learners employing training data and integrate the weaker ones into a strong learner. Weak learners are the individual techniques such as DT, ANN, or SVM. The prediction of the properties of objects/materials, such as concrete, using ensemble ML algorithms can be effectively carried out.

Recently, the application of ensemble ML algorithms to forecast the performance of materials with high accuracy has attracted great interest from researchers. The popularity of ensemble ML techniques has increased and several studies have found that these types of algorithms can more accurately predict outcomes compared to individual ML approaches [47]. This research focused on the application of various machine learning algorithms, including bagging, multi-boosting, rotation boosting, and random sub-spaces, to determine landslide vulnerability and comparison between results. The accuracy of the technique (MLPNN) was evaluated using an ensemble-learning approach. It was found that the multi-boosting technique was more effective than the other approaches. The application of ensemble ML approaches is listed in

Table 2. Ensemble ML algorithms used in the literature.

Concrete Type	Properties	Techniques	References
High-performance concrete	Compressive strength	BANN	[48]
		GBANN
		Adaptive boosting	[49]
		RF	[32]
		Gradient tree boosting	[15]
Recycled aggregate concrete	Modulus of Elasticity	RF + SVM	[50]
Corrosion of concrete sewer	Microbially induced concrete corrosion	Bagging/BoostingMLPNN/RBFNN/CHAID/CART	[51]
Corrosion of concrete sewer	Microbially induced concrete corrosion	Ensemble RF	[51]
High-performance concrete	Compressive strength	Adaptive boosting
RC panels	Failure modes	GBML	[52]
Lightweight self-compacting concrete	Compressive strength	RF	[53]

.

This research described the application of various ML approaches to forecasting the CS of high-performance concrete containing waste material (fly ash). The novelty, as well as the significance of this study, is twofold. The aim was based on challenging the execution of network-based DT, MLPNN models considering boosting and bagging with AdaBoost, and random forest, gradient boosting as an ensemble technique for forecasting. Additionally, this research considered the performance of tree-based and network-based models, as well as the correlation between the findings from both tree-based and network-based and learning, using several ML basic learners. More precisely, this study includes a description of the correct parameters for collecting appropriate component models and can be considered as a critical metric for model design. Second, this research compared and implemented ensemble learning methodologies to other learning models (individual) to forecast the strength of HPC using waste material fly ash as a crucial ingredient. To the authors’ knowledge, no equivalent study utilizing ensemble machine learning modeling for HPC exists in the literature. These learner programs were written in Anaconda Navigator software version 1.9.12 using Jupyter Notebook and Python 3.7.

2. Database Description

The database required to estimate concrete strength utilizing waste materials was gathered from the literature [54] and is summarized in Appendix A. Seven characteristics were utilized to forecast the CS of the final product, including days, cement, fine aggregate, binder water, and superplasticizer, coarse aggregate, as well as waste items, such as fly ash. Their distribution correlations with the compressive strength’s input parameters are shown in Figure 1.

The input variables had a significant effect on the model’s outcomes. The employed parameters used to run the models for the prediction of CS of high-performance concrete also played a positive role in the prediction. The Spyder environment from Anaconda software (version 3.7) was used to run the various models with the help of Python coding. For input parameters, the distributions are shown in Figure 2. The obtained data (results), in the form of their performance of the employed technique, was severely affected by the parameters. In an artificial approach, ML is effective in neuron-based artificial techniques to forecast the number of properties by analyzing the important concentration of parameters. Python applies the ML approaches and plots the correlation with the help of the Seaborn command. Figure 3 represents the maximum values of parameters in predicting its outcome CS. It was found and can be seen that the contours indicate individual influences on strength. Moreover, applying these concentrations in performing experimental work yields the maximum outcome. The ranges, as well as the data variables employed to run the model, can be seen in

Table 3. Description of input variable ranges.

Variables Used	Abbreviation Used	Minimum Value	Maximum Value
Input variables
Binder	CEM	134.7	540
Fine aggregate Coarse aggregate	FA CA	594 801	945 1125
Fly ash	FA	0	200.1
Water	W	140	228
Superplasticizer Age	SP AG	0 1	28.2 365
Output variable
Compressive strength	Fc	6.27	79.99

and

Table 4. Statistical indication of parameters.

Statistics	Cement	Fly Ash	Water	Superplasticizer	Coarse Aggregate	Fine Aggregate	Days	Strength
Count	471	471	471	471	471	471	471	471
Mean	298.08	62.59	181.88	5.02	1004.04	793.17	47.84	31.60
Std	100.69	64.88	18.01	5.49	74.17	73.86	65.53	14.74
Min	134.7	0	140	0	801	594	1	6.27
25%	229.7	0	167.55	0	961.2	758.3	14	19.73
50%	281	90	186	4.6	1006	792	28	31.35
75%	349	118.3	192	9.5	1056	850	56	40.87
Max	540	200.1	228	28.2	1125	945	365	79.99

, respectively.

3. Machine Learning Methods

Various researchers, when forecasting and evaluating the behavior of materials, commonly use ML algorithms. In this study, the prediction component of high-performance concrete’s CS was addressed by the deployment of machine learning-based methodologies including artificial neural network, random forest, decision tree, and gradient-boosting network. The selection of these approaches was based on their popularity and high accuracy in prediction. However, another learner was then applied for modeling the concrete’s strength. The flow diagram indicating the process of ensemble learning is shown in Figure 4.

3.1. Overview of Machine Learning Algorithms

3.1.1. DT Algorithm

A decision tree is a subset of machine learning that has a tree-like structure composed of branches and nodes. The nodes with outgoing corners are referred to as inner nodes, whereas the nodes without outgoing corners are referred to as leaves. Through an inner node corresponding to a given function, the instance used for regression or classification is separated into numerous classes. The input parameters are treated as a special function during the training step. The stimulant for the decision tree is a method that is seen as a decision-maker tree from supplied instances. By reducing the fitness function, the conducted technique determines the best decision tree. Due to the absence of classes in the given datasets, the regressor model was used in this study for the target parameter using the independent variable. For each parameter, the dataset is segmented at multiple places. The performed technique determines the difference between the expected and actual values for the pre-specified fitness function at each division point. The mistakes in the divided region are equal across the variables, and the parameters with the lowest fitness function values are chosen as the split point. This practice is repeated indefinitely to improve the model.

3.1.2. Random Forest (RF) Regressor

The random forest model is a classification and regression technique that has been the focus of numerous studies [32,53]. Shaqadan [55] employed the RF model to forecast the concrete’s CS. The primary distinction between RF and DT is that in DT a single tree is produced but, in RF, a forest of trees is generated and non-similar data are arbitrarily allocated to all the trees. For each tree, the database is assigned in the form of rows and columns, with distinct column and row dimensions determined [56]. The growth of every tree is based on the multiple factors mentioned below:

• The frame of the given data is two-thirds of the total data collected randomly for each tree. This is referred to as bagging. Forecasted parameters are chosen freely and the node-splitting algorithm uses the finest split on these parameters.
• For each tree, the out-of-bag error is determined using the remaining data. Additionally, errors from each tree are accumulated to determine the ultimate out-of-bag error rate.
• Each tree displays a regression, and the model chooses the prediction with the most votes out of all the trees in the forest. These can be zeroes or ones. Forecasting probability is defined as the proportion of 1’s obtained.

Random forest (RF) is considered one of the most sophisticated ensemble approaches. It combines the advantages of variable importance measures (VIMs) with a small number of model parameters, as well as robust overfitting resistance [57,58]. It uses a decision tree as its base predictor to the random forest. Models constructed using RF can produce acceptable results along with the default setting of the parameter [59]. By utilizing RF, it is possible to minimize the number of possible combinations of base predictors and parameter values to one. Many applications for utilizing RF can be found in different domains, such as ecology [60,61,62] and bioinformatics [63,64,65], but it has seldom been applied to concrete in civil engineering [66].

3.1.3. ANN Approach

ANN approaches simulate the microstructure (neurons) of a biological nervous system [67]. ANNs are computational processing information networks emulating the biological connection between neurons. In artificial neural networks, the models perform a major role in performing neural activities [68]. The motivation in developing artificial neural networks (ANNs) originated from emulation of the human brain to implement neural intelligent tasks analogous to that of the human brain [69]. Neurons are the human brain’s basic unit, which underpins the capacity to memorize and think based on experience. These cells are present in the form of neuron networks. Generally, these cells receive input from other sources, perform non-linear neural activities and generate finalized output. In a similar way to other fields, ANNs are also utilized in engineering applications [70]. Recently, these neural activities have been found to be successful in civil engineering structural detection applications, identification behavior of the structural system, observation of groundwater, and prediction of mechanical properties.

ANNs are comprised of a large number of dense parallel linkages. These cells receive data in the form of weighted inputs and then employ an activation function to communicate the weighted output to other neurons, resulting in the activation of a cascade of neurons. These neural activities are made up of one or more layers of activity. The multilayer perceptron network is commonly employed in brain processes such as memory and learning. These input parameters, which have been established based on previous experience, are processed using neural layer processing systems. The perception reaction is generated based on the number of input variables, the number of input layers, and the result. The input, hidden, and output layers are the three layers of a network with the addition of three layers. The hidden layer can be located between the input layer and the output layer, and may be part of a larger network of hidden layers than the input layer. A single hidden layer can be used to solve some of the problems with perceptrons; nevertheless, it is more important and more successful to use more than one hidden layer [71]. Figure 5 is a representation of a neural network architecture, which includes one input layer, two hidden layers, and one output layer, among other elements. Every neuron in the layers, except the input layer, begins by analyzing a linear combination in conjunction with a bias. The neurons then compute nonlinear functions in the buried layers of their input, which they then feedback to the computer. It is necessary to point out that a sigmoid function is a non-linear function [72].

ANNs mainly consists of five parts, including input, activation functions, sum function, weights, and output. ANNs can train their model by adapting its weight factor from outside the network or by adjusting within the layer in response to input [74]. The external source is used to provide input parameter information while the input’s outcome set can be described by the weight of the values. The submission of inputs and weights with bias is characterized as a sum function. The mathematical expression of the weighted sum function is calculated using Equation (1).

$${\left(net\right)}_j={{\displaystyle \sum }}_{i=1}^n{w}_{ij}{x}_i+b$$

(1)

The activation function processes the net input which is found after submission from the sum function and determines the output of the cell. Normally sigmoid, ramp and Gaussian functions are the most popular activation functions. A schematic diagram of a multilayer perceptron, showing input, sum function with sigmoid activation, and output, is shown in Figure 6 [75]. The output of j neuron based on input variables is computed by employing sigmoid functions as follows. The sigmoid function is a non-linear activation function, except the input layer activates in every layer. The overall net out-based output of j neuron is obtained using Equation (2).

$${\left(out\right)}_j=f{\left(net\right)}_j=\frac{1}{1+e^{-\alpha {\left(net\right)}_j}}$$

(2)

The MLPNN feed-forward algorithm is used to execute the ANNs model in this study. Both the hidden layers and the changing neuron with two in number are taken into consideration while evaluating the superior performance of MLP [76]. In addition, a linearly activating function, along with a non-linear transit function, is used between the hidden layer and the input layer to verify the relationship between the input and output parameters. Furthermore, the database derived from the literature was subdivided into two sets, namely the training set and the testing set. Because the effect of overfitting data is a significant concern in machine learning, this was done to mitigate the effect of overfitting. It was proposed in a prior study that 80 percent of the randomly generated data should be used for training while only 20% of it should be used for testing [77].

3.1.4. Gradient Boosting Algorithm

Boosting is an effective approach that combines many simple base classifiers to create a committee whose output is substantially better than that of any base classification [9]. The core concept of boosting is to sequentially introduce new models to the ensemble model. A new, poor basic learner model is trained on each particular iteration about the failure of the entire ensemble that has so far been learned [78]. Gradient boosting constructs the model stage-wise, as do other boosting approaches, and generalizes it by optimizing an arbitrarily variational loss function [79]. Using the gradient descent approach, the problem of minimization is solved and a prediction model is generated in the form of an ensemble of weak prediction models. Thus, it can be used for classification or regression-based problems [79].

GBR is a simplification of gradient booster and consists of three elements: a loss function (to be optimized), a weak learner (to predict), and an additive model (to incorporate weak learners to minimize the loss function) [80]. The key idea behind this algorithm is to create a maximum correlation between the new basic teachers and the negative gradient of loss function associated with the entire ensemble [80]. The loss function may be arbitrary, but to provide a better intuition, the learning method can lead to consecutive error-fitting when the error function is the classic square-error loss. In general, the researchers are responsible for selecting the loss function; they have a rich range of loss functions and the possibility of applying their task-specific losses. The mathematical descriptions of GBR are as follows (see Equation (3)). Consider an advanced type model [79]:

$$F\left(x\right)={\displaystyle \sum }_{m=1}^M{\gamma }_m{h}_m\left(x\right)$$

(3)

where ℎ_m (x) are the root functions, which are typically called poor students in the boosting aspect. GBR uses fixed-size decision trees as slow learners. Decision trees have several capabilities which enhance their ability to handle mixed-type data and to model complex functions. Like other boosting algorithms, GBR develops the additive model in a forward stage-wise fashion, as represented in Equation (4) [79]:

$$F_m\left(x\right)=F_{m-1}\left(x\right)+{\gamma }_m{h}_m\left(x\right)$$

(4)

The decision tree in that stage ℎ_m (x) is taken to reduce the loss function L, given the model F_m-1 and its fit F_m−1(x_i) (see Equation (5)).

$$F_m\left(x\right)=F_{m-1}\left(x\right)+argmin {\displaystyle \sum }_{i=1}^{n}L\left(y_i,F_{m-1}\left(x_i\right)-h\left(x\right)\right)$$

(5)

GBR tries numerically to solve this error function problem via gradient descent. The GBR gradient is the negative gradient of the loss function evaluated in the existing model F_m−1 that can be determined for any differentiable loss function, as indicated in Equation (6). The main gains of GBR are:

• normal management of mixed form data,
• high predictive control,
• output space robustness (via robust loss functions)
• supports various loss functions.

$$F_m\left(x\right)=F_{m-1}\left(x\right)+{\gamma }_m{\displaystyle \sum }_{i=1}^n{\nabla }_{F}L\left(y_i,F_{m-1}\left(x_i\right)\right)$$

(6)

3.2. Bagging and Bossing Approaches

Ensemble approaches are used to optimize data mining and machine learning operations. Additionally, these strategies assist in mitigating unneeded training data concerns by merging and pooling numerous suboptimal prediction models (i.e., sub-models of components). By carefully changing training data, it is possible to generate a large number of sub-models/classifier components (1, 2, …, M). This could lead to a more productive learner. The optimal predictive model, on the other hand, can be built by averaging combine procedures on verified sub-models. Bagging, along with the bootstrap resampling strategy and benefit collation, is recognized as one of the most prevalent ensemble modeling strategies. The component models are replaced by the first training set (bootstrap tests up to the training set size). Many data points may appear several times in product models, while others may not exist at all. Finally, the outcome is obtained by averaging the component model outputs.

The boosting technique generates a cumulative model, which results in more accurate component development than an individual model. Meanwhile, boosting positions dependent sub-models in the final model using weighted averages of dependent sub-models. The present work used MLPNN, decision trees, and random forests with gradient boosting, as well as boosting and bagging, to predict the mechanical properties of commonly used concrete.

Parameter Tuning for Ensemble Learner

The tuning parameters for the model employed in ensemble procedures can be (i) variables correlated with the ideal number of sample learners and (ii) learning rates and other characteristics that specifically impact ensemble techniques.

In this study, the optimal range of sub-models was determined, bagging and boosting ensemble models (20 each) with 10, 20, 30, ..., 200 component sub-models were constructed for base learners, and correlation with high coefficient values was used to choose the best model. Figure 7 illustrates the relationship between ensemble model accuracy and the number of component sub-models. According to the report, the ensemble model with boosting exhibited a high correlation coefficient with the expected aspect. Additionally, decision trees with bagging and boosting as ensemble models of 20 and 130 sub-models had a substantial relationship, as illustrated in Figure 7a,b. Similarly, ensemble machine learning with 20 and 40 ensemble models demonstrated a significant effect, as illustrated in Figure 7c,d.

Initially, the research showed that using ensemble modeling improved the accuracy of both models. However, employing 200 or more component sub-models for the bagging ML technique and 10 or more for the boosting algorithms yielded more accurate results.

Table 5. Detailed information of the sub-models of ensemble approaches.

Approaches Used	Ensemble Techniques	Machine Learning Methods	Ensemble Models	Optimum Estimator	R-Value
Individual	-	DT	-	-	0.80
	-	MLPNN	-	-	0.77
Ensemble	Bagging	Decision tree- bagging	(10, 20, 30…200)	20	0.87
		Multilayer perceptron neuron network- bagging	(10, 20, 30…200)	20	0.83
Ensemble	Boosting	Decision tree- Adaboost	(10, 20, 30…200)	130	0.89
		Multilayer perceptron neuron network- Adaboost	(10, 20, 30…200)	40	0.80
Modified learner		Random forest	(10, 20, 30…200)	130	0.89
Boosting regressor		Gradient boosting regressor	(10, 20,…30.200)	50	0.84

describes the architectures used in the ensemble models. It is worth noting that the learning rate for the study was 0.9 when both ML ensemble ML methods, bagging, and boosting algorithms, were used.

3.3. 10 K-Fold Cross-Validation Using 10 K-Fold Method

Typically, k-fold cross-validation is used to reduce the random sampling bias associated with training and to hold out data samples. According to Kohavi’s assessment, a ten-fold validation strategy produces reliable variance while consuming the least computing time [81]. The performance of a model that classifies a fixed amount of data into 10 subsets is evaluated using a ten-fold cross-validation procedure. It selects a distinct data subset for testing purposes and trains the selected model using the remaining data subsets in each of the 10 rotations of model development and validation. The test subset is processed to ensure that the algorithm, as displayed in Figure 8, is accurate. The algorithm’s accuracy is then reported as the average of the accuracy gained from ten models across ten validation cycles.

3.4. Evaluation of Models Using a Statistical Measure

To better assess the effectiveness of the individual and ensemble learner prediction models, three types of indicators are available, which are as follows:

$$\mathrm{MAE}=\frac{1}{n}{\displaystyle \sum }_{i=1}^n\left|x_i-x\right|$$

(7)

$$\mathrm{MSE}=\frac{1}{n}{\displaystyle \sum }_{i=1}^n{\left(y_{pred}-y_{ref}\right)}^2$$

(8)

$$\mathrm{RMSE}=\sqrt{\sum \frac{{\left(y_{pred}-y_{ref}\right)}^2}{N}}$$

(9)

$$\mathrm{RMSLE}=\sqrt{\frac{1}{N}\backslash {\displaystyle \sum }_{i=1}^N(\log \left(y_i+1\right)-\log {\left(y_i+1\right)}^2 }$$

(10)

4. Model Result

4.1. The Outcome of the DT Model

The forecasted outcome of compressive strength of concrete from a supervised machine-learning algorithm produced an effective result, as shown in Figure 9. Moreover, the DT algorithm was modeled with ensemble algorithms. The anticipated result from the DT approach, without the use of ensemble approaches, demonstrated a strong relationship with an outcome of R² = 0.803 with its error distribution, as shown in Figure 9a,b. The error distribution in Figure 9b shows that the average error of the testing set was approximately 4.41 MPa. In addition, the data shows that 89 percent of the data of the test set had an average error less than 10 MPa. The distribution of error data also shows that around 9 percent of the error lay between 10 to 15 MPa. Furthermore, as shown in Figure 9b, the distribution exhibited a maximum error of 22.84 MPa and a minimum error of 0.036 MPa. Individually, DT was anticipated to be a favorable model; however, when DT was combined with ensemble techniques, it yielded higher accuracy with maximum R², as indicated in Figure 9c–f. Figure 9c,d show the DT with bagging ML approach, together with a regression model and a distribution of errors, respectively. The ensemble-bagging model predicted a positive outcome with a coefficient of correlation (R²) equal to 0.87 and a lower error in the testing data set than the other models, as illustrated in Figure 9c,d. The bagging ensemble algorithm had higher accuracy, showing 95.74 percent error below 10 Mpa, as indicated in Figure 9d. Similarly, the ensemble model with boosting algorithm showed a significant response with R² = 0.89 (Figure 9e), compared to ensemble bagging with R² = 0.87 with less error, as illustrated in Figure 9f. Both the models, after ensembling, produced a positive response showing less error, with an average error of approximately 4.41 MPa for the decision tree model, 3.54 MPa for the decision tree with bagging algorithm, and 2.96 MPa for the Adaboost algorithm. Overall, the bagging algorithm showed 19.72 MPa and the boosting algorithm showed 32.87 MPa model enhancement compared to the individual model. The trend analysis of the models is shown in

Table 6. Statistical evaluation of an individual with ensemble DT model.

Analysis	DT	DT-Bagging	DT-AdaBoost
Average EDT	4.41	3.54	2.96
Minimum EDT	0.036	0.006	0.057
Maximum EDT	22.84	19.82	20.53
Results lies below 10 MPa	84	90	90
Results between 10 MPa and 15 MPa	8	3	1
Results between 15 MPa and 20 MPa	1	1	1
Results between 20 MPa and 25 MPa	1	0	0
Total data in testing data	94	94	94
Average below 10 MPa	89.36	95.74	95.74
Results between 10 MPa and 15 MPa	8.51	3.19	3.19
Results between 15 MPa and 20 MPa	1.06	1.06	1.06
Results between 20 MPa and 25 MPa	1.06	0	0

.

4.2. MLPNN Model Outcomes

MLPNN networks belong to a class of supervised learning in artificial intelligence approaches, and their applications in concrete demonstrate an appreciable relationship between forecasting strength with actual strength. The outcome of the concrete prediction via MLPNN with its error distribution of the testing data with a coefficient of correlation R² = 0.77 is demonstrated in Figure 10a,b. The MLPNN distribution yielded an average error value of 5.24 MPa for the test set, with a minimum error value of 0.09 MPa and a maximum error value of 19.57 MPa. However, a significant enhancement of R² was observed using ensemble bagging and boosting techniques. The MLPNN-bagging algorithm showed R² = 0.833 with an average error of approximately 3.94 Mpa, as shown in Figure 10c,d. Whereas, MLPNN with boosting techniques produced a higher correlation of R² = 0.80 with an average error of approximately 4.42 Mpa, presented in Figure 10e,f. Even though the ensemble model trains itself on the weak learner to obtain a strong learner this ultimately produces a better model [82]. The data analysis of the bagging and boosting algorithm models in terms of errors shows that most of the data points of the predicted model had errors below 10 MPa. The bagging and boosting algorithms demonstrated a 92.55 and 94.68 percent enhancement of the model compared to the individual model without ensemble technique.

Table 7. Statistical results of the individual with ensemble MLPNN model.

Analysis	MLPNN	MLPNN-Bagging	MLPNN-Adaboost
Average of EMLPNN	5.24	3.94	4.42
Minimum of EMLPNN	0.09	0.05	0.02
Maximum of EMLPNN	19.57	22.30	22.77
Result lies below 10 MPa	76	87	89
Result lies between 10 MPa and 15 MPa	14	5	2
Result lies between 15 MPa and 20 MPa	4	1	1
Result lies between 20 MPa and 25 MPa	0	1	2
Total data in testing data	94	94	94
Average result below 10 MPa	80.85	92.55	94.68
Average result between 10 MPa and 15 MPa	14.89	5.31	2.11
Average result between 15 MPa and 20 MPa	4.25	1.06	47.34
Average result between 20 MPa and 25 MPa	0	1.06	4.22

presents the results of the statistical analysis of testing data performed using MLPNN ensemble modeling.

4.3. RF Model Outcome

The RF algorithm is a combination of bagging and random feature target in ensemble machine learning techniques, which is regarded as an efficient and user-friendly approach for producing forecasting models [83]. Figure 11 illustrates the anticipated accuracy level for the supervised learning technique in HPC. It can be seen in Figure 11a that the RF-ensemble model shows a strong correlation of R² = 0.89 with a robust predicted outcome. Additionally, the anticipated outcome of the RF model can also be analyzed using the error distribution, as illustrated in Figure 11b. The error distribution showed an average error value of 3.17 MPa of test data. Additionally, the 96.80 MPa result of the test set indicates that the error was less than 10 MPa, suggesting that it is accurate for nonlinear forecasting of strength for normal concrete.

Table 8. Statistical analysis of RF.

Statistical Analysis	RF
Average EMLPNN	2.89
Minimum EMLPNN	0.06
Maximum EMLPNN	20.39
Entries lies below 10 MPa	91
Entries lies between 10 MPa and 15 MPa	2
Entries lies between 15 MPa and 20 MPa	1
Entries lies between 20 MPa and 25 MPa	0
Total data in testing data	94
Average below 10 MPa	96.80
Average between 10 MPa and 15 MPa	2.12
Average between 15 MPa and 20 MPa	1.06
Average between 20 MPa and 25 MPa	0

contains statistical information on the data used in the random forest regression.

4.4. GB Model Outcome

The response of prediction by GBR is illustrated in Figure 12. The ensemble-based algorithms produced a strong correlation of R² = 0.84 with less error compared to individual learners, as shown in Figure 12b. In addition, the statistical analysis revealed that 88 data points of the testing set showed lower error than 10 Mpa, with a maximum and minimum error of approximately 23.97 and 0 MPa. Moreover, if evaluated in terms of percentage, then 93.61 MPa represents a clearly lower error, as shown in

Table 9. Statistical analysis of GB.

Statistical Analysis	GB
Average EMLPNN	3.59
Minimum EMLPNN	0.00
Maximum EMLPNN	23.97
Entries lies below 10 MPa	88
Entries lies between 10 MPa and 15 MPa	4
Entries lies between 15 MPa and 20 MPa	1
Entries lies between 20 MPa and 25 MPa	1
Total data in testing data	94
Average below 10 MPa	93.61
Average between 10 MPa and 15 MPa	4.25
Average between 15 MPa and 20 MPa	1.06
Average between 20 MPa and 25 MPa	1.06

.

4.5. K-Fold Cross-Validation Approach

It is important to achieve the required accuracy of the models that are being used for the prediction of the models. This is a crucial component for ensuring the effectiveness of anticipated models, and validation is critical in this regard. The process of validation is used to evaluate the accuracy of data by mixing it randomly, and the K-fold validation test was used in this research to evaluate model accuracy. This strategy is used to eliminate the bias associated with randomly picking the training database from the training database. It separates the samples of data obtained from the experimental approach into ten equal subgroups using a random number generator. It absorbs nine out of ten subsets to shape the effective learner, with the remaining subset being used to test the model’s predictions. The process is repeated approximately ten times to achieve the average accuracy of the repetitions performed. Many researchers consider that the use of this ten-fold cross-validation approach is effective in demonstrating both the generality and reliability of a model’s performance [81].

Figure 13 depicts the error measurement-based validation test of the nonlinear model and ensemble model (RF and GBR) for error estimation. The model validation is checked by different error indicators, including mean absolute error (MAE), root mean square error (RMSE), and root mean square log error (RMSLE), mean absolute error (MAE), and root mean square log error (RMSE), as presented in Figure 13. Even though there were fluctuations reported in the results, the accuracy remained high, as demonstrated in Figure 13a–f. The result of K-fold cross-validation for the decision tree model via MAE, RMSE, and RMSLE produced an error below 10MP, as shown in Figure 13a–f and

Table 10. Result of K-Fold cross-validation for ensemble models.

Folds	DT-Bagging			DT-Boosting
	MAE	RMSLE	RMSE	MAE	RMSLE	RMSE
1	7.812	0.173	8.822	7.259	0.159	8.665
2	7.156	0.134	7.950	8.502	0.117	9.150
3	3.970	0.033	5.746	4.854	0.0429	7.342
4	5.316	0.075	8.306	3.620	0.0422	4.943
5	4.850	0.058	7.441	4.865	0.0519	5.970
6	5.308	0.079	7.289	4.761	0.0413	6.305
7	2.924	0.034	4.906	2.667	0.0328	3.946
8	9.307	0.103	11.095	6.800	0.0526	8.433
9	7.684	0.124	11.759	5.761	0.0878	8.926
10	6.622	0.085	9.458	6.495	0.0659	8.018

, respectively. Similarly, the MLPNN with bagging and Adaboost showed the same trend by showing minimal errors (see

Table 11. K-Fold cross-validation result of ensemble models.

Folds	MLPNN-Bagging			MLPNN-Boosting
	MAE	RMSLE	RMSE	MAE	RMSLE	RMSE
1	5.249	0.125	8.200	5.255	0.109	6.294
2	7.378	0.117	8.200	6.319	0.091	8.234
3	5.721	0.070	6.598	6.537	0.059	8.834
4	3.888	0.077	5.832	3.872	0.043	5.124
5	7.794	0.074	11.817	7.075	0.077	8.093
6	4.412	0.042	5.840	4.45	0.031	6.428
7	2.693	0.019	2.940	2.242	0.046	3.358

). This was due to their lower bias in predicting a model. Moreover, random forest regression and gradient boosting regression showed a strong correlation in terms of k-fold cross-validation. It can be seen in Figure 13e–f and

Table 12. K-Fold cross-validation result of ensemble models.

Folds	RF-Bagging			GBR-Boosting
	MAE	RMSLE	RMSE	MAE	RMSLE	RMSE
1	6.136	0.151	8.197	7.628	0.126	9.229
2	7.274	0.117	8.158	8.614	0.137	10.723
3	1.511	0.010	2.032	6.008	0.072	9.055
4	4.492	0.038	5.196	7.217	0.142	14.321
5	4.483	0.056	5.807	5.500	0.067	6.347
6	4.769	0.054	5.555	5.497	0.056	6.783
7	2.688	0.031	4.175	5.799	0.081	7.901
8	6.420	0.051	8.628	5.774	0.054	10.006
9	6.059	0.094	9.990	7.432	0.131	9.404
10	6.016	0.053	7.719	5.215	0.044	6.912

that the error indicators for model validation showed less error than 10 MPa.

4.6. Results Evaluation of the Employed Models

Model evaluation of the ensemble model with individual algorithms was also conducted using statistical analysis through various error indicators. It can be seen in

Table 13. Statistical evaluation.

Approaches Use	ML Methods	MAE	MSE	RMSE	MSLE
Individual learner	Decision tree	5.40	55.70	7.46	0.052
	Multilayer perceptron neuron network	4.57	37.34	6.11	0.049
	Decision tree-bagging	4.19	34.51	5.87	0.034
Ensemble learning bagging	Multilayer perceptron neuron network- bagging	4.41	33.49	5.78	0.043
	Decision tree-Adaboost	3.53	24.28	4.92	0.029
Ensemble learning boosting	Multilayer perceptron neuron network- Adaboost	4.39	39.29	6.26	0.045
Modified Ensemble	Random forest	3.26	22.26	4.71	0.026
Boosting ensemble	Gradient boosting	4.11	33.60	5.79	0.042

that individual models of DT and MLPNN exhibited higher errors, as compared to their ensemble approaches. This means that ensemble methods not only provide accurate predictions but also help narrow the error range between anticipated and actual findings.

5. Limitations and Future Work

This study considered the vast variety of ML approaches and their comparative performance in terms of predicting the strength property of HPC. However, it does have certain limitations. The data included is indispensable for the efficiency of the forecasted models. The range of the database employed in this study was limited to 471. In addition, this research was also limited to compressive strength prediction and did not consider the flexural and corrosive behavior of concrete at high temperatures. Indeed, proper database and testing must be applied as they are vital elements for engineering applications. This study was based on a wide range of data sets with eight input variables; however, the database and input parameters can be increased to generate a better response of the employed models.

It is recommended that additional variables, such as elevated temperature, humidity, and other environmental conditions, are added as input variables to evaluate the performance of the model for the prediction of required outcomes. As concrete properties are severely affected by environmental conditions, the impact under different conditions should be explored with other deep ML algorithms, such as the convolutional neural network (CNN) and the recurrent neural network, and using a restricted Boltzmann machine (RBM).

6. Conclusions

The objective of this paper was to use and analyze the accuracy of individual learners (MLPNN, DT) with ensemble learners, using techniques such as bagging, boosting and Python Anaconda. In this approach, the data was trained and tested and a high correlation was obtained between predictions and targets. The following conclusions are drawn from our analysis:

• The result of individual learners, DT and ANN, showed a strong correlation between predictions and targets with R² = 0.80 and R² = 0.77, respectively. However, ensemble learner with bagging and boosting and mostly boosting with Adaboost for DT outburst from the individual learner produced a stronger correlation R² = 0.899, and ANN with bagging also produced a stronger correlation R² = 0.833.
• Optimization of ensemble models was conducted with 20 models ranging from 10 to 200 estimators (sub-models). A decision tree with boosting (ensemble = 130) and random forest (ensemble = 130) provided a robust strong correlation with R² = 0.89.
• It is evident that using ensemble learner with a weak learner showed less average error compared to the individual learner. Furthermore, K-fold cross-validation was used to validate models with coefficients of correlation, mean square error, and root mean square error. All the models had low MAE and RMSE errors and a high correlation R². Fluctuations in validation were noticed, using K-fold validation to acquire data in steps and then performing validation on unknown data.
• Statistical analysis was also performed by means of MAE, MSE, RMSE, and MSLE. All ensemble learners produced less error compared to the individual learner, with random forest bagging giving a lesser error compared with MAE, MSE, RMSE, and MSLE. RF and AdaBoost are supervised learning algorithms that yielded strong relationships between prediction and targets.