Mechanical Performance Prediction for Sustainable High-Strength Concrete Using Bio-Inspired Neural Network

: High-strength concrete (HSC) is a functional material possessing superior mechanical performance and considerable durability, which has been widely used in long-span bridges and high-rise buildings. Unconﬁned compressive strength (UCS) is one of the most crucial parameters for evaluating HSC performance. Previously, the mix design of HSC is based on the laboratory test results which is time and money consuming. Nowadays, the UCS can be predicted based on the existing database to guide the mix design with the development of machine learning (ML) such as back-propagation neural network (BPNN). However, the BPNN’s hyperparameters (the number of hidden layers, the number of neurons in each layer), which is commonly adjusted by the traditional trial and error method, usually inﬂuence the prediction accuracy. Therefore, in this study, BPNN is utilised to predict the UCS of HSC with the hyperparameters tuned by a bio-inspired beetle antennae search (BAS) algorithm. The database is established based on the results of 324 HSC samples from previous literature. The established BAS-BPNN model possesses excellent prediction reliability and accuracy as shown in the high correlation coefﬁcient (R = 0.9893) and low Root-mean-square error (RMSE = 1.5158 MPa). By introducing the BAS algorithm, the prediction process can be totally automatical since the optimal hyperparameters of BPNN are obtained automatically. The established BPNN model has the beneﬁt of being applied in practice to support the HSC mix design. In addition, sensitivity analysis is conducted to investigate the signiﬁcance of input variables. Cement content is proved to inﬂuence the UCS most signiﬁcantly while superplasticizer content has the least signiﬁcance. However, owing to the dataset limitation and limited performance of ML models which affect the UCS prediction accuracy, further data collection and model update must be implemented.


Introduction
High-strength concrete (HSC) is a type of cementitious material that has uniaxial compressive strength (UCS) larger than 40 MPa [1][2][3]. The HSC composite exhibit outstanding mechanical strength, considerable durability, low permeability, and compact density. In addition, it satisfies special uniformity and performance requirements, which is superior to ordinary fabricated concrete [4][5][6]. HSC has been widely applied in long-span bridges because it sustains superior dead and live loading with fewer bridge piers and thus WCA) was developed for the prediction of the compressive strength of concrete [62]. The grey wolf optimizer (GWO) was implemented with ELM to predict the compressive strength of concrete with partial replacements for cement [63]. It also successfully predicts the behaviour of channel shear connectors in composite floor systems at different temperatures [64]. A Support Vector Machine (SVM) coupled with Firefly Algorithm (FFA) was performed for the shear capacity estimation of angle shear connectors [65]. The Beetle Antennae Search (BAS) is another feasible meta-heuristic algorithm to tune BPNN architecture with fast convergence, stability in local optimization and uncomplicated implementation [41,66,67]. Therefore, BAS algorithm is chosen to tune the hyperparameters of BPNN. Some robust optimisers are also proposed recently such as adaptive hybrid evolutionary firefly algorithm (AHEFA), hybrid differential evolution and symbiotic organisms search (HDS), and evolutionary symbiotic organisms search algorithm (ESOS) [68][69][70].
In this study, the focus is on predicting the UCS of HSC using BAS-BPNN and understanding the sensitivity ranking of varying influencing factors upon the strength performance of HSC. Different from the traditional ML models, this study develops a novel ML model comprising BPNN and BAS architectures based on a total of 324 experiment data from the literature. The BAS algorithm possesses fast convergence which is beneficial to analysis on the basis of a large database. This pioneering research supplies a novel method to predict the mechanical strength of HSC for advanced engineering construction and application.

Dataset
A total of 324 HSC data samples are collected from previous literature [71] (listed in the Appendix A). Type 1 ordinary Portland cement (OPC) is used as binder material. Silica sand is incorporated as fine aggregate (FA) and the gravel with the size less than 20 mm is served as coarse gravel aggregate (CA). A polycarboxylate-based superplasticizer (SP) with a density of 1.06 g/cm 3 is also introduced for adjusting the cement fluidity and segregation performance.
The specific statistics of the input and output variables are summarised in Table 1 based on the database (Appendix A). All the five influencing variables comprise the content of cement, fine and coarse aggregates, water, and SP. The correlation coefficient distribution is computed, as shown demonstrated in Figure 1. According to the result, the UCS is highly correlated with cement. For input variables, most of the correlations are relatively low (less than 0.5), suggesting that these variables will not produce multicollinearity problems [72][73][74].

BPNN
The artificial neural network (ANN) is one of the commonly used machine learning models, which comprises many categories such as recurrent neural networks (RNN) and feedforward neural network (FFNN). The FFNN includes the Back-propagation neura network (BPNN), which is widely employed to solve problems in the field of building materials and construction [42,75,76]. Back propagation (BP) is a popular approach to ad just the weights and bias of the model, which is composed of an input layer, one or mor hidden layers, and one output layer. The BP process will compare the actual outputs and predicted outputs to obtain the optimal weight and threshold values of the network. The output (O) of a neuron is computed as follow where w j represents the weight value of the jth input neuron (x j ) in the previous layer; b is the bias value of the output neuron; f denotes the activation function. In this study, the following active function was used mainly due to its superior performance [75]: In the backpropagation process, the method computes the gradient of the error func tion with respect to the weights of the neural networks. The training iteration will stop when the mean square error (MSE) between the actual and predicted outputs become smaller than a defined threshold. The topology of the backpropagation process is shown in Figure 2.

BPNN
The artificial neural network (ANN) is one of the commonly used machine learning models, which comprises many categories such as recurrent neural networks (RNN) and feedforward neural network (FFNN). The FFNN includes the Back-propagation neural network (BPNN), which is widely employed to solve problems in the field of building materials and construction [42,75,76]. Back propagation (BP) is a popular approach to adjust the weights and bias of the model, which is composed of an input layer, one or more hidden layers, and one output layer. The BP process will compare the actual outputs and predicted outputs to obtain the optimal weight and threshold values of the network. The output (O) of a neuron is computed as follow where w j represents the weight value of the jth input neuron (x j ) in the previous layer; b is the bias value of the output neuron; f denotes the activation function. In this study, the following active function was used mainly due to its superior performance [75]: In the backpropagation process, the method computes the gradient of the error function with respect to the weights of the neural networks. The training iteration will stop when the mean square error (MSE) between the actual and predicted outputs become smaller than a defined threshold. The topology of the backpropagation process is shown in Figure 2.

BAS
The BAS algorithm is a recently proposed metaheuristic optimization algorithm [77] It is inspired by the hunting behavior of the longhorn beetle with its two long antennae The beetle gradually moves to the food source (the global optimum). Therefore, the con centration of odour is represented by the objective function at position x. In a multi-di mensional space, the global optimum (source point) lies in the position with the best ob jective value. The beetle's searching behaviour is given by: where and represent the areas in the right-hand side and left-hand side, respec tively; is the position at an th time instant. denotes the length of the beetle's an tennae at ith iteration. denotes a unit vector that is randomly normalized, which is ex pressed as b = rnd(k, 1) ‖rnd(k, 1)‖ (5 where k denotes the dimensionality of the position; rnd(·) is a random function The beetle's detecting behaviour is determined using the following equation: where sign(·) is the sign function; represents the step size at the ith iteration, which is updated using the following formula: where is the attenuation coefficient of the step size. The flowchart of BAS is shown in Figure 3 and the pseudocode of tuning hyperpa rameters of BPNN using BAS is presented in Figure 4.

BAS
The BAS algorithm is a recently proposed metaheuristic optimization algorithm [77]. It is inspired by the hunting behavior of the longhorn beetle with its two long antennae. The beetle gradually moves to the food source (the global optimum). Therefore, the concentration of odour is represented by the objective function at position x. In a multidimensional space, the global optimum (source point) lies in the position with the best objective value. The beetle's searching behaviour is given by: where x r and x l represent the areas in the right-hand side and left-hand side, respectively; x i is the position at an ith time instant. d i denotes the length of the beetle's antennae at ith iteration. b denotes a unit vector that is randomly normalized, which is expressed as b = rnd(k, 1) rnd(k, 1) where k denotes the dimensionality of the position; rnd(·) is a random function The beetle's detecting behaviour is determined using the following equation: where sign(·) is the sign function; δ i represents the step size at the ith iteration, which is updated using the following formula: where η is the attenuation coefficient of the step size. The flowchart of BAS is shown in Figure 3 and the pseudocode of tuning hyperparameters of BPNN using BAS is presented in Figure 4.

Performance Evaluation
In this study, Root-mean-square error (RMSE) and Correlation coefficient (R) are used to evaluate the performance of the proposed model. RMSE and R are calculated as follows

Performance Evaluation
In this study, Root-mean-square error (RMSE) and Correlation coefficient (R) are used to evaluate the performance of the proposed model. RMSE and R are calculated as follows

Performance Evaluation
In this study, Root-mean-square error (RMSE) and Correlation coefficient (R) are used to evaluate the performance of the proposed model. RMSE and R are calculated as follows where n denotes the number of data samples; y * i is the predicted value; y i represents the actual value; where and are the mean value of predicted and observed values, respectively.

Determination of Architecture of BPNN
The hidden layer and the number of neurons in each hidden layer are optimised using BAS in this study. To tune these hyperparameters, 10-fold cross validation (CV) was performed in the training set ( Figure 5). The training set is divided into 10 folds, in which 9 folds are used to tune the number of neurons by BAS, and the performance of the BPNN model with the optimal architecture is validated in the remaining fold. After repeating 10 times (for each time, a different fold is selected as the validation fold), the average neuron number is selected as the final neuron number used in this study. Finally, 30% of the data in the test set are used to test the performance of the BPNN with optimal architecture.
where n denotes the number of data samples; * is the predicted value; represe actual value; where and are the mean value of predicted and observed values, respectively.

Determination of Architecture of BPNN
The hidden layer and the number of neurons in each hidden layer are optimis ing BAS in this study. To tune these hyperparameters, 10-fold cross validation (CV performed in the training set ( Figure 5). The training set is divided into 10 folds, in 9 folds are used to tune the number of neurons by BAS, and the performance of the model with the optimal architecture is validated in the remaining fold. After repea times (for each time, a different fold is selected as the validation fold), the average n number is selected as the final neuron number used in this study. Finally, 30% of th in the test set are used to test the performance of the BPNN with optimal architectu

Results of Hyperparameter Tuning
In this study, the number of neurons in each layer is tuned using the BAS algo In each fold, the RMSE obtained by the BPNN (with optimal neuron number of thi is plotted in Figure 6. The smallest RMSE values versus iterations corresponding to ing hidden layers are shown in Figure 7, which presents the process of neuron n tuning. It can be seen that the RMSE decreases to its minimum value within 40 iter suggesting that BAS has high efficiency in finding the optimal number of neuron mately, the final hidden layer is 1 and the corresponding optimal neuron number i

Results of Hyperparameter Tuning
In this study, the number of neurons in each layer is tuned using the BAS algorithm. In each fold, the RMSE obtained by the BPNN (with optimal neuron number of this fold) is plotted in Figure 6. The smallest RMSE values versus iterations corresponding to varying hidden layers are shown in Figure 7, which presents the process of neuron number tuning. It can be seen that the RMSE decreases to its minimum value within 40 iterations, suggesting that BAS has high efficiency in finding the optimal number of neurons. Ultimately, the final hidden layer is 1 and the corresponding optimal neuron number is 24.   Figure 8 shows the actual values (blue line), predicted values (red point), and errors between the actual and predicted UCS (yellow bar graph). It can be observed that although several large noises are observed, most of the errors are pretty small on the training set ( Figure 8a) and test set ( Figure 8b). This result indicates that the BAS-BPNN model is highly accurate. The correlation between the actual and predicted UCS is visualized in Figure 9. High prediction accuracy is observed on the training set ( Figure 9a) and test set (Figure 9b), as indicated by the high R values (0.9971 and 0.9893 on the training and test sets, respectively) and low RMSE values (0.7167 MPa and 1.5158 MPa on the training and test sets, respectively). Compared with previously published papers [42,51], the obtained results show much higher accuracy (R is around 0.99), which might be attributed to the model performance or the accuracy and size of the database. Furthermore, no overfitting problems take place as the test set RMSE (and R) is close to that on the training set. Owing to the inherent stochastic properties of the BAS algorithm, the statistical outcomes of extra 20 run times are also reported in Table 2 to verify the robustness of the introduced ML model.   Figure 8 shows the actual values (blue line), predicted values (red point), and errors between the actual and predicted UCS (yellow bar graph). It can be observed that although several large noises are observed, most of the errors are pretty small on the training set ( Figure 8a) and test set (Figure 8b). This result indicates that the BAS-BPNN model is highly accurate. The correlation between the actual and predicted UCS is visualized in Figure 9. High prediction accuracy is observed on the training set ( Figure 9a) and test set (Figure 9b), as indicated by the high R values (0.9971 and 0.9893 on the training and test sets, respectively) and low RMSE values (0.7167 MPa and 1.5158 MPa on the training and test sets, respectively). Compared with previously published papers [42,51], the obtained results show much higher accuracy (R is around 0.99), which might be attributed to the model performance or the accuracy and size of the database. Furthermore, no overfitting problems take place as the test set RMSE (and R) is close to that on the training set. Owing to the inherent stochastic properties of the BAS algorithm, the statistical outcomes of extra 20 run times are also reported in Table 2 to verify the robustness of the introduced ML model.  Figure 8 shows the actual values (blue line), predicted values (red point), and errors between the actual and predicted UCS (yellow bar graph). It can be observed that although several large noises are observed, most of the errors are pretty small on the training set ( Figure 8a) and test set (Figure 8b). This result indicates that the BAS-BPNN model is highly accurate. The correlation between the actual and predicted UCS is visualized in Figure 9. High prediction accuracy is observed on the training set ( Figure 9a) and test set (Figure 9b), as indicated by the high R values (0.9971 and 0.9893 on the training and test sets, respectively) and low RMSE values (0.7167 MPa and 1.5158 MPa on the training and test sets, respectively). Compared with previously published papers [42,51], the obtained results show much higher accuracy (R is around 0.99), which might be attributed to the model performance or the accuracy and size of the database. Furthermore, no overfitting problems take place as the test set RMSE (and R) is close to that on the training set. Owing to the inherent stochastic properties of the BAS algorithm, the statistical outcomes of extra 20 run times are also reported in Table 2 to verify the robustness of the introduced ML model.

Variable Importance
Global sensitivity analysis (GSA) is combined with the developed BPNN model to analyse the variable importance ( Figure 10). It can measure the impact on the proposed BAS-BPNN output when the input value changes within its value range [79]. The data sample is represented as x, and x a , a ∈ {1, . . . , M} denotes an input variable through its range with L levels (M is the number of input variables). And y represents the UCS value which is predicted by the BPNN. According to the range of x a and L levels, the input variable x a can be divided into i values, namely, x ai , i = {1, . . . , L}. The respective sensitivity response of each input variable is calculated by Equation (10). Afterward, the relative importance of each variable is calculated by Equation (11).
where a is the input variable that needs to be analysed;ŷ a,i , i = {1, . . . , L} stands for the sensitivity response indicator for x ai , i = {1, . . . , L}; R a is the relative importance of the variable. It can be observed that UCS of HSC is the most sensitive to contents of cement and water with importance ratios of 44.9% and 34.9%, respectively. This is mainly duo to the water-to-cement ratio, which is crucial to the development of concrete strength. It is interesting to note that superplasticiser (importance ratio = 2.7%) is not as important as other influencing variables. This may be caused by insufficient content of superplasticiser in the concrete mixtures. It is worthwhile to note that the importance of input variables is calculated on the basis of the data set collected in this study, as listed in the Appendix A.
BAS-BPNN output when the input value changes within its value range [79]. The data sample is represented as x, and x , ∈ {1, … , } denotes an input variable through its range with L levels ( is the number of input variables). And represents the UCS value which is predicted by the BPNN. According to the range of x and L levels, the input variable x can be divided into values, namely, x , = {1, … , }. The respective sensitivity response of each input variable is calculated by Equation (10). Afterward, the relative importance of each variable is calculated by Equation (11).
where is the input variable that needs to be analysed; , , = {1, … , } stands for the sensitivity response indicator for x , = {1, … , }; is the relative importance of the variable.
It can be observed that UCS of HSC is the most sensitive to contents of cement and water with importance ratios of 44.9% and 34.9%, respectively. This is mainly duo to the water-to-cement ratio, which is crucial to the development of concrete strength. It is interesting to note that superplasticiser (importance ratio = 2.7%) is not as important as other influencing variables. This may be caused by insufficient content of superplasticiser in the concrete mixtures. It is worthwhile to note that the importance of input variables is calculated on the basis of the data set collected in this study, as listed in the Appendix A.

Comparison of the BAS-BPNN Model with Other ML Models
To seek the optimal ML model and further verify the strength of the established BAS-BPNN model in the prediction of UCS of HSC, its prediction performance is compared with several widely used ML models [80]: Support vector machine (SVM), random forest (RF), K-nearest neighbours (KNN), logistic regression (LR), and multiple-linear regression (MLR). Among these models, the hyperparameters of SVM, RF, and KNN are also tuned by BAS. The tuned hyperparameters with their empirical scopes, initial values, and final values are listed in Table 3. The hyperparameter tuning process of these models is shown in Figure 11. It can be seen that all RMSE curves can converge within 50 iterations, indicating the high searching efficiency of the BAS algorithm. In the first 20 iterations, the RMSE obtained by SVM decreases less significantly in comparison with that obtained by other ML models. This implies the initial hyperparameters of SVM are close to the optimal hyperparameters.  Table 3. The hyperparameter tuning process of these models is shown in Figure 11. It can be seen that all RMSE curves can converge within 50 iterations, indicating the high searching efficiency of the BAS algorithm. In the first 20 iterations, the RMSE obtained by SVM decreases less significantly in comparison with that obtained by other ML models. This implies the initial hyperparameters of SVM are close to the optimal hyperparameters.  Figure 11. Hyperparameter tuning by different models on the training set.
The prediction errors of different ML models are compared on the test set using a boxplot, as shown in Figure 12. The lower edge of the box represents the first quartile, and the upper edge is the third quartile. The median is demonstrated as a red line in the box. The lower and upper whiskers are the 1.5 IQR minus the first quartile and 1.5 IQR above the third quartile, respectively (IQR is the interquartile range). All the other data points are defined as outliers in this study. It can be observed that BPNN has the smallest third quartile, indicating that most of the errors obtained by BPNN are relatively small. Although few outliers were observed in BPNN, the general prediction performance was the best among these ML models. The advantage of BAS-BPNN is also verified by comparing different ML models using a Taylor plot that shows in Figure 13, indicating three model evaluation indices (standard deviation, RMSE, and R). The ML model will be the most The prediction errors of different ML models are compared on the test set using a boxplot, as shown in Figure 12. The lower edge of the box represents the first quartile, and the upper edge is the third quartile. The median is demonstrated as a red line in the box. The lower and upper whiskers are the 1.5 IQR minus the first quartile and 1.5 IQR above the third quartile, respectively (IQR is the interquartile range). All the other data points are defined as outliers in this study. It can be observed that BPNN has the smallest third quartile, indicating that most of the errors obtained by BPNN are relatively small. Although few outliers were observed in BPNN, the general prediction performance was the best among these ML models. The advantage of BAS-BPNN is also verified by comparing different ML models using a Taylor plot that shows in Figure 13, indicating three model evaluation indices (standard deviation, RMSE, and R). The ML model will be the most realistic if the distance between the ML model and the point labelled "Actual" is the shortest. It can be seen that BPNN is the closest to the "Actual point", suggesting BPNN performs better in terms of standard deviation, correlation coefficient, and RMSE. Generally, the boxplot and Taylor plot present a similar phenomenon, ranking the accuracy of ML models as BP, SVM, RF, MLR, LR, and KNN. This is controlled according to the model complexity and database suitability. According to the "no free lunch" (NFL) theorem of machine learning, there is no single model that performs universally superior to other models for any dataset. Therefore, based on the dataset used in this study, BPNN is the optimal prediction model. the boxplot and Taylor plot present a similar phenomenon, ranking the accuracy of ML models as BP, SVM, RF, MLR, LR, and KNN. This is controlled according to the model complexity and database suitability. According to the "no free lunch" (NFL) theorem of machine learning, there is no single model that performs universally superior to other models for any dataset. Therefore, based on the dataset used in this study, BPNN is the optimal prediction model.

Conclusions
In this study, the BPNN model with the BAS algorithm being used to tune the hyperparameters was established to predict the UCS of HSC. The proposed BAS-BPNN model was developed based on a collected dataset containing over 300 HSC samples with the boxplot and Taylor plot present a similar phenomenon, ranking the accuracy of ML models as BP, SVM, RF, MLR, LR, and KNN. This is controlled according to the model complexity and database suitability. According to the "no free lunch" (NFL) theorem of machine learning, there is no single model that performs universally superior to other models for any dataset. Therefore, based on the dataset used in this study, BPNN is the optimal prediction model.

Conclusions
In this study, the BPNN model with the BAS algorithm being used to tune the hyperparameters was established to predict the UCS of HSC. The proposed BAS-BPNN model was developed based on a collected dataset containing over 300 HSC samples with

Conclusions
In this study, the BPNN model with the BAS algorithm being used to tune the hyperparameters was established to predict the UCS of HSC. The proposed BAS-BPNN model was developed based on a collected dataset containing over 300 HSC samples with different mixtures. The BPNNs with 1, 2, and 3 hidden layers were compared and the ultimate optimal architecture is one hidden layer with 24 neurons. The results show that BAS has high efficiency in tuning hyperparameters of BPNN and the obtained BAS-BPNN model is highly accurate (R = 0.9893, RMSE = 1.5158 MPa on the test set). Besides, the BAS-BPNN is superior by comparing its prediction performance with other widely used ML models (SVM, RF, KNN, LR, and MLR). In addition, the importance ranking of the input variables through GSA was implemented showing that cement and water are the most significant variables to the UCS of HSC. Generally, the findings in this study can be used in practice to support the HSC mix design.
It is noted that only five input variables are considered in this study, which inevitably influences the diversity and size of the database. Therefore, more samples containing varying raw materials such as fly ash, slags, and other solid wastes will be incorporated in the future to further improve the generalisation ability of the BAS-BPNN model. Also, other active functions, advanced machine learning models, and optimization algorithms (e.g., AHEFA, HDS, and ESOS) can be applied for performance comparison. An Adaptive Neuro-Fuzzy Inference System can be used to determine the most influencing parameters to further verify the findings in this study [81,82].