Predicting the Compressive Strength of Concrete Containing Binary Supplementary Cementitious Material Using Machine Learning Approach

Several advantages of supplementary cementitious materials (SCMs) have led to widespread use in the concrete industry. Many various SCMs with different characteristics are used to produce sustainable concrete. Each of these materials has its specific properties and therefore plays a different role in enhancing the mechanical properties of concrete. Multiple and often conflicting demands of concrete properties can be addressed by using combinations of two or more SCMs. Thus, understanding the effect of each SCM, as well as their combination in concrete, may pave the way for further utilization. This study aims to develop a robust and time-saving method based on Machine Learning (ML) to predict the compressive strength of concrete containing binary SCMs at various ages. To do so, a database containing a mixture of design, physical, and chemical properties of pozzolan and age of specimens have been collected from literature. A total of 21 mix design containing binary mixes of fly ash, metakaolin, and zeolite were prepared and experimentally tests to fill the possible gap in the literature and to increase the efficiency and accuracy of the ML-based model. The accuracy of the proposed model was shown to be accurate and ML-based model is able to predict the compressive strength of concrete containing any arbitrary SCMs at ay ages precisely. By using the model, the optimum replacement level of any combination of SCMs, as well as the behavior of binary cementitious systems containing two different SCMs, can be determined.


Introduction
The increasing demand in consumption of concrete as the second most consumed material in the world, environmental pollution, the need for optimal utilization of materials, and the positive effects of using supplementary cementitious materials (SCMs) on the properties of concrete have led to the widespread use of these materials in the concrete industry. These materials need to have sufficient amorphous aluminosilicates which react with calcium hydroxide in the presence of water to form one or more hydration products: calcium silicate hydrate (C-S-H), calcium aluminate hydrate (C-A-H), and calcium aluminosilicate hydrate (C-A-S-H) [1,2]. Since SCMs have a very small amount of embodied CO 2 which is defined based on the total amount of CO 2 produced in the extraction and transportation of raw materials and their manufacture into the final product, they are susceptible to producing sustainable concrete [3].
There are several SCMs that each have specific properties and therefore play a different role in enhancing the mechanical properties of concrete, for example, Silica fume shows a However, the variation of the mechanical properties of concrete containing ZE [28,29] and reduction in workability of the mixture [30][31][32][33] are the main drawbacks of widespread utilization of this SCM. Prior research related to the use of ZE in ternary systems is scarce and the knowledge of their performance is limited.
The current study aims to provide a model based on an artificial neural network (ANN) to predict the compressive strength of concrete containing two types of pozzolans. For this purpose, 192 data from previous research have been selected carefully and parameters such as mix design, physical and chemical properties of pozzolan, and age of specimens have been considered as influential factors in the compressive strength of ordinary concrete. In addition, in order to increase the efficiency and accuracy of the proposed model, extensive experimental research was conducted to fill the gaps in the literature. A total of 21 OPC mix designs, including binary mixes, were prepared by substantially replacing cement with FA, MK, and ZE. The compressive strength tests were conducted on concrete mixtures. Moreover, the other goal is to optimize the replacement level of SCMs in ordinary concrete with the objective of achieving comparable mechanical properties. Moreover, being aware of the behavior of binary cementitious systems containing two different SCMs is the other main goal of the present study.

Artificial Neural Network
Toward the estimation of the compressive strength of ordinary concrete containing two different types of SCMs, a robust approach called Multi-Layer Perceptron (MLP) is used. One of the main benefits of using MLP is the simplification of the utilization and improvement of the accuracy of results [12,34]. Incomplex operating elements that work alongside make MLP function. In nature, the performance of the human brain, which is a neural network, is regulated by the way in which the components are interconnected [35,36]. Thus, it is feasible to develop a simulated structure like natural networks, and obtain the relation among its components by adjusting the weights of each connection. Subsequently, by adjusting the weights of each connection or in other words, training the neural network, applying a particular input results in a specific output. Minimizing the difference between the output and the real result, i.e., target, is the main objective of training. This is done by changing the weight during the learning process and continuing until the error function is less than the specified limit. Training is a repetitious strategy by initializing the weight values, predicting the output of the network, and calculating the corresponding error. Error is relatively high at the first step, since weights are randomly privileged. The main objective of learning in an ML-based method is the acquisition of the weights that leads to the lowest error range. In most artificial networks, the number of weights is high and so there is no direct method to find the weights [37]. Determining weights by trying and error also wastes time and effort. One of the efficient methods to asset the least sets of errors more quickly during network training is the gradient descent approach. Gradient descent, as the name implies, utilizes the error gradient to reduce the error [38,39]. The error is completely affected by the output of the network, and it depends on the weighted output of the hidden neurons, and it depends on the weights. Therefore, by moving toward the input layer and adjusting the weights, the difference between the output of the network and target results may be reduced. This method is known as backpropagation which is a gradient descent algorithm in which the weights of a network move in the opposite direction to the performance function slope. The hidden neurons can compute their error to adjust the weights according to the error signal [34,40].
The following assumptions can be considered for an MLP network: 1.
Simple elements known as neurons are responsible for the processing of information, 2.
Processed information is passed neuron over connection link, 3.
An associated weight is considered for each connection link, 4.
Inputs are transmitted from a predefined activation function in neurons and outputs are determined.
The configuration of an MLP network, along with the learning algorithm and the activation function applied in each neuron, is defined as a network. Implementation of the neural networks may decrease the number of experiments and save time and cost [41,42].

Dataset
A deep and careful survey of the literature was done in order to develop a network to estimate the compressive strength of the concrete containing two types of SCMs. The dataset includes about 192 samples with 19 distinguished features. The collected dataset contained information about water content (W), cement (C), fine aggregate (F), coarse aggregate (G), binary SCMs (SCM1 and SCM2), the chemical composition of each SCMs including SiO 2 , CaO, Fe 2 O 3 , Al 2 O 3 , MgO, and physical properties of binary SCMs, i.e., specific surface (SS), and age of specimens. The mold of specimens was considered, so the compressive strength was converted to a 150 mm × 300 mm cylinder standard mold. The compressive strength (f c ) was considered as the output of the network. Figure 1a shows the marginal histogram of binary SCMs percentages. It is worth noting that any type of pozzolan is considered a SCM as long as its physical and chemical characteristics are known. As can be seen, the numbers of data points indicating the implementation of SCMs of more than 30% are lower. In other words, the main focus of previous studies was on replacing cement with SCMs by 30% of weight or less. Therefore, the dataset is limited to experiments with the maximum usage of 30% for each SCM. This results in reducing the dataset to 142. It is worth noting that Figure 1 indicates the replacement level in percent, while in the proposed network, the effect of SCMs is considered as a replacement weight to fully cover any arbitrary concrete mix design.
Furthermore, in order to increase the number of data points in the dataset and fill the possible gap in the previous experimental research, numerous studies were performed in the laboratory. The experimental study was designed to determine the effect of SCMs such as MK, ZE, and FA on different percentages of cement replacement. By doing so, the number of data points increased to 226. Thus, the marginal histogram of binary SCMs in the dataset changed into Figure 1b. Table 1 shows statistical parameters for the dataset. A diverse range of SCMs was investigated in the literature along with the current experimental study. The chemical composition of various SCMs are plotted in Figure 2 on a Al 2 O 3 -SiO 2 -CaO ternary figure. As can be seen, a wide range of pozzolanic materials are considered as the influential parameters on the compressive strength of concrete containing binary SCMs.

Material
A type I Portland cement with the chemical composition summarized in Table 2 was used in all mixtures. Clean, well-graded, and natural fine and coarse aggregate with unit weights of 2.61 and 2.68, and water absorption of 1.8 and 1.4%, respectively, were used. Tap water was used for making and curing all concrete samples (Figure 3b). Reaching a constant slump in each mix design demands the utilization of a polycarboxylic acidbased high-range water reducer (Carboxal HF5000). The mixing procedure was performed according to the ASTM C192 [43]. A commercially-available MK with chemical properties as shown in Table 3 was procured for use in this study. The FA produced at DRIK company in accordance with the specification listed in Table 4 was used. Natural ZE was provided by a local manufacturer with the chemical specification shown in Table 5.

Mix Proportions and Test Method
In order to understand the effect of binary pozzolan and enrich the collected dataset, 21 distinct mix designs were considered to have a constant water/binder ratio of 0.45 and total binder content of 350 kg/m 3 . SCMs contain MK+ZE, MK+FA, and FA+ZE, in which a proportion of Portland cement was replaced with the SCMs. The replacement levels for SCMs were up to 50% with 5% intervals. These mix designs are shown in Table 6. The mixture codes were assigned based on the inclusion of pozzolan replacement, i.e., Metakaolin (MK), Zeolite (ZE), and Fly ash (FA). For instance, the mix coded MK5ZE5 was made with 5% metakaolin and zeolite replacement. For each mix design, the compressive strength of concrete was conducted at 3, 7, 28, and 90 days of age. A cylindrical 150 × 300 mm mold was used for the compressive strength test according to ASTM C39 [44] (Figure 3a).

MLP Modeling
Generally, the procedure of indicating a complex real-world event as a combination of mathematical expressions is called modeling [45]. Governing the suitable network configuration in which the lowest error and highest accuracy can be obtained is vital. To do so, a trial and error procedure is utilized to ascertain the optimal number of neurons in the middle layer, which is called a hidden layer. For each network with a specific number of neurons in the hidden layer, the mean squared error (MSE) indicating the performance of the network is calculated 30 times.
The network with the lowest MSE is considered to be an optimum network with a specific number of neurons in the hidden layer. Changing the weights matrix during the training step using an iterative procedure and continuing this until performance reaches the specified goal is the most vital part of network learning. The MSE error in the initial step is relatively high since weights are selected randomly. Finding weights by trial and error that result in the lowest MSE would require a great deal of time and effort [45]. One of the efficient approaches to encountering the least sets of errors within the model learning step is the gradient descent method. Since the error is related to the output of the network, and it depends on the weights, updating weights in each step results in precise outcomes. After some steps, the accuracy of the network for validation data remains constant and the optimal configuration of the network along with its optimum weights matrix will be determined. The optimal configuration of the MLP network, along with its performance, is shown in Figure 4. The most important and effective parameters, such as concrete mix design, physical and chemical properties of both pozzolans, and age, are considered in the proposed network. So long as the MLP network is trained, the compressive strength of concrete containing binary SCMs can be estimated. Training the MLP network was done using the linearly normalized input and by the implementation of the Levenberg-Marquardt (LM) algorithm owing to suitable convergence, high accuracy, and less time consumption [37]. Data is randomly segregated into 3 distinguished parts, namely training, validation, and test. In the proposed MLP network, 70% of data are assigned for training, and two 15% remaining data are considered for validation and test. It was shown that the aforementioned ratio has the best performance [46]. Two commonly used activation functions of TANSIG (y = 1−e −2x 1+e −2x ) and PURELIN (y = x) were used in the hidden and output layer, respectively. Once the desired network performance is obtained, the learning procedure is considered completed.

Mix design
Chemical and Physical properties of SCM1 Chemical and Physical properties of SCM2 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2  The performance of the MLP networks with reference to predicting the compressive strength of concrete containing binary SCMs is shown in Figure 4b. The best validation performance was acquired as 0.0017 at the 27th epoch. The quality of the prediction as a function of the correlation coefficient, R, for all data is demonstrated in Figure 5a, revealing the correlation between the target (experimental f c ) and the MLP network result. The overall response with a correlation coefficient close to 1 verified that the network computed the outcomes with reasonable precision. The comparison of the predicted compressive strength using MLP network (output) and experimental f c (target) along with MSE of target and output is depicted in Figure 5b. It can be concluded that the network is able to estimate the compressive strength of concrete containing binary SCMs with an acceptable error. The histogram of error is plotted in Figure 6. As it is obvious, more than 42% and 94% of data is predicted with an error of less than 2% and 10%, respectively.  The statistical error values for the estimated compressive strength of concrete containing binary SCMs obtained from the MLP network are described as root mean square error (RMSE), Nash-Sutcliffe efficiency (NSE) coefficient, mean absolute percentage error (MAPE), and correlation coefficient (R). The aforementioned statistical measures can be calculated using Equation (1). These statistical indicators, including MSE, are compared in Table 7 according to all data points. Zero or near to zero are ideal values for all statistical parameters, except for NSE and R, while the ideal value for NSE and R is one. RMSE stipulates the deviation between the experimental results and estimated outcomes of the MLP network. Both the estimation error and the ratio of the error to the experimental value are reflected in MAPE [39,47]. Assessment of the estimation capability of the MLP network is determined using the NSE coefficient. The statistical metrics in Table 7 show that the results of the MLP network in estimating the compressive strengths of concrete containing binary SCMs are close to the experimental results in a satisfactory manner. This further validates the acceptability of the proposed MLP model.
where f c andf c are the experimental compressive strengths and estimated outcomes of the MLP network, and thef c andf c parameters are the averages of the experimental and the estimated values, respectively.

Experimental Compressive Strength
The experimental results of the compressive strength of specimens with different percentages of pozzolan at various ages are listed in Table 8. The effect of a binary combination of SCMs on the compressive strength of concrete at the age of 3, 7, 28, and 90 days is depicted in Figure 7. MK is shown to be more beneficial in combination with FA considering 15, 22, and 11% increased compared with the control specimen in MK2.5FA2.5, MK5FA5, and MK7.5FA7.5, respectively. This is in accordance with the results of Grist et al. [48]. As can be seen in Figure 7, all the binary mixtures, regardless of the contained pozzolan, have the same optimum replacement level. The optimal replacement level for binary mix design is determined to be 10% of cement weight. Since the effect of SCMs becomes more pronounced with a rise in time, the improvement in 90-day compressive strength is reported herein. The 90-day compressive strength of concrete specimens containing MK and FA has increased by 38% compared to the control specimen at the same age. The amount of enhancement for concrete containing MK and ZE is 35%, and for concrete with ZE and FA ash is 32%.

Prediction of the Compressive Strength
With a focus on estimating the compressive strength of concrete containing binary SCMs, including physical and chemical properties, an MLP network was trained and its performance was assessed. As results indicated, the network could estimate the compressive strength of concrete containing binary SCMs with suitable accuracy which is sufficient in practical use. One of the main advantages of machine learning approaches is the ability to solve complex problems with numerous affecting parameters, especially in the engineering field. As it is discussed, in the current study, there are 19 affecting parameters on the compressive strength of concrete containing binary SCMs. Finding a suitable, accurate, and time-consuming method to estimate the compressive strength according to the inputs is simple thanks to the machine learning approaches. Furthermore, these approaches, after ensuring their accuracy and performance, can be used to produce new results based on new input parameters. This is called generalization, in which a new dataset (unseen data) that is costly or impossible to experiment is fed into the network and results are estimated using a previously-learned machine learning approach.
Since the main objective of the current study is to predict the compressive strength of concrete containing binary SCMs with various chemical and physical properties, the generalization feature of the MLP network will be used. In order to determine the effect of SCMs replacement level, types, and properties, the percentage of substituting cement with SCMs and their pozzolanic characteristics are considered as variables, and the variation in the compressive strength due to changes in these parameters is determined. Utilizing the MLP network outcomes, a wide range of concrete mixtures can be evaluated. Therefore, the concrete mix design is assumed to be constant and the parameters of the mix design are chosen to be around their median [37]. The assumed mix design is summarized in Table 9. It is worth noting that the proposed MLP network is able to estimate the compressive strength of concrete containing any known or unknown pozzolanic material. In other words, since the proposed network determines the strength of concrete by using the physical and chemical properties of pozzolans, it can also estimate the compressive strength of concrete containing an SCM that may be introduced in the future. In the first step, in order to determine the effect of replacement level, 6 common and well-known SCMs are used to develop new results. The physical and chemical properties of these SCMs are listed in Table 10. The replacement level of SCMs was presumed to be between 2.5 and 30% with 2.5% intervals. In addition, the age of specimens is considered to be at 56 days. The results of MLP network prediction are depicted in Figure 8.  As can be seen, the general trend of variation in the compressive strength of concrete containing binary SCMs indicates that there is an optimum level of replacement for each specific combination of pozzolans. For instance, in a concrete mixture with FA and MK replacement, the maximum compressive strength may be obtained for an FA replacement level of less than 20% and an MK replacement level of less than 12%. Moreover, the effect of an increase in the FA replacement is more significant than the MK percentage. Figure 8b shows the changes in the compressive strength of concrete made with a combination of FA and RA. As can be seen, the effect of RA replacement level in improving the compressive strength is less than FA. The main reason for this trend may be attributed to the higher pozzolanic reactivity of FA compared with RA. The optimum level for this combination of SCMs is around 20% for FA and 2.5 to 20% for RA. This is in accordance with the results of [49][50][51] that consider the improvement as a result of the synergic effect of using binary pozzolans.
According to Figure 8c, it can be said that both SCMs, i.e., FA and SF, are almost equally effective in improving the compressive strength of concrete. The higher reactivity of SF, due to higher surface area and higher amount of SiO 2 , leads the compressive strength to be further improved in the combination of FA-SF concrete. Higher percentage levels of SF in the presence of the lower amount of FA replacement increases the compression strength of concrete, which is in accordance with the experimental results obtained by [15,52]. For concrete specimens containing FA and SL (Figure 8d), the highest compressive strength is obtained for the replacement level of pozzolans lower than 30%. Moreover, it can be concluded that the pozzolanic effect of both FA and SL is almost the same since the compressive strength in the replacement levels of less than 15% has lower fluctuation. Almost the same results were observed in the experiments of Jeong et al. [24], in which the combined effect of FA and SL at various replacement levels were investigated. It was shown that increasing replacement levels of SL lead to neglectable changes in the compressive strength.
The combined effect of using ZE with other SCMs is rarely studied. In Figure 8e, the effect of implementation of a binary combination of FA-ZE was evaluated using the results of the MLP network. As can be seen, there is an optimum level of replacement in order to reach the maximum compressive strength. The optimal replacement level for both SCMs is around 10% of cement weight. The experimental results (Figure 7c) indicate the suitable replacement level as 5% for each SCMs. The reason for incompatibility may be attributed to the different physical and chemical properties of FA considered in experimental and machine learning approach. However, the fact that there is an optimum replacement level is observed in both experimental and machine learning methods.
The other general trend in Figure 8 is a reduction in the compressive strength in higher replacement levels. In almost all the cases, the minimum compressive strength occurs in 30% replacement level for two studied SCMs. Adding higher amounts of SCMs may result in the dilution effect, which is a reduction in the hydration reaction owing to the lack of sufficient cement content in the mixture [45,53].
In order to understand the effect of the chemical composition of SCMs on the compressive strength of concrete, several predictions are conducted using the proposed MLP network. For better comparison, in these predictions, the concrete mix design, replacement level, and physical characteristics of SCMs are assumed to be constant (Table 11). The mix design is the same as Table 10, the replacement level for both SCMs is considered to be 10%, and the second pozzolan used in the mixture is assumed to be SF and constant during the generalization. Moreover, since the summation of chemical composition of a SCMs need to be 100%, those combinations of SiO 2 , CaO, and Al 2 O 3 which exceed 93.12%, i.e., (100% − (Fe 2 O 3 + MgO)), is omitted from the generalization outcomes. This may result in 5770 distinguished mix designs at various ages. Changes in the compressive strength of concrete containing binary SCMs against the chemical properties of pozzolans are shown in Figure 9. The empty area in this figure indicates the impossible outcome of the unseen data, i.e., the summation of SiO 2 , CaO, and Al 2 O 3 exceeds 93.12%. As can be seen, an increase in the age of concrete specimens results in an enhancement in the compressive strength of concrete. This obvious trend once again validates the performance of the MLP network and demonstrates its accuracy in predicting unseen data.  As can be seen from Figure 9a-d, an increase in the amount of SiO 2 and CaO and at the same time reduction in the percentage of Al 2 O 3 results in an enhancement in the compressive strength of concrete containing SCMs. This is in accordance with the experimental results of Kasaniya et al. [2]. In their tests, it was shown that level of reactivity of FA depends on the amount of SiO 2 +CaO+Al 2 O 3 as well as the particle size of SCMs. Furthermore, the regions in Figure 9 with higher compressive strength indicate Class C fly ashes. The comparison between Class F and Class C fly ashes was done by several researchers [54][55][56]. The results of their research confirm the reliability of the outcomes of the MLP network in predicting the compressive strength of concrete containing binary SCMs. Moreover, time plays a vital role in highly reactive pozzolans. In other words, an increase in the compressive strength of concrete containing a highly reactive pozzolan occurs at a higher rate compared with pozzolans with a lower amount of SiO 2 . The same trend can be observed from Figure 9a-d.

Developing a Software to Predict the Results
One of the most suitable and simplest ways to use the results of a machine learning method in practice is to implement the weights obtained from the network in a numerical system and in the form of user-friendly software. This may be achieved by using a graphical user interface (GUI) in the Matlab environment. With such an approach and using the developed software, there is no need to perform complex and time-consuming calculations, and by implementing the optimal weights obtained from the network, the results can be estimated with appropriate accuracy. This may help engineers to achieve the results without conducting experimental tests or computing numerous complex equations. Figure 10 demonstrates the main GUI. As can be seen, the compressive strength of concrete containing binary SCMs at any age between 3 to 365 can be estimated by considering the concrete mix design, along with the chemical composition of pozzolans.

Conclusions
The current experimental study was carried out to fill the existing gap in the literature on the evaluation of the compressive strength of concrete containing binary SCMs. The effect of various replacement levels of three different common pozzolans, namely Metakaolin, Zeolite, and Fly ash in concrete mixtures, was done. It was shown that the optimal replacement level for binary mix design is determined to be 10% of cement weight.
In addition, an accurate and comprehensive database of previous research on the effect of using binary pozzolans on the compressive strength of concrete was collected. The database contains 19 important factors on the compressive strength of concrete containing binary SCMs. Using the MLP method, an attempt was made to develop a comprehensive, reliable, and accurate model for predicting the compressive strength of concrete containing binary SCMs. The accuracy of the model in terms of MSE was 0.0017 for validation data. Furthermore, more than 42% and 94% of data were predicted with an error of less than 2% and 10%, respectively.
By ensuring the accuracy of the proposed model, the unseen results based on the generalization technique were obtained. The effect of various combinations of SCMs with any arbitrary chemical composition in any age between 3 to 365 days can be predicted with high accuracy using the MLP network proposed in this paper. In order to show the capability of the MLP network, several simulations were done and the results were compared with the proven fact and previous experimental tests. The outcomes of the MLP network demonstrate a reliable precision in estimating unseen data and can be used for further prediction of any concrete mixture with any combination of SCMs. Finally, to facilitate the utilization of the proposed MLP network, user-friendly software was developed based on the prediction procedure of the machine learning method. The proficiency and competence of this tool has been successfully proven.

Data Availability Statement:
The data presented in this study are available on request from the corresponding author.

Conflicts of Interest:
The authors declare no conflict of interest.

Abbreviations
The following abbreviations are used in this manuscript. The order is based on their appearance: