Interpretable Machine Learning for Compressive Strength Prediction of Fly Ash-Based Geopolymer Concrete

Abstract

Fly ash-based geopolymer concrete (GPC) is a sustainable alternative to conventional cementitious materials; however, its compressive strength is governed by complex and highly correlated mixture parameters, making experimental optimization expensive and data-driven modeling challenging. While machine learning (ML) techniques have been widely applied to predict GPC strength, most studies prioritize predictive accuracy without explicitly addressing multicollinearity among input variables, which can distort feature importance, reduce model stability, and limit engineering interpretability. This study proposes a multicollinearity-integrated and interpretable ML framework that systematically embeds correlation diagnostics and structured feature screening within the modeling pipeline rather than treating interpretability as a post-processing step. Multiple conventional and ensemble learning algorithms were comparatively evaluated using cross-validation to ensure generalization robustness. The proposed framework achieved a maximum coefficient of determination (R²) of 0.96 with low prediction error, outperforming baseline regression models while demonstrating improved stability under correlated input conditions. Unlike existing studies that rely solely on black-box optimization, the integrated interpretability analysis revealed physically consistent dominance of curing temperature, alkali content, and water-related parameters in governing strength development. By explicitly coupling predictive performance with multicollinearity mitigation and engineering-oriented interpretability, this work advances beyond accuracy-driven ML applications and provides a robust and transparent decision-support tool for sustainable geopolymer mix design.

1. Introduction

Concrete is the most widely used construction material worldwide, owing to its versatility, durability, and relatively low cost. However, the dominant use of ordinary Portland cement (OPC) as the primary binder in concrete has raised serious environmental concerns. Cement production is an energy-intensive process. It is responsible for approximately 5–8% of global anthropogenic CO₂ emissions, mainly due to clinker production and fossil fuel consumption. As global infrastructure demand continues to increase, reducing the environmental footprint of cement-based materials has become a critical challenge for sustainable development in the construction sector [1].

In response to these concerns, geopolymer concrete has emerged as a promising alternative to OPC-based systems. Geopolymers are inorganic binders formed through the alkaline activation of aluminosilicate-rich materials, such as fly ash, ground granulated blast furnace slag (GGBFS) and other industrial by-products. Unlike OPC, geopolymer binders do not rely on high-temperature clink erization, enabling substantial reductions in CO₂ emissions while simultaneously valorizing industrial waste streams [2]. From a sustainability perspective, geopolymer technology aligns with circular economy principles by transforming by-products into value-added construction materials and reducing reliance on virgin raw resources.

Geopolymer concrete has gained considerable attention due to its environmental benefits and mechanical performance. In addition to its behavior under static loading, geo-polymer concrete also exhibits notable performance under dynamic loading conditions. Studies have shown that it provides adequate resistance to impact and vibrations, effectively absorbing applied energy. Therefore, investigating its strength and behavior under dynamic conditions is particularly important for both structural and industrial applications.

Among various precursor materials, fly ash has received particular attention due to its wide availability and favorable chemical composition. Large quantities of fly ash are generated annually by coal-fired power plants. Only a limited fraction is effectively utilized in construction applications. When activated under alkaline conditions, fly ash can form a dense aluminosilicate network. It results in geopolymer concretes with mechanical properties comparable to, and in some cases exceeding those of, conventional OPC concrete [3,4,5]. The incorporation of GGBFS has also been shown to enhance early-age strength development, especially under ambient curing conditions, further improving the practical applicability of geopolymer systems [6].

Despite these advantages, the mechanical performance of geopolymer concrete is governed by a complex interaction of multiple parameters. These include precursor composition, fineness of fly ash, alkaline activator type and concentration, activator to binder ratio, curing temperature and duration, and the presence of chemical admixtures [7]. The nonlinear and interdependent nature of these variables makes it challenging to develop generalized mixed design rules using traditional empirical approaches. As a result, reported findings in the literature often exhibit significant variability, limiting the transferability of experimental results to broader engineering practice.

To address this complexity, machine learning (ML) techniques have increasingly been adopted to model and predict the compressive strength of geopolymer concrete. ML models are capable of capturing nonlinear relationships and interactions among multiple variables and have demonstrated high predictive accuracy. However, many ML-based approaches suffer from limited interpretability, particularly when input variables exhibit strong linear interdependencies. In such cases, model predictions may be accurate, yet the extracted feature importance measures become unreliable, effectively turning the models into black-box predictors with limited physical insight.

From a sustainability-oriented engineering perspective, interpretability is as important as predictive accuracy. Understanding which material parameters most strongly influence compressive strength supports more efficient mix design, reduces unnecessary material usage and facilitates the rational optimization of geopolymer systems with lower environmental impact. Therefore, there is a clear need for modeling frameworks that balance predictive performance with transparency and robustness, especially when dealing with high-dimensional datasets characterized by multicollinearity.

Recent studies have extensively explored machine learning (ML) techniques for predicting the compressive strength (CS) of fly ash-based geopolymer concrete (GPC), highlighting the nonlinear dependency between mixture constituents, curing conditions, and strength development. For instance, Ansari et al. compared conventional ML models (Linear Regression, Artificial Neural Networks) with ensemble techniques such as AdaBoost and reported a substantial improvement in predictive performance using ensemble learning (R² = 0.944), demonstrating the superiority of boosting-based approaches over traditional regression models [8]. Similarly, Gupta et al. showed that Random Forest Regression outperformed KNN and Linear Regression in predicting GPC compressive strength, further confirming the robustness of ensemble-based frameworks for highly nonlinear concrete datasets [9].

More advanced ensemble and boosting strategies have also been proposed. Ahmad et al. employed ANN, AdaBoost, and boosting techniques, concluding that boosting algorithms achieved the highest accuracy (R² ≈ 0.96), and incorporated k-fold validation to improve generalization reliability [10]. Dash et al. introduced a firefly-optimized hybrid ensemble model that integrates multiple base learners through stacking. Their study combined metaheuristic hyperparameter tuning with Sobol and FAST global sensitivity analyses, demonstrating that curing temperature and water content dominate strength variability. These studies collectively indicate a strong trend toward hybrid and optimized ensemble frameworks to maximize prediction accuracy [11].

Beyond predictive accuracy, recent research has begun addressing model interpretability. Hu et al. applied SHapley Additive exPlanations (SHAP) to interpret eXtreme Gradient Boosting predictions for FRP-confined concrete columns, identifying parameter importance and nonlinear functional relationships [12]. Likewise, Nguyen et al. proposed a structured three-stage framework that transitions from ML prediction (LR, DNN, ResNet) to global sensitivity analysis and finally to empirical formula derivation, thereby attempting to bridge the gap between black-box ML models and practical engineering applicability [13].

Despite these significant advances, two critical issues remain insufficiently addressed in the current literature. First, while ensemble and hybrid models consistently improve predictive accuracy, most studies prioritize performance metrics (R², RMSE, MAE) without systematically addressing multicollinearity among input variables—a common characteristic of geopolymer concrete datasets due to correlated binder compositions, activator ratios, and curing parameters. Second, interpretability is often treated as a post-processing step (e.g., SHAP or sensitivity analysis) rather than being structurally integrated into the modeling framework to enhance both robustness and engineering transparency.

Therefore, the existing body of research reveals a methodological gap between high-accuracy ensemble modeling and structured interpretability with explicit treatment of correlated input features. The present study aims to address this gap by integrating multicollinearity mitigation strategies within the ML pipeline and systematically evaluating their impact on both predictive performance and interpretability. In doing so, this work moves beyond incremental accuracy improvement and contributes to a more reliable and engineering-oriented modeling framework for geopolymer concrete strength prediction.

The novelty of this study lies in the development of an interpretable and dependency-aware machine learning framework for estimating the compressive strength of sustainable geopolymer concrete. Unlike conventional ML approaches that prioritize prediction accuracy alone, this work introduces a multi-objective NSGA-II–based feature selection strategy that explicitly accounts for linear interdependencies among input variables. By simultaneously optimizing predictive accuracy and feature independence, the proposed framework enhances both model reliability and interpretability.

Using an extensive dataset comprising multiple mix designs and curing parameters, the study systematically evaluates twelve hyperparameter-tuned machine learning models trained on optimized feature subsets. The results not only confirm the dominant influence of industrial by-products such as GGBFS and key activator-related parameters but also demonstrate that high predictive accuracy can be maintained while significantly reducing input redundancy. From a sustainability standpoint, the proposed methodology supports more informed and resource-efficient geopolymer mix design, reinforcing the role of geopolymer concrete as a viable low-carbon alternative to conventional cement-based materials.

2. Materials and Methods

This study aims to enhance the interpretability and robustness of machine-learning models in the domain of compressive strength prediction of geopolymer concrete by reducing linear interdependence among input features while preserving predictive accuracy.

To ensure a fair and reproducible evaluation framework, the dataset was first divided into training and testing subsets using a single 80/20 holdout split. The same partition was consistently preserved across both the baseline models and the feature-selected models to eliminate performance variability arising from data partitioning differences.

Given the high degree of linear interdependence among geopolymer constituents, a multi-objective feature selection strategy based on the NSGA-II algorithm was implemented. Unlike conventional single-objective selection approaches, the proposed framework simultaneously considers four criteria: predictive performance, multicollinearity reduction, correlation control, and feature subset compactness. These objectives were combined using weighted aggregation to balance model accuracy with statistical stability.

The weight coefficients (0.15, 0.30, 0.40, 0.15) were determined empirically through pre-liminary exploratory experiments to achieve a stable trade-off between predictive performance and multicollinearity mitigation. The selected configuration provided consistent convergence behavior across independent runs. The aim was not to impose theoretical optimality, but to establish a practically balanced configuration suited to the characteristics of the dataset.

Feature selection was performed exclusively on the training set to prevent information leakage. After subset identification, twelve machine learning algorithms were trained using identical training–testing partitions to ensure a valid comparison between baseline and optimized feature configurations.

Hyperparameter optimization was conducted using Tree-structured Parzen Estimator (TPE) search with 50 evaluation iterations per model. This budget was selected as a compromise between computational feasibility and convergence stability, and preliminary runs indicated diminishing performance gains beyond this threshold.

Although the dataset consists of 274 experimental samples, it represents a consolidated compilation of geopolymer concrete mixtures with varying binder compositions, activator ratios, and curing conditions. Therefore, the results should be interpreted within the scope of this dataset, and broader generalization requires further cross-dataset validation.

2.1. Data Collection

To achieve robust predictions and reliable feature interpretation, it is essential to curate and pre-process datasets carefully and ensure their credibility. A key dataset in this field is provided with a manuscript titled “Effect of Alkaline Activators and Other Factors on the Properties of Geopolymer Concrete Using Industrial Waste”, developed with a gene expression programming (GEP) approach by Pham and Nguyen [14]. It is publicly available through the Mendeley Data Repository [15]. The dataset focuses on mixtures incorporating industrial by-products such as fly ash and ground granulated blast furnace slag, offering a solid basis for exploring geopolymer concrete properties through machine learning techniques.

2.2. Exploratory Data Analysis

The dataset contains 19 independent and 1 dependent variable(s) (columns) and 274 data points (rows). To provide a comprehensive overview of the dataset employed in this study, the descriptive statistics of the input variables are presented in

Table 1. Descriptive statistics of input features.

Variable	Mean	SD	Min	Max
Fly Aah (FA)	255.2143	91.8794	120.000	530.000
GGBFS	164.6996	99.8852	0.000	360.000
Coarse Aggregate	991.2118	199.9069	525.4000	1276.8000
Fine Aggregate (Sand)	744.3048	84.6714	514.000	196.4000
Na2SiO3 Solution	149.2435	26.7160	76.2500	102.8571
NaOH	59.8875	11.7165	30.500	102.8571
Na2O	18.4245	3.9115	11.2100	28.8700
SiO2	44.5342	7.9886	22.4200	57.80600
Water (1)	86.287	15.5516	42.6238	112.8214
Concentration	8.4689	2.2129	2.0000	16.000
Water (2)	39.9111	10.0020	13.4200	73.6000
NaOH (Dry)	19.9764	5.2394	3.4960	38.4000
Additional water	20.6293	53.1391	0.0000	170.000
Superplasticizer	11.7201	7.3765	0.0000	18.000
Total water	153.8571	62.0876	56.0438	303.5336
Initial curing time	0.2894	0.4543	0.0000	1.000
Initial curing temp	37.3040	21.2549	24.000	100.000
Initial curing rest time	0.9048	0.3181	0.0000	2.0000
Final curing temp	28.3297	2.6942	24.000	30.000

.

Fly ash is a byproduct of coal combustion in thermal power plants. It is a fine powder mainly composed of silica, alumina, and iron oxides. With its pozzolanic properties, it reacts with calcium hydroxide in the presence of water to form cementitious compounds, thereby improving workability, reducing heat of hydration, enhancing long-term strength, and supporting sustainability by recycling industrial waste. Ground granulated blast furnace slag (GGBFS) is generated as a by-product of the iron and steel industry. It also exhibits cementitious and pozzolanic characteristics that enhance durability and mechanical performance over time. The incorporation of these materials in concrete is strongly aligned with sustainable construction practices and the objectives of green development [16].

In the present dataset, Concentration (M) NaOH denotes the molar concentration of the sodium hydroxide solution used as the alkaline activator. This parameter describes the amount of NaOH dissolved per liter of solution and is used to characterize the alkalinity level of the activating medium. The concentration of NaOH is known to play an important role in controlling the dissolution of aluminosilicate precursors and the overall reaction environment during geopolymer formation. Defining concentration in terms of molarity is a common and well-established practice in geopolymer and alkali-activated material studies, and the same convention has been adopted in this dataset.

Sodium silicate solution (Na₂SiO₃) is obtained by combining alkaline compounds with silica sources and water. Owing to its soluble silica content, it plays a key role in the geopolymerization process, stabilizing the structure and supporting the formation of three-dimensional silicate–aluminate networks [17]. Water within this solution reduces viscosity and improves particle dispersion, thereby enhancing the workability of fresh mixtures. The SiO₂/Na₂O molar ratio is particularly critical for both strength and durability [18]. Sodium hydroxide (NaOH), which is prepared by dissolving solid NaOH in water, acts as a strong alkaline activator that breaks down aluminosilicate structures, releasing reactive ions to accelerate polymerization and strength development [19]. However, its concentration must be carefully controlled to avoid negative effects such as thermal cracking or reduced workability. Sodium oxide (Na₂O) further increases sodium ion availability, strengthening silicate–aluminate gels and promoting the dissolution of precursors, though excessive contents may harm durability [20].

Silicon dioxide (SiO₂) provides the main source of reactive silica, enabling the formation of sodium aluminosilicate hydrate (N-A-S-H) gels that contribute to strength, durability, and low permeability [21]. Geopolymer systems also reduce CO₂ emissions by replacing Portland cement with recycled pozzolanic materials and alkaline activators [21], while dense gel networks improve resistance against carbonation and aggressive ions, contributing to sustainability. Water primarily acts as a reaction medium [17], while its total amount, including any additional water, must be optimized to balance workability and strength. The molar concentration and dry mass of NaOH [22] provide precise control of activator-to-binder interactions. Superplasticizers enhance flow without excess water [23], though their efficiency depends on compatibility with alkaline conditions.

Curing conditions are equally critical. Initial curing time and temperature strongly affect early geopolymer gel formation and strength gain, with moderate heat curing between 60 and 80 °C being particularly beneficial. A short resting period before heat application can reduce thermal stresses. Final curing temperature [3] governs matrix densification, durability, and efflorescence prevention. Additionally, sodium silicate concentration [5] influences network formation, setting behavior, and compressive strength through its control of the Si/Na ratio.

The variables included in the dataset were chosen mainly for their known effects on the mechanical strength of geopolymer concrete. At the same time, materials that contribute to sustainability, such as industrial by-products like fly ash and GGBFS, were also considered when selecting the features.

2.3. The Experimental Process

Following the collection of the data, a 4-stage experiment was conducted to fulfill the research aim and objectives (Figure 1). Appendix A provides a more comprehensive step-by-step tree diagram of all experimental process (pipeline). The stages of the process were Initialization, Feature Selection with Genetic Algorithm, Performance Assessment of ML Models, Visualization and Verification. This section provides the details of each stage.

2.3.1. Stage 1: Initialization

The process starts with loading the dataset from a CSV file. A check is made following this to detect if there is any missing data in the dataset. In the case of missing data, rows that contain null values would be removed. In fact, in our dataset, there were no rows with missing data.

In the following phase, the Standard Scaler (z-score normalization) X_scaled = (X − μ)/σ, where μ is the mean and σ is the standard deviation, is applied to the dataset. In many machine learning tasks, features often have different scales, which can negatively affect model performance. The Standard Scaler addresses this by standardizing each feature to have a mean of zero and a standard deviation of one. This ensures that all variables contribute equally to the learning process, without being dominated by those with larger magnitudes. By preserving the structure of the data while balancing feature scales, Standard Scaler facilitates more stable and efficient model training. This stage ensures data quality and normalizes features for consistent algorithm performance.

The following phase involves parameter configuration. A set of parameters for the feature selection process is configured in this phase. First, the minimum number of variables to retain (default: 5) is provided by the user. Following this, parameters related to the Genetic Algorithm Population size (default: 60), Number of generations (default: 40), Crossover probability (default: 0.7), and Mutation probability (default: 0.2) are set. Finally, the weights for each objective, VIF, RMSE, R², and Total Multiple R (MultiR) are provided by the user. These weights are used for transforming the 4-objective optimization problem into a single objective optimization problem. Weights also determine the focus of the feature selection, either by prioritizing one of the objectives (i.e., minimizing VIF) or by giving equal importance to each objective. In our study, the objective weights were provided as twelve weight combinations prioritizing both maximizing R² and minimizing RMSE, while controlling the effects of VIF and Total Multiple R. The weight combinations are provided in

Table 2. Weight combinations for objectives.

Scenario	Weights Combinations (VIF, R2, RMSE, MultiR)
S1	(0.05, 0.45, 0.45, 0.05)
S2	(0.05, 0.50, 0.40, 0.05)
S3	(0.05, 0.40, 0.50, 0.05)
S4	(0.10, 0.40, 0.40, 0.10)
S5	(0.10, 0.35, 0.45, 0.10)
S6	(0.10, 0.45, 0.35, 0.10)
S7	(0.15, 0.35, 0.35, 0.15)
S8	(0.15, 0.40, 0.30, 0.15)
S9	(0.15, 0.30, 0.40, 0.15)
S10	(0.20, 0.30, 0.30, 0.20)
S11	(0.20, 0.35, 0.25, 0.20)
S12	(0.20, 0.25, 0.35, 0.20)

.

The following phase was correlation analysis and the generation of a correlation plot for the entire dataset based on Pearson Correlation Coefficient values (Figure 2). Figure 2 shows the correlation matrix of 19 key variables related to geopolymer concrete. The relationships between the variables are color-coded based on the correlation coefficient, with red indicating a positive relationship and blue indicating a negative relationship. A very strong negative correlation (r = −0.92) is observed between FA and GGBFS, indicating a strong inverse relationship between the two constituents. Na₂SiO₃ also shows a high positive correlation with NaOH and total water, since they are often used together in alkaline solutions. The values of water (1), added water, and total water also show relatively high correlations. Most variables related to curing time and temperature (e.g., final curing temperature) are less correlated with other variables, indicating that these parameters are determined independently of the composition of the raw materials. Overall, this matrix accurately shows the internal relationships of geopolymer concrete variables.

Following this phase, the dataset with all features is evaluated with a Random Forest Algorithm using 10-fold cross-validation. In this stage, metrics related to linear dependence and model accuracy are provided with several graphs.

2.3.2. Stage 2a: Feature Selection with Genetic Algorithm (Multi-Objective Optimization)

In this study, we have developed a Genetic Algorithm (GA)-based feature selection method to identify the most relevant input variables. The method optimizes a multi-objective fitness function that balances prediction accuracy, linearity/linear dependence, and the number of selected features. Each solution is represented as a binary chromosome, and standard GA operators such as selection, crossover, and mutation are used. The best-performing feature subset is selected based on repeated runs and average model performance. This approach is adaptable to any regression model and improves both efficiency and interpretability compared to traditional methods. This approach can be used to tackle high VIF scores caused by possibly redundant features. The GA based algorithm developed focuses on the following 4 objectives:

(1)

Minimizing VIF

(2)

Maximizing R²

(3)

Minimizing RMSE

(4)

Minimizing Total Multiple R

The following elaborates on each objective:

• Minimizing VIF (Variance Inflation Factor):

High multicollinearity arises when independent variables are highly correlated, making it difficult to isolate the individual effect of each predictor. This condition increases the standard errors of the regression coefficients and undermines the model’s reliability. The Variance Inflation Factor (VIF) is commonly used to detect such issues; VIF values greater than 10 typically indicate problematic multicollinearity, suggesting the need to remove or combine variables. Rather than merely treating input features as isolated variables, this objective seeks to examine the underlying dependency structure between them. A high VIF implies that a variable carries redundant information already encoded by other predictors, which not only distorts coefficient estimates but also undermines the clarity of model interpretation. By minimizing VIF, we aim to ensure that each selected variable contributes unique and non-overlapping information, thereby promoting a more stable and trustworthy model structure.

• Maximizing R² (Coefficient of Determination):

This objective is rooted in the notion that a strong predictive model should be capable of capturing the majority of the variability present in the response variable(s). A high R² does not merely signal good performance—it indicates alignment between the model’s internal representation and the natural structure of the data. Optimizing for R² thus reflects an effort to build models that are not just mathematically accurate, but also conceptually coherent with the phenomenon being modeled.

• Minimizing RMSE (Root Mean Square Error):

While R² reflects the proportion of variance explained, RMSE directly captures the magnitude of prediction error in the same unit as the response. It penalizes large deviations more severely than smaller ones, making it particularly sensitive to outliers and noise. Minimizing RMSE ensures that the model’s predictions remain not only close to observed values but also consistent and reliable across diverse instances, without major fluctuations in performance.

• Minimizing Total Multiple R (Custom Metric):

Beyond standard metrics, we introduce Total Multiple R as a tailored indicator of cumulative linear dependence between input features and the target(s). Total Multiple R measures the overall interdependence among selected variables by summing the Multiple Correlation R values for each variable. The calculation method for the Total Multiple R metric is provided in

Table 3. Calculation method for Total Multiple R.

Single Variable Multiple R: Calculated for each Independent Variable (IV) separately

Step 1: Define regression problem as Equation (1) Step 2: Calculate R2 (Coefficient of Determination) according to Equations (2)–(4) Step 3: Calculate Multiple Correlation R according to Equation (5)   where R_i ∈ [0, 1]

Total Multiple R: Calculated as an overall metric as given in Equation (6)

Properties:   - Range: [0, V]   - Minimum: 0 (perfect independence of IVs)   - Maximum: V (complete redundancy of IVs)

.

For each independent variable x_i, a regression model is constructed using the remaining V-1 independent variables as given in Equation (1):

$${\mathrm{y}}_{\mathrm{i}}={\mathsf{\beta }}_0+{\sum }_{\mathrm{j}=1,\mathrm{i}\ne \mathrm{j}}^{\mathrm{V}}{\mathsf{\beta }}_{\mathrm{j}}{\mathrm{x}}_{\mathrm{j}}+\mathsf{\epsilon }$$

(1)

where y_i denotes the i-th original variable treated as the dependent variable in the corresponding regression model, the remaining x_j are used as independent variables (IVs) of the original dataset, V is total number of independent variables, β₀ is intercept, β_i is regression coefficient and ε is residual error term. The calculation of coefficient of determination (R²) is done according to Equations (2)–(4).

$${\mathrm{R}}_{\mathrm{i}}^2=1-\left(\mathrm{S}{\mathrm{S}}_{\mathrm{r}\mathrm{e}\mathrm{s},\mathrm{i}}\right)/\left(\mathrm{S}{\mathrm{S}}_{\mathrm{t}\mathrm{o}\mathrm{t},\mathrm{i}}\right)$$

(2)

$$\mathrm{S}{\mathrm{S}}_{\mathrm{r}\mathrm{e}\mathrm{s},\mathrm{i}}={\mathsf{\Sigma }}_{\left(\mathrm{k}=1\right)}^{\mathrm{n}}{\left({\mathrm{y}}_{\mathrm{i},\mathrm{k}}-{\hat{\mathrm{y}}}_{\mathrm{i},\mathrm{k}}\right)}^2$$

(3)

$$\mathrm{S}{\mathrm{S}}_{\mathrm{t}\mathrm{o}\mathrm{t},\mathrm{i}}={\mathsf{\Sigma }}_{\left(\mathrm{k}=1\right)}^{\mathrm{n}}{\left({\mathrm{y}}_{\mathrm{i},\mathrm{k}}-{\stackrel{-}{\mathrm{y}}}_{\mathrm{i}}\right)}^2$$

(4)

y_i,k is the observed value of variable i for sample k, ${\hat{\mathrm{y}}}_{\mathrm{i},\mathrm{k}}$ is the predicted value from regression, ${\stackrel{-}{\mathrm{y}}}_{\mathrm{i}}$ is the mean of variable i and n is the number of samples. The multiple correlation coefficient R_i is calculated as Equation (5).

$${\mathrm{R}}_{\mathrm{i}}=\sqrt{\left({\mathrm{R}}_{\mathrm{i}}^2\right)}$$

(5)

Total multiple R (Total_R) is calculated as an overall metric as given in Equation (6).

$${\mathrm{T}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l}}_{\mathrm{R}}={\mathsf{\Sigma }}_{\mathrm{I}=1}^{\mathrm{V}}{\mathrm{R}}_{\mathrm{i}}$$

(6)

As the range of Total Multiple R is [0, Number of Independent Variables], high total correlation (Total Multiple R) values may reflect a scenario where certain features dominate the output space, leading to overfitting and sensitivity to minor changes in those dominant predictors. By minimizing this metric, we aim to balance the influence of individual features and reduce the risk of overly deterministic or fragile model behavior.

The objective function, which aims to fulfill the 4 objectives mentioned above, takes independent variables of the ML problem and objective weights as inputs, trains the Random Forest models with different combination of independent variables, then calculates the values of VIF, R², RMSE and Total Multiple R, and provides the weighted sum of these as the objective function output.

The optimization algorithm with 4 objectives has been transformed into a single objective optimization problem using the Weighted Sum Method, as explained in [24], by forming a convex combination of the objectives with the weights provided by the user. In runtime, the user is given an option to run the feature selection process either focusing on one of the competing objectives, i.e., minimization of VIF, maximization of R², minimization of RMSE, and minimization of Total Multiple Correlation R or choosing a balanced approach. When the user selects one of these 4 approaches, a predefined set of weights is provided for that specific focus. When the user selects the balanced approach, equal weights of 0.25 are assigned for each objective. In this specific study, twelve weight combinations, provided in Table 2, were used, prioritizing both maximizing R² and minimizing RMSE, while controlling the effects of VIF and Total Multiple R.

NSGA-II is implemented as the Genetic Algorithm (GA) variant for this multi-objective problem. Random Forest is used inside the genetic algorithm for training as the ML model that is evaluated with different combinations of independent variables. A fixed set of hyperparameters for the Random Forest model is used when evaluating the model with different variable combinations. The minimum number of independent variables to keep was a constraint of the implemented GA (default: 5) and is input by the user before the algorithm initiates. The user is also asked to input hyperparameters of the GA, such as the number of generations [10–100] and population size [20–200].

Following the completion of the multi-objective genetic algorithm optimization process, the algorithm produces a set of non-dominated solutions known as the Pareto front. These solutions represent optimal trade-offs among the four competing objectives: minimization of VIF, maximization of R², minimization of RMSE, and minimization of Total Multiple Correlation R. The Pareto front analysis phase involves a comprehensive evaluation of all solutions, and the selection of a single optimal solution based on the user-defined focus and preferred weights set. The set of all Pareto optimal solutions forms the Pareto front, which represents the best possible trade-offs achievable among the competing objectives. Solutions in the Pareto front have the property that any improvement in one objective necessarily requires degradation in at least one other objective.

2.3.3. Stage 2b: Pareto Front Analysis for Multi-Objective Variable Selection

Each individual in the Pareto front is represented as a binary vector b ∈ {0,1}^V, where V is the total number of available variables, bi = 1 indicates variable i is selected, and bi = 0 indicates exclusion. The pseudocode for Pareto front analysis is provided in

Table 4. Pseudocode for Pareto front analysis.

ALGORITHM: Pareto Front Analysis and Optimal Selection INPUT:   pareto_front: Set of non-dominated solutions from NSGA-II   X_scaled: Standardized feature matrix   y: Target variable   weights: [w1, w2, w3, w4] objective weights OUTPUT:   optimal_solution: Selected best compromise solution   pareto_df: DataFrame with all solutions and metrics   best_by_objective: Dictionary of best solutions per objective PROCEDURE: 1. Initialize result storage   pareto_solutions ← [] 2. FOR EACH individual IN pareto_front DO   (a) Decode binary representation      variable_mask ← individual.binary_vector      selected_vars ← DECODE(variable_mask)      n_vars ← COUNT(selected_vars)   (b) Extract subset data      X_subset ← X_scaled[:, selected_vars]   (c) Calculate VIF      max_vif ← CALCULATE_MAX_VIF(X_subset, selected_vars)   (d) Evaluate model performance (5-fold CV)      r2, rmse ← EVALUATE_RANDOM_FOREST(X_subset, y)   (e) Calculate Total Multiple R      total_r ← 0      individual_r_scores ← []      FOR EACH var_idx IN selected_vars DO      r_value ← CALCULATE_SINGLE_R(var_idx, X_subset)      individual_r_scores.APPEND(r_value)      total_r ← total_r + r_value      END FOR      avg_r ← total_r/n_vars   (f) Store comprehensive metrics      solution ← {      ‘solution_id’: i,      ‘n_variables’: n_vars,      ‘max_vif’: max_vif,      ‘r2’: r2,      ‘rmse’: rmse,      ‘total_multiple_r’: total_r,      ‘avg_multiple_r’: avg_r,      ‘variables’: selected_vars,      ‘individual_r_scores’: individual_r_scores,      ‘fitness’: individual.fitness.values      }      pareto_solutions.APPEND(solution) 3. Create DataFrame   pareto_df ← DATAFRAME(pareto_solutions) 4. Identify best solutions by individual objectives   best_by_objective ← {      ‘vif’: pareto_df[pareto_df[‘max_vif’].IDXMIN()],      ‘r2’: pareto_df[pareto_df[‘r2’].IDXMAX()],      ‘rmse’: pareto_df[pareto_df[‘rmse’].IDXMIN()],      ‘total_r’: pareto_df[pareto_df[‘total_multiple_r’].IDXMIN()]   } 5. Calculate balanced (i.e., weighted) scores for all solutions   FOR EACH solution IN pareto_df DO      // Normalize objectives      norm_vif ← 1/(1 + solution[‘max_vif’]/100)      norm_r2 ← solution[‘r2’]      norm_rmse ← 1/(1 + solution[‘rmse’]/10)      norm_total_r ← 1/(1 + solution[‘total_multiple_r’]/10)      // Weighted aggregation      balanced_score ← w1 × norm_vif +             w2 × norm_r2 +             w3 × norm_rmse +             w4 × norm_total_r      solution[‘balanced_score’] ← balanced_score   END FOR 6. Select optimal solution   optimal_solution ← pareto_df[pareto_df[‘balanced_score’].IDXMAX()] 7. RETURN optimal_solution, pareto_df, best_by_objective END ALGORITHM

.

In the Pareto front analysis, following the initial 3 steps, Step 4 identifies the best solutions for each objective, and Step 5 calculates the weighted (balanced) scores for all possible solutions (i.e., feature subsets). Finally, the optimal solution is selected based on these scores in Step 6. The best solutions for each of the 12 scenarios tested were calculated using this analysis.

2.3.4. Stage 3: Performance Assessment of ML Models with Selected Features

Once the feature selection is complete and the optimal subset of features is selected based on commonly selected features in 12 scenarios, a series of ML models is trained with hyperparameter optimization using this optimal subset. In this stage, the hyperparameter optimization is accomplished through the HyperOpt (version 0.2.7) Python library. This section first presents a brief overview of each of the models evaluated in this stage, and following this, an overview of hyperparameter optimization is provided.

XGBoost: XGBoost is a Gradient Boosting framework that builds decision trees sequentially, with each new tree attempting to reduce the residual errors of the previous ones. Its learning mechanism incorporates both first- and second-order gradients, allowing for precise optimization during training. What sets XGBoost apart is its built-in regularization, which helps prevent overfitting, especially in high-dimensional datasets often encountered in structural and materials engineering. This algorithm has shown superior predictive accuracy in complex regression problems, such as modeling the rheological properties of self-compacting concrete.

CatBoost: CatBoost is another Gradient Boosting algorithm specifically designed to handle categorical features more effectively than traditional models. İt introduces a novel technique known as ordered boosting, which reduces prediction shift and leakage that can occur during training. Furthermore, CatBoost internally transforms categorical variables into numerical formats without the need for external encoding. This capability makes it highly suitable for construction datasets that include multiple qualitative inputs. Although its training time is typically longer than that of other boosting methods, it often achieves comparable or better accuracy.

LightGBM: LightGBM is a Gradient Boosting algorithm developed for efficiency and scalability. It uses a leaf-wise growth strategy, which typically results in deeper trees and better accuracy compared to level-wise methods. Additionally, it employs techniques like Gradient-based One-Side Sampling (GOSS) and Exclusive Feature Bunding (EFB) to reduce computational cost and memory usage. İn engineering applications involving large and heterogeneous datasets, LightGBM offers an excellent balance between training speed and prediction performance, making it an attractive option for real-time modeling.

K-Nearest Neighbors: KNN is a non-parametric, instance-based learning algorithm commonly used for regression and classification tasks. It predicts outcomes by identifying the closest data points in the feature space and aggregating their responses. Although not frequently employed. KNN often serves as a benchmark due to its simplicity and interpretability, particularly when transparency in decision-making is required.

Random Forest: Random Forest is an ensemble learning algorithm that constructs multiple decision trees using random subsets of data and features. The final prediction is obtained by aggregating the outputs of individual trees through majority voting or averaging.

Gradient Boosting: In practical engineering problems where data complexity and non-linearity are prevalent, Gradient Boosting has emerged as a reliable modeling strategy. It functions by building a predictive structure layer by layer, correcting prior errors iteratively, which allows for capturing subtle patterns hidden in the data. Rather than relying on a single deep tree, the model constructs a sequence of shallow ones, each fine-tuning the shortcomings of its predecessor. This stepwise refinement not only improves accuracy but also allows engineers to regulate overfitting by adjusting parameters like learning rate and maximum depth. It is particularly valuable in applications involving material strength prediction or load estimation, where precision is essential and training data may be limited.

Decision Tree: For tasks requiring clarity and explainability, Decision Trees serve as an intuitive analytical framework. Their logic mimics human decision-making: sequential, rule-based, and visually traceable. While they may not offer the highest predictive power on their own, they are instrumental in uncovering key variable interactions and segmenting data in a transparent way. This interpretability is especially advantageous in engineering domains, where understanding cause–effect relationships between inputs and outputs can inform both model development and domain-specific decisions.

Extra Trees: In scenarios where model training speed and generalization are crucial, the Extra Trees algorithm offers an efficient yet robust solution. Randomly selecting both features and splitting thresholds, it eliminates the exhaustive search typical of standard decision trees, which drastically reduces training time. Although the model sacrifices fine-tuned decisions at each node, it gains stability and noise tolerance through sheer ensemble diversity. This makes Extra Trees highly practical in settings where data may include measurement uncertainties or where rapid iterations are needed across many design alternatives.

MLP Neural Network: The Multilayer Perceptron (MLP) is a class of artificial neural networks particularly effective for learning nonlinear relationships in engineering systems. With its layered architecture consisting of interconnected neurons distributed across input, hidden, and output layers, MLP can model complex mappings between input features and output responses. Its strength lies in the backpropagation algorithm, which fine-tunes the internal weights based on the discrepancy between predicted and actual values. This adaptability allows MLPs to capture intricate behavioral patterns in domains such as material science, structural health monitoring, or energy consumption forecasting, even when analytical formulations are not feasible.

Ridge Regression: When dealing with multicollinearity among predictors or limited data in high-dimensional spaces, Ridge Regression serves as a disciplined approach to prevent model overfitting. By introducing a penalty term proportional to the square of the coefficients, it constrains the magnitude of parameter estimates, stabilizing the solution. This regularization not only improves predictive generalization but also allows for a more robust interpretation in data-driven engineering studies. Especially in optimization tasks involving multiple correlated inputs, Ridge Regression provides a mathematically sound alternative to standard linear models by managing trade-offs between bias and variance.

Linear Regression: Linear Regression remains a foundational method in predictive modeling, offering clarity and interpretability in understanding relationships between variables. Its principle assumes that the target variable can be approximated as a weighted sum of the input features, with residuals following a normal distribution. Despite its simplicity, linear regression is valuable in preliminary assessments, sensitivity analyses, or when establishing baseline models for comparison. In engineering contexts, it enables practitioners to quickly identify dominant factors affecting system behavior, while also setting the stage for more advanced modeling if required.

Lasso Regression: Lasso Regression, like Ridge, is a type of penalized linear regression used for feature selection and preventing overfitting. The main difference from Ridge is that Lasso’s penalty can drive some coefficients exactly to zero. This feature allows Lasso to remove unimportant features from the model, creating a simpler and more interpretable model.

Elastic Net: Elastic Net is a combination of Ridge and Lasso regression. This algorithm simultaneously uses both L1 penalty (like Lasso) and L2 penalty (like Ridge). This combination allows Elastic Net to perform feature selection (like Lasso) and effectively manage multicollinearity issues (like Ridge), making it suitable for many datasets.

SVR (Linear): Linear SVR is a version of Support Vector Machine designed for regression problems. Instead of trying to find a line that minimizes error, Linear SVR tries to find a line that keeps most of the data within a specific “margin area”. This makes the model less sensitive to outliers.

SVR (RBF): RBF SVR is also used for regression, but it uses a kernel function called the Radial Basis Function (RBF). This function allows SVR to model complex nonlinear relationships between data. In other words, RBF SVR can find a “line” (actually a curve or hyperplane) that best approximates non-linear data, offering more flexibility than Linear SVR.

In machine learning, hyperparameters are external settings defined by the user before the training process begins. Unlike model weights, which are learned from data, hyperparameters control the overall behavior of the algorithm. They influence aspects such as model complexity, training speed, and generalization ability. Examples include the learning rate, the depth of a decision tree, or the number of estimators in ensemble methods. Selecting appropriate hyperparameters is a critical step, as it directly impacts the model’s accuracy and stability. Hyperparameter optimization aims to systematically determine these parameters to maximize the model’s performance. This process helps the model produce more stable and accurate results by preventing potential problems such as over- or under-fitting. Once the feature selection is finalized and the set of optimal features is determined, training with hyperparameter optimization is performed for each ML model (Random Forest, XGBoost, LightGBM, etc.) through the Hyperopt library [25]. The Hyperparameter Optimization was accomplished using the HyperOpt (version 0.2.7) python package’s Tree-structured Parzen Estimator (TPE), which is used as the optimization algorithm in this stage. Following the training stage, the models are evaluated on a separate holdout set.

2.3.5. Stage 4: Visualization and Verification

This stage involves the generation of convergence plots, Pareto front plots, Feature Importance Plot for the most accurate ML model, in addition to a series of verification checks that are also made to ensure the final VIF values are not extreme, and the linear dependency problem has been resolved.

3. Results

The experiment started with providing the initial parameter values to the Genetic Algorithm as indicated in

Table 5. Initial Parameter Values for the Experiment.

Parameter	Value
GA Population size	60
GA Generations	40
Objective weights	12 scenarios
Dataset variables	19
Minimum variables to keep	5
Cross-validation folds	5
Random state	42

. The algorithm used the exact same values for all runs of the algorithmic pipeline, which runs 12 times for different weight scenarios given in Table 2.

3.1. Feature Selection via Genetic Algorithm

Table 6. Optimal solutions for genetic algorithm runs with different weight scenarios.

Scenario	Weights (VIF, R2, RMSE, MultiR)	N Vars	Max VIF of Solution	R2	RMSE	Total Multi-R
S1	(0.05, 0.45, 0.45, 0.05)	9	36.01	0.9156	3.3044	8.2536
S2	(0.05, 0.50, 0.40, 0.05)	9	36.01	0.9156	3.3044	8.2536
S3	(0.05, 0.40, 0.50, 0.05)	9	36.01	0.9156	3.3044	8.2536
S4	(0.10, 0.40, 0.40, 0.10)	7	11.76	0.9142	3.3294	5.7507
S5	(0.10, 0.35, 0.45, 0.10)	7	11.76	0.9142	3.3294	5.7507
S6	(0.10, 0.45, 0.35, 0.10)	7	11.76	0.9142	3.3294	5.7507
S7	(0.15, 0.35, 0.35, 0.15)	7	11.76	0.9142	3.3294	5.7507
S8	(0.15, 0.40, 0.30, 0.15)	7	11.76	0.9142	3.3294	5.7507
S9	(0.15, 0.30, 0.40, 0.15)	7	11.76	0.9142	3.3294	5.7507
S10	(0.20, 0.30, 0.30, 0.20)	7	11.76	0.9142	3.3294	5.7507
S11	(0.20, 0.35, 0.25, 0.20)	7	11.76	0.9142	3.3294	5.7507
S12	(0.20, 0.25, 0.35, 0.20)	7	11.76	0.9142	3.3294	5.7507

below presents a summary of outputs of 12 genetic algorithm pipeline runs, for the 12 weight combination scenarios that focus on performance optimization while taking into account tackling variable interdependency. N vars indicates the number of features retained on scenario completion, Max VIF of the solution indicates the Max VIF value exhibited along with all retained variables, R², RMSE, Total Multi-R indicates the R², RMSE, Total Multi-R values obtained as result of training with baseline (non-fine-tuned) Random Forest Model for that scenario. Naturally as the number of selected features decreases, due to both the decrease in the predictive power of the model and total correlations among independent variables, the Max VIF, R² and Total Multi-R values have decreased and the RMSE values have increased.

Table 7. Variable selection stability across weight scenarios.

Variable	Times Selected	Scenarios
Concentration(M) NaOH	12/12	S1, S2, S3, S4, S5, S6, S7, S8, S9, S10, S11, S12
NaOH (Dry)	12/12	S1, S2, S3, S4, S5, S6, S7, S8, S9, S10, S11, S12
GGBFS	12/12	S1, S2, S3, S4, S5, S6, S7, S8, S9, S10, S11, S12
Initial curing temp (C)	12/12	S1, S2, S3, S4, S5, S6, S7, S8, S9, S10, S11, S12
Coarse Aggregate	12/12	S1, S2, S3, S4, S5, S6, S7, S8, S9, S10, S11, S12
Na2O (Dry)	9/12	S4, S5, S6, S7, S8, S9, S10, S11, S12
Initial curing rest time (day)	9/12	S4, S5, S6, S7, S8, S9, S10, S11, S12
Additional water	3/12	S1, S2, S3
Initial curing time (day)	3/12	S1, S2, S3
FA	3/12	S1, S2, S3
Na2SiO3	3/12	S1, S2, S3

provides the selected features (variables) for each scenario. The set of Concentration(M) NaOH, NaOH (Dry), GGBFS, Initial curing temp (C), Coarse Aggregate formed a solid foundation by being selected in all scenarios. The Na₂O (Dry) and Initial curing rest time (day) were selected in ¾ of all scenarios, indicating that these variables can also play a significant role in predictions. Additional water, initial curing time (day), FA, Na₂SiO₃ were selected only in ¼ of all scenarios, indicating that the contribution of these variables to the predictive ability of the model is limited. A total of 11 features were selected as a result of this selection process; the remaining eight features appear to have either zero or a considerably very low impact on the predictive ability of the model.

Every variable selection happens as a result of the Pareto front analysis process. The weighting scheme helps in transforming the solution from a multi-objective to a single-objective optimization problem. The balanced score is calculated first by applying a min-max normalization to all calculated objective values, then the weighted sum (ws) of normalized values of each objective was calculated according to the formula below.

ws = (1 − normVIF) ∗ w1 + (normR²) ∗ w2 + (1 − normRMSE) ∗ w3 + (1 − normMultiR) ∗ w4

(7)

Table 8. S4 Pareto front optimal solutions.

Solution ID	N Vars	Max VIF	R2	RMSE	Total Multiple R	Balanced Score (ws)	Optimal
1	5	1.7866	0.8902	3.7692	2.9706	0.2000	False
2	5	2.5093	0.8974	3.6401	3.1931	0.4082	False
3	5	2.6997	0.8927	3.729	3.1785	0.2662	False
4	5	2.8235	0.9032	3.5359	3.4096	0.576	False
5	5	2.9639	0.8905	3.7688	3.1313	0.2005	False
6	6	3.1136	0.9109	3.3943	4.3889	0.7919	False
7	7	3.7254	0.9122	3.3698	5.4745	0.8129	False
8	7	11.6182	0.9124	3.3646	5.6581	0.8129	False
9	7	11.7611	0.9142	3.3294	5.7507	0.8673	True
10	10	26.2126	0.9158	3.304	9.2227	0.8487	False
11	6	26.8715	0.9116	3.3814	4.6817	0.7932	False
12	9	36.0141	0.9156	3.3044	8.2536	0.8548	False
13	9	37.8616	0.9142	3.3308	7.8009	0.8189	False
14	10	162.9658	0.9164	3.2878	9.4148	0.7855	False
15	10	167.6364	0.917	3.2781	9.3437	0.8011	False

provides Pareto front optimal solutions, calculated weighted sum scores (Balanced Score) and selected optimal solution for scenario S4 (chosen as an example as it is the first scenario representing the lowest VIF score obtained). In the table, the column N Vars provides the number of variables (features) selected for that optimal solution. The table also provides the maximum VIF value observed for the variables in that selected subset, along with R² and RMSE. Total Multiple R values observed in the training with the Random Forest model (Random Forest model was the only model used in all generations of the GA).

The algorithm also provides the best solution per objective for each scenario. The best solutions per objective for scenario S4 are shown in

Table 9. Best solution per objective for S4.

Criterion	Solution ID	N Vars	VIF	R2	RMSE	Multi-R
Min VIF	1	5	1.79	0.8902	3.7692	2.9706
Max R2	15	10	167.64	0.917	3.2781	9.3437
Min RMSE	15	10	167.64	0.917	3.2781	9.3437
Min Multi-R	1	5	1.79	0.8902	3.7692	2.9706

.

The solution providing the Best Max VIF score was Solution 1, the solution providing the Best R² was Solution 15, the solution providing the Best RMSE was again Solution 15, and the solution providing the Best Total Multiple R was Solution 1. According to the weights provided for this scenario S4 (0.10, 0.40, 0.40, 0.10), the Best Solution (the Balanced Solution) was selected as Solution 9, with a Balanced Score of 0.8673. The feature subset in the selected solution (Solution 4) included Concentration(M)NaOH, NaOH (Dry), GGBFS, Initial curing temp(C), Coarse Aggregate, Na₂O (Dry) and Initial curing rest time (day). The Pareto front plot from the perspective of different objectives of the selected solution is provided in Figure 3.

Figure 3 indicates how variables representing objectives correlate with each other in the selected Pareto front. The Pareto front visualization in Figure 3 illustrates the trade-off relationships among multicollinearity (VIF), predictive accuracy (R²), prediction error (RMSE), and feature redundancy (Total MultiR). Each red diamond represents a non-dominated solution where no objective can be improved without degrading at least one other objective. The optimal solution, indicated by the red star, was selected using the predefined weighted aggregation function. The results demonstrate that although slightly higher R² values can be achieved at the expense of increased multicollinearity and redundancy, the selected optimal solution provides a balanced compromise by maintaining high predictive performance while controlling feature interdependence. This confirms the effectiveness of the proposed multi-objective optimization framework in identifying robust and parsimonious feature subsets without relying solely on predictive accuracy.

As explained previously, the set consisting of Concentration(M)NaOH, NaOH (Dry), GGBFS, Initial curing temp(C) and Coarse Aggregate formed a solid foundation by being selected in all scenarios. Thus, it was decided to use these five features in the further ML training and evaluation stages. Figure 4 presents the correlation plot of selected variable and target variable (28 d cubic compressive strength (MPa)).

The Pearson correlation analysis indicated that the 28-day compressive strength exhibits weak-to-moderate linear relationships with the input variables. Most importantly, the absence of strong individual correlations suggests that compressive strength is governed by complex multivariate interactions rather than a single dominant factor, supporting the need for the use of advanced machine learning approaches capable of capturing nonlinear relationships. Among the predictors, NaOH concentration showed the highest positive correlation (r = 0.33), followed by NaOH dry mass (r = 0.25), indicating that alkaline activator properties play a relatively more important role in strength development. Despite the fact that we had concerns regarding keeping the GGBFS within the selected set, as GGBFS content and initial curing temperature exhibited very weak correlations (r = 0.07), suggesting that their individual linear contributions to strength variation are limited within the studied range and a strong negative correlation was observed between GGBFS and coarse aggregate (r = −0.92), indicating severe multicollinearity between these variables (which would in turn increase VIF), we finally decided to keep this variable as it is selected in all 12 scenarios in GA optimization process, as it might possibly have a high contribution to the predictive power of the models.

3.2. Baseline Model Evaluation

The ML pipeline involved 30 runs of baseline model evaluation and multi-model fine-tuning. In the first stage, by considering the Random Forest as the algorithm for baseline evaluation, the model was first evaluated in each run of the ML pipeline with the full input variable set. The dataset was divided into training (80%) and holdout (20%) subsets using a fixed seed in each of the 30 runs to ensure the baseline model and fine-tuned models use the same data splits. The model training and hyperparameter evaluation were performed using 5-fold cross-validation exclusively on the training data. Appendix B provides the performance metrics of the baseline model. In the metrics, the cross-validated R² represents the average out-of-fold predictive performance, while the holdout R² reflects the final generalization performance on an independent, unseen dataset. The best performing run for the fine-tuned models was for seed 29; thus, in this section, we focus on the baseline model evaluation metrics for the baseline model run for the same seed 29. The R² value for the holdout set evaluation was 0.9306 with RMSE = 3.1516 and MAE = 2.2696. The R² value for the test fold of 5-fold cross-validation was 0.8933 with RMSE of 3.6735. (Figure 5).

The VIF values of each variable were calculated to control interdependence between features used in the baseline Random Forest model. Figure 6 presents the VIF bar graph in descending order, 10 features such as Superplasticizer, Additional water, Water (2), NaOH, Total water, Water (1), NaOH (Dry), Na₂SiO₃, SiO₂ (Dry), Na₂O (Dry) exhibit extremely high VIF values (>1000).

The R² value for the test set was also high for the Random Forest model, which is known to be a robust model for high VIF values of predictor variables. In fact, ML models become very weak in terms of interpretability when multiple independent variables are strongly correlated (as multiple variables explain the same signal). In such a case, the Random Forest will often pick one of them for splits, ignore the other, or alternate randomly across trees. This leads to three subtle effects: (i) Feature importance becomes unreliable; Gini or impurity-based importance gets “diluted” across correlated features. None of them look important, even though the group is; (ii) Interpretability lowers, and it would not be possible to say “X₁ matters more than X₂” when they are shadow-boxing the same information. (iii) Model size and noise increase slightly, redundant features mean more candidate splits, which means more randomness leading to marginal inefficiency. In order to ensure robustness in model evaluation, increase efficiency in accuracy and achieve explainability in the next stage, the pipeline involved 30 runs of fine tuning of a set of ML models with the exact seeds and data splits used for baseline model evaluation.

3.3. Evaluation of Hyperparameter Optimized Models

A set of ML models was evaluated with 30 runs of the fine-tuning pipeline using the selected features in the following stage. The dataset is split into train and holdout sets with an 80/20 ratio using the pre-determined seeds, and only the train set is used in the training process with 5-fold cross-validation. The 5-fold cross-validation was conducted on the training dataset to estimate out-of-sample predictive performance, and the trained model was subsequently evaluated on an independent holdout dataset that was not involved in either model fitting or validation. The evaluation metrics for all 30 runs are presented in Appendix C. The cross-validated (CV) R² represents the mean performance across the out-of-fold test partitions, whereas the holdout R² (Holdout) reflects performance on a single, unseen realization of the data distribution

Table 10. Average metrics for fine-tuned models (30 runs).

Model	Holdout R2	Holdout RMSE	Holdout MAE	CV R2	CV RMSE
CatBoost	0.9237	2.995	2.224	0.9152	3.2209
XGBoost	0.9167	3.1488	2.2867	0.906	3.3978
Gradient Boosting	0.9054	3.3722	2.3904	0.8996	3.5108
ExtraTrees	0.8894	3.6799	2.9701	0.8782	3.9005
LightGBM	0.8776	3.8502	2.7892	0.8449	4.406
Random Forest	0.8716	3.9903	3.1352	0.8597	4.2089
SVR	0.8513	4.2611	3.2999	0.8354	4.5054
MLP	0.8064	4.8227	3.9577	0.8579	4.1708
Decision Tree	0.7752	5.1786	4.1551	0.775	5.2726
KNeighbors	0.7142	5.9684	4.5852	0.6532	6.688
Lasso	0.6156	6.8971	5.6716	0.6184	6.9271
ElasticNet	0.6154	6.8994	5.6747	0.6185	6.9271
Ridge	0.6154	6.8981	5.674	0.6185	6.9266
Linear Regression	0.6153	6.8976	5.6743	0.6183	6.9264

and Figure 7 and Figure 8 summarizes average performance of the hyperparameter-optimized models after 30 runs of the pipeline, each run with 1000 TPE iterations of the Hyperopt library. The models were trained with the set consisting of five optimal input variables determined as a result of the GA-based feature selection process explained in the previous section.

When the average performance across the out-of-fold test partitions is compared, CatBoost appeared as the best-performing model, followed by XGBoost and Gradient Boosting. This is followed by ExtraTrees, LightGBM and Random Forest. The results indicate the dominance of ensemble models in exhibiting the best performance, where boosting-based models performed slightly better than the bagging-based ones. Among the foundation models, SVR appeared as the best performing model, followed by MLP, Decision Tree and KNN. Linear models, regardless of being the penalized ones or simple linear regression, exhibited the lowest performance among all models. Comparison of the average CV R² and holdout R² values revealed no indication of overfitting and no evidence of bias arising from an unbalanced holdout split, as both metrics exhibited consistent and comparable predictive performance.

The best overall holdout performance was exhibited in the 30th run (seed = 29) (

Table 11. Performance metrics for fine-tuned models (30th run).

Model	Holdout R2	Holdout RMSE	Holdout MAE	CV R2	CV RMSE	Seed
CatBoost	0.9627	2.3113	1.7425	0.9182	3.2227	29
GradientBoosting	0.9228	3.3233	2.2458	0.889	3.7145	29
LightGBM	0.9137	3.5147	2.7174	0.8439	4.439	29
ExtraTrees	0.9136	3.5163	2.857	0.8814	3.8748	29
RandomForest	0.8775	4.1863	3.4466	0.8557	4.2775	29
SVR	0.8618	4.4467	3.3173	0.8513	4.3612	29
MLP	0.8262	4.987	4.1373	0.8431	4.4839	29
DecisionTree	0.8061	5.2672	3.9692	0.8175	4.7925	29
KNeighbors	0.7876	5.513	4.4698	0.6658	6.5232	29
Lasso	0.5746	7.8024	6.6356	0.6567	6.6273	29
Ridge	0.5745	7.8033	6.6362	0.6569	6.6262	29
ElasticNet	0.5745	7.8029	6.638	0.6569	6.6259	29
LinearRegression	0.5744	7.8041	6.6365	0.6567	6.6277	29

) by the fine-tuned CatBoost model. In this run, the fine-tuned CatBoost model was able to achieve a holdout R² score of 0.9627 and RMSE of 2.3113 with only five input variables (i.e., set of Concentration(M)NaOH, NaOH (Dry), GGBFS, Initial curing temp(C), Coarse Aggregate). Feature importances plot for CatBoost is given as Figure 9. Figure 10 provides a predicted vs. actual plot for all algorithms for this run of the pipeline for the holdout set.

3.4. Feature Importance Analysis

In the feature importance analysis, the focus is on evaluating how each input variable contributes to the predictive performance of the best-performing model for estimating the output feature. CatBoost was chosen for this examination as it outperformed the other models. We would like to note that reported importance rankings are model-dependent interpretations rather than universal causal claims. The measure of importance for each variable was determined by assessing the change in the model’s error when that specific variable was used in the prediction process. The algorithm uses each model’s built-in feature_importances attribute to get the feature importance values for each feature. The results show that GGBFS has the highest contribution to the predictive ability by a significant margin. This indicates the important role of blast furnace slag in the mechanical properties of geopolymer concrete. In addition, NaOH (Dry) also has a significant contribution to the predictive ability of the model. In addition, Concentration, Initial Curing Time, and the amount of Coarse Aggregate were identified as factors affecting the concrete strength.

Although GGBFS exhibited a very weak linear correlation with compressive strength (r = 0.07), it was identified as the most important feature by the CatBoost model. This apparent discrepancy arises because Pearson correlation measures only the individual linear relationship between a predictor and the target variable, whereas CatBoost feature importance reflects the overall predictive contribution of a variable within a multivariate and nonlinear modeling framework. In particular, GGBFS showed a strong negative correlation with coarse aggregate (r = −0.92), indicating substantial multicollinearity, which can suppress its apparent marginal correlation with strength. However, tree-based models such as CatBoost can effectively capture nonlinear and interaction effects between variables. Therefore, the high importance of GGBFS suggests that it plays a critical role in combination with other parameters, even though its individual linear association with strength appears weak. This finding highlights the limitation of relying solely on linear correlation analysis and demonstrates the ability of the proposed machine learning framework to uncover complex multivariate relationships governing compressive strength.

4. Discussion

In this study, various machine learning models were used to predict the compressive strength of geopolymer concrete. Boosting-based ensemble models, particularly CatBoost, XGBoost, and Gradient Boosting, achieved the best performance, followed by bagging methods, while SVR was the strongest among base models and linear models performed the worst; moreover, the consistency between CV R² and holdout R² indicated reliable generalization without overfitting or holdout bias.

For the 30th run (seed = 29), the Random Forest model trained with the reduced feature set (five variables) achieved (holdout) R² of 0.8775 on the unseen data, while the baseline Random Forest model trained with the full feature set (19 variables) achieved R² of 0.9306 under the same conditions. Based on the 30th run, although the baseline model exhibited a slightly higher performance (ΔR² ≈ 0.06), the overall predictive performances remained comparable. This result along with the average Random Forest performance of R² of 0.8716 when compared with the baseline model performance, indicate that the hyperparameter optimization was able to largely compensate for the reduction in input dimensionality, and that decreasing the number of features from 19 to 5 did not result in a substantial or practically significant deterioration in predictive performance, while greatly enhancing the robustness in model interpretability.

A comparison of our findings with the existing literature reveals both alignment with established trends and notable methodological distinctions. Previous studies have consistently reported that parameters such as NaOH molarity, GGBFS proportion, and curing conditions exert a marked influence on geopolymer compressive strength; however, these investigations were often restricted to specific experimental setups or relatively small datasets. In this study, a larger and more diverse dataset comprising 274 mix designs was analyzed, enabling the confirmation of these known effects within a broader parameter space. By combining machine learning models with a multi-objective genetic algorithm, we were able to optimize predictive accuracy while reducing redundancy among variables, ensuring interpretability and thereby offering a practical tool that complements and extends the scope of earlier experimental research.

A comparison of the existing literature and the present study on geopolymer compressive strength prediction is given in

Table 12. Comparative summary of existing literature and the present study on geopolymer compressive strength prediction.

Literature	Data/Experimental Conditions	Methods/Analysis	Key Findings
Pham and Nguyen (2024) [14]	Broadly compiled industrial geopolymer mixes (various FA/GGBFS blends).	Gene Expression Programming (GEP)/experimental data compilation.	Highlighted the strong effect of alkaline activators and raw material composition.
Hardjito and Rangan (2005) [3]	Low-calcium FA, controlled curing (high temperature).	Experimental mechanical tests; microstructural analysis.	Demonstrated that heat curing accelerates early strength gain, with responses varying by mix composition.
Wattanasiriwech et al. (2017) [26]	Sequential curing at 90 °C, followed by humid and then dry curing at 40 °C.	Experimental strength testing.	Showed that multi-stage curing can produce consistently high strengths.
Assi (2016) [27]	Experimental tests and microstructural evaluation.	Reported that silica fume-based systems are less sensitive to curing conditions than sodium silicate systems.	Reported that silica fume-based systems are less sensitive to curing conditions than sodium silicate systems.
Present Study	274 experimental records covering 19 variables (FA, GGBFS, Na2SiO3, NaOH, curing parameters, aggregates, water, SP, etc.).	Data preprocessing, correlation and network analysis, VIF calculation, multi-objective GA for feature selection, hyperparameter tuning; models: RF, XGBoost, LightGBM, CatBoost, SVR, MLP, etc.	Best performance achieved with CatBoost (R2 = 0.9583). GGBFS identified as most influential feature; NaOH (Dry) is also critical for estimation.

.

This study introduced a combined approach of machine learning and multi-objective genetic algorithms to achieve both high predictive accuracy and stable variable selection. By modeling the effects of various input parameters across a broad dataset, multicollinearity is reduced through GA-based feature selection, enhancing interpretability. The analysis also considered ambient curing conditions, capturing their interactions with other mix components through correlation and network-based approaches. The models reproduce known trends, quantify optimal ranges, and examine interactions among mix design factors. Feature importance results from CatBoost highlight Na₂SiO₃ and dry NaOH as key predictors, consistent with experimental observations. Overall, this work reinforces established patterns while providing a data-driven predictive framework that combines feature selection with GA and optimized machine learning models. The study confirms trends at a larger scale and introduces a methodological contribution by simultaneously optimizing feature stability and prediction accuracy within a single workflow.

5. Conclusions

The production of ordinary Portland cement is a major source of CO₂ emissions, whereas geopolymer concrete significantly reduces environmental impact by utilizing industrial by-products such as fly ash and slag. At the same time, geopolymer concrete can achieve mechanical properties comparable to conventional concrete when key parameters, including precursor composition, activator type, and curing conditions, are properly optimized. These characteristics demonstrate that geopolymer concrete offers an effective balance between environmental sustainability and structural performance.

This research was focused on reinforcing the reliability and transparency of machine learning frameworks used to estimate geopolymer concrete compressive strength. Through implementing a dependency-aware feature selection protocol, we mitigate the negative effects of multicollinearity in a dataset (with many linearly dependent features) while maintaining high predictive precision of the model. The methodology followed a three-fold objective: first, engineering a specialized multi-objective NSGA-II genetic algorithm that filters features based on their linear independence; second, restructuring a high-dimensional geopolymer dataset into an optimized variable subset; and finally, benchmarking the predictive efficacy of twelve hyperparameter-tuned models trained on this refined input dataset. The results have shown that the number of linearly dependent features can be efficiently reduced without sacrificing the prediction accuracy for this dataset. It should be emphasized that the absence of a substantial loss in predictive accuracy between the baseline model trained on the full feature set and the models trained on the reduced feature subset can be largely attributed to the hyperparameter optimization applied to the latter, which enabled the models to maintain comparable performance despite the lower input dimensionality.

The features that have an influence on compressive strength prediction were observed as GGBFS, NaOH (Dry), Concentration, Initial curing time and Coarse Aggregate. This highlights the potential of industrial by-products like ground granulated blast furnace slag (GGBFS) as sustainable alternatives to conventional aggregates. Higher NaOH concentrations, particularly in the 8–16 M range, generally improve strength due to enhanced dissolution of aluminosilicate species.

Geopolymer concrete is particularly attractive for industries requiring high durability and resistance to aggressive environments, such as infrastructure, marine constructions, and precast concrete elements.

To emphasize the practical relevance of the findings, the best-performing ensemble model achieved an R² of 0.96 on the holdout dataset, with consistently low prediction error across cross-validation folds, confirming its robustness. The integration of correlation diagnostics enabled the selection of a reduced and physically consistent feature subset, mitigating the instability commonly caused by multicollinearity in geopolymer datasets. This structured integration of interpretability within the modeling pipeline improves confidence in feature importance rankings and enhances engineering transparency compared to conventional black-box ML applications.

Nevertheless, the proposed framework is constrained by the moderate dataset size and reliance on a limited number of experimental sources, which may affect external validity. Model performance may also exhibit sensitivity to data partitioning strategies. Future work should focus on expanding the dataset with independent experimental campaigns, testing transferability across different precursor systems, and validating the framework under varying curing and environmental conditions. From a practical standpoint, the developed approach provides a transparent decision-support tool for geopolymer mix design, enabling more efficient parameter prioritization and reducing experimental trial-and-error efforts.

Future research should extend the proposed framework by incorporating larger and multi-source experimental datasets to enhance external validity and improve generalization capability. The integration of additional geopolymer systems (e.g., blended precursors or alternative activator types) would allow evaluation of model transferability across different material compositions. Moreover, combining the current multicollinearity-aware pipeline with advanced explainable AI techniques and uncertainty quantification methods could further strengthen interpretability and reliability. The development of hybrid physics-informed or mechanistic-ML models may also improve extrapolation performance beyond the training domain. Finally, implementing the framework within an optimization environment for automated mix proportioning would enable direct decision-support applications for sustainable geopolymer concrete design in practical engineering settings.