A Novel Combination of PCA and Machine Learning Techniques to Select the Most Important Factors for Predicting Tunnel Construction Performance

Abstract

Numerous studies have reported the effective use of artificial intelligence approaches, particularly artificial neural networks (ANNs)-based models, to tackle tunnelling issues. However, having a high number of model inputs increases the running time and related mistakes of ANNs. The principal component analysis (PCA) approach was used in this work to select input factors for predicting tunnel boring machine (TBM) performance, specifically advance rate (AR). A reliable and precise forecast of TBM AR is desirable and critical for mitigating risk throughout the tunnel building phase. The developed PCAs (a total of four PCAs) were used with the artificial bee colony (ABC) method to predict TBM AR. To assess the created PCA-ANN-ABC model’s capabilities, an imperialist competitive algorithm-ANN and regression-based methods for estimating TBM AR were also suggested. To evaluate the artificial intelligence and statistical models, many statistical evaluation metrics were evaluated and generated, including the coefficient of determination (R²). The findings indicate that the PCA-ANN-ABC model (with R² values of 0.9641 for training and 0.9558 for testing) is capable of predicting AR values with a high degree of accuracy, precision, and flexibility. The modelling approach utilized in this study may be used to other comparable studies involving the solution of engineering challenges.

1. Introduction

In mechanized tunneling designs, estimating the tunnel boring machine (TBM) performance is considered a vital task before selecting the machine and conducting the project. It is important for the project schedule, management of relevant issues, and cost estimation of tunneling projects [1,2]. Many techniques/formulas have been proposed theoretically and empirically by previous investigators for predicting different factors related to TBM performance (e.g., penetration rate, PR and advance rate, AR) [3,4,5,6,7,8]. These empirical and theoretical techniques mostly used one or two predictors relevant to rock material and mass properties such as strength and joint conditions [9,10,11]. Hence, the mentioned techniques according to previous studies [12,13,14] are not good or strong enough to provide a suitable level of TBM performance prediction.

Aside from empirical and theoretical techniques, statistical-based models such as multiple and simple regression have been employed for assessing TBM performance [15,16,17]. In simple regression models/equations, researchers evaluated TBM performance such as PR using only one predictor, mainly from the rock mass and material properties (such as rock strength or a rock mass classification system) [18,19]. In multiple regression models/equations, the use of a minimum of two effective parameters on TBM performance was carried out by several researchers to predict TBM PR [15,16,17,20]. However, according to several studies such as [17,21], statistical-based techniques cannot always investigate complex systems. Additionally, the performance capacity of the mentioned models is not at a satisfactory level [9,12], while a satisfactory and acceptable degree of prediction is required for TBM performance in order to minimize the risks associated with tunneling project costs.

Several scholars have presented different artificial intelligence (AI)-based and machine learning (ML)-based techniques for solving TBM-related issues [2,13,22,23,24,25]. Some of the techniques proposed in this regard are support vector machine (SVM), particle swarm optimization (PSO), fuzzy inference system (FIS), and artificial neural network (ANN). These methods have been generally utilized to approximate the TBM field penetration index (FPI), TBM PR and TBM AR. The Athens Metro tunnel database was used by Benardos and Kaliampakos [26] to develop an ANN-based model to calculate TBM AR. In another project, Yagiz et al. [13] introduced an ANN approach for the estimation of the PR of TBM. On the other hand, in the case of the same datasets, Mahdevari et al. [15] intended an ML approach, namely support vector regression, to solve a TBM problem. In predicting FPI values, Feng et al. [27] and Adoko and Yagiz [28] tried to solve this problem by proposing deep learning and FIS techniques, respectively. A group of other authors tried to solve performance of TBM using other single intelligent approaches like group modeling of data handling, genetic-based and neuro-fuzzy [2,9,29,30,31]. On the other hand, some other scholars developed advanced intelligent techniques for this problem which are based on a combination of at least two different intelligent models [1,10,14,32,33]. In these combined models, there is a base technique and an optimization algorithm for improving prediction capacity of the base technique in estimating TBM performance. It is crucial to note that AI and ML approaches have been highly employed to solve difficulties in science and engineering [34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52].

After reviewing the related AI and ML works, it was found that the ANN model is one of the frequently used techniques in evaluating and predicting TBM performance. However, the ANN model is connected with several limitations that lead to a low-performance prediction for this model. These limitations include slow learning rate [53,54]. Therefore, this study aims to increase the prediction model’s ability by training the ANN using the artificial bee colony (ABC) algorithm. The capability of the ABC-ANN model is compared with another hybrid ANN and statistical-based models to choose more accurate techniques in the area of TBM AR. It is significant to note that the principal component analysis (PCA) is applied to reduce the number of input parameters for predicting TBM AR. As far as the authors know, this is the first time using the PCA in the area of TBM performance to make a more straightforward technique. Then, the PCA-ANN-ABC model results will be discussed and compared at the end of this research.

2. Methodology Background

2.1. ANN

Among many ANN structures studied, the most common use is the hidden multi-layer feed-forward network structure. The main structure of a feed-forward (FF) model usually consists of three distinct layers [55]. Each layer is made up of nodes called neurons. In an FF model, data travels in only one direction, from input to output neurons, through the hidden nodes. Each artificial node of the network receives the signal and passes the signal through an activation function to estimate the output. Every neuron’s output will be the input for the subsequent neuron. An ANN can be trained multiple times to improve its performance capacity. During the data training, the architecture of an ANN and its connection weights are altered iteratively to reduce imperfections of the predicted data [56,57].

When the architecture of an FF neural network is defined, the parameters include the activation function, layer No. and the neuron No. per layer; in the next step, there is a need to adjust the ANN weights and biases. These parameters should be intended perfectly to receive the best prediction capacity (i.e., highest performance prediction). A complete explanations about ANNs and their process in performing prediction tasks can be found in previous investigations [58,59,60,61].

2.2. ABC

A colony-based optimization algorithm, namely ABC, was proposed by Tereshko [62] which works according to bee colony behavior in real life. In this optimization technique, there are two different behaviors for bee colonies: source abandonment and food source recruitment [62,63]. According to Tereshko [62], the basic components of ABC can be described in the following:

⮚

Food source selection: In this stage, bee colonies will be searched for different sources of food and their characteristics. Some include nectar taste, the distance between bee colony and the hive, and energy richness [64].

⮚

Employed forager: an employed forager works in the food source from which it currently feeds. The responsibility of the employed forager is to share the food supplier information with other bees waiting in the hive [63,64].

In the ABC algorithm, when there are values for several parameters such as the quantity of nectar and the position of the food source, an optimization process for finding the possible answers for the problem can be performed [64]. A random number of bees will be selected in the ABC algorithm, where they are searching within the search space for possible and practical solutions. Independent artificial bees work together and exchange information to find/optimize more accurate results/solutions [65]. Actually, these bees, using knowledge and sharing data, focus on promising areas and gradually leave low-hope areas. Overall, in each iteration of the ABC algorithm, the artificial bees will improve their capabilities in finding more accurate solutions [66]. The process of the ABC algorithm will continue until the best answers/solutions that the user can define from the beginning. The role of this algorithm in a combined ABC-ANN is actually to optimize the ANN weights and biases, which will increase the performance capacity of the hybrid system. The general process of the ANN technique optimized by the ABC algorithm is shown in Figure 1.

2.3. Principal Component Analysis (PCA)

PCA’s main idea is to diminish the number of interdependent variables to preserve their variations in the set of remaining parameters. This result is achieved using a principal components transformation function on the main parameters. Principal components are not related to each other and are arranged to contain the highest variance of the main variables [67,68]. One of the most critical aspects of the PCA approach is choosing the number of PCs. This approach defines the accuracy according to the data and the desired results. Then the total changes in the number of principal components are selected based on the cumulative percentage. The expected accuracy is deemed to be 80% to 90% of the overall variations. The rate of incremental changes is defined from Equation (1), where l_k is the eigenvalue [67].

$$Percent of variation=\frac{l_k}{{\sum }_{j=1}^p{l}_k}$$

(1)

3. Materials and Methods

3.1. Case Study and Data Collection

To predict TBM AR in this study, a tunnel site and project in Malaysia were selected to be studied. This tunnel aims to transfer water from one state (i.e., Pahang) to another (i.e., Selangor). The primary rock type in the research area was granite, and the tunnel has an overburden range of 100–1400 m. The total length of this tunnel is 44 km which was excavated using 2 different construction techniques namely (1) drilling and blasting and (2) mechanized excavation or TBM. In the case of mechanized excavation, three TBMs were used to excavate approximately 35 km of the tunnel with a diameter of 5.2 m.

In this study, the database was prepared based on the previous suggestions/investigations. The mentioned database, which comprises 1286 datasets, covers laboratory test results and field observations. In terms of laboratory tests, uniaxial compressive strength (UCS) and Brazilian tensile strength (BTS) tests were conducted and their results were recorded. In this case, more than 100 rock block samples were transferred to the laboratory and the samples for the mentioned tests were prepared and tested. These laboratory tests were carried out according to ISRM guidelines [69]. In the tunnel site, several rock mass properties, i.e., rock mass rating (RMR), rock quality designation (RQD), and weathering zone (WZ) were measured for each panel which is 10 m of the tunnel site. Various values were obtained for the mentioned parameters. For example, as presented in

Table 1. Characteristics of the system inputs and output in this investigation.

Factor	Unit	Type	Max	Min	Average	STD
RQD	%	Input	95	10	53.79	27.85
UCS	MPa	Input	193	45.4	134.68	44.30
RMR	-	Input	95	46	72.74	15.71
BTS	MPa	Input	15.68	4.69	10.26	4.04
WZ	-	Input	3	1	1.68	0.69
TFPC	kN	Input	497.67	91.34	303.26	77.76
RPM	rev/min	Input	11.95	4.54	8.91	2.26
TBM AR	m/h	Output	2.57	0.20	1.08	0.57

Max: maximum; Min: minimum; STD: standard deviation.

, minimum values of 10%, 45.4 MPa, 46, and 4.69 MPa were obtained for RQD, UCS, RMR and BTS parameters, respectively, while their maximum values were measured as 95%, 193 MPa, 95 and 15.68 MPa. In terms of WZ, three zones were observed as fresh with a grade of 1 in the analysis, slightly weathered with a grade of 2 in the analysis, and moderately weathered with a grade of 3 in the analysis. As shown in Table 1, revolutions per minute (RPM) and thrust force per cutter (TFPC) were recorded by the TBMs in the tunnel and used in this study as two important machine parameters. The system output based on Table 1 is TBM AR or AR which is in the range of 0.20–2.57 m/h. It is significant to note that the database description in Table 1 is not considered as 1286 datasets, and it is only 1205 data samples. The difference in datasets is related to outliers (or unusual values in the data) that the authors attempted to identify and remove. This is considered as a mandatory task when there is a large amount of data, i.e., several data samples in this research. Outliers are able to increase the variability in the database, which produces an amount of modeling error (i.e., difference between measured AR and predicted AR) and decreases statistical power.

A total of 81 data samples were identified as outliers from the whole data (i.e., inputs and output) through the use of boxplot rules [70]. Boxplots are a simple way to depict a five-number summary, including the lowest (min), highest (max), first (Q1), second (Q2), and third quartiles (Q3). This technique calculates the range (Max–Min) and IQR (Q3 Q1), which can help us distinguish normal data from outliers. The statistical information presented in Table 1 is based on the whole database after removing outliers.

In statistical terms, correlation or dependence is defined as any statistical association (causal or non-causal) between two arbitrary variables or bivariate data. Generally, any statistical association can be implied as a correlation; yet, it usually denotes the degree of a linear correlation between a pair of parameters [71]. A correlation matrix is demonstrated as a table that reflects the correlation coefficients between different variables in which each cell depicts the correlation between a pair of variables. A correlation matrix summarizes data that can be utilized as input data for more complex analyses and as a diagnostic tool for advanced analyses [72].

Table 2. Correlation matrix for input variables.

Name	UCS	BTS	RQD	RMR	WZ	TFPC	RPM
UCS	1
BTS	0.8	1
RQD	0.71	0.67	1
RMR	0.77	0.73	0.77	1
WZ	−0.12	−0.11	−0.22	−0.23	1
TFPC	−0.72	−0.67	−0.64	−0.72	−0.05	1
RPM	−0.76	−0.78	−0.68	−0.69	0.02	0.68	1

and Figure 2 show the correlation matrix values for the input parameters.

3.2. Study Steps

The flowchart of the study and its different steps to solve problems related to TBM AR are presented in Figure 3. According to this flowchart, the model’s inputs and output were measured in the laboratory and the tunnel site after investigating the data needed. Then, to propose a simpler model of interest and importance, the PCA technique was used to generate new input parameters. The new inputs are actually a combination of the other inputs presented in Table 1. After that, the modeling stage of this study was started by constructing the feed-forward ANN and ANN optimized by ABC models to predict TBM AR.

On the other hand, a multiple regression model and an ANN-imperialist competitive algorithm (ICA) technique were built for comparison purposes. The developed models were evaluated and the more accurate predicted model was selected and introduced for AR estimation. Finally, the effects of all model predictors were investigated and reported using another statistical technique.

3.3. Research Methodology

The number of input parameters used is seven, and the output parameter is the actual AR (m/h). Given that many input parameters in ANNs increase the error, PCA can be employed to orthogonalize the input variables relative to each other. The input density diagram is shown in Figure 4. MINITAB version 14.0 software was utilized to analyze the input parameters using PCA. The effects of the parameters are shown in

Table 3. Eigen Analysis correlation matrix for establishing input variables by PCA.

Parameter	Inputs
	PCA 1	PCA 2	PCA 3	PCA 4	PCA 5	PCA 6	PCA 7
Eigenvalue	4.6162	1.0666	0.3896	0.3151	0.2323	0.1937	0.1865
Proportion	0.659	0.152	0.056	0.045	0.033	0.028	0.027
Cumulative	0.659	0.812	0.867	0.913	0.946	0.973	1

. The graph of the effect of PCA parameters, which expresses the eigen analysis correlation matrix, is shown in Figure 5.

According to Table 3, the conversion of seven input parameters into four variables of PCA 1 to PCA 4 using PCA results in approximately 91% of the data; due to data convergence, better results can be reached. The resulting input variables using PCA are listed in

Table 4. Relationship between the principal components and input parameters.

Variable	Unit	PCA 1	PCA 2	PCA 3	PCA 4	PCA 5	PCA 6	PCA 7
RQD	(%)	0.397	0.138	0.369	0.737	0.048	0.331	0.181
UCS	(MPa)	0.424	−0.031	−0.151	−0.159	−0.433	0.433	−0.629
RMR	(-)	0.417	0.128	0.317	−0.019	−0.338	−0.766	−0.092
BTS	(MPa)	0.412	−0.035	−0.517	−0.129	−0.293	0.056	0.675
WZ	(-)	−0.074	−0.941	0.07	0.23	−0.21	−0.083	0.009
TFPC	(kN)	−0.388	0.235	−0.499	0.573	−0.383	−0.202	−0.174
RPM	(rev/min)	−0.405	0.146	0.469	−0.185	−0.648	0.256	0.276

. Considering that the number of input parameters has decreased, PCA’s resulting four input parameters were used to model the ANN (Table 4). Also, considering that AR output data’s statistical behavior should be evaluated, its histogram is plotted in Figure 6. This indicates that 966 samples from the sample set are in the range of 0.2 to 1.56; approximately 80% of the data volume falls within this range. The probability plot diagrams for determining the normal distribution of the AR parameter (see Figure 6) show that their statistical behavior’s AR output parameter follows the normal distribution.

In this research, the new feed-forward ANN is selected and used. To start the modeling, the data should be randomly classified into two groups to reduce the effects of excessive errors. Of the 1205 data samples, 70% (844 samples) were chosen and applied for training, and 30% (361 samples) were chosen for network testing. This is in line with some of the previous studies [73,74,75,76]. In the proposed model, four input parameters were obtained through the PCA technique. Thus, the trained ANNs in the input layer have four nodes, while in the output layer, it has only one node (Figure 7). Hidden two-layer networks have been used for modeling. For an ANN model, the No. of hidden layers and the No. of neurons in the hidden layers vary on the problem. Accordingly, the trial-and-error technique was applied to obtain the ideal structure (i.e., the structure that best represents the data). A standard heuristic method for the maximum No. of nodes is given in Equation (2) [77]:

$$N_H\le 2N_I+1$$

(2)

where: N_H is the No. of nodes in hidden layer, and N_I is the No. of model inputs.

Based on Equation (2), the total No. of N_H should be equal to or less than 9. Therefore, different architectures have a maximum of two hidden layers and a maximum of nine trained neurons. The employed architectures are given in

Table 5. The trained ANN architectures.

Num	Topology	Num	Topology	Num	Topology	Num	Topology	Num	Topology
1	1-1	5	2-1	9	3-1	13	4-1	17	5-1
2	1-2	6	2-2	10	3-1	14	4-2	18	5-2
3	1-3	7	2-3	11	3-3	15	4-3	19	5-3
4	1-4	8	2-4	12	3-4	16	4-4	20	5-4

which include a total number of 20 models. In all models, the hyperbolic tangent stimulation function and the Levenberg–Marquardt training algorithm were used. In optimizing the ANN weights, the ABC algorithm as a strong optimization model was applied to optimize the ANN weights and ANN biases. ABC produces the least calculation error for the trained architecture. The properties of the ABC algorithm parameters are displayed in

Table 6. The initialization parameters used in the ABC algorithm.

Number of Bees	Source Number of Bees	Max of Cycle Number	Onlooker Number
10	5	50	5

.

Among the trained models to determine the AR output parameter, the model with the 4-5-4-1 topology based on RMSE, R, and other statistical indices has been selected as the best model in Equations (3)–(5). The characteristics of the said model are provided in

Table 7. Neural network characteristics as the best model.

Neural Network’s Features
Number of Inputs	Number of Outputs	Number of Hidden Layers	Number of Nodes in Hidden Layers	Transfer Function	Training Algorithm
5	1	2	5-4	tansig	Translim

. The results of the best approach in the training and test sections are shown in

Table 8. Statistics of the best ANNs combined with the ABC algorithm using the 4-5-4-1 topology.

Step	Statistical Index	PCA-ABC-ANN 2L (5-4)
Train	RMSE	0.11
	AAE	0.12
	VAF%	96%
Test	RMSE	0.12
	AAE	0.13
	VAF%	96%

. The statistical indices employed for the performance evaluation of the topologies include Root Mean Squared Error (RMSE), Average Absolute Error (AAE), and Variance Account Factor (VAF), which are defined according to Equations (3)–(5) [78].

$$\mathrm{RMSE}={\left[\frac{1}{n}{\displaystyle \sum }_{i=1}^n{\left(P_i-O_i\right)}^2\right]}^{\frac{1}{2}}$$

(3)

$$\mathrm{AAE}=\frac{\left|{\sum }_{i=1}^n\frac{\left(O_i-P_i\right)}{O_i}\right|}{n}\times 100$$

(4)

$$\mathrm{VAF}=\left[1-\frac{var\left(O_i-P_i\right)}{var\left(O_i\right)}\right]\times 100$$

(5)

According to Table 8, PCA-ABC-ANN 2L (5-4) networks have the lowest error in RMSE, AAE, and VAF indices and have the highest value of R². For networks with the 4-5-4-1 topology, the values of R² in training, testing, and all modes for the two AR outputs are 0.9641, 0.9558, and 0.9617, respectively. The error criteria for training and test samples were computed using data rates in the main range of parameters and not in the normal range [−1, +1], which is sometimes used in the literature. Figure 8 illustrates the PCA-ABC-ANN cost graph. The ANN performance is demonstrated in Figure 9 and Figure 10 for the three phases of training, validation, and testing.

The model’s predicted values against their values from experiments for the training and test data are shown in Figure 11 and Figure 12 to visualize the performance of the PCA-ABC-ANN 2L (5-4). As a result, the values predicted by the PCA-ABC-ANN model are very close to the line y = x, indicating a high accuracy capacity of this model.

In most published articles on ANN models, it is common for the authors to provide the optimal ANN technique without any data about the proposed ANN weight values. Any structure without the ANN model’s final weight values is of little value to experienced researchers and engineers. A proposed ANN architecture must be accompanied by (quantified) weight values to be useful [63]. After finding a stable and final model in determining the AR output parameter, it is necessary to generate an information table for weights and biases so that the final modeling can be determined at any time, even with new data. 

Table 9. Final bias and weight values of the developed PCA-ABC-ANN technique.

IW					b1
1.0000	0.0048	−1.0000	−0.6979		−0.3815
−0.7198	0.0175	−0.0788	−0.6253		0.9675
−1.0000	0.1828	−0.8656	−0.4221		−0.6311
−1.0000	0.1674	1.0000	0.5205		0.7721
−0.4443	0.3115	1.0000	−0.5144		0.5210
LW1					b2
−0.8369	0.6940	1.0000	−0.0327	0.1430	0.9120
−0.4156	0.0648	1.0000	0.4767	−0.3320	−0.4661
−0.0545	0.6210	−1.0000	−0.6569	0.3856	0.6196
1.0000	−1.0000	−0.0477	1.0000	0.7499	0.1558
LW2					b3
0.3997	0.4148	−0.5161	−0.0175		0.1067

IW: Weight values for the input layer, LW1: Weight values for the first hidden layer, LW2: Weight values for the second hidden layer, b1: Bias values for the first hidden layer, b2: Bias values for the second hidden layer, b3: Bias values for the output layer.

displays the ABC algorithm’s final weight and bias for both hidden layers.

3.4. Validation of the Developed Model

Multiple linear regression models were used to validate the ANN-based techniques. In the multiple regression approach, two or more individual variables have a significant influence on the dependent variable (Equation (6)) [79]:

$$y=f\left(x_1,x_2,\dots \right)\to y=a_0+a_1{x}_1+a_2{x}_2+\cdots$$

(6)

where y is the dependent or output parameter; x₁, x₂, … are the model inputs or predictors; and a₁, a₂, a₃, … are the coefficients related to each input [79,80,81]. A series of regression equations were investigated for input and output variables. These equations are shown in

Table 10. Multiple linear regression equations with independent parameters.

Models	R2%	Parameters	Equation Numbers
AR = 0.6405 − 0.002603 RQD − 0.001639 UCS − 0.006375 RMR − 0.00789 BTS − 0.00058 WZ + 0.002999 TFPC + 0.04854 RPM	91.79	7	(7)
AR = 0.6373 − 0.002600 RQD − 0.001639 UCS − 0.006364 RMR − 0.00788 BTS + 0.003002 TFPC + 0.04858 RPM	91.68	6	(8)
AR = 0.4662 − 0.002889 RQD (%) − 0.007356 RMR − 0.01365 BTS + 0.003159 TFPC + 0.05414 RPM	91.39	5	(9)
AR = 0.4266 − 0.009943 RMR − 0.01539 BTS + 0.003239 TFPC + 0.06152 RPM	90.66	4	(10)
AR = 1.7366 − 0.016681 RMR − 0.02506 BTS + 0.09089 RPM	82.45	3	(11)
AR = 0.5165 − 0.05625 BTS + 0.12752 RPM	73.29	2	(12)
AR = −0.7618 + 0.20618 RPM	67.04	1	(13)

with one to seven independent variables. Among all the equations, the most appropriate coefficients of the multiple linear regression model for the AR output parameter involve an equation with seven parameters presented in Equation (7). Also, the statistical indices for the best multiple linear regression model for all the samples are given in

Table 11. Statistical results of bee algorithms, colonial competition, and multiple linear regression model for all samples.

Step	Statistical Index	PCA-ABC-ANN 2L (5-4)	PCA-ICA-ANN 2L (5-4)	MLR 7
All	RMSE	0.11	0.16	0.16
	AAE	0.12	0.17	0.16
	VAF%	96%	92%	92%

.

For another evaluation of the model, the colonial competition algorithm is combined with an ANN. The colonial competition approach is a random population-based technique inspired by the idea of humans’ socio-political evolution. In this algorithm, several colonial countries, together with their colonies, search for finding the best answers to solve problems [82,83]. The initial answers in ICA algorithm are also known as “country”. Like ABC, in ICA, these countries try to improve themselves for finding better results/solutions. They are able to search within the search environment similar to the ABC technique. Since the main purpose of this study is not proposing a model based on ICA, the authors decided to stop explaining about this technique herein. The full description of this algorithm is available elsewhere [82,84,85,86].

For using ICA instead of ABC in the hybrid system, the same ANN with a 4-5-4-1 topology was used and the TBM AR values were predicted through the system. A series of PCA-ICA-ANN models with several values of important parameters in ICA (e.g., No. of countries and No. of imperialists) were built. The mentioned analyses and modeling were conducted using a trial-and-error process. According to the obtained results in this regard, the best parameters of ICA were investigated as presented in

Table 12. Characteristics of the imperialist competitive algorithm in the ANN model.

Models’ Name	Employed Initialization Parameters in ICA
	Number of Countries	Number of Imperialists	Number of Decades
PCA-ICA-ANN 2L (5-4)	500	50	250

. It can be seen that the best PCA-ICA-ANN model has values of 500, 50 and 250 for the No. of countries, the No. of imperialists and No. of decades, respectively. In addition, RMSE, AAE, and VAF values related to the developed models in this study to estimate TBM AR are presented in Table 11.

The results of PCA-ABC-ANN, PCA-ICA-ANN, and MLR models are shown in Figure 13, Figure 14 and Figure 15 for the AR prediction. According to these figures and Table 11, the results show that the ABC-optimized ANN model has higher accuracy than the ICA-optimized ANN and multiple linear regression models. It was found that by developing the PCA-ICA-ANN model, the performance capacity of the developed MLR equation can be improved for approximately 0.01 of R². It means that the difference between these models is very low and there is no point in setting the PCA-ICA-ANN model by spending much effort and having sufficient AI knowledge. However, in case of the PCA-ICA-ANN model, the story is different. The more significant difference and higher performing capacity can be achieved by proposing this technique. R² of 0.9617 was obtained for the selected model and it is obviously better than the other proposed models in this study to predict TBM AR. The Taylor diagram was also used to evaluate and select the best model (Figure 16). According to the diagram, the accuracy of the PCA-ABC-ANN model is higher than the other models. This model is a combination of PCA technique and a hybrid ANN algorithm and can enjoy the advantages of all mentioned models in predicting TBM AR.

4. Sensitivity Analysis (SA)

Considering the PCA-ABC-ANN model exhibited superior performance compared to the PCA-ICA-ANN and regression models, an SA was conducted using the model to identify the relative impact of each predictor parameter on the TBM AR in granitic rock mass. In this way, Lek’s proposed profile method [87,88] was applied in MATLAB version 2018 software. The method attempted to analyze each input while considering other values as constants. Details on the theory and implementation of the method are available in the literature [87,88]. In this work, a total number of 1205 data samples were used to analyze the effects of each PCA on the TBM AR. According to Figure 17, all four combined and generated parameters from the input data, including RQD, UCS, RMR, BTS, WZ, TFPC, and RPM, have a desirable effect on the network output. It is important to note that the results obtained from this analysis are in line with some of the previous studies in this field [89,90,91].

5. Conclusions

The following conclusion remarks can be extracted from this study:

• Increasing the number of inputs in the ANN increases the number of errors. PCA reduces the inputs, which improves the results in the combined PCA-ANN-based models for determining AR output.
• After using the PCA results to reduce the ANN inputs, in order to optimize the weights of the ANN, the ABC algorithm was used to select the best structure for the ANNs and produce the least errors in the models. For this purpose, 20 models with different topologies were used, and the model with the 4-5-4-1 topology offered acceptable results.
• The PCA-ABC-ANN model combined with the Levenberg–Marquardt learning algorithm and the hyperbolic tangent transfer function were more capable and accurate in predicting TBM AR values. In this model, the R² values for TBM AR in the training and test stages were as 0.9641 and 0.9558, respectively, indicating the model’s high accuracy. On training data, RMSE, AAE, and VAF% of the PCA-ABC-ANN model for TBM AR values were 0.11, 0.12 and 96%, respectively, while in testing data, they were 0.12, 0.13, and 96%. The statistical indices of VAF, AAE, and RMSE presented in Table 7 indicate the model’s negligible error.
• To assess the accuracy of the ABC-optimized technique, it is compared with the ICA algorithm. For the PCA-ICA-ANN model on all data, RMSE, AAE, and VAF% for TBM AR values were 0.16, 0.17, and 92%, respectively.
• The authors have used a statistical model with seven input variables. For the MLR model on all data, RMSE, AAE, and VAF% for TBM AR values were 0.16, 0.16, and 92%, respectively.
• According to the evaluation results, the ABC algorithm received a higher accuracy level than the ICA algorithm followed by the MLR model.
• The modeling procedure introduced in this study regarding reducing the number of inputs using PCA can be implemented in the other similar fields.
• The models in this study were developed for a granitic rock mass, which includes simple geological conditions. Therefore, these models should be used in very similar conditions if very close performance is needed. Of course, the error is higher if different geological conditions are examined.