Water Leakage and Crack Identification in Tunnels Based on Transfer-Learning and Convolutional Neural Networks

Abstract

In order to solve the problems of long artificial time consumption, the inability to standardize the degree of damage, and the difficulty of maintaining data in traditional tunnel disease detection methods, this paper proposes the use of Residual Network (ResNet) models for tunnel water leakage and crack detection. ResNet proposes a residual learning framework to ease the training of networks that are deeper than those previously used. Furthermore, ResNet explicitly reformulates the layers as learning the residual functions of the reference layer inputs, rather than learning the unreferenced functions. The ResNet model is built on the Tensorflow Deep Learning (DL) framework and transfer-learning is used to optimize the model. The ResNet-V1 can be obtained by pre-training in ImageNet. The fully connected layers of the ResNet-V1 were modified to four classifications of tunnel disease. Then, the SoftMax function is used to recognize the tunnel diseases. Four network structures have been chosen, i.e., ResNet34 and ResNet50, with and without Transfer-learning, respectively. Those models were selected for testing and training on the sample dataset, and these four network structures were compared and analyzed using five types of evaluation indicators, which are the confusion matrix, accuracy, precision, recall ratio and F1. In identifying tunnel cracks and water leakage, the accuracy of ResNet50 and ResNet34 using the transfer-learning were 96.30% and 91.29%, and the accuracy of ResNet50 was 5.01% higher than that of ResNet34; for the network structure without the transfer-learning, the accuracy of ResNet50 was 90.36% and ResNet34’s accuracy was 87.87%. These data show that the accuracy of ResNet50 is higher than that of ResNet34 with or without the transfer-learning, and the deep structure framework is superior in the identification of tunnel diseases; secondly, comparing the network structures with and without the transfer-learning, it can be found that using the Transfer-Learning can improve the ResNet network’s accuracy for tunnel disease identification. The experiments and reliability analysis demonstrate the intelligent tunnel disease identification method proposed in this paper, and its good robustness and generalization performance. This method can be used for the rapid identification of cracks and water leakage in a tunnel survey, construction and maintenance, which has practical engineering implications for tunnel disease detection.

1. Introduction

With the rapid development of China’s economy, the infrastructure construction of railways continues to advance rapidly, and a series of complex geological, oversized, buried, and extra-long tunnel projects are being or will be implemented [1]. Tunnel disease is a phenomenon in a tunnel that may prevent normal use. This includes water leakage, cracks and tunnel frost damage. These not only cause damage to the infrastructure in the tunnel but also significantly shorten the service life of the tunnel, so the accurate identification of the tunnel disease is crucial to ensure tunnel safety [2,3]. Currently, the two most used approaches for detecting tunnel diseases are artificial identification and machine learning [4]. Traditional tunnel disease detection methods rely on artificial statistics, but artificial statistics is costly and slow, and the detection results are inevitably affected by human factors, making it difficult to obtain a uniform standard. Traditional artificial detection includes threshold segmentation [5], edge detection [6,7] and area growth [8], but this method is often constrained by the difference between the fracture pixels and the background pixels, and can only achieve better recognition results when the difference is obvious, and the choice of threshold value when using threshold segmentation methods can directly affect the recognition effect with large uncontrollable factors. In the field of machine learning, a method for crack edge detection based on the Gabor filter was presented by Roberto Medina [9], which is rotation-independent and can detect cracks independent of the growth direction; integral channel features were used to redefine the markers that constitute the cracks, which can achieve complex crack recognition [10]. While statistical machine learning can achieve better recognition results, it often requires the pre-processing of images to obtain abstract features of the tunnel disease, and the subsequent need for constant verification and modification to ensure that the validity of the features is a heavy workload [11,12,13,14]. The existing means of detecting tunnel engineering defects lags far behind current developmental needs. As artificial intelligence is an inevitable trend in the future of tunnel construction, the development of tunnel disease monitoring-technology based on intelligent fields has great practical engineering significance.

The deep learning method uses convolutional neural networks (CNNs) to extract features from tunnel diseases images, so the model learns to abstract tunnel diseases. This approach enables the automatic identification of tunnel cracks and tunnel leakage, making up for the shortcomings of traditional detection methods. DL is a branch of machine learning of which CNNs are an important method, and in recent years, CNNs have been used with great success in computer vision, natural language processing and speech [15,16,17,18]. With the continuous advancement of research in the field of geotechnical engineering, The deep supervised target detection network (DSOD) and ResNet model was combined to achieve intelligent recognition of rock properties, which has good robustness and generalization performance [19]. Full convolutional networks were used to learn the features of weak targets in a complex background and use residual learning in the network with acceleration network optimization, model parameters, etc., but the filter results generated by this method can cause the target signal increase in intensity and size. The expanded phenomenon causes the size and shape information of the original target to become lost [20]. Full convolutional network (FCN) was used to achieve the identification of water leakage in shield tunnels, and the method has superior robustness in overcoming pipeline occlusion [21]; in addition, the FCN was also used in the recognition algorithm, which not only achieves the identification of rock-concrete fractures, but also provides real-time statistics on their geometric length, width, and area [22]; the residual network structure was used to construct an over-prediction network model for depth feature extraction of water-rich fracture zones, which provides a basis for decision making in practical engineering [23]. At present, with the continuous development of DL, different model structures have been evolved and generated in CNNs, such as VGG, ResNet, FCN, R-CNN, etc. [24].

Different model structures can be used for tunnel disease identification in different fields to achieve better results, such as using Mask R-CNN to identify bridge cracks, using SC-Net to identify corrosion cracks in reinforced concrete [25,26], etc. These latest studies have achieved a good recognition effect. Considering the actual situation, tunnel cracks and water leakage account for a small proportion of the sample images, especially the crack images which do not obviously belong to the fine feature tunnel disease which requires deeper neural networks to extract more advanced abstract features. In this paper, the ResNet model structure is introduced, which cannot only use the deep network to extract small features but also solve the problem that the deep network is difficult to train. The Tensorflow DL framework is used to build the ResNet model, a transfer-learning mechanism is introduced, and the ImageNet dataset is used to pre-train the ResNet model to obtain the pre-trained ResNet-v1 model, keeping the front-end parameters and weights of the model unchanged and modifying the fully connected layer (FC) of the model to adapt to the four divisions of tunnel damage in this paper, followed by a 4:1 ratio. The new prediction model was then tested using the divided test set. Finally, the test results were analyzed using five evaluation metrics: confusion matrix, accuracy, precision, recall, and F1.

2. DL Based Tunnel Disease Identification

CNNs are very widely used models in DL; they are feedforward neural networks with local connectivity and shared weights in which a large number of neurons are organized in a certain way to produce responses to overlapping regions in the visual field [24]. The ResNet model is one of the cross-connected models of convolutional neural networks, proposed by Kaiming He [27]. Deeper levels of the network are required to continuously extract more abstract features since water leakage and cracks in tunnels represent a small proportion of pixels in the picture. The ResNet model was chosen for the implementation of the detection of tunnel diseases to prevent the problem of vanishing gradients or exploding gradients in deeper network layers. The recognition of tunnel diseases by the model consists of two main steps: one is the establishment of the sample dataset, and another is the construction of the model structure.

2.1. Dataset Construction

Through the statistical analysis of tunnel diseases, cracks and water leakage are the most important diseases, representing more than 86% of the tunnel diseases. This paper therefore focuses on the analysis of cracks and water leakage and considers the fact that other equipment in the tunnel, such as lining joints and pipelines, may interfere with the detection of diseases. This paper divides the sample dataset into four categories, namely Label 1, Label 2, Label 3, and Label 4, as shown in Figure 1.

Adequate sample data optimizes the model in two ways: first, it improves the abstract representation of the features of the network model, and second, it improves the network’s ability to generalize data and avoid overfitting the model [28]. The majority of the tunnel disease image dataset used in this paper was obtained from random photographs taken by field personnel on the tunnel surface, and a small portion of the images were retrieved from the network, for a total of 600 sample data. Since the number of obtained samples is limited, image augmentation is used to increase the size of the dataset by making a series of random changes to the training images to produce similar but different training samples. For example, we can crop the image in different ways so that the objects of interest appear in different locations, thus reducing the model’s dependence on where the objects appear: we can also adjust factors such as brightness and color to reduce the model’s sensitivity to color, etc. The training images were rotated once every 30°, 12 times in total. Additionally, a total of 7600 image samples were then obtained after image augmentation, among which the image samples were divided into a training set (6080) and a test set (1520) by a category in a 4:1 ratio [29]. The classification and number of tunnel diseases is shown in

Table 1. Classification and number of tunnel diseases.

Types of Tunnel Diseases	Sample Sets	Training Sets	Test Sets
Label 1	2100	1680	420
Label 2	2020	1616	404
Label 3	1780	1424	356
Label 4	1700	1360	340
Total	7600	6080	1520

.

2.2. Construction of ResNet Models in Tunnel Disease Identification

2.2.1. ResNet

In 1962, Hubel and Wiesel proposed the concept of receptive fields by studying the visual cortical cells of cats [30,31]; in 1979, Fukushima proposed a neurocognitive machine model based on the concept of receptive fields which is considered to be the first convolutional neural network implemented [32]; in 1998, LeCun et al., combined convolutional layers and in 2012, Krizhevsky et al. proposed the AlexNet convolutional neural network using modified linear units as the activation function and achieved excellent results in the ImageNet image evaluation [33,34], which is an important milestone in the history of DL development. The feature extraction process from input to output of a standard convolutional neural network is shown in Figure 2.

As shown in Figure 2, the convolutional neural network identifies the tunnel lesions in four main steps. Firstly, the input to the CNN is actually a three-dimensional neuron (one matrix for each of the color channels in RGB). The output of the convolutional layer is then obtained by multiplying the convolutional kernel with the digital matrix and then summing it, which achieves the initial extraction of the tunnel disease features. As the feature maps obtained after convolution are still large, a pooling layer must be used to reduce the dimensionality of each feature map in order to retain the most useful image information. Finally, the extracted features are aggregated in a fully connected layer, and the score for each category is obtained by a classifier based on which the type of tunnel disease is predicted.

On the surface, the more layers of convolution there are in a CNN, the more likely the model is to acquire higher-level abstract features. However, overly deep CNNs are generally difficult to train, due to the exponential decay or growth of the cumulative backpropagation error signal as the number of layers of the neural network increases, whereas ResNet introduce the cross-layer connections and constructs the residual modules that solve the degradation problem compared to ordinary neural networks [35]. The following is a brief description of its principle in the general network back propagation process—the loss function and gradient values are shown in Equation (1) [36,37]:

$$\left\{\begin{array}{c}Loss=F\left(X,W\right)\\ \frac{\partial Loss}{\partial X}=\frac{\partial F\left(X,W\right)}{\partial X}\end{array}\right.$$

(1)

The $F\left(X,W\right)$ is the loss function of the network and is used to estimate the inconsistency between the predicted value F of the model and the real value Y, where W is the parameter of the neural network and X is our given input. Taking the loss function as the optimization goal of the entire network, the optimized network parameter W is obtained by the multiple derivation of input X, and the error can reach the convergence state; finally, the model has a good prediction effect. Similarly, it extended the loss function of the network to a multi-layer neural network and the value of the gradient was introduced according to the chain rule, as shown in Equation (2) [36,37].

$$\left\{\begin{array}{c}Loss=F_n\left(X_n,W_n\right);L_n=F_{n-1}\left(X_{n-1},W_{n-1}\right);\dots L_2=F_1\left(X_1,W_1\right)\\ \frac{\partial Loss}{\partial X_i}=\frac{\partial F_n\left(X_n,W_n\right)}{\partial X_n}\ast \dots \ast \frac{\partial F_n\left(X_{i+1},W_{i+1}\right)}{\partial X_{i+1}}\end{array}\right.$$

(2)

We can see that the gradient of the front layer network becomes smaller and smaller as the error is back propagated. ResNet cleverly introduces a residual structure, as shown in Figure 3.

From Figure 3, the ResNet modifies the output layer $H\left(X\right)=F\left(X\right)$ to $H\left(X\right)=F\left(X\right)+X$, at which point the gradient value of the multi-layer neural network changes from Equation (2) to Equation (3) [27]:

$$\frac{\partial X_{i+1}}{\partial X_i}=\frac{\partial X_i+\partial F\left(X_i,W_i\right)}{\partial X_i}=1+\frac{\partial F\left(X_{i+1},W_{i+1}\right)}{\partial X_i}$$

(3)

We thus learn that the gradient does not disappear from Equation (3), even if the network is very deep.

2.2.2. Pre-Training Model ResNet_V1 Based on Transfer-Learning

Transfer-learning is a new type of machine learning approach that uses existing knowledge to solve problems in related domains, concentrating on the target task rather than learning all source and target tasks simultaneously [38]. In transfer-learning, the roles of the source and target tasks are no longer symmetrical, and unlike traditional machine learning, which has to be retrained for different target tasks, transfer-learning does not [39]. The difference between traditional machine learning and transfer-learning can be seen in Figure 4.

Transfer-learning can find the same parameter information between the source and target domains, allowing the data of the source domain and the data of the target domain share some model parameters. This can be applied to a new and different dataset through feature extraction and parameter tuning, thus avoiding costly data labelling efforts and greatly improving the learning performance. There are four main applications of transfer-learning: firstly, the new (target) dataset is small and similar to the original training dataset; secondly, the new (target) dataset is small and different from the original training dataset; thirdly, the new (target) dataset is large and similar to the original training dataset; and fourthly, the new (target) dataset is large and different from the original training dataset. We chose the second scenario in transfer-learning since the number of new training samples cannot reach our desired value. We added a new fully connected layer to the remaining pre-training layer with the number of classifications matching the tunnel disease category we set for the target dataset, then randomized the weights of the new FC layer and froze all the pre-trained network weights, and finally only the parameters of the new fully connected layer was trained after feature extraction. This approach avoids the problem of overfitting caused by the small size of the new dataset; this process is shown in Figure 5.

ImageNet is a large visual database for visual object recognition software research which contains more than 14 million images covering more than 20,000 categories, among which more than a million images have explicit category annotations and annotations of object locations in the images [34]. Transfer-learning is generally performed in two ways: either fine-tuning or fixing weights [38]. Fixing weights is the way we choose to: use ResNet_V1 as the pre-training model obtained after training ResNet on the ImageNet dataset; fix the ResNet_V1 correlation layer and modify the network output layer for the type of tunnel disease recognition; use the method of fixed weights to avoid the update of the ResNet model parameters after each round of training by the structural layer after fine-tuning, enabling us to accelerate the network training. Tunnel disease identification is a multi-classification problem (c > 2) for which the SoftMax function was chosen as the classification layer of the convolutional neural network which takes the output of the fully connected layer as input and exponentiates it, the SoftMax function is defined as Equation (4) [34].

$$\left\{\begin{array}{c}{V}_i=x\ast w+b\\ S_i=\frac{e^{V_i}}{{{\displaystyle \sum }}_j^c{e}^{V_j}}\end{array}\right.$$

(4)

x is the input value of the previous layer of neurons; w is the weight; b is the bias; $V_i$ represents the output of the classifier’s pre-stage output unit; i indicates the category index; the total number of categories is C; $S_i$ is the ratio of the index of the current element to the sum of the indices of all elements; SoftMax converts the output of multiclassification into a relative probability with a positive value and a sum of 1. In summary, the process of tunnel disease identification with the transfer-learning-based ResNet model in this paper is given in Figure 6.

2.3. Model Evaluation Indicators

The confusion matrix directly reflects the model’s classification of tunnel defects. In the sample dataset, we already know the labels of the images in the real case, and we can also use the known labels to determine which predictions the trained model makes on the test set are correct and which are wrong. For this, we can introduce four basic indicators: one is the number of true positives (TPs), where the true value is positive and the model considers it to be positive; second, the number of false negatives (FNs), for which the true value is positive and the model considers it to be negative; third, the number of false positives (FPs), for which the true value is negative and the model considers it to be positive; fourth, the number of true negatives (TNs), for which the true value is negative and the model considers it to be a negative quantity true negative (TN). Combining the above four indicators in a table gives the confusion matrix for the dichotomous problem, as in

Table 2. Confusion matrix for binary classification problems.

Confusion Matrix		True Value
		Positive	Negative
Predicted value	Positive	TP	FP
	Negative	FN	TN

.

We can learn that the larger the number of TPs and TNs and the smaller the number of FPs and FNs, the better the model predicts the classification. However, the statistics in the confusion matrix are individual numbers, and sometimes it is difficult to measure the recognition effect of the model just by counting the number of individuals in the face of a large amount of data. We therefore introduced four secondary indicators, namely accuracy (ACC), precision (P), recall (R) and F₁, to analyze the overall data to value the model [40].

ACC is expressed as the proportion of samples correctly classified by the model as a percentage of the total number of samples, for all categories of measure, as in Equation (5) [41]:

$$ACC=\frac{TP+TN}{TP+FP+FN+TN}$$

(5)

P is the number of samples for which the model predicts positive tunnel disease, as a proportion of the actual samples that are also positive. The precision rate measures the ability of the model to discriminate between negative samples; the higher the precision rate, the better the model is at discriminating between negative samples, as in Equation (6) [41]:

$$P=\frac{TP}{TP+FP}$$

(6)

R is the proportion of all samples that are actually positive cases that are predicted to be positive. The higher the recall, the better the model’s ability to discriminate between positive samples, as in Equation (7) [41]:

$$R=\frac{TP}{TP+FN}$$

(7)

F₁ is the summed average of the precision and recall in the range [0, 1], and the higher the value is, then the more robust the model, as in Equation (8) [41]:

$$F_1=\frac{2PR}{P+R}$$

(8)

3. Experimental Analysis of the Model

To verify the effectiveness of transfer-learning and model selection in this paper, two models, ResNet34 and ResNet50, were selected using a dataset of 7600 samples with and without transfer-learning base on the training set, and a test set was used to compare the recognition effectiveness of the four models. The superiority of the ResNet50 model for tunnel disease recognition was confirmed using confusion matrix, ACC, P, R, and F₁ score.

3.1. Training Model

The ResNet shallow network structures are ResNet18 and ResNet34, while the deeper ones are ResNet50, ResNet101, ResNet152, etc. The number after ResNet represents the number of layers that need to be updated with parameters through training. The residual module is introduced in both structures, the shallow network structure uses the BasicBlock residual module, and the deeper network structure uses the BottleNeck residual module, as shown in Figure 7.

ResNet34 and ResNet50 were selected as network structures for comparison: the number of training iterations was 5000, the learning rate was 0.0002, and the momentum decay parameter was 0.9. As the number of iterations increases, the accuracy rate shows an increasing trend, indicating that the ResNet network can be selected for the identification of tunnel diseases. Within a certain range, the number of iteration steps is positively correlated with the model recognition accuracy, and the test set image is tested once every iteration and its accuracy is recorded. The maximum accuracy of the transfer-learning based ResNet50 on the test set can be seen to be 96.30%, and the maximum accuracy of the transfer-learning-based ResNet34 on the test set is 91.29%; the accuracy of ResNet50 without transfer-learning-based learning was up to 90.36%, and the accuracy of ResNet34 without transfer-learning based learning was up to 87.87%, as shown in Figure 8. Indicating that the transfer-learning model of ResNet50 was better than ResNet34 at identifying tunnel diseases, and the transfer-learning improved the accuracy of the model in extracting the tunnel disease features, and the model robustness is better.

3.2. Results and Analysis

The transfer-learning-based ResNet34 and ResNet50 models were compared and the sample data identified on the tunnel distress test set were represented as a confusion matrix. As shown in Figure 9 ResNet34 identified water leakage with 27 samples identified as tunnel sheet splices and 13 samples identified as tunnel lines, while ResNet50 identified water leakage with 2 samples identified as tunnel sheet splices and 10 samples identified as tunnel lines; ResNet34 identified cracks with 27 samples identified as tunnel sheet splices and 3 samples identified as tunnel lines; ResNet34 identified cracks with 27 samples and 3 samples with tunnel lines, and ResNet50 identified cracks with 9 samples and 2 samples with tunnel lines. For sample 1 and sample 2, these were not tunnel diseases. When ResNet34 identified sample 1, 13 samples were incorrectly identified as tunnel lines, and 40 samples were incorrectly identified as tunnel cracks, while ResNet50 identified that in sample 1, 8 samples were incorrectly identified as tunnel lines, and 13 samples were incorrectly identified as tunnel cracks. Similarly for sample 2, when using ResNet34 for identification, 29 samples were incorrectly identified as the tunnel sheet splices, 4 samples were incorrectly identified as tunnel water leakage, and 20 samples were incorrectly identified as tunnel cracks. When using ResNet50, the number of wrong samples for identifying tunnel sheet splices and tunnel cracks is greatly reduced, and the number of wrong samples for tunnel leakage is 0. It can be seen that the transfer-learning-based ResNet50 is more stable when classifying tunnel deceases.

We further compared the transfer-learning-based ResNet34 and ResNet50 models on the labeled data using P, R, and F₁. As shown in

Table 3. Assessment indicators for each label transfer-learning ResNet34.

Labels	P/%	R/%	F1
1	87.07	81.14	0.84
2	86.88	92.37	0.90
3	88.76	98.75	0.93
4	91.18	83.78	0.87

and

Table 4. Assessment indicators for each label transfer-learning ResNet50.

Labels	P/%	R/%	F1
1	95.00	94.55	0.95
2	95.30	94.59	0.95
3	96.63	99.42	0.98
4	95.59	94.20	0.95

, the precision and recall rates of the transfer-learning-based ResNet50 are higher than those of ResNet34, and it can be seen that ResNet50 is more capable of distinguishing between the positive and negative samples. From Table 3 and Table 4, it can be seen that R is smaller when P is larger and P is smaller when R is larger; indicating that P and R are in fact contradictory measures. This cannot allow for a comprehensive judgement of the model with a single comparison of P and R, so we introduce the value of F₁ that takes into account both P and R to allow both P and R to be maximized and balanced. As shown in Figure 10, the F₁ of the transfer-learning-based ResNet50 model is over 94% for each label, and is higher than that of ResNet34 for each label.

In summary, it can be seen that the transfer-learning-based ResNet50 has a better ability to identify tunnel diseases, and we can build a transfer-learning-based ResNet50 model and use convolutional neural networks to identify tunnel diseases.

4. Conclusions

With the development of artificial intelligence, deep learning has made great progress in tunnel engineering in recent years. Based on existing research results, this paper selects a network model that has not been previously used in the identification of tunnel diseases, which broadens the research direction for the identification of tunnel diseases. Using the unique cross-layer connection of ResNet, a residual module is constructed which realizes the fusion of local features and abstract features of cracks so as to retain detailed features of cracks and greatly improve the detection accuracy. Secondly, the method of transfer learning is introduced, combined with the model structure, and the main conclusions can be obtained through experimental verification as follows:

(1)

In the model structure used in the past, the neural network can only identify a single tunnel disease, such as water leakage or cracks. The disadvantage of this method is that it needs to train two models which reduce the work efficiency and increase the time cost. In order to identify the two tunnel diseases of cracks and leaking water at the same time, a new fully connected layer with the same classification number as the tunnel diseases category was set up in this paper, which improves the efficiency of tunnel detection.

(2)

The experimental data show that the accuracy rate of Res-Net50 without the transfer learning method is 90.36%, the accuracy rate of Res-Net50 with transfer learning method is 96.3%; the accuracy rate of Res-Net34 without transfer learning method is 87.87%; and the accuracy of Res-Net34 using the transfer learning method is 91.29%. It can be seen that using the transfer learning and fine-tuning training methods, a model with superior performance can be obtained in a shorter time.

(3)

On the one hand, compared with the structure of the ResNet50 model and the ResNet34 model, the ResNet50 model is more complex, so it takes longer to train the ResNet50 network structure. On the other hand, according to the experimental data obtained from the test set, the five evaluation indicators of ResNet50 are higher than those of ResNet34, so the recognition accuracy of ResNet50 is the best. Considering that in practical engineering applications, due to the non-real-time change of cracks, engineering detection pays more attention to the model with the best crack recognition accuracy, so the improvement of the recognition rate has more practical use value.

(4)

The research results of this paper can be applied to the automatic detection of tunnel cracks and water leakage in combination with photographic equipment which can not only reduce labor costs but also improve the efficiency of detection works. However, due to the complex tunnel environment, there are not only two types of tunnel diseases which are water leakage and cracks; in the next study, more types of tunnel diseases should be considered. Additionally, the way to obtain sample data in the article is too simple, mainly from on-site photography; in the future research, the sample data collected will come from different time and different places, and a mature database will be established to further improve the recognition effect of the model.