Automatic Classification of Crack Severity from Cross-Section Image of Timber Using Simple Convolutional Neural Network

Featured Application

Automatic evaluation for timber in construction. Currently, humans conduct this evaluation. It is tedious and time-consuming. CNN is suitable because it can incorporate subjective human evaluations and reduce time and human effort.

Abstract

Cedar and other timbers used for construction generally undergo a high-temperature drying process after being harvested to maintain their quality. However, internal cracks occur during this process. This is an issue because it deteriorates the structural performance, such as buckling strength and joint durability of the timber. Since preventing these internal cracks is difficult, their severity must be examined manually. Currently, the length, thickness, and area of the cracks on a cross-sectional surface of square timber are measured using calipers. However, this process is time-consuming and labor-intensive. Therefore, we employed a convolutional neural network (CNN), widely used in artificial intelligence applications, to automatically evaluate the severity of cracks from cross-sectional images of timber. A novel CNN was constructed and experimentally evaluated in this study. The average classification accuracy was 85.67%.

1. Introduction

Timber used as building material, such as cedar, goes through a drying process after being cut from the woods. This drying process primarily improves its strength. Additionally, drying is essential to stabilize the quality of timber: it prevents changes in size and shape, and the growth of bacteria. Furthermore, the weight reduction due to lower moisture content leads to lower transportation costs. High-temperature drying is the generally employed process [1]; however, it leads to significant internal cracking invisible from outside [2], as shown in Figure 1.

Figure 1. Pictures of the internal cross-section of timber. (a) Timber samples with critical cracks not found from outside; (b) good condition dried timber.

Cracking is a problem because it deteriorates structural performance parameters, such as buckling strength and joint durability [3,4]. Therefore, many researchers have investigated the mechanism of cracking propagation during the drying process [5,6]. Since completely preventing cracking is impractical, we need to evaluate the severity of these cracks in the timber used for construction.

To assess the severity of internal cracks, workers check a cross-sectional view of a square piece of timber and measure the length, thickness, and area of the cracks using calipers. However, this method of examination is time- and labor-intensive. Therefore, to facilitate the automatic evaluation of internal cracks, we propose a convolutional neural network (CNN) [7,8]. CNNs are widely used in image recognition applications using artificial intelligence. If such a computational system is realized, the effort and time required for the manual inspection of timber will reduce. The aim of this study was to develop a CNN capable of classifying crack severity in lieu of a worker, as shown in Figure 2. To check timber quality, the timber specimen is first cut as shown in Figure 2a. Next, a special silver paint is sprayed onto the timber piece to highlight the cracks on the cross section. Since visual inspection is labor-intensive, as shown in Figure 2c, the CNN is used to evaluate the cracks, as shown in Figure 2d. Finally, with the proposed system, human intervention for timber inspection can be completely avoided in the future.

Figure 2. Schematic of the proposed system. (a) Timber samples; (b) silver-painted cross sections highlight cracks; (c) human inspection of crack severity; (d) automatic classification of crack severity using the proposed CNN; (e) finally, the computer will substitute the inspection task.

Previous studies have attempted to estimate the state of invisible internal cracks from the propagation characteristics of acoustic vibrations [9,10,11]. Some studies employed CNNs to recognize defects in timber. A method to automatically detect cracks in wooden surfaces has been previously reported [12]. In addition, a method for the automatic detection of defects in wood, such as splits, knots, and molds, has been developed [13,14]. Unlike previous research, our study attempted to automatically evaluate the level of crack severity in a cross section of timber that has been cut and made directly visible. Additionally, we developed a CNN to detect crack severity but not defects.

The rest of this paper is organized as follows: Section 2 describes the preparation of square cedar cross-sectional images used in this study. Section 3 describes the structure of the proposed CNN. Section 4 discusses the evaluation experiments and their results. Finally, Section 5 concludes the study with a summary and direction for future work.

2. Preparation of Timber Cross-Section Image

As shown in Figure 3, a 3000 mm-long high-temperature, dried cedar square timber was cut at four positions from P1 to P4. Then, a silver spray (ROVAL RS-420ML) was applied to enhance the visibility of the cracks, as shown in Figure 2. Silver-painted images at positions P1–P4 were captured with a scanner (Canon CanoScan LiDE 700F) for 32 identical cedar timbers. Therefore, a total of 128 images (32 pieces × 4 positions) were obtained. Note that only one photo was taken at each cutting position.

Figure 3. Schematic of the 3000 mm-long cedar timber used for obtaining images. It has a square cross-section area of 105 × 105 mm². Silver paint is sprayed to enhance the visibility of the cracks in the cross section.

Figure 4 shows the images of before and after being painted; S5 in Figure 4 shows that the silver spray can highlight even thin cracks on the surface of the cross section. In this study, coating the timber pieces with silver paint was considered the first step in our study.

Figure 4. Silver-painted cross-sectional images.

A total of 128 images were classified by timber quality control experts as “Bad,” “Good,” and “Neutral” based on crack severity. Figure 5 shows a portion of the entire classification set. The images in the “Bad” category have many large cracks, while the “Good” ones have almost no cracks. Accordingly, the “Neutral” category has images with crack severity between those of the “Good” and “Bad” categories. From the dataset of 128 images, 49 “Bad,” 33 “Good,” and 46 “Neutral” images were obtained.

Figure 5. Examples of expert’s classification based on crack severity. (a) Potentially risky to use in construction due to large cracks. (b) It is clearly safe. (c) It is safe with almost no major cracks.

3. Simple CNN Configuration

Figure 6 shows the schematic of a simple CNN architecture. The input image is a grayscale image of 500 × 500 pixels. As shown in Figure 6, convolution operations are performed three times (Conv1–Conv3) [7]. The output of the CNN is the classification result of an input image, which can be “Bad,” “Good,” or “Neutral.” This implies that only one class with the highest value in the output of the Soft-Max layer is considered as the classification result.

Figure 6. Architecture of the proposed simple CNN.

Table 1 and Figure 7 illustrate the feedforward calculation of the CNN. A breakdown of the calculations for the size of the feature maps and the output by convolution operations is provided. “BatchNorm” refers to the batch normalization operation [15]. It adjusts the scale between different mini batches and prevents network overfitting (over-adaptation to training data). Most of the recently proposed deep networks, such as ResNet [16] and DenseNet [17], employ batch normalization. ReLU (Rectified Linear Units) operation [18] is commonly used in CNNs. ReLU operation has been adopted in many well-known CNNs from around 2010, such as Alexnet [19], Vgg [20], and Googlenet [21]. For instance, ResNet [16] is applied for object detection [22] in an RGB image. By contrast, the proposed shallow CNN deals with gray-scale images. The famous pre-trained CNNs [16,17,19,20,21] are trained with RGB color images of the ImageNet, including 1000 kinds of objects such as fish, houses, flowers, tigers, and so on. In the instance that the number of available training data is small, pre-trained CNNs would be applied by fine tuning (transfer-learning); however, pre-trained CNNs need large memory capacity; therefore, they are not suitable for edge computing. The proposed shallow CNN is suitable for memory saving and computational resources.

Table 1. CNN feedforward calculation.

Layer	Operation	Filters’ Set Num	Filter Size	Stride	Output
Input	-	-	-	-	500 × 500 × 1
Conv1	Conv->BatchNorm->ReLU	50	4 × 4 × 1	2 × 2	249 × 249 × 50
Max pool	Max pooling	-	3 × 3	2 × 2	124 × 124 × 50
Conv2	Conv->BatchNorm->ReLU	50	2 × 2 × 50	1 × 1	123 × 123 × 50
Conv3	Conv->BatchNorm->ReLU	50	3 × 3 × 50	2 × 2	61 × 61 × 50
Fc1	Affine->ReLU	-	-	-	500
Fc2	Affine->ReLU	-	-	-	250
Fc3	Affine->Soft-Max	-	-	-	3

Figure 7. CNN feedforward flow.

4. Validation of the Proposed Simple CNN

Experiments were conducted to evaluate the classification accuracy of the CNN for the 128 images (Bad: 49, Good: 33, Neutral: 46) described in Section 2. It is not always possible to collect enough data to evaluate a machine learning model [23]. Therefore, the training image data were augmented by flipping the images left and right, up and down, and rotating them to increase the size of the training dataset virtually. This augmentation of training image data allows the CNN to learn the common features required for inference.

Furthermore, we employed repeated random subsampling validation. This generates datasets comprising various combinations of training and test images and evaluates their accuracy. We employed this because the available data is limited.

4.1. Training of CNN

The training dataset was augmented, as shown in Figure 8. Random left-to-right flipping (Figure 8a), up-and-down flipping (Figure 8b), and random clockwise rotation over 0–360° (Figure 8c) were performed on the training images.

Figure 8. Augmentation of CNN training dataset. (a) Left–right reflection is performed in 50% of images; (b) top-down reflection is performed in 50% of images; (c) random clockwise rotation from 0–360° is performed for 100% of cases in the training phase.

CNN training was divided into First Half and Last Half, as shown in Table 2. It also enumerates the training conditions for the CNN.

Table 2. CNN training condition.

	First Half	Last Half
Solver	SGDM (Stochastic Gradient Descent with Momentum)
Learn Rate	10−3
Total Epochs/Iterations	300	50
Mini batch Size	256	100
Augmentation	Random Left–Right Reflection (50%), Random Top-Down Reflection (50%)
	Random rotation ranged in [0, 360] degree	No rotation
CPU	Intel core i9 10980XE
Main Memory	98 GB
OS	Windows 10 64 bit
Development Language	MathWorks, MATLAB (R2022a)
GPU	Nvidia RTX A6000 (VRAM 48 GB, 10752 cuda cores)

In Last Half, training was performed with unrotated images. In the repeated random subsampling validation, the 128 images were divided into 113 images for training and 15 images for testing, as shown in Figure 9.

Figure 9. Overview of repeated random subsampling validation.

Note that we determined the hyperparameters shown in Table 2 and CNN configuration illustrated in Figure 6 and Figure 7 in a heuristic manner.

After training the CNN, its classification accuracy was evaluated using a set of 15 test images not used for training. Figure 10 shows the change in loss during the training phase for each trial with different datasets. We adopted a cross-entropy error function [24] for loss calculation to minimize error. We confirmed that the loss decreased as the epoch progressed. Thereby, we concluded that learning was performed appropriately in all trials.

Figure 10. Loss in each epoch.

4.2. Testing the CNN

Table 3 shows the IDs of the data images not used to train the CNN and to evaluate its accuracy in each trial. Figure 11a shows the classification accuracy for test images not used for training the CNN (different datasets). In all the trials, the accuracy was higher than 70%, and the mean accuracy was 85.67%. As demonstrated in the confusion matrix in Figure 11b, misjudged “Bad” and “Good” images were classified as “Neutral” in all the trials.

Table 3. CNN testing results.

Trial	Bad Image IDs for Test					Good Image IDs for Test					Neutral Image IDs for Test					Accuracy
1	5	16	21	28	31	17	19	20	24	29	2	21	41	42	45	0.8000
2	22	32	33	42	45	5	13	14	15	17	6	18	19	30	46	1.0000
3	14	16	34	42	47	3	4	5	12	17	3	19	40	42	46	0.7333
4	7	18	19	44	48	8	16	19	21	31	6	19	23	27	38	0.8667
5	2	12	29	35	41	1	6	14	28	29	7	10	14	23	43	0.8667
6	4	9	13	16	49	7	12	27	31	33	3	5	27	28	41	1.0000
7	2	5	6	20	33	7	9	15	27	30	1	7	20	22	26	0.8000
8	7	24	32	36	42	1	6	14	15	24	1	8	22	41	43	0.9333
9	4	27	35	37	47	1	2	5	16	29	1	2	9	10	17	0.8667
10	30	33	39	43	48	5	7	8	13	14	3	8	11	21	30	0.8000
11	7	9	18	32	47	3	6	17	25	27	7	12	19	37	44	0.9333
12	15	24	26	33	36	2	3	14	19	27	10	15	20	25	27	0.9333
13	18	34	36	39	45	4	16	17	21	22	1	17	31	39	42	0.8667
14	25	26	29	39	48	2	5	6	15	18	3	4	12	31	37	0.8667
15	12	16	23	27	42	2	9	20	25	26	4	25	36	42	45	0.7333
16	17	21	40	41	49	1	6	11	20	22	2	6	18	24	44	0.8000
17	2	8	16	23	44	19	20	28	29	32	8	14	24	29	32	0.8000
18	17	18	21	32	40	18	23	30	32	33	10	20	22	26	33	0.8667
19	12	15	35	39	47	2	3	7	13	33	9	12	15	42	44	0.8667
20	25	32	36	39	46	3	7	16	22	25	4	16	20	22	27	0.8000
–	–					–					–					Average: 0.8567 Std: 0.0758

Figure 11. Test results. (a) Accuracy during each trial; (b) confusion matrix showing that the CNN did not confuse between “Bad” and “Good” images.

Figure 12 shows the misclassified images. Figure 12a,b display actual “Bad” and “Good” images, respectively. This shows that the CNN performs classification like a human because such distinguishment includes boundary fuzziness [25]. However, in the future, we will explore suitable methodology to judge the order of “bad,” “neutral”, and “good”.

Figure 12. Misjudged Images (a) “Bad” images classified as “Neutral”; (b) “Good” images classified as “Neutral”.

Figure 13 shows the misclassified images as well. Figure 13a,b display actual “Neutral” images classified as “Bad” and “Good” images, respectively.

Figure 13. Misjudged Images (a) “Neutral” images classified as “Bad”; (b) “Neutral” images classified as “Good”.

The proposed system could distinguish the images between “Bad” and “Good.” However, the Neutral-like images were hard to classify. On the other hand, misjudged “Bad” images look like “Neutral.” Therefore, “Neutral” evaluated images are close to “Good.” By increasing training data, the CNN can remove “Bad” specimens. This system could be a decision support system for the human quality management judge.

5. Conclusions

This study developed a CNN for the automatic evaluation of the severity of internal cracks in a cross section of square timber. We experimentally confirmed that the average classification accuracy of the CNN was 85.67%. Additionally, the CNN did not confuse between “Good” and “Bad” images and performed human-like classification. In this paper, we illustrated that the CNN could distinguish between “Bad” and “Good.” This result implies the humans’ task will only be to find out obviously “Bad” from “Neutral”-judged specimens. Future work includes the development of a portable device for practical applications. Furthermore, we will attempt to realize a CNN that can classify the severity of cracks even from images without silver spray paint. For that purpose, we will collect many training datasets of the un-sprayed images.

We expect to share image data and code with academic non-commercial researchers who are willing to collaborate with us.