Quantitative Analysis of Bolt Loosening Angle Based on Deep Learning

Abstract

Bolted connections have become the most widely used connection method in steel structures. Over the long-term service of the bolts, loosening damage and other defects will inevitably occur due to various factors. To ensure the stability of bolted connections, an efficient and precise method for identifying loosened bolts in a given structure is proposed based on computer vision technology. The main idea of this method is to combine deep learning with image processing techniques to recognize and label the loosening angle from bolt connection images. A rectangular steel plate was taken as the test research object, and three grade 4.8 ordinary bolts were selected for study. The analysis was conducted under two conditions: manual loosening and simulated loosening. The results showed that the method proposed in this article could accurately locate the position of the bolts and identify the loosening angle, with an error value of about ±0.1°, which proves the accuracy and feasibility of this method, meeting the needs of structural health monitoring.

1. Introduction

Steel structural facilities have three primary connection methods: welding, bolting, and riveting. Among these, the most commonly used is bolting, or bolted connections. Bolted connections have several advantages over welding and riveting, such as a more straightforward structure, easy installation, a low cost, and reliable quality. Due to these advantages, bolted connections are widely used in various fields, including building structures, mechanical products, power equipment, transportation, aerospace engineering, and more [1]. However, bolted connections often occupy critical parts of steel structures, and their connection parts are continuously exposed to damp air, acid rain, and other harsh environments, making them susceptible to external damage like chemical corrosion. Additionally, due to lateral loads, mechanical shocks, and vibrations, bolted connection parts often operate in a damaged state for extended periods, which gradually reduces the pre-tightening force. This can eventually cause loosening or detachment of the connection joints, leading to structural damage [2].

Bolted connections are widely applied in civil, mechanical, and aerospace engineering steel structures. Bolt loosening is one major threat to structural safety. Caused by vibration or repetitive loads, the loosened bolt may not be easily identified initially but could weaken the structural integrity and lead to catastrophic failure.

To mitigate the structural damage caused by bolt loosening, human inspection is one of the most dominant methods in the engineering practice, particularly in civil engineering. For example, the Federal Highway Administration (FHWA) in the United States [3] requires routine bridge inspection every two years, during which the bolts of steel connections are inspected, recorded, and monitored by trained engineers. However, human inspection is expensive and prone to error, especially if the bolt loosening is subtle. In addition, some structural bolts are located in inaccessible places, making it more challenging to inspect bolt loosening promptly.

To date, extensive research efforts have been made to leverage advanced technologies for detecting and monitoring bolt loosening. Some of the applications include piezoelectric sensors [4,5], vibration-based methods [6,7], and electromechanical impedance [8,9]. Despite their successes, one commonality in these methods is that physical sensor deployments and/or cabling work are required, leading to cost increments for monitoring systems in the case of sensing a large number of steel bolts.

Non-contact vision-based methods, on the other hand, have shown great promise for monitoring bolt loosening. Such technologies rely on consumer-grade digital cameras to collect images of a structural surface at different inspection periods. Then, the image features related to bolt loosening (e.g., rotation of the bolt head) can be extracted through computer vision algorithms. Vision-based bolt loosening methods are appealing as they are non-contact, low-cost, and easy to deploy. Through the integration of other image collection platforms, such as unmanned aerial vehicles, these vision-based approaches could be ideal candidates for monitoring the steel bolts of civil structures that are inaccessible to human inspectors.

In this paper, we propose a vision-based approach for bolt loosening detection. The proposed approach can reliably identify loosened bolts and quantify the rotation angle of a bolt head. A laboratory experiment is conducted using a simple bolted steel plate to validate the performance of the proposed approach. The results indicate the effectiveness of the proposed approach for bolt loosening inspection in engineering applications.

The rest of the paper is organized as follows. Section 2 reviews the related work in the literature; Section 3 illustrates the methodology and technical background; Section 4 describes the experimental test setup and discusses the results; and Section 5 concludes the study.

2. Related Work

The health monitoring of bolted steel connections using vision-based methods is not new and has been well investigated. In 2004, Mazzeo et al. [10] developed a machine-learning method to identify the absence of bolts between rails and sleepers. A few years later, Marino et al. [11] improved the detection accuracy of the same work by combining the discrete wavelet transform technique with a neural network. As a result, the proposed method can achieve an accuracy of 99.6% in detecting visible bolts and 95% in detecting missing bolts. Liu et al. [12] investigated a novel machine vision approach to detect the absence of a bolt installed between the safety chain and the brake beam in freight trains. The proposed method shows promising results even when the collected image is subjected to a more complex background. Nevertheless, as mentioned above, these studies can only offer binary detection results on a bolt’s presence/absence. Whether the bolt is subjected to loosening is out of the scope of these studies.

Instead of inspecting the absence of a bolt, an increasing interest has been reported in the research community in quantifying bolt loosening through vision-based approaches. Two research groups (Cha et al. [13] and Park et al. [14]) conducted the earliest attempts in this field, after which more related work can be found, particularly in the fields of civil and mechanical engineering. In the rest of this section, a literature review is performed by discussing nine articles published in peer-reviewed journals from 2015 to 2019.

Cha et al. [13] investigated a vision-based approach for detecting loosened bolts. The proposed approach first utilized the Hough transform method to extract edge features of the bolt boundary, such as the dimensions of bolt heads and wrench holes. Based on such dimensions, the loosened and non-loosened bolts were classified by a support vector machine (SVM) to identify the loosened bolt effectively. Later on, follow-up work was conducted by Ramana et al. [15] through the usage of the Viola–Jones algorithm and a revised SVM. As a result, the proposed method enabled a better robustness of bolt loosening detection against a broader scope of image angles and distances.

Park et al. [14] applied the Hough transform method, Canny edge detector, and a series of computational algorithms to identify the rotation of bolt heads from images. A laboratory test was conducted on a bolted ring-shaped flange joint to validate the performance of this approach. Then, the proposed method was revised by Huynh et al. [16] by adding deep learning technology to enable a quasi-autonomous approach for bolt loosening detection. A field deployment on a box girder bridge was performed using an uncrewed aerial vehicle to validate the performance of this method.

Zhao et al. [17] developed a deep learning-based method to detect loosened bolts. The method utilized a convolutional neural network to recognize a unique texture pattern on a bolt head and then tracked the movement of this pattern once the bolt rotated. Zhang et al. [18] modified and applied such a concept for tracking the change in bolt height before and after bolt loosening. As a result, loosened bolts could be identified from images taken at particular camera shooting angles.

Sun et al. [19] proposed a binocular vision-based method to detect the bolt loosening on a bogie component in a train. Using edge extraction and 3D reconstruction technologies, the method can track subtle changes in the bolt head after loosening. Wang et al. [20] demonstrated a novel detection method for bolt loosening inspection in the oil and gas industry. The established method first utilized a convolutional neural network for bolt recognition. Then, the bolt rotation angles could be identified using the Hough transform line detection technique. Kong et al. [21] reported an image registration methodology to detect loosened bolts in steel connections. The method overlapped images of the steel bolts before and after loosening and found the differential image features caused by bolt-head rotation.

In reviewing the above-related works, one major challenge in existing methods is to robustly identify and quantify bolt loosening damage against texture changes of the structural surface. This is a concern in transferring such technologies from laboratory to practice application as bolted steel connections in the field may be contaminated by unexpected textures such as rust, dust, and corrosion marks. Particularly in civil engineering, the surfaces of steel connections could vary dramatically due to environmental factors like temperature and humidity during intervals between inspection periods. Such a texture change on the structural surface would provoke intensity redistribution at/around steel bolts in the collected images, leading to the possible failures of these methods.

The main contribution of this study is to investigate a robust vision-based approach to reliably detect loosened bolts against texture changes of the structural surface. Furthermore, through the geometric transformation technique, the loosened bolts can be quantified by automatically recovering the rotation angle of the bolt heads. To the authors’ best knowledge, this is the first effort reported in the literature on the vision-based detection and quantification of bolt loosening damage against surface texture changes.

3. Bolt Loosening Detection Method Based on Computer Vision Technology

3.1. Overall Detection Process

This paper integrates deep learning algorithms and digital image processing technologies, designing a bolt-loosening detection framework based on computer vision technology to achieve bolt-loosening recognition and angle calculation. The overall detection process is divided into two steps: Bolt detection and positioning based on the YOLOv2 algorithm and bolt loosening identification based on image feature matching. The specific process steps are shown in Figure 1.

3.2. Bolt Detection and Positioning Based on Deep Learning

The KLT corner tracking algorithm is an abbreviation for the Kanade–Lucas–Tomasi tracking algorithm [18,22], and it is one of the classic corner-tracking algorithms. This algorithm assumes that the target only produces consistent small displacements in the video stream, and the grayscale variation in the target is not significant.

There are currently two directions for the task of object detection. One is based on traditional algorithms, but it has not been widely adopted due to redundant selection boxes and low efficiency. The other is based on deep learning algorithms, such as the two-stage series of Fast r-cnn, Faster r-cnn algorithms, and the one-stage series of SSD and YOLO algorithms.

YOLOv2 [23] is the second-generation algorithm of the YOLO series, and its detection performance is excellent. Based on the engineering reality that bolts belong to the small target category, this paper proposes an improved YOLOv2 model, as shown in Figure 2.

Using transfer learning, the convolutional residual network Resnet18 [24], which has a deep network depth and a low computational load, and can extract rich features, is used as the backbone network for YOLOv2. The advantage of introducing the residual network is to reduce the gradient vanishing or gradient explosion problem caused by the excessive depth of the network. The structure of the Resnet18 residual is shown in Figure 3.

The residual module is represented by Formula (1):

$$y=F\left(x,{\omega }_i\right)+x$$

(1)

In the above formula, $x$ is the input of the residual module, $y$ is the output of the residual module, $F\left(x,{\omega }_i\right)$ represents the residual mapping, and ${\omega }_i$ is the weight coefficient of the ith layer. When the second layer is included, Formula (2) is used for calculation.

$$F={\omega }_2\sigma \left({\omega }_{1}x\right)$$

(2)

In the above formula, ${\omega }_1$ is the weight coefficient of the first layer, ${\omega }_2$ is the weight coefficient of the second layer, and $\sigma$ represents the activation function.

A smartphone with a dual-lens setup (16 million pixels + 12 million pixels) was used to capture the image dataset required for training the bolt detector. The dataset was then preprocessed with unified numbering and other preprocessing tasks. The experimental equipment was a computer with an Intel(R)Core(TM)i5 processor and 8.00 GB of memory. The software environment was the Windows 10 system, with the overall algorithm framework written on the Matlab platform.

The test bolt image data were annotated using the Image Labeler in Matlab 2016a software. The dataset was expanded through data augmentation. A dataset of 1000 bolt images was obtained, with 80% used as the training set and 20% as the validation set. The algorithm used the SGDM, with a learning rate set to 0.001, MinbatchSize set to 8, and MaxEpochs set to 20. Feature extraction was performed using the Resnet18 deep residual network to train the bolt detector based on the YOLOv2 model.

Hyperparameters are fixed parameters set before a model begins training, determining the efficiency of network training and the detection performance of the model. This paper established hyperparameters based on the device’s memory capacity and the relevant literature [25,26,27]. The hyper-parameters are listed in

Table 1. The setting of hyper-parameters.

Catagary	Value
Optimizer	SGDM
numEpochs	80
miniBatchSize	8
LearningRate	0.001
warmupPeriod	1000
12Regularization	0.0005
penaltyThreshold	0.5

.

After the training was completed, the model’s accuracy was tested using a test set to obtain Precision and Recall data plotted on a coordinate axis. The area enclosed by this curve and the axes represent the model’s accuracy; the larger the area, the better the model’s detection performance. The graph shows that the Precision–Recall curve of the bolt detection model can achieve a Recall rate of 80% or higher without compromising its accuracy. As the Recall gradually increases, the detection accuracy experiences a slight reduction and eventually stabilizes around 0.95. Through calculations, it was determined that the average Precision (Average Precision) of the bolt detection model reached 0.95 during Recall, increasing from 0 to 1. As shown in Figure 4, the trained bolt-detection model achieved an average accuracy of up to 95%, meeting the detection requirements.

3.3. Bolt Loosening Identification Based on ORB Image Matching Algorithm

ORB, which stands for Oriented FAST and Rotated BRIEF, is a new key point detection and feature description algorithm proposed by Rublee in 2011 [28]. This algorithm merges the speed advantages of FAST key point detection [29] with the stability benefits of BRIEF’s binary feature description [30]. The integration of these two characteristics results in ORB’s impressive speed and stability. In this study, we use this algorithm to detect and extract feature points from two images of bolts before and after loosening. We then match the corresponding feature points and their pixel positions. Through MLESAC [31], we estimate the geometric transformation matrix of the matched points, thereby identifying the loosening angle of the bolt.

(1)

oFAST Feature Extraction Algorithm

The principle of the FAST key point extraction algorithm is to use the fluctuation in grayscale differences between pixel points to achieve the goal of rapid feature point extraction. As shown in Figure 5, we take 16 bolt pixel points on the circumference with p as the center and r as the pixel radius and compare the difference with the grayscale value of the central pixel.

If there are consecutive $n$ pixel points that satisfy Formula (3), then:

$$N={\displaystyle \sum _{q\forall \left(circle\left(p\right)\right)}\left|I_{\left(q\right)}-I_{\left(p\right)}\right|}>{\epsilon }_d$$

(3)

In the formula: $I_{\left(q\right)}$ represents the pixel value of a certain point on the circumference, $I_{\left(p\right)}$ represents the pixel value of the center of the circle, ${\epsilon }_d$ is a predefined threshold, and $circle\left(p\right)$ represents the set of all pixel points on the circumference with pixel point p as the center.

While the FAST algorithm detects extremely fast, the extracted key points do not include related information such as direction and scale. To address this shortcoming, Rublee proposed an improved oFAST algorithm. After determining the bolt key points, the algorithm adds direction to the detected vital points using the grayscale centroid method [18], thus resolving the lack of directionality in the FAST algorithm.

(2)

rBRIEF Key Point Description Algorithm

After extracting key points using oFAST, the image’s critical information must undergo relevant description. The primary idea of the BRIEF key point description algorithm is to randomly select several pairs of points near the extracted vital points and combine the grayscale values of these point pairs into a binary string. This string is then used as the feature descriptor for that critical point.

Suppose a smooth image is selected. Within its domain of size S × S, the description is carried out using the binarization Formula (4):

$$\tau \left(p;x,y\right)=\{\begin{array}{cc}1,& I\left(p,x\right)<I\left(p,y\right)\\ 0,& I\left(p,x\right)\ge I\left(p,y\right)\end{array}$$

(4)

where $I\left(p,x\right)$ represents the grayscale pixel value of the smoothed image domain at a certain pixel location.

If $n$ pairs of points are selected, the resulting feature vector is converted into a $n$-dimensional binary string descriptor, as shown in Formula (5):

$$f_n\left(p\right)={\displaystyle \sum _{1\le i\le n}{2}^{i-1}}\tau \left(p;x_i,y_i\right)$$

(5)

where $n$ can be 128, 256, or 512 dimensions, occupying 16 bytes, 32 bytes, and 64 bytes, respectively.

Since BRIEF does not possess rotational invariance, Rublee proposed the improved rBRIEF algorithm. The BRIEF descriptor is rotated by adopting the primary direction of the critical points.

(3)

Estimating Transformation Matrix Using MLESAC Algorithm

After matching the features of the images of the bolt before and after loosening, the loosening angle of the bolt can be determined by calculating the geometric transformation matrix of the bolt images. The MLESAC (Maximum Likelihood Estimation SAmple Consensus) algorithm estimates the geometric transformation matrix between images. It can transform the maximum likelihood estimation problem into a problem of minimizing a cost function, as shown in Formula (6):

$$C=-{\displaystyle \sum _i\left(\gamma \left(\frac{1}{2\mathsf{\pi }{\sigma }^2}\right)\exp \left(-\frac{e_i^2}{2{\sigma }^2}\right)+\left(1-\gamma \right)\frac{1}{\nu }\right)}$$

(6)

where $\gamma \left(\frac{1}{2\mathsf{\pi }{\sigma }^2}\right)\exp \left(-\frac{e_i^2}{2{\sigma }^2}\right)$ represents the error pattern of Gaussian distribution, indicating the probability of inliers; $\left(1-\gamma \right)\frac{1}{\nu }$ represents the error pattern of uniform distribution, indicating the probability of outliers. $\nu$ is the size of the search window in the matching algorithm, and $\gamma$ is the mixing factor of the two error patterns.

Figure 6 shows the deviation in pixel coordinates for each pair of feature points when the bolt is loosened by 10°.

4. Case Study Analysis

To validate the bolt-loosening detection method proposed in this paper, based on computer vision technology, the experimental setup utilized a rectangular steel plate with a thickness of 10 mm, a length of 230 mm, and a width of 100 mm as the object of study. Three grade 4.8 standard bolts were used to construct a model simulating a critical joint in steel structures. An intelligent smartphone (model: Samsung S10e; dual rear camera with 16 million + 12 million pixels; sensor type: CMOS sensor; camera aperture: f/1.5–2.4 + f/2.2) mounted on a tripod was used to capture images of the experimental model.

The external environment was set to natural daylight to simulate the lighting conditions of steel structures under actual working conditions. During the shooting process, the smartphone was fixed on a stand approximately 30 cm from the bolted joint plate. The built-in auto-focus feature of the phone was used for image data collection. The bolts on the joint plate were numbered for identification purposes. To simplify the calculation of angles, the recognized angles were rounded to the nearest whole number. The experimental setup is shown in Figure 7.

4.1. Categorization of Bolt-Loosening Angles

After mounting the bolts on the joint steel plate, they were numbered. To simulate the loosening of bolts in steel structures in actual engineering projects, this study utilized a protractor to demarcate several smaller-angle intervals at the bolt holes of the joint plate, precisely at 0°, 5°, 10°, and 15°. Additionally, to verify various loosening scenarios, larger angle intervals of 0°, 20°, 40°, and 60° were also marked. The simulation of bolt loosening in real engineering situations was achieved by manually aligning the bolts with these angle markings and rotating them counterclockwise. The divisions of these angle intervals are illustrated in Figure 8.

4.2. Bolt-Loosening Recognition Analysis

The joint model was placed on a horizontal workbench, with the smartphone fixed on a tripod under natural lighting conditions to collect image data. Initially, an image of the model before the bolt loosening was captured with the smartphone positioned directly in front of the joint model (the lens perpendicular to the model). Subsequently, the bolts were rotated counterclockwise to a certain angle by the angle scale, and another image of the model after the bolt loosening was taken in the same manner. Both images have a resolution of 3024 pixels × 3024 pixels—a comparative illustration of the bolts before and after loosening is presented in Figure 9.

After acquiring the bolt image data, the two images, one before and one after loosening, were input into the algorithm designed in this paper. This process involved detecting, locating, and identifying bolt loosening in the images, with the specific workflow illustrated in Figure 10.

The figure shows that the detector precisely located the bolts in both images (a) and (b). Image (a) shows the bolt before loosening, with a green box displaying “bolt1: 0°”, indicating the initial bolt image data. Image (b) represents the bolt after loosening, with a yellow box displaying “bolt1: test”, signifying this image’s processing of bolt-loosening recognition. Images (c) and (d) depict the cropped target areas of the bolt before and after loosening, respectively. This cropping retains the image of the bolt head while eliminating background and other distracting information, thus reducing image processing demands and enhancing the algorithm’s efficiency. Images (e) and (f) show the detection of a large number of feature points in the grayscale bolt images; feature points in the pre-loosening image are marked with red “o”s, and those in the post-loosening image with green “+” signs. The detailed detection effect of these feature points is shown in images (g) and (h).

Subsequently, feature point information is extracted and matched, with each matched pair connected by a yellow line, as shown in image (i). This matching identifies feature points that indicate rotation at the bolt head and some around the bolt, although there are numerous mismatched pairs. Image (j) overlays the pre- and post-loosening images, and after removing mismatched pairs, the rotation trajectory of the red and green feature points becomes apparent.

The original image (b) is re-marked for bolts that have loosened. The original locating rectangle is changed to red, and the degree of loosening is displayed at the top of the image, as shown in image (k), with a red box displaying “bolt1: 14°”.

Following the operational process established for recognizing loosening in a single bolt, this procedure was sequentially applied to images of multiple bolts to test for loosening recognition. To verify the timeliness of the algorithm, a stopwatch was used during testing, and it was observed that the recognition of loosening in images of multiple bolts took approximately 5 s. Given the inherent errors in manually loosening bolts, an alternative approach was adopted. Images of bolt heads were cropped and then superimposed onto the bolt-hole positions on the joint plate to create composite images. This process, executed by the computer, simulated the bolts being loosened counterclockwise, thus achieving ideal loosening image data, as shown in Figure 11.

In Figure 11, the left side displays images of bolts manually loosened, while the right side shows computer-simulated loosening conditions. In the simulated loosening condition images, the angles of counterclockwise rotation are precisely controlled by the computer, ensuring that the algorithm’s detection results match the actual degrees of loosening. In the case of manual loosening, there were inevitable errors; for example, Bolt 3’s loosening angles of 4°, 9°, and 14° were each 1° off the ideal angles of 5°, 10°, and 15°. However, Bolt 2, which was not manipulated during the experiment and remained tightened, showed a detection result consistent with reality, displaying 0°. For Bolt 1, in conditions 1 and 2, the loosening angles were 21° and 39°, differing by 1° from the ideal angles. However, in condition 3, the detected loosening angle of 60° perfectly aligned with the ideal value, meeting the detection requirements. The comparative analysis of the two sets of images indicates that the algorithm’s angle recognition results are generally consistent with the actual conditions.

Based on

Table 2. Comparative Analysis of Manual Loosening Angle Error (photos taken by mobile phone).

Conditions	Angle	Bolt 1	Bolt 2	Bolt 3
1	Manual loosening angle	20.0000°	0.0000°	5.0000°
	Algorithmic Recognition Angle	21.3417°	0.0288°	4.2185°
	error value	1.3417°	0.0288°	0.7815°
2	Manual loosening angle	40.0000°	0.0000°	10.0000°
	Algorithmic Recognition Angle	39.2511°	0.0588°	9.2995°
	error value	0.7489°	0.0588°	0.7005°
3	Manual loosening angle	60.0000°	0.0000°	15.000°
	Algorithmic Recognition Angle	60.3005°	0.0034°	13.6516°
	error value	0.3005°	0.0034°	1.3484°

, the error in manual loosening is evident in the recognition of 15° and 20° angles, with an approximate error of around 1.3°. The errors at other angles are slightly smaller. Bolt 2, which remained tightened throughout and did not undergo manual loosening, exhibited a maximum error of only 0.0588°, closely aligning with the actual 0° loosening condition. This indicates that, if the loosening angle of a bolt can be precisely controlled, the recognition results under actual working conditions can meet basic accuracy requirements.

In

Table 3. Comparative Analysis of Manual Loosening Angle Error (photos taken by computer simulation).

Conditions	Angle	Bolt-1	Bolt-2	Bolt-3
1	Manual loosening angle	20.0000°	0.0000°	5.0000°
	Algorithmic Recognition Angle	20.0232°	0.0098°	5.0346°
	error value	0.0232°	0.0098°	0.0346°
2	Manual loosening angle	40.0000°	0.0000°	10.0000°
	Algorithmic Recognition Angle	39.9675°	0.0060°	10.0400°
	error value	0.0325°	0.0060°	0.0400°
3	Manual loosening angle	60.0000°	0.0000°	15.000°
	Algorithmic Recognition Angle	60.0073°	0.0084°	15.0066°
	error value	0.0073°	0.0084°	0.0066°

, all cases involve computer-simulated loosening, resulting in a maximum error of only 0.0400°. This error is nearly identical to the pre-set 10° angle in the computer simulation, effectively validating the precision of the bolt-loosening recognition algorithm presented in this paper. This demonstrates that, with precise control and accurate simulation, the algorithm can reliably detect and quantify bolt loosening to a high degree of accuracy.

5. Conclusions

This paper proposes a bolt-loosening detection method based on computer vision technology, implemented through algorithms developed on the MATLAB platform. The core idea integrates deep learning algorithms with image processing techniques to identify and calculate the loosening angles of bolts in joint model images. Firstly, a bolt detector trained using deep learning algorithms automatically identifies and frames bolts in captured images. Secondly, image processing techniques are applied to crop the bolt images within the confidence region, reducing the computational load in later stages. The cropped bolt images are then processed using the ORB and MLESAC algorithms for feature-point matching and pixel coordinate deviation calculations. This process effectively identifies and calculates the loosening angles of bolts in the images. Finally, experiments on joint model components were conducted to validate the feasibility of the proposed method. This experimental study lead to the following conclusions:

(1)

The method effectively identified loosened bolts in steel joint plates by training a bolt detector based on the YOLO algorithm and combining it with the ORB image-matching and MLESAC algorithms. It calculated their loosening angles, thereby proving its feasibility;

(2)

The algorithm efficiently processes bolt images obtained from experiments, identifying the location of the bolts, cropping images to include only the bolt targets, and, finally, calculating and marking the loosening angles on the images within less than 5 s, demonstrating the method’s efficiency;

(3)

Comparing manually loosened bolt images with computer-synthesized bolt-loosening images, it is evident that the angle values identified by the algorithm nearly match the actual values when the loosening angles of bolts are precisely controlled. The maximum error in loosening angles is approximately ±0.1°, confirming the algorithm’s accuracy in angle calculations.

Despite these achievements, the algorithm designed in this paper has certain limitations, which will be addressed in future research:

(1)

The algorithm’s processing time has yet to achieve near-real-time efficiency. Further improvements in device configuration and algorithm processing will be made to enhance the recognition performance, aiming for live detection capability;

(2)

Most bolt loosening in steel structures involves small angles, but scenarios of 360° or its multiples are not excluded, rendering the current algorithm infeasible. To enhance the practicality of the algorithm, future research will consider classifying bolts based on the length of the screw exposed after loosening, thereby addressing the limitations of the current method;

(3)

The issue of failing to consider the variability in tightened bolts’ angles when methods are used in production and installation processes should also be considered. According to the literature [32,33], the Bayesian method, as a typical method to overcome variability, could be considered. Due to the variability in tightened bolts, it is hard to estimate the actual angle of the loosened bolts, so the Bayesian-based method can be applied to update/predict the actual values of angles.