2.3. Evaluation of GPR Image Detection Model Based on YOLOv5 Algorithm
2.3.1. Evaluation Metrics of the Target Detection Algorithm
Judging the detection effectiveness of target detection algorithms usually requires evaluating some quantitative evaluation metrics so as to determine the algorithm’s merit from an objective perspective. To quantify the effectiveness of the YOLOv5 algorithm for GPR image target detection, we chose some commonly used metrics to evaluate the model, which include accuracy (P), recall (R), average precision (AP), F1 score, inference time, and model size.
These metrics are defined as follows:
where
TP is the number of correctly detected targets,
FP is the number of non-targets that the detector considers to be targets, and
FN is the number of non-targets that the detector considers to be targets.
Generally, the precision rate (P) is the detection accuracy of the target detector at an IoU of 0.5, which is the proportion of correct targets detected by the target detector to all detected targets, the recall rate (R) is the recall rate of the target detector at an IoU of 0.5, which is the proportion of correct targets detected by the target detector to all targets in the data set. While precision and recall are mutually exclusive to some extent, i.e., high recall has lower precision and low recall has higher precision, in order to balance the indicators of recall rate and accuracy rate, we more often use average precision (AP). AP is the area of the PR curve in the range of 0–1, compared to single precision and recall, AP reflects the sum of precision at different recall rates and better expresses the detection performance of the detector.
2.3.2. Performance Comparison of Different Target Detection Algorithms in GPR Image Detection
We divided the data set processed according to
Section 2.1, i.e., 295 GPR images, into training set, verification set, and test set corresponding to a ratio of 7:2:1, and trained the sets on a GPU server with a graphics card of Tesla V100 using the YOLOv5s network model in YOLOv5 version 6 mentioned in
Section 2.2 with 300 epoch iterations.
We also compared other excellent target detectors such as Faster-RCNN and SSD in the same training scenario to verify that the YOLOv5 algorithm is more suitable for GPR real-time detection experiments. Due to the small number of samples of non-parabolic features, the detection results of parabolic features are quasi in this paper to compare the detection performance of each detector. The comparison results are shown in
Table 1.
As can be found in
Table 1, the YOLOv5 network model has a detection accuracy of 89.3%, a recall of 70.6%, an average accuracy of 80.5%, an F1 score of 0.65, an inference time of 11 ms per frame, and a model size of 13.7 MB.
For the Faster RCNN algorithm, YOLOv5 algorithm has 12.1% less recall, but a 39.4% higher detection accuracy, 9% higher average accuracy, and 0.03 higher F1 score; notably, its inference time is 43.9 ms faster, and its model size is reduced by 94.3 MB. Through data comparison, it can be seen that YOLOv5 algorithm is more suitable for the real-time requirements of GPR than the Faster RCNN algorithm.
The YOLOv5 algorithm is also 1.4 ms slower than the SSD algorithm in terms of inference time, but it also meets the real-time requirements of the GPR detection system, and the YOLOv5 algorithm has a 2.8% higher detection accuracy, 31.4% higher recall, 20.1% higher average accuracy, 0.11 higher F1 score, and, most importantly, a 77.4 MB smaller model size. The data comparison shows that the YOLOv5 algorithm is more suitable for in-vehicle system deployment than the SSD algorithm, and it presents fewer false detections and omissions.
Overall, the detection speed of our YOLOv5 algorithm meets the requirement of real-time detection and has more balanced recall and accuracy compared with classical algorithms such as Faster RCNN and SSD; therefore, we obtain better detection performance in the target detection task of GPR images, and there are fewer errors and misses, which can better meet the needs of GPR image target detection in real-time. This means that we have better detection performance in GPR image target detection tasks, and fewer false and missed detections.
2.3.3. Comparison of the Effectiveness of Different Target Detection Algorithms in GPR Image Detection
We use YOLOv5 algorithm and Faster RCNN, SSD algorithm to detect GPR images and compare their effectiveness in detecting subsurface soil-situated foreign objects, including their correct recognition rate, missed detection rate, and false detection rate. The following
Figure 4 comparison chart shows the effectiveness of YOLOv5 algorithm and the other two target detection algorithms in actual detection. In
Figure 4, (a) is the original GPR image, (b) is the marker map of the GPR image feature curves, (c) is the detection effect of YOLOv5 algorithm, (d) is the detection effect of Faster RCNN algorithm, and (e) is the detection effect of SSD algorithm.
In
Figure 4, there are several small parabolic features, two large parabolic features, and three non-parabolic features in the GPR image. Regarding the leakage phenomenon, the YOLOv5 algorithm detected the two large parabolic features, three non-parabolic features, and most of the small parabolic features. The Faster RCNN algorithm detected two large parabolic features and most of the small parabolic features but missed two non-parabolic features. The SSD algorithm detected two large parabolic features but missed three non-parabolic features and some small parabolic features were missed. In terms of misdetection, the YOLOv5 algorithm misidentifies a background feature as a small parabolic feature, the Faster RCNN algorithm misidentifies a non-parabolic feature as a parabolic feature, and the SSD misidentifies a background feature as a parabolic feature.
Overall, the YOLOv5 algorithm and the other two algorithms were able to detect most of the parabolic and non-parabolic features, and although the Faster RCNN algorithm had a higher confidence level for the target foreign matter features, the YOLOv5 algorithm had better regression accuracy for the GPR image-based target foreign matter compared to the other two target detection algorithms. The center of the upper border of the detection frame is roughly near the parabolic feature vertex, which means that the detected subsurface soil-situated foreign objects’ position on the image coordinates can be roughly determined by the detection frame with better regression accuracy, which provides a certain theoretical basis for our research on the precise location of soil-situated foreign bodies.
2.4. Building an Automatic Parabola Extraction and Detection System for Ground-Penetrating Radar Images Based on YOLOv5 Algorithm
In order to improve the accuracy and efficiency of identifying target objects in GPR images, we use a machine learning algorithm to automatically extract the foreign object feature curves of the ground-penetrating radar image in real time. The hardware components of the indoor real-time detection system built in this study include the GPR instrument, laptop, Android cell phone, and data cable; the software components include the GPR instrument equipped with MALA Controller software and YOLOv5 network structure. The GPR real-time detection system was built as shown in
Figure 5.
According to the entire process of detecting the target by the ground-penetrating radar, we built a ground-penetrating radar image target feature curve automatic extraction detection system based on the ground-penetrating radar and YOLOv5 algorithms using artificial intelligence technology to achieve complex data-processing; reconstruct the GPR image coordinates; ascertain the target objects’ buried location information, GPS information, and other parameters of the acquisition process; and perform real-time rapid detection and identification of target characteristic curves of GPR images obtained during the detection process. The detection and recognition process of the constructed automatic extraction and detection system of the target characteristic curves of ground-penetrating radar images is shown in
Figure 6.
In our experiments, we use MALA Controller software to image the GPR data, and the cell phone screen is read through the Android Debug Bridge (ADB) in order to obtain GPR images in real time. After acquiring the GPR images, the image pixel coordinates are re-established by finding the imaging section of the GPR image on the cell phone screen. The burial depth of the soil-situated foreign objects is obtained by using the linear interpolation method. Then, the value of the horizontal coordinate of the ground location in the GPR image is screenshotted, and the value of the horizontal coordinate in the GPR image is identified by using the OCR algorithm. Consequently, the position obtained using the linear interpolation method and the starting point position identified by OCR are summed. Finally, the positions obtained by the linear interpolation and the starting point positions identified by OCR are summed to obtain the distance positions of hard foreign objects in the GPR images along the measuring line in the detection process.
We acquire the GPR images by continuously reading the MALA Controller software interface in real time with the Android Debug Bridge (ADB), a development tool used for communication between computers and Android devices, which consists of three components: client (ADB client), server (ADB server), and service process (ADBD). The client (ADB client) runs on the computer and is used to send commands to the server (ADB server); the server (ADB server) runs on the computer as a client of the service process (ADBD) and is used to send commands to the Android device; the service process (ADBD) runs on the Android device and is used to execute commands on the Android device.
We first send the command os.system(“adb shell screencap/sdcard/radar picture.png”) to the computer client (ADB client) to communicate with the server (ADB server) through TCP port 5037; then, the server (ADB server) will communicate with the Android device through USB serial port or TCP. At this time, the service process (ADBD) will intercept the screen of the Android device in real time so as to obtain the GPR images recorded by the running MALA Controller software and store them in the memory card of the Android device. Similarly, the computer client (ADB client) sends the command os.system(“adb pull/sdcard/radar image.png”) to transfer the GPR image stored in the memory card of the Android device to the computer. The above operation realizes the real-time acquisition of GPR images. The principle of real-time acquisition of GPR images is shown in
Figure 7.
During the test, the GPR system uses a trigger wheel to control the triggering of the GPR pulse, transmitting and receiving an electromagnetic wave pulse every step. MALA Controller software communicates with the GPR, and the MALA Controller updates the image every time the GPR receives a new pulse, while the ABD tool takes continuous screenshots of the cell phone screen, thus realizing real-time data acquisition of the whole detection process of GPR.
2.5. Indoor Testing Based on the Detection System
We conducted indoor detection experiments in the Key Laboratory of Agricultural Machinery and Equipment of the Ministry of Education, which was built on common farmland soil platform in the south of China. The soil platform has dimensions of 6 m in length, 2 m in width, and 1.5 m in depth, and contains a batch of ferrous experimental materials, such as steel bars and iron plate materials. The high-dynamic GPR MALA GX750 HDR instrument was selected to detect the target objects buried inside the soil. In order to achieve real-time detection, we used MALA Controller, a cell phone software developed by MALA, to assist us to understand and interpret the detection results more intuitively, as well as obtain the GPR image data in real time.
In the soil platform in the Key Laboratory of Agricultural Machinery and Equipment of the Ministry of Education in the south, we used rebar as an example, whose diameters were 5 cm, 4 cm, and 3 cm, with lengths of 15 cm. The depth of the rebar was 15 cm, and the buried positions were 50 cm, 150 cm, and 250 cm from the starting point; the specific buried positions of the rebar are shown in
Figure 8.
We obtained the GPR image data recorded by MALA Controller software in real time by taking screenshots of the cell phone screen with the ABD tool, and part of this process is shown in
Figure 9.
Figure 9a shows a screenshot of the computer reading the interface of the MALA Controller software when the GPR instrument scans above the position of the first piece of rebar;
Figure 9b shows a screenshot of the computer reading the interface of MALA Controller software when the GPR instrument has completely passed the buried position of the first piece of rebar, but before reaching the buried position of the second piece of rebar;
Figure 9c is a screenshot of the computer reading the MALA Controller software interface when the GPR instrument reaches the top of the second piece of rebar;
Figure 9d is a screenshot of the computer reading the MALA Controller software interface when the GPR instrument reaches the top of the third piece of rebar; and
Figure 9e is a screenshot of the computer reading the MALA Controller software interface when the GPR instrument has completely crossed the buried position of the third piece of rebar. A screenshot of MALA Controller software interface is shown in
Figure 9e.
After we acquired the GPR images using MALA Controller software and intercepted the cell phone screen with ADB tool, according to the characteristics of the MALA Controller software interface, the image pixel coordinates were re-established after finding the imaging part of the GPR in the MALA Controller software interface. The image pixel coordinates before re-establishment are shown in
Figure 10.
We found the GPR-imaging section in the MALA Controller software interface and established new image pixel coordinates based on this. The re-established image pixel coordinates are shown in
Figure 11.
Next, we choose a certain length of pixel vertical coordinates as the unit pixel depth; for example, note that the pixel vertical coordinate corresponding to 0.00 in the vertical coordinate is , while the actual burial depth is ; note that the pixel vertical coordinate corresponding to 0.45 in the vertical coordinate is , while the actual burial depth is . Then, in the image pixel coordinate system, the difference between and describes the difference between and with respect to the local coordinate difference, which is the depth of 0.45 m.
We read the pixel coordinates of the YOLOv5 prediction frame on the image; find the pixel vertical coordinate Y of the foreign object, i.e., the peak of the hyperbola; and then use linear interpolation to solve for the burial depth of the foreign object in the local coordinate D. The solution formula is shown in Equation (5).
Since, in the process of the GPR detection of underground foreign objects, the depth coordinate axis in the image does not change with the change in the GPR moving distance, then the burial depth obtained by selecting the 0.00 point under the new pixel coordinate system as the starting point is the burial depth in local coordinates.
However, when the GPR probes more than a certain distance along the survey line, i.e., when the GPR receives more than a certain number of columns of signal data, the MALA Controller software will only image the last N columns of signal data and refresh the horizontal coordinates of the MALA Controller software interface at the same time. Therefore, when the GPR moves more than a certain distance, the value of the horizontal axis of the MALA Controller software interface will change, i.e., the distance represented by the starting point in the new pixel coordinate system will change. Therefore, in order to obtain the distance of the target foreign object in the actual GPR moving along the measuring line’s direction, in addition to the above-mentioned linear difference method, the distance of the GPR moving along the measuring line direction corresponding to the starting point under the new pixel coordinate system should be added. Take
Figure 12 as an example: at this point the distance represented by the point 0.00 of the new pixel coordinates is 0.51 m.
Since the position of the measuring line distance axis of the ground-penetrating radar in the GPR image is fixed, we only need to identify the coordinates of the starting point and the ending point of the measuring line distance axis in the image and read out the pixel coordinates of the starting point and the ending point of the measuring line distance axis and the hyperbola vertex of the soil-situated hard foreign matter; then, we can obtain the measuring line distance of the soil-situated hard foreign matter through the linear interpolation method.
In order to solve the above problem, we first obtain the digital images containing only the starting and ending points of the distance axis of the measurement line corresponding to 0.51 and 3.34 in
Figure 12 by cutting the images, and then use the recognition engine of Tesseract-OCR to perform OCR text recognition on the cut digital picture, OCR text recognition is then performed on the clipped digital images using Tesseract OCR’s recognition engine. Thus, the distance of the measured line from the start point of the axis and the end one are obtained.
Finally, the distance between the local coordinates of the starting point and the end point of the line of sight distance axis identified by OCR, as well as the starting point and the end point of the line of sight distance axis, and the pixel coordinates of the target foreign object in the ground-penetrating radar image are used to obtain the position of the soil hard foreign object along the measurement line by linear interpolation method. The position equation of the target foreign object in the GPR displacement direction is shown in Equation (6).
where X is the pixel horizontal coordinate of the target foreign object parabola in the GPR image,
is the new pixel horizontal coordinate of the start point of the distance axis of the measurement line,
is the new pixel horizontal coordinate of the end point of the distance axis of the measurement line, S is the distance of the target foreign object in local coordinates along the measurement line,
is the distance of the start point of the distance axis of the measurement line in local coordinates along the measurement line,
is the distance of the end point of the distance axis of the measurement line in local coordinates along the measurement line.
The abscissa in the GPR image is shown in
Figure 13, we recognize that the starting point of the distance axis of the current survey line is 0.51 and the ending point is 3.34 through the OCR text recognition algorithm; then, we can read that the new pixel abscissa of the image corresponding to the starting point (0.51) is 0, the new pixel abscissa of the image corresponding to the end point (3.34) is 400, and the new pixel abscissa of the image corresponding to the hyperbola vertex of the soil-situated hard foreign object is 100. Consequently, the lateral distance (dist) of the soil-situated hard foreign object is shown in Equation (7).
Similarly, through the observation of the depth coordinate axis and the reading of pixel coordinates, the burial depth of soil-situated hard foreign matter can be obtained with the help of linear interpolation method
The burial depth and displacement position obtained by linear interpolation method are shown in
Figure 14.