Performance Comparison of Feature Detectors on Various Layers of Underwater Acoustic Imagery

Abstract

Image feature matching is essential in many computer vision applications, and the foundation of matching is feature detection, which is a crucial feature quantification process. This manuscript focused on detecting more features from underwater acoustic imageries for further ocean engineering applications of autonomous underwater vehicles (AUVs). Currently, the mainstream feature detection operators are developed for optical images, and there is not yet a feature detector oriented to underwater acoustic imagery. To better analyze the suitability of existing feature detectors for acoustic imagery and develop an operator that can robustly detect feature points in underwater imageries in the future, this manuscript compared the performance of well-established handcrafted feature detectors and that of the increasingly popular deep-learning-based detectors to fill the gap in the literature. The datasets tested are from the most commonly used side-scan sonars (SSSs) and forward-looking sonars (FLSs). Additionally, the detection idea of these detectors on the acoustic imagery phase congruency (PC) layer was innovatively proposed with the aim of finding a solution that balances detection accuracy and speed. The experimental results show that the ORB (Oriented FAST and Rotated BRIEF) and BRISK (Binary Robust Invariant Scalable Keypoints) detectors achieve the best overall performance, the FAST detector is the fastest, and the PC and Sobel layers are the most favorable for implementing feature detection.

1. Introduction

In recent years, human and industrial development has become more closely related to the ocean, with more infrastructure built off the coast and more renewable energy obtained from the deep sea. To more safely and efficiently explore the ocean, AUVs are becoming the predominant underwater exploration equipment, which can effectively replace divers for port and offshore infrastructure maintenance and can also perform exploration tasks in uncharted waters. When the AUVs perform exploration tasks in deep water, sonar sensors play an important role, with the forward-looking sonar (FLS) assisting AUVs to avoid obstacles for their safety and the side-scan sonar (SSS) providing high-resolution views of the seafloor to help researchers understand the deep-sea operating area [1]. With the continuous development of the electronics industry and remote-sensing technology, the detection performance of sonar equipment has been dramatically improved. The two main sonar types, FLS and SSS, can provide rich underwater acoustic imageries. The number and resolution of these images are constantly increasing. Therefore, it is crucial and far-reaching to design an efficient vision-processing algorithm to extract the information from underwater acoustic images, which, in turn, can serve deep-sea exploration activities. Image matching, as a principal technology in computer vision, applies the knowledge of graphics and corresponding mathematical theory to analyze the correspondence of grayscale, content, structure, texture, and semantic information in one or more groups of images, finally, to find similar image targets. Typical matching methods can be divided into region-matching and feature-point-matching approaches [2]. Image matching is the essential work of image analysis and processing and also a fundamental prerequisite for tasks such as target tracking, visual navigation, map stitching, and 3D reconstruction [3].

The above tasks are bound to be faced in practical ocean engineering tasks, so there is an opportunity and far-reaching significance to developing an effective acoustic image-matching algorithm. Underwater acoustic imageries are different from natural light images in that they have low contrast, a narrow grayscale range, high spatial correlation of adjacent pixels, and poor quality. Traditional matching algorithms designed for optical images, such as SIFT (scale-invariant feature transform) [4] and SURF (speeded-up robust features) [5], cannot effectively perform the matching task of acoustic images. The contrast between the large and small size of underwater acoustic images and the lack of the number of targets of interest leads to the inappropriateness of area-based matching methods for underwater task scenarios and considering the real-time requirements of underwater detection tasks, and the current mainstream research is based on feature-point-matching methods [6,7], which usually include feature detection, feature description, and outlier rejection. Among them, feature detection is the basis of the whole process, representing the salient structural information in the image. It also determines the number of correct matching pairs and the accuracy of the point position in the two images to be matched. In recent years, more research results have been achieved in the feature description stage, but many bottlenecks have not been solved in the feature detection stage. Meanwhile, compared with the development of optical image matching technology, the technique in the field of underwater acoustic imagery matching is lagging, which is mainly limited by the imaging shortcomings of underwater acoustic imageries and external factors such as the lack of samples available for research [8].

In recent years, with the development of the electronics industry and the update of sensor technology, the quantity and quality of underwater acoustic imageries available to researchers have been guaranteed. Meanwhile, the research on sonar image feature detection has gradually started to be emphasized. When a search is performed on the Web of Science using the subject term “sonar image feature detection” and setting the search date from 2000 to the present, the following report could be derived, as shown in Figure 1. There has been a significant increase in the number of publications and citation frequency of related research since 2013, which may be related to the rise of deep-learning techniques within the field of computer vision, where new techniques have brought opportunities for underwater acoustic imagery feature detection tasks.

Figure 1. Status of publications related to sonar image feature detection since 2000.

In this paper, a framework of comparative feature detection algorithms was proposed to find a high-quality feature detection scheme so as to fundamentally advance the underwater acoustic image matching technology to meet the needs of the downstream tasks such as the underwater visual navigation, seabed mapping, and environment reconstruction. Moreover, the relevant experimental reports are also summarized to help newcomers in this field as well as researchers who are proficient in image processing but do not understand sonar images to participate in acoustic perception tasks. Finally, some potential applications of deep-learning methods in this research field are summarized.

The rest of the manuscript is organized as follows: Section 2 briefly describes the research related to sonar image matching and the recent developments; Section 3 presents the feature detection approach taken in this paper; Section 4 describes the experimental setup, including dataset introduction, evaluation criteria, and parameter settings; Section 5 describes and compares the experimental results; Section 6 discusses the experimental shortcomings and outlook; finally, conclusions are drawn in Section 7.

2. Related Work

In natural light images, feature points are generally selected from pixel points that are easily distinguishable and visually salient in the image, such as straight-line segment intersections, corner points, or local center-of-gravity points [2]. Compared with natural light images, underwater acoustic images have poor imaging quality, severe noise interference, grayscale distortion, and weak edge features, all of which lead to the inefficiency of feature detectors initially designed for optical images. In addition, due to the significant differences between acoustic and optical imaging mechanisms, sonar images are usually accompanied by a narrow grayscale range, high spatial correlation of adjacent pixels, and insignificant grayscale variation, and there is also no publicly available and mature scheme for preprocessing sonar images. This situation also leads to almost failure of feature matching in the final underwater acoustic imageries, as evidenced by the low number of feature points detected and the absolute low matching accuracy.

In the study of underwater acoustic image matching, Daniel et al. [8] proposed to use acoustic shadow features and a segmentation algorithm to achieve SSS image matching, which is based on the geometric distribution characteristics of shadows in SSS images, echoes, and their corresponding critical position information. However, the authors used a manual combination of features rather than individual feature detectors, so the overall pipeline design lacked flexibility. Khater [9] proposed an SSS image feature detection method based on SUSAN (smallest univalue segment assimilating nucleus) and Harris corner point information, which can achieve better detection when the sonar image features are stable and uniform. However, in general, there is a large amount of noise and distortion on sonar images, and features are scarce and difficult to extract, so the generalization performance of this method is insufficient, and the application space in underwater scenes is limited. P. Vanish [6] investigated the effect of the classical SIFT algorithm on SSS images. The results showed that when the image features are rich and stable, SIFT could accomplish the SSS image-matching tasks. However, when there are apparent noise interference and image distortion in the sonar image, the matching effects of SIFT will also be significantly reduced. Zhang [10] proposed a SIFT-like feature detector and descriptor MBS-SIFT that could robustly cope with noise interference in multibeam imaging sonar images, which could more accurately capture the features of underwater objects compared with the classical SIFT method. Tao [11] proposed a combined SSS image-matching method in a priori position information, and SURF detectors are used to detect feature points from the SSS image that are stable to the affine transform for matching, and finally, the results are corrected with the help of RANSAC (random sample consensus). The results show that the matching algorithm is time-consuming and has high accuracy. However, this method depends on a priori position data, which has limited application in practical underwater scenarios. Shang [12] proposed a way to automatically match and mosaic SSS images. First, the image overlap region between adjacent strips is automatically determined based on the sonar detection track lines and the width of the scanned strips. Then in the overlap region, the SURF detector is used to detect the feature points. Finally, the geographic coordinates of the feature points are used to constrain the method, which uses multiple constraint information to complete the matching between the SSS image pair, and the matching results are better but very dependent on external information. Peter King [7] compared the performance of several classical handcrafted feature-matching algorithms in matching sonar images using a pre-collected dataset. The results show that the SIFT and SURF methods match better when the sonar images are feature-rich and stable. In the complex landscape background, only the SURF method could barely complete the matching. However, the authors did not deeply dissect the feature detection link and only evaluated each algorithm regarding the overall matching effect. Tueller [13] compared the performance of seven commonly used feature detection algorithms on synthetic sonar image datasets with different background attributes by training an SVM (support vector machine) classifier to evaluate the performance of the detectors. However, the study only covered several traditional feature detectors and tested a single type of image. Ansari [14] investigated the detection effectiveness of two algorithms, SIFT and SURF, in underwater image matching, and the results showed that SIFT and SURF in feature extraction and matching of SSS images offer good performance; but this is limited to scenes where stable textures and independent targets exist, and their performance sharply degrades when hard sediments and rocks are encountered.

Although the above studies have achieved feature detection of sonar images in their respective assumed scenarios, it is evident that the detection is less than ideal, and there are strict requirements for the quality of the acoustic imageries themselves and the feature richness of the detected scenes, which reduce the generalization performances of these methods. When AUVs carry sonar equipment for underwater detection tasks, the idealized characteristics of acoustic images cannot be guaranteed in most cases due to the complexity and unknown nature of the underwater environment, so it is necessary to develop a robust acoustic imagery feature detection method, which is also the cornerstone for designing an overall algorithm for underwater acoustic imagery matching.

With the rapid development of deep-learning techniques within the field of image processing, CNN (convolutional neural network) based feature detectors have been generated, which can be used to extract shallow, in-depth, and combined multidimensional features in images to achieve feature point detection. Zhou [15] proposed to match underwater acoustic images using deep-learning-based detectors, feature descriptors, and style transfer algorithms, and the matching effect was quantified and visualized. The experimental results showed that even though the learned detectors and descriptors were obtained based on the optical image dataset training, they surpassed the traditional handcrafted operators, such as the SIFT, in terms of the overall effect. When the style transfer method was introduced, the matching results of acoustic images were further improved, and this study can provide a reference for the design of deep-learning-based matching algorithms for underwater acoustic images in the future.

The above review of related studies shows that the feature point-based matching method is more robust and flexible and is the leading way of feature matching for underwater acoustic images at present. This kind of method can also work well when there are complex geometric transformations and external interference between two images; therefore, it has been more widely used in underwater acoustic imagery matching. However, most of the literature studies used the existing optical matching methods due to the fact that it is very challenging to develop a high-precision acoustic matching algorithm with particular generalization performance, so it is necessary to individually design two links of feature detection and feature description, each of which has a significant impact on matching results. Obviously, the first step of these processes is to find out a high-performance feature detection scheme according to the characteristics of underwater acoustic imagery.

According to the literature survey, existing literature does not independently develop an analysis of feature detection on underwater acoustic imagery, which is the motivation of this manuscript. We noticed that Oliveira [16] conducted research on the feature extraction of acoustic imagery involved in underwater navigation tasks, and the tested algorithms involved SURF, ORB, and BRISK. Their work achieved preliminary results, but the conclusion is limited to underwater navigation tasks, and the evaluated methods are not comprehensive enough to generalize to other underwater tasks, such as 3D reconstruction or underwater mapping.

In contrast, the research in this paper has expanded with multiple applications, devices, detection methods, and evaluation methods, aiming to obtain a more comprehensive solution. Meanwhile, the work in this manuscript is more focused, as reflected by the fact that the research content only explores the feature detection aspect in depth, as it is the cornerstone of feature description and matching. In addition to filling gaps in the existing literature, the study of this paper has made a more valuable contribution: on the one hand, due to the scarcity of underwater acoustic data, the research on deep-learning-based matching methods is still not mature enough, and many algorithms need further experimentation and validation—this manuscript carried out valuable work in this area; on the other hand, the research pipeline designed is flexible and can be updated according to the actual mission requirements to cope with the complex and changing underwater environment information.

The overall research pipeline proposed in this manuscript is shown in Figure 2. First, input underwater acoustic data, such as from FLS and SSS, and visualize them as a set of digital images. Meanwhile, the images can be pre-processed to generate regions of interest (ROIs) Second, extract layers ready for feature detection, such as grayscale, gradient, and PC layers. Third, set up detectors for feature extraction, such as handcrafted and deep-learning-based. Then, the best feature detection scheme tailored to the current scene is used for downstream tasks such as matching, mapping, and tracking.

Figure 2. Research pipeline proposed in this manuscript.

3. Methodology

Current feature point-based matching methods usually use grayscale or gradient information to detect keypoints, which are a set of potential feature points. However, this kind of method is easily affected by contrast, grayscale distortion, and noise, which are negative factors commonly found in underwater detection scenarios, and they will make the detection of keypoints unstable and inaccurate. On the one hand, due to the wide noise distribution, some points will be incorrectly detected as keypoints, whereas some correct keypoints may not be detected; on the other hand, low contrast and grayscale distortion phenomena on acoustic images will directly affect the number of detected feature points. Inaccurate feature point detection results will directly affect the design of subsequent descriptors, affecting the final matching results.

Therefore, to address these problems, this manuscript proposed to apply the PC (phase congruency) principle to underwater acoustic imagery feature detection. The PC layer is not affected by image illumination and contrast, and is robust to noise interference; it also provides rich texture, edge, and structure information consistent with human vision, so using the PC layer in underwater acoustic imagery feature detection tasks has the suitable space and apparent advantages.

To more clearly and intuitively understand the effect of PC processing of underwater acoustic imagery, Figure 3 shows the PC layer and original grayscale layer of the raw SSS image while comparing them with the classical gradient layers, which involve the Sobel, Scharr, and Laplacian layers.

Figure 3. Schematic diagram of various layers of underwater acoustic imagery.

3.1. Phase Congruency Layer

Phase Congruency Solving

The principle of PC started in 1981 when Oppenheim and Lim [17] first revealed the effect of phase information on image features, and subsequent researchers have extended PC in various directions [18,19,20]. One of the more classical PC-solving algorithms in image processing is the PC based on the multiscale multidirectional log-Gabor filter proposed by Kovesi in 2003 [21]. The log-Gabor filter is expressed as Equation (1):

$$\mathrm{LG}\left(\widehat{w}\right)=\exp \left(\frac{-{\left. [\ln \left(\widehat{w}/{\widehat{w}}_0\right)\right]}^2}{2{\left. [\ln \left({\sigma }_{\widehat{w}}/{\widehat{w}}_0\right)\right]}^2}\right)$$

(1)

assuming that $I\left(x,y\right)$ denotes the underwater acoustic imagery, and $M_{n\overline{o}}^e$ and $M_{n\overline{o}}^o$ represent the even- and odd-symmetric log-Gabor wavelets at the $n_{th}$ scale and ${\overline{o}}_{th}$ direction, respectively. The two wavelet functions are convolved with the image signal to obtain the corresponding components $e_{n\overline{o}}\left(x,y\right)$ and $o_{n\overline{o}}\left(x,y\right)$, respectively, and the overall computational process is described as Equation (2):

$$\left. [e_{n\overline{o}}\left(x,y\right),o_{n\overline{o}}\left(x,y\right)\right]=\left. [I\left(x,y\right)\times M_{n\overline{o}}^e,I\left(x,y\right)\times M_{n\overline{o}}^o\right]$$

(2)

The amplitude and phase of the underwater acoustic imagery after wavelet transform with scale $n$ and direction $\overline{o}$ are as follows:

$$A_{n\overline{o}}\left(x,y\right)=\sqrt{e_{n\overline{o}}{\left(x,y\right)}^2+o_{n\overline{o}}{\left(x,y\right)}^2}$$

(3)

$${\varphi }_{n\overline{o}}\left(x,y\right)=\arctan \left. [o_{n\overline{o}}\left(x,y\right)/e_{n\overline{o}}\left(x,y\right)\right]$$

(4)

Combining the analysis results in each direction, the PC solving of the underwater acoustic imagery can be obtained as Equation (5):

$$P{C}_2\left(x,y\right)=\frac{{\displaystyle \sum _n{\displaystyle \sum _{\overline{o}}{W}_{\overline{o}}\left(x,y\right)⌊A_{n\overline{o}}\left(x,y\right)\mathsf{\Delta }{\mathsf{\Phi }}_{n\overline{o}}\left(x,y\right)-\widehat{T}⌋}}}{{\displaystyle \sum _n{\displaystyle \sum _{\overline{o}}{A}_{n\overline{o}}\left(x,y\right)+\xi }}}$$

(5)

where $W_{\overline{o}}\left(x,y\right)$ indicates the two-dimensional (2D) frequency expansion weight factor, and $\mathsf{\Delta }{\mathsf{\Phi }}_{n\overline{o}}\left(x,y\right)$ is the 2D phase deviation function.

3.2. Classic Gradient Layers

3.2.1. Sobel Layer Solving

The Sobel operator is a discrete differentiation operator used to compute the approximate gradient of the image grayscale function. Furthermore, it is also a joint Gaussian smoothing plus differentiation operator with solid noise resistance.

Assume an underwater acoustic image $I$, derive it in both horizontal and vertical directions using the Sobel operator, and then perform a convolution operation on $I$ applying a kernel of size $K$. To ensure the accuracy and operation speed, $K$ is usually taken as 3, where $G_x$ and $G_y$ denote the convolution results in the horizontal and vertical directions, respectively, and the calculation formula is

$$G_x=\left. [\begin{array}{ccc}-1& 0& +1\\ -2& 0& +2\\ -1& 0& +1\end{array}\right]\times I$$

(6)

$$G_y=\left. [\begin{array}{ccc}-1& -2& -1\\ 0& 0& 0\\ +1& +2& +1\end{array}\right]\times I$$

(7)

Then, for any pixel on the underwater acoustic imagery, the gradient of its location can be calculated by using Equation (8).

$$G=\sqrt{G_x^2+G_y^2}$$

(8)

3.2.2. Scharr Layer Solving

The Scharr operator has enhanced its difference extraction capability based on the Sobel operator, and the solution principle is similar to that of Sobel. In general, the size of the edge detection filter of the Scharr operator is set to $\left(3\times 3\right)$, and the detection is also divided into two directions, which are calculated as Equations (9) and (10):

$$G_x=\left. [\begin{array}{ccc}-3& 0& 3\\ -10& 0& 10\\ -3& 0& 3\end{array}\right]\times I$$

(9)

$$G_y=\left. [\begin{array}{ccc}-3& -10& -3\\ 0& 0& 0\\ 3& 10& 3\end{array}\right]\times I$$

(10)

where $I$ represents the input underwater acoustic imagery, and $G_x$ and $G_y$ indicate the image gradient values obtained by solving.

3.2.3. Laplacian Layer Solving

The Laplacian operator is a kind of a spatial sharpening filter, which is a second-order differential operator in n-dimensional Euclidean space with rotational invariance. This operator usually considers the point whose second-order differential point of the pixel point value is 0 as an edge point. For the underwater acoustic imagery $I\left(x,y\right)$, the Laplace solving process of the image is

$${\nabla }^{2}I\left(x,y\right)=\frac{{\partial }^{2}I}{\partial x^2}+\frac{{\partial }^{2}I}{\partial y^2}$$

(11)

To find the second-order derivatives of the image $I\left(x,y\right)$ in the $x$ and $y$ directions, the second-order Laplacian operator expressing can be obtained as follows:

$${\nabla }^{2}I\left(x,y\right)=I\left(x,y+1\right)+I\left(x,y-1\right)+I\left(x+1,y\right)+I\left(x-1,y\right)-4I\left(x,y\right)$$

(12)

Finally, the filter mask of the Laplacian operator can be solved using Equations (11) and (12).

$$G_1=\left. [\begin{array}{ccc}0& 1& 0\\ 1& -4& 1\\ 0& 1& 0\end{array}\right]$$

(13)

It can be seen from Figure 3 that the PC is well-adapted to the nonlinear features of underwater acoustic imagery. Specifically, it can well contain the structural feature information in the raw acoustic imagery, and the preservation of edges is more desirable, while it can also effectively resist noise interference. On the contrary, the operators’ performances based on gradient information are poor, and it is not easy to adapt to the nonlinear gradient features on acoustic imagery; it is also more sensitive to the interference of noise. In the next section, this research will quantitatively and qualitatively evaluate the performances of the feature detectors on various layers from experiments.

4. Experiment

4.1. Experimental Method Setting

In terms of experimental method settings, several mainstream feature detection operators, including single-scale detection operators, multiscale detection operators, and deep-learning-based detectors, are selected in this manuscript, as depicted in Table 1. The purpose of the experiments is to fully explore the best performance of the detection operators of various modes on the underwater acoustic imagery layers and to verify whether the deep-learning-based feature detector will bring some opportunities for underwater acoustic imagery feature detection task.

Table 1. Introduction to detectors used in the experiment.

Sequences	Name	Time	Creator	Characteristic
1	AKAZE [25]	2011	Pablo F Alcantarilla	Multiscale detector
2	BRISK 1 [26]	2011	S Leutenegger	Multiscale detector
3	FAST [27]	2010	Edward Rosten	Single-scale detector
4	Harris [28]	1988	CG Harris	Single-scale detector
5	ORB 2 [29]	2011	GARY R.Bradski	Multiscale detector
6	Shi-Tomasi [30]	1994	J.SHI & C. Tomasi	Single-scale detector
7	SIFT [4]	2004	Lowe	Multiscale detector
8	SURF [5]	2008	Bay	Multiscale detector
9	SAR-SIFT [31]	2014	Flora Dellinger	Multiscale detector
10	KeyNet [23]	2019	Axel Barroso	Learned detector

¹ Detector used by BRISK is FAST-9-16. ² Detector used by ORB is Oriented FAST.

The above-mentioned feature detectors have their own advantages; however, underwater scenes are often complex and variable, resulting in detected sonar images that may have noise interference, grayscale distortion, translation transformations, rotation transformations, scale transformations, mixed rotation and scale transformations, viewpoint transformations, and slight distortions, which make it tough for a single detector to cope with. What is worse, these transformations are often nonlinear, irregular, and difficult to describe in mathematical language. Therefore, it is necessary to further explore a scheme for underwater acoustic detection scenarios that should concentrate on the advantages of existing detection operators and be able to effectively deal with the limitations of images themselves while meeting the real-time needs of underwater applications.

The detectors and layers set up for the experiments in this research are separately shown in Table 1 and Table 2. The detectors for sequences 1 to 9 are implemented using the benchmark OpenCV library [22], and a KeyNet [23] detector for sequence 10 is implemented using the benchmark Kornia library [24], both with default parameters to facilitate comparison and update for future researchers. In terms of the detection scene setting, three types of feature detection layers are selected, which are the grayscale, gradient, and PC layers, where the gradient layers are computed by the Sobel, Scharr, and Laplacian operators.

Table 2. Detection layers used in the experiment.

Layer Name	Gradient Layers	PC Layer	Grayscale Layer
Operator	Sobel	PC	Pixel
	Scharr
	Laplacian

4.2. Dataset

To evaluate the detection accuracy and processing time of different detectors on various layers, a random dataset of underwater acoustic imageries was selected in this paper. These images were obtained from various types of SSS and FLS sensors, and they are depicted in Figure 4. For the internal target richness, the dataset can be classified into simple and complex scenes; in terms of external factors, it can be classified into general, scale-transformed, noise-interfered, grayscale-transformed, and other phenomena. The division of this dataset is specified in Table 3, and the test samples are described as follows:

Figure 4. Schematic diagram of experimental dataset, the serial numbers 1–16 in the figure caption correspond to the 16 samples above. For the analysis of their characteristics, please refer to the above.

Table 3. Characteristics division of dataset scenarios.

Characteristics	Simple Scenes	Complex Scenes
General	2, 11	9, 14
Scale-transformed	1	13
Noise-interfered	5, 12	8, 10
Grayscale-transformed	6, 7	3, 4
Others	15	16

• This image was provided by the T-SEA Marine Technology company [32] and originated from an SSS device with an image size of 112 × 186. The content of this image is the wreck site without any pre-processing of the image.
• Image 2 is a scale transformation of image 1 with a scale-transformation factor of 2. The transformed image size is 224 × 372, and the purpose of the transformation is to check the scale-invariant performances of the detectors.
• Image 3 was provided by Peng Cheng Laboratory (PC LAB) [33] from an SSS device. The image size is 429 × 424, and the image content is a “ridge-like zone” on the seafloor. This image is used to test the detection ability of the detector for protruding targets in a monotonous background, without any pre-processing of the image.
• Image 4 was provided by PC LAB from an SSS device. The image size is 337 × 425, and the image content is a seafloor protrusion with severe acoustic shadowing. This image is used to check the detection capability of the detector for targets with severe acoustic shadowing in a sandy background, without any processing of the image.
• Image 5 was provided by PC LAB from an SSS device. The image size is 210 × 305, and the image content is a depressed area on the seafloor. This image is used to check the detection capability of the detector for depressed targets in a monotonous background, without any pre-processing of the image.
• Image 6 was provided by PC LAB from an SSS device. The image size is 245 × 264, and the image content is a tubular object on the seafloor. This image is used to check the detection capability of the detector for tubular targets in a monotonous background, without any pre-processing of the image.
• Image 7 was provided by PC LAB from an SSS device. The image size is 286 × 258, and the image content is a submarine tubular target with significant grayscale distortion. This image is used to check the detector’s detection capability for tubular targets under the influence of grayscale distortion, without any pre-processing of the image.
• Image 8 was provided by PC LAB from an SSS device. The image size is 495 × 374, and the image content is a rocky area of the seafloor with strong noise interference. The image was denoised using a median filter in order to check the detector’s feature extraction capability under strong noise interference.
• Image 9 is the original image of image 8 without any pre-processing of the image, the purpose of which is to compare with the test results of image 8.
• Image 10 was provided by PC LAB from an SSS device. The image size is 446 × 371, and the image content is a seabed area with a lot of noise interference. The image is intended to complement the test scenes of test images 8 and 9. The targets in image 10 are mostly raised, and no pre-processing of the image has been performed.
• Image 11, from [34], is an FLS image with an image size of 116 × 127. The image contains a tire laid underwater, and the image has significant noise interference. This image is used to check the detection ability of the detector under substantial noise interference, and the image is denoised using a median filter.
• Image 12 is the original image of image 11, without any pre-processing of the image, which is intended to be compared with the test results of image 11.
• Image 13, from [34], is an FLS image with an image size of 44 × 50. The image content is a steel frame laid underwater, without any pre-processing of the image.
• Image 14 is obtained based on image 13 by scale transformation with a scale factor of 2. The transformed image size is 88 × 100. The purpose of the transformation is to check the scale-invariant performances of the detectors.
• Image 15 is from the Sound Metrics database [35] and is a DIDSON (dual-frequency identification sonar) image with an image size of 351 × 359. The image content is a floor tile underwater, and the image is used to check the feature extraction capability of the detectors for DIDSON images without any pre-processing of the image.
• Image 16 was provided by DeepVision AB company [36] from an SSS image device with an image size of 800 × 400. The image content is a low-texture, low-contrast depressed region of the seafloor. The image was designed to check the detectors’ feature extraction capability for low contrast and texture areas without any pre-processing of the image.

Next, some representative underwater acoustic imageries are taken to demonstrate the display of each layer to help the reader visualize the information represented by various layers. The demonstration results are described in Figure 5 and Figure 6.

Figure 5. Display of various layers of some side-scan sonar images.

Figure 6. Display of various layers of some forward-looking sonar images.

Finally, the feature detectors in Table 1 were used for three gradient layers, one PC layer, and one grayscale layer. Quantitative and qualitative evaluation criteria were used to find the feature detector with the best detection performance, the layer best suited for feature detection, and the best combination of the two; these criteria will be given in the next section.

4.3. Evaluation Criterion

In terms of the evaluation criterion, this paper introduced the Number of keypoints (N), Precision (P), Distribution (D), and Time consumption (T). It is worth mentioning that these evaluation criteria are developed for the region of interest (ROI), not the whole image, which is mapped by experts in the field of underwater acoustics. The detection performance of the detector is reflected by comparing the number of features correctly identified within the ROI, the precision, and the distribution of feature points; the real-time performance of the detector is reflected by comparing the time consumed for feature point detection. In traditional optical image-matching studies, the metric used to evaluate the performance of detector is the feature repeatability, which requires prior knowledge of the image pair corresponding to the transformation relationship. However, due to the scarcity of underwater acoustic datasets, the corresponding image pairs cannot be effectively obtained, so the test dataset used in this manuscript is single, random, and irregular, including both SSS and FLS image data. Moreover, the image size is not adjusted and is kept in its raw state to maximize the fairness of the test.

Firstly, experts in the field of sonar image interpretation are requested to manually map the ROIs, as shown in Figure 7. These ROIs all have a clear detection value, and the closer the feature detection effect is to these regions, the better the detector’s performance. The keypoints that fall into these regions are considered valid feature points, and the detector’s performance is accordingly evaluated. In addition, this manuscript fine-tuned the thresholds of some detectors during the experiment to ensure that the number of their detection points will not explode, making the final evaluation results in a valid range.

Figure 7. Schematic diagram of ROIs (regions delineated by red line) on tested images drawn by experts. The serial numbers 1–16 in the figure correspond to the 16 samples in Section 4.2.

4.3.1. Number of keypoints (N)

N is computed to evaluate the richness of the detection effect. It directly affects the starting quantity of subsequent matches and the accuracy of the solution of the transformation matrix.

$$N={\displaystyle \sum _{i=1}\left[\left(x_i,y_i\right)\in RO{I}_s\right]}$$

(14)

where $\left(x_i,y_i\right)$ represents the coordinate point that falls within the ROIs.

4.3.2. Precision (P)

P is used as the criterion to evaluate the accuracy of detection effect, which is calculated as Equation (15):

$$P=\frac{N}{N_{all}}$$

(15)

where $N_{all}$ indicates the number of detected keypoints on the whole image.

4.3.3. Distribution (D)

The distribution criterion is used to measure the uniformity of the detection results, since an ideal feature detector should detect feature points that are uniformly distributed. For the distribution measure, the traditional histogram idea is used to divide the whole image into a set of uniform regions. In the experimental part, the width and height of each region are 0.1 of the entire imagery, and there are 100 regions in total. Since this work only considered the homogeneity of feature points inside the ROI, where the area is covered in red in the figure, only the feature points inside the ROI are counted, and only the regions ($bins$) overlapping with the ROI with an area larger than 0 are considered, while the rest of the regions are ignored, as shown in Figure 8. After counting the number of keypoints $n_i$ in each small area, the chi-square test is used to check the distribution of keypoints. Specifically, this part of the research proposed the hypothesis that the distribution of feature points in the $H_0$: ROI follows a two-dimensional uniform distribution. Therefore, the probability $p_i$ that a feature point falls in each cell region is equal to the ratio of the area $S_i$ of the region to the total area $S_{ROI}$ of the ROI. Then the expectation of the number of feature points in each small area $i$ is calculated as Equation (16):

$$E_i=p_i\times N=\frac{S_i}{S_{ROI}}\times N$$

(16)

where $N$ is the total number of keypoints in the ROI. From the above, the chi-square statistic can be calculated based on the actual distribution of keypoints and the expected distribution, which is computed as follows:

$${{\rm X}}^2={\displaystyle \sum _i\frac{{\left(n_i-E_i\right)}^2}{E_i}}$$

(17)

Figure 8. Schematic diagram of Distribution (D) calculation. The area enclosed by the red line is the ROI to be calculated, and the detected feature points are represented by green circles. Image is offered by DeepVision AB company [36].

The probability is then calculated based on the value of the chi-square distribution function $cdf$ at ${{\rm X}}^2$ for a sample with degrees of freedom $N_{bin}-1$. Then the final calculation formula is derived, where the significance level parameter was set to 0.05.

$$D=1-cdf\left({{\rm X}}^2,N_{bin}-1\right)$$

(18)

4.3.4. Time consumption (T)

T denotes the time consumed to detect per keypoint and is computed as

$$T=\frac{T_{all}}{N_{all}}$$

(19)

where $T_{all}$ denotes the total time taken to detect all keypoints ($N_{all}$) on the whole image.

4.4. Testing Environment

All tests were conducted under a 64-bit Windows 10 operating system with an Intel Core i7-10750 2.60 GHz processor, 16 GB of physical memory, and an NVIDIA GeForce RTX2060 graphics card. All algorithms were compiled using Python 3.7. To ensure the reliability of the test results and avoid the influence of coarse differences, the evaluation results were averaged over ten test results.

5. Experiment Results

The evaluation criterion can evaluate the detectors in four aspects: richness, accuracy, distribution uniformity, and timeliness, and the evaluation scope is more comprehensive. In order to more intuitively find the most suitable feature detector for underwater acoustic images, this paper proposed a normalized evaluation strategy that normalizes the first three evaluation criteria, the number of keypoints (N), Precision (P), and Distribution (D), into each index of the evaluation algorithm, and then multiplies them by their respective weights to calculate the final evaluation score. Each index is assigned a score (0–10) according to ranking order (0–10), and then the final score $S$ is calculated as follows:

$$S={\displaystyle \sum w_i{s}_i}=w_N{s}_N+w_P{s}_P+w_D{s}_D$$

(20)

where $s_i$ is the score of each individual metric of the feature detector, and $w_i$ is the weight value corresponding to the individual metric. $s_N$ is the metric score obtained from the evaluation of the number of keypoints, $w_N$ is the weight value corresponding to $s_N$, $s_P$ is the metric score obtained from the evaluation of precision, $w_P$ is the weight value corresponding to $s_P$, $s_D$ is the metric score obtained from the evaluation of the distribution, and $w_D$ is the weight value corresponding to $s_D$. $S$ is the final score of the performance evaluation of each detector. Considering the practical applications in ocean engineering, the experts suggest that the $w_N$, $w_P$, and $w_D$ distributions take the values 0.35, 0.60, and 0.05, respectively. The results on each test image are presented in the form of a table, where, for the evaluation of gradient layers, it selected the one with the greatest Precision (P) among the three layers for presentation. The specific experimental results are shown in Table 4, Table 5, Table 6, Table 7, Table A1, Table A2, Table A3, Table A4, Table A5, Table A6, Table A7, Table A8, Table A9, Table A10, Table A11, Table A12 and Table A13. In order to accommodate the layout of the manuscript, three typical table results were selected for analysis in this section; please refer to Appendix A for the completed test results. The highest scoring items are marked with *.

Table 4. Comparison of Number of keypoints (N), Precision (P), and Distribution (D) of image 6.

	Layers	Grayscale Layer			Gradient Layer (Sobel) *			PC Layer *
Detectors		N	P	D	N	P	D	N	P	D
AKAZE		41	0.8913	0.0378	100	0.5882	0.0217	46	0.5349	0.2422
BRISK		559	0.4815	0.0000	1088	0.4131	0.0000	441	0.4608	0.0000
FAST		332	0.4618	0.0000	836	0.3696	0.0038	331	0.4206	0.0009
Harris		9	0.4286	0.3444	11	0.2973	0.0539	66	0.3158	0.5465
ORB *		332	0.8448	0.0000	322	0.7368	0.0000	315	0.7360	0.0000
Shi-Tomasi		43	0.4300	0.9462	43	0.4300	0.9879	50	0.5000	0.8790
SIFT		151	0.4535	0.3539	184	0.4044	0.3060	185	0.4780	0.0000
SURF		255	0.4374	0.0135	299	0.3705	0.0390	236	0.4574	0.0140
SAR-SIFT		12	0.7059	0.0000	30	0.3409	0.0000	295	0.3203	0.0000
KeyNet		159	0.3850	0.7262	170	0.3020	0.5512	160	0.3922	0.3252

Table 5. Comparison of Number of keypoints (N), Precision (P), and Distribution (D) of image 8.

	Layers	Grayscale Layer			Gradient Layer (Laplacian)			PC Layer *
Detectors		N	P	D	N	P	D	N	P	D
AKAZE		291	0.6063	0.0000	16	0.8000	0.0241	334	0.6095	0.0000
BRISK		614	0.6221	0.0000	1254	0.6006	0.0000	2234	0.5935	0.0000
FAST		194	0.6978	0.0000	924	0.6067	0.0000	1410	0.5975	0.0001
Harris		123	0.6089	0.0000	105	0.6069	0.0000	381	0.5443	0.0240
ORB *		344	0.6880	0.0000	288	0.6973	0.0000	324	0.6480	0.0000
Shi-Tomasi		70	0.7000	0.0091	69	0.6900	0.0008	67	0.6700	0.6352
SIFT		950	0.5675	0.0000	77	0.5461	0.0000	1363	0.5623	0.0000
SURF		1125	0.6134	0.0109	522	0.5781	0.0000	1487	0.6165	0.0000
SAR-SIFT *		146	0.8795	0.0000	2278	0.6142	0.0000	2322	0.6180	0.0000
KeyNet		1059	0.5970	0.9472	1021	0.6241	0.9671	1119	0.6159	0.0957

Table 6. Comparison of Number of keypoints (N), Precision (P), and Distribution (D) of image 10.

	Layers	Grayscale Layer			Gradient Layer (Scharr)			PC Layer *
Detectors		N	P	D	N	P	D	N	P	D
AKAZE		436	0.9316	0.0000	707	0.8559	0.0000	45	0.9783	0.0059
BRISK		7804	0.7963	0.0000	9992	0.7930	0.0000	614	0.8899	0.0000
FAST		5541	0.7354	0.0000	7810	0.7223	0.9939	390	0.8387	0.0000
Harris		719	0.6548	0.0423	4	0.6667	0.7437	56	0.8000	0.0405
ORB *		491	0.9820	0.0000	470	0.9514	0.0000	432	0.9841	0.0000
Shi-Tomasi		84	0.8400	0.0927	73	0.7300	0.2023	84	0.8400	0.0234
SIFT		994	0.7821	0.0000	1231	0.7245	0.2000	46	0.8519	0.0000
SURF		2400	0.7846	0.0000	3292	0.7735	0.0000	123	0.9111	0.0000
SAR-SIFT		203	0.9144	0.0000	3	1.0000	0.0000	3015	0.7320	0.0000
KeyNet		1197	0.7389	0.9982	1365	0.7708	0.9445	668	0.7687	0.0024

Table 7. Statistical table of results of experimental tests. (n) indicates that the number of times with optimal performance is n. Total number of tests is 16.

Test items	Detector	Layer
Best	ORB 1 (7)	PC (7)
Next best	BRISK2 (6)	Sobel (5)

¹ Specific detection operator is Oriented FAST. ² Specific detection operator is FAST-9-16.

Table 4, Table 5 and Table 6 are utilized as typical schematic tables to illustrate the statistical contents. The horizontal axis of each table is used to illustrate the performance comparison of detection layers, and the vertical axis of each table is used to illustrate the performance comparison of detectors. The performance evaluation metrics include the Number of keypoints (N), Precision (P), and Distribution (D). Finally, all the metrics are weighted to determine the optimal detector and detection layer.

From the statistical results in Table 7, it can be seen that the most suitable detectors for the underwater acoustic imagery are ORB and BRISK, which appear seven and six times, respectively. It is worth mentioning that both detectors use the detection algorithm of the FAST series.

In terms of detection layers, the PC layer has the best performance, appearing seven times, followed by the classic Sobel layer appearing five times. This result shows the PC layer’s great potential in underwater acoustic imagery feature detection.

Next, the paper evaluated the detection speed of various detectors and detection layers, and the results show that the FAST detector and PC layer have an apparent advantage. In comparison, grayscale layers are the slowest to detect. The specific detection time difference is demonstrated in Figure 9.

Figure 9. Comparison of average time consumed to detect single feature point.

As shown in Figure 10, images with good visual perception of the detection results are also labeled as optimal items in quantitative statistical Table 4, Table 5 and Table 6, which are the quantitative evaluation results for images 6, 8, and 10, respectively, fully validating the validity and rationality of our proposed evaluation framework.

Figure 10. Effects of feature detection on PC layer using ORB detector; feature points are highlighted with green. The serial numbers 1–16 in the figure correspond to the 16 samples in Section 4.2.

In the following, this manuscript will use the detection scheme that is the ORB detector with the PC layer, which achieves the best performance for the feature detection of the acoustic imagery dataset. The detailed detection effects are shown in Figure 10. In order to observe the details of feature detection, the feature points detected by the PC layer are mapped to the original image.

As shown in Figure 10, it can be seen that the ORB detector has an overall good detection effect on the PC layer, which can resist the noise interference on the underwater acoustic imageries, and the detected feature points can be accurately around the target area and more uniformly distributed. This detection method allows for effective feature detection on underwater acoustic imageries while meeting the need for real time.

In addition, the detection effect of images 1 and 2 shows that the scheme can resist scale transformation; the detection effect of image pairs 8 and 9, and image pairs 11 and 12 indicates that the scheme based on the PC layer does not need to denoise the acoustic images in advance and can achieve good results by directly detecting the raw images. It is worth mentioning that this almost noise-independent feature detection scheme can bring significant help to the underwater sonar target tracking tasks [37,38,39], which, in turn, can effectively enhance the autonomous capability of AUVs.

However, this detection scheme is currently unable to overcome the challenges posed by acoustic shadowing, as shown in test images 4, 7, and 16. Moreover, the detection results on small-scale forward-looking sonar images are also unsatisfactory, such as image 13. Both of these are the directions for continued research in the future.

In order to compare the detection effects of the detector on various layers, Figure 11 displays the detection effects of the ORB detector and grayscale layer, Sobel layer, Scharr layer, Laplacian layer, and PC layer combinations. In order to visually compare the detailed effects of feature detection, the detection performances on each layer are mapped to the original image. As can be seen in Figure 11, the PC layer has stronger noise resistance than the gradient layer, whereas richer feature points can be detected in the PC layer than in the grayscale layer. This detection scheme can achieve a good balance between the precision and velocity, making it more suitable for underwater sensing.

Figure 11. Comparison of detection effects of ORB detector on PC layer, grayscale layer, Sobel layer, Scharr layer, and Laplacian layer.

6. Discussion

In this manuscript, we designed feature detection experiments from several scientific perspectives, including the sonar-type differences, scale transformation, noise interference, detector design methods, and the properties of detection layers. In addition to the intuitive optimal results above, there are many valuable research insights that can be found from the experimental results: (1) KeyNet, a deep-learning-based feature detector, is not the optimal choice, which may be related to its lack of pretraining on acoustic datasets. In the future, if more underwater acoustic datasets can be collected and fine-tuned on the original model, the final detection results may be better. (2) Both the ORB and BRISK detectors perform well, and the underlying detection operators they used are based on the FAST derivation, which is very clever. In addition, the FAST detector achieved the best results in velocity evaluation, so developing better FAST-like detectors may present opportunities for underwater acoustic imagery feature detection. (3) Layers are the basis of feature detection. In the experiments of this paper, the best results were achieved by PC-layer-based detection, which achieves a balance between accuracy and speed. The PC layer has excellent resistance to noise interference and brightness variations, which is well-suited for applications in underwater acoustic matching tasks. In the future, if a feature detector based on PC layers can be developed, it will become more convenient to engage in underwater perception tasks. (4) The test results show that the combination of the ORB detector and PC layer is not perfect. They are not sensitive to the line features in the image, and how to enhance their line feature extraction capability will be essential work in the future because it will affect underwater applications such as underwater pipeline maintenance and underwater cable laying. (5) From the comparison of all the experimental results, the detection results of various detectors on side-scan sonar images are better than those on forward-looking sonar, which may be due to the higher resolution and better quality of side-scan sonar images, which are closer to the optical images in terms of image display effects. In the future, it is worth considering how to use deep-learning methods to achieve domain transfer between acoustic and optical images and reduce the imaging differences between the two, which will also provide an idea for underwater acoustic imagery detection and matching research.

7. Conclusions

This manuscript systematically investigated how to more effectively detect feature points on underwater acoustic imagery by enumerating the most commonly used detectors for experiments and comparing their effects on the grayscale, gradient, and PC layers. Tests were conducted on a dataset containing side-scan sonar and forward-looking sonar images, and a new evaluation framework was proposed. The experimental results show that the feature detection using the ORB detector and PC layer is the best, and this combined detection mode can ensure the detection accuracy while also reducing the interference of image noise, and has obvious advantages in terms of speed. It is worth mentioning that this manuscript is the first comprehensive proposal to use the PC layer for underwater acoustic imagery feature detection. Throughout the testing process, no assumptions were made on the sonar type and imagery attributes, and the test results proved that the evaluation framework proposed in this manuscript has good generalization performance.

The research work in this manuscript selected typical acoustic images in practical ocean applications, analyzed the characteristics of each image and classified them, and, finally, displayed the experimental report of each of them in detail to provide reference for practitioners. The conclusions can also provide new ideas for novices in the field of acoustic image processing and researchers who are proficient in image processing but do not understand acoustic images. However, there can be more detector attempts beyond the test methods set in this research, as well as the use of methods such as the hyperparameters to explore the optimal parameter settings for the detector instead of the default parameters. For these cases, the evaluation framework in this manuscript preserves the pipeline and deep-learning interfaces to facilitate testing.

In the future, the research will introduce deep-learning-based assessment metrics while expanding the underwater acoustic dataset to form a more comprehensive and objective assessment framework and open-source it in the community for researchers to use; moreover, it will consider the performance of feature detectors and feature descriptors when combined to conclude a complete solution for underwater acoustic imagery matching.