Environmental Perception Method for Unmanned Surface Vehicles Based on Sea–Sky Line Detection

Abstract

This paper is dedicated to solving the environmental perception system problem of unmanned surface vehicles (USVs) experiencing adverse sea conditions and complex mission scenarios. First, the functionalities and characteristics of each subsystem in the USV environmental perception system under different mission scenarios are analyzed, and an efficient and stable environmental perception system is designed. Second, the static and dynamic characteristics of the sea–sky line are investigated, along with the impacts on each subsystem of the environmental perception system when the USV experiences six-degree-of-freedom motion on the sea surface. Based on the above analysis, a sea–sky line detection method based on the radar–electro-optical system is designed. This method utilizes the features of the radar and electro-optical subsystems to redefine the region of interest, effectively suppressing interference from non-sea–sky line edges, thereby improving detection efficiency and accuracy. Furthermore, a sea–sky line-based target detection algorithm is proposed, which confines the search area to the vicinity of the detected sea–sky line, significantly reducing false detections caused by sea clutter and noise. Sea trials demonstrate that the proposed method enhances the accuracy and real-time performance of USV environmental perception. The proposed systematic approach offers a practical solution for improving the robustness of USV environmental perception in complex marine environments. Sea trials have shown that the method improves the effectiveness of target information by 3.61%.

1. Introduction

Compared to manned vessels, USVs offer distinct advantages in performing missions that are highly dangerous, repetitive, or conducted in harsh environments [1,2,3,4]. Through complete or intermittent autonomous control by onboard computers, USVs achieve a high degree of automation. This autonomy enables them to undertake complex maritime tasks effectively while significantly reducing personnel operational risks.

The intelligent systems of USVs include the environmental perception system, the control system, the path planning system, the communication system, and the human–computer interaction system. The environmental perception system [5,6], as the core of USV intelligence, is dedicated to acquiring information about surrounding maritime targets. The control system [7,8] is responsible for governing the USV’s steering gear and propulsion system, ensuring that the vessel follows the intended trajectory. The path planning system [9,10] is tasked with determining the optimal path for the USV based on the current mission requirements. The communication system [11] facilitates data exchange between the USV and shore-based equipment. Through the communication system, the human–computer interaction system enables the issuance of commands and remote control of the USV.

USVs play significant roles in both military and civilian domains. Militarily, they are extensively deployed for anti-submarine warfare, coastal defense, mine detection, and counter-piracy missions [12]. Their ability to sustain prolonged patrol and surveillance operations, unimpeded by human fatigue or emotional factors, makes them a vital asset for enhancing maritime security and combat capabilities. Military USVs are equipped with advanced sensors and communication systems, enabling efficient intelligence gathering, threat detection, and strike missions, thereby providing substantial support to naval operations. In civilian applications, USVs are primarily used for hydrological monitoring, maritime search and rescue, and seabed exploration. For hydrological monitoring, USVs carry various sensors to conduct real-time observation of marine environmental parameters—such as water quality, temperature, and salinity—delivering essential data for oceanographic research and resource management. In maritime search and rescue operations, their rapid response and long-range search capabilities allow for quick deployment in emergencies, improving mission efficiency and success rates for life-saving missions. Furthermore, USVs contribute significantly to seabed exploration [13] by using advanced sonar and imaging technologies for topographic mapping, resource prospecting, and marine biological studies, thereby supporting scientific discovery and oceanographic resource development. As shown in Figure 1, the ‘Tianxing-1′ USV is performing an area search mission [14].

In sea–sky background images, the boundary between the sky and the sea surface is called the sea–sky line. As a crucial feature in such images, the sea–sky line helps distinguish the sky from the sea and facilitates target detection [15,16]. During USV missions, effective sea–sky images acquired by electro-optical devices generally comprise three main components: the sky, the sea–sky line, and the sea surface. Unlike terrestrial scenarios, where targets may appear anywhere in the image, requiring a full-image search, targets in valid sea–sky images are predominantly located near the sea–sky line. This characteristic allows for rapid positioning of marine targets by leveraging the sea–sky line’s location. Consequently, confining the detection area to the vicinity of the sea–sky line substantially reduces the ROI, mitigating interference from sea clutter and noise, lowering computational load, and improving both the efficiency and accuracy of target extraction.

Sea–sky line detection methods primarily include algorithms based on linear fitting [17,18], image segmentation [19], gradient saliency [20], transform domains [21], and information entropy [22]. Under simple sea–sky backgrounds, linear fitting-based algorithms demonstrate satisfactory performance. However, in complex backgrounds, selected candidate points are easily disturbed. Image segmentation-based methods struggle to determine optimal thresholds and exhibit limited adaptive noise resistance.

Gradient-saliency-based detection algorithms are prone to erroneous outcomes because various disturbances exhibit edge features similar to those of the sea–sky line. Although transform-domain methods can address sea–sky line detection in complex backgrounds, they significantly increase computational complexity, thereby failing to meet the real-time requirements of USV applications and limiting their practicality [21]. In descending order of computational complexity, common transform-domain techniques include the Laplace transform, wavelet transform, shearlet transform, and Hough transform. While information-entropy-based algorithms exhibit strong environmental adaptability, they also entail substantial computational overhead.

2. Environmental Perception System for USVs

Environmental perception technology is a core component of USVs, primarily used to acquire information about surrounding maritime targets. This system holds a crucial position in the development of USVs. By detecting sea surface targets, the environmental perception system enables a USV to build a situational awareness map of its surroundings, thereby achieving a more comprehensive and in-depth understanding of its current operational environment. This provides essential support for the USV’s autonomous navigation and mission execution [23,24]. As shown in Figure 2, the environmental perception system for a USV mainly comprises Marine radar [25,26], a perception computer, electro-optical equipment, and other related devices.

The environmental perception system’s functionality extends beyond providing situational awareness; it directly influences the USV’s mission planning and execution capabilities. By analyzing and interpreting the current situational map, the USV can formulate specific task instructions and support intelligent mission planning. The data and information provided by the perception system are vital to the USV’s decision-making processes, furnishing accurate key information—such as target identification, tracking, and localization—that enhance the vessel’s operational efficiency and mission accomplishment capability [27,28].

Furthermore, the environmental perception system plays a significant role in ensuring the USV’s safety. By monitoring the surroundings in real time, the system enables the USV to promptly detect the direction, position, and motion trajectories of other surface vessels, thereby enabling effective obstacle avoidance and ensuring safety during navigation [29]. For military USVs, the environmental perception system can lock onto critical sections of enemy vessels, enabling precise strikes against key enemy equipment and systems. This precision strike capability is of great significance for enhancing the combat effectiveness of USVs in military operations.

2.1. Radar–Electro-Optical System Composition

2.1.1. Marine Radar

Marine radar plays a vital role in the perception system of USVs. It emits microwave pulses into its surrounding area. When these pulses encounter objects such as buoys, islands, ships, or coastlines, they scatter. The radar determines target information by receiving a portion of the backscattered waves. Within the perception system of USVs, radar detection typically serves as the initial step in the overall detection process and is commonly used for long-range target detection. The Marine radar provides target information, including distance, azimuth, speed, and heading. Although marine radar exhibits significant detection errors, with range inaccuracies potentially exceeding several tens of meters, the azimuth information of targets is relatively accurate. Typically, the deviation between the azimuth of a target detected by the radar and its actual azimuth does not exceed 0.3 degrees. Consequently, utilizing the marine radar to cue electro-optical equipment for target pointing is a reliable strategy. By leveraging target information from marine radar, USVs can achieve situational awareness and localization of their surroundings, providing fundamental data for subsequent decision-making and mission execution. Despite its inherent detection errors, the Marine radar remains an indispensable and critical component within the perception system of USVs.

2.1.2. Perception Computer

The main modules of the perception computer include the communication module, radar module, dehazing module, detection and tracking module, data acquisition module, and decision-making module. The communication module is responsible for transmitting information between various sensors and the perception computer, as well as facilitating interaction with the planning system [30,31]. It ensures the reliable transmission of sensor data and effective communication between the perception computer and other systems. The radar module processes target information detected by the marine navigation radar. It classifies, processes, and analyzes this target information, eliminating false targets caused by factors such as sea waves. Through the radar module, the USV obtains accurate target information, providing foundational data for subsequent processing and decision-making. The dehazing module performs fog detection using visible-light information from the electro-optical equipment. Based on the detection results, it assesses the sea fog conditions in the USV’s environment. If heavy fog is present, the dehazing module preprocesses electro-optical images to enhance image quality, thereby improving the accuracy and stability of target detection and tracking. The detection and tracking module first performs detection on visible-light information from the electro-optical sensor to obtain the pixel coordinates of various targets in the image. Subsequently, it classifies and identifies these targets. Using the targets’ pixel coordinates, it predicts the area where a target is likely to appear in the next frame, thereby determining the ROI position. The module then performs detection again, this time only within the ROI, to achieve real-time target tracking. During maritime missions, the data acquisition module primarily collects target-image information. Once the perception computer successfully identifies a target, the acquisition module captures the image information of the current frame. The decision-making module constructs an overall situational map using target information detected by various sensors. Based on the current mission and this situational map, the decision-making module conducts analysis and issues commands to the sensors to fulfill specific task requirements. This module is also responsible for handling various exceptional cases, such as deck wetness and target loss, to enhance the perception system’s robustness in tasks like target tracking.

2.1.3. Electro-Optical Equipment

The electro-optical equipment primarily comprises a visible light sensor, an infrared sensor, a laser sensor, a stabilized pan-tilt platform, and an inertial navigation system (INS). The visible-light sensor acquires visible-light images of targets at varying distances using multi-level zoom. Visible-light imagery is one of the most critical information sources within the perception system. By processing these images, the system can determine the target’s category and provide essential data for dynamic target tracking.

Infrared sensors are categorized into cooled and uncooled types, with cooled sensors offering superior imaging performance. They are primarily employed for nighttime operations. When a USV executes missions at night, the visible-light sensor fails to capture clear imagery, whereas the infrared sensor compensates for this limitation by enabling nighttime imaging.

The laser sensor is utilized for precise target positioning. As the laser sensor and the visible light sensor are coaxially installed, when the electro-optical system locks onto a target at the center of the visible light image, the laser beam emitted by the laser sensor can accurately hit the target, thereby acquiring the distance between the target and the USV. By integrating the USV’s current longitude and latitude, heading information, and the azimuthal orientation of the electro-optical equipment, the precise longitude and latitude of the target can be determined.

Compared to the detection information provided by marine radar, the data from the laser sensor is significantly more accurate, typically with errors not exceeding 5 m. Consequently, the laser sensor becomes a key method for USVs to acquire precise target location information.

The stabilized pan-tilt platform plays a vital role in two main aspects. Firstly, based on target information from the marine radar, the platform adjusts the pointing direction of the electro-optical equipment to ensure the target remains within the field of view of the visible-light sensor. Secondly, during high-speed navigation of the USV, the platform stabilizes the target at the image center, mitigating the effects of vessel pitch and roll. The stabilized platform possesses horizontal and vertical degrees of freedom. The horizontal mechanism enables high-speed 360-degree free rotation. Upon receiving target pointing information, the platform can rapidly rotate horizontally to orient the electro-optical equipment toward the target, ensuring the target remains visible to the visible-light sensor. Under adverse sea conditions, such as significant wave impact, the platform can predict target position and react promptly to maintain target stability within the image.

2.2. Radar–Electro-Optical System Perception Strategy

Figure 3 shows a schematic diagram of information flow within the USV environmental perception system. The entire perception process begins with the marine radar scanning the surrounding waters of the USV. The radar detects targets and transmits information such as their distance and bearing to the perception computer. The perception computer then processes this data, filtering out false targets caused by factors such as sea clutter, to obtain accurate target information.

Based on the current mission requirements and the situational map, the perception computer selects key targets and sends guidance commands to the electro-optical system. Upon receiving the commands, the electro-optical system controls the stabilized pan-tilt platform to rotate, ensuring the target enters the field of view of the visible light sensor. The system simultaneously calculates and sets the optimal zoom level based on the target’s distance. Once pointed, the electro-optical system captures the optical image and returns it along with its status data to the perception computer.

The perception computer first processes the image using the dehazing module if necessary. The cleaned image is then passed to the detection and tracking module. This module identifies the target, obtains its pixel coordinates and offset from the image center, and sends the offset data back to the electro-optical system as guidance instructions. The electro-optical system uses this information to keep the target centered in the field of view, enabling stable, real-time tracking.

Throughout the tracking process, the decision-making module assesses the tracking stability. Once stable tracking is confirmed, the perception computer commands the laser sensor to fire, acquiring the target’s precise distance. Concurrently, the data acquisition module records all relevant information for each frame, including the timestamp, target type, geographic coordinates, distance, and USV posture data. This entire process enables the USV to detect, track automatically, and record targets.

Furthermore, mission instructions can be sent from a shore-based host computer to the USV’s perception computer via a wireless link. Conversely, the perception computer’s display interface can be transmitted back to the host computer, allowing remote personnel to monitor mission execution in real time.

3. Analysis of Sea–Sky Line Characteristics

3.1. Static Characteristic Analysis of the Sea–Sky Line

During the mission execution of USVs, the effective sea–sky background images acquired by the electro-optical equipment are primarily composed of two parts: the sky and the sea surface. As shown in Figure 4, the sea–sky background image and its corresponding gray-scale image are presented. Through image analysis, a significant gray-scale difference is observed between the sky and sea surface regions. Consequently, a distinct boundary line, namely the sea–sky line, exists at the junction of the sky area and the sea surface area. Furthermore, it is found that maritime targets are located near the sea–sky line within the sea–sky background image.

To more intuitively observe the pixel gray values in the sky and sea-surface regions of a sea–sky background grayscale image, a grayscale visualization is applied to the image. Figure 5 shows the grayscale visualization effect of the sea–sky background grayscale image. This visualization more directly reflects the gray value of each pixel in the sea–sky background grayscale image. Analysis of the 3D grayscale histogram reveals that the pixel gray values in the sky and sea surface regions fluctuate within their respective narrow ranges. Due to sea clutter, the 3D grayscale histogram shows that the fluctuation of gray values in the sea surface region is greater than in the sky region. Furthermore, at the junction between the sky and sea-surface regions, the gray values exhibit a distinct, cliff-like abrupt change; this location is identified as the sea–sky line.

The grayscale histogram of the sea–sky background image is shown in Figure 6. Due to the abrupt, cliff-like change in gray values at the junction between the sky and sea surface regions, the histogram shows that the gray values are concentrated in two distinct regions. These two areas represent the pixel counts for the sea surface and sky portions, respectively. Furthermore, between these two regions, there is a specific area where the number of pixels accounts for less than 1% of the total. This special area is also present in all 2967 images tested.

3.2. Dynamic Characteristic Analysis of the Sea–Sky Line

During missions, USVs experience six-degree-of-freedom motions: surge, sway, heave, roll, pitch, and yaw. These motions shift the position of the sea–sky line in images captured by the visible light sensor. To study how six-degree-of-freedom motions affect the sea–sky line, we established the imaging relationship of the visible light sensor under different USV states. Figure 7 shows the imaging geometry of the electro-optical equipment on the USV in its initial state.

In the initial state of the USV, the central axis of the visible light sensor is parallel to the sea surface. Under this condition, the intersection line between the electro-optical image plane and the plane containing the central axis of the visible light sensor lies along the horizontal midline of the image, denoted as segment $EF$. The visible light sensor is located at point $P$, which is at a height $h$ above the sea surface, and its vertical projection onto the sea surface is at point $O$. The vertical field of view of the visible light sensor is $2\beta$. The sea–sky line in the image is segment $SL$, and the angle between plane $PSL$ and the horizontal plane is $\alpha$. The radius of the Earth is $R$, and its center is point $G$. The angle between plane $PSL$ and plane $PMN$ is $\epsilon$, which represents the terrestrial refraction. Due to the Earth’s curvature, the sea surface is actually curved. A plane $CDMN$ is tangent to this curved surface at points $M$ and $N$. The distance from point $P$ to segment $MN$ is $d$, which is segment $PT$. To derive the sea–sky line equation, the following deduction is performed based on the subsequent information.

In triangle $PGT$, the following parameters are known: $\angle GPT={\displaystyle \frac{\pi }{2}}-\epsilon -\alpha$, $\angle GTP={\displaystyle \frac{\pi }{2}}-\epsilon$, $PG=R+h$, $TG=R$. Based on geometric relations, the following can be derived:

$${\displaystyle \frac{R+h}{\sin (\frac{\pi }{2}-\epsilon )}}={\displaystyle \frac{R}{\sin (\frac{\pi }{2}-\epsilon -\alpha )}}$$

(1)

$${\displaystyle \frac{R+h}{\cos \epsilon }}={\displaystyle \frac{R}{\cos (\epsilon +\alpha )}}$$

(2)

Expanding the term $\cos (\epsilon +\alpha )$ in the above equation yields:

$$\cos \epsilon \cos \alpha -\sin \epsilon \sin \alpha ={\displaystyle \frac{R}{R+h}}\cos \epsilon$$

(3)

$$\cos \alpha -\tan \epsilon \sin \alpha ={\displaystyle \frac{R}{R+h}}$$

(4)

Transforming the terms $\sin \alpha$ and $\cos \alpha$ in the above equation yields:

$${\displaystyle \frac{1-{\tan }^2(\frac{\alpha }{2})}{1+{\tan }^2(\frac{\alpha }{2})}}-{\displaystyle \frac{2\tan \epsilon \tan (\frac{\alpha }{2})}{1+{\tan }^2(\frac{\alpha }{2})}}={\displaystyle \frac{R}{R+h}}$$

(5)

Solving for $\tan ({\displaystyle \frac{\alpha }{2}})$ in the above equation yields:

$$\tan ({\displaystyle \frac{\alpha }{2}})={\displaystyle \frac{h+R}{h+2R}}(\sqrt{{\tan }^2\epsilon +{\displaystyle \frac{h^2+2hR}{{(h+R)}^2}}}-\tan \epsilon )$$

(6)

$$\alpha =2\arctan ({\displaystyle \frac{\sqrt{{\tan }^2\epsilon {(h+R)}^2+h^2+2hR}-(h+R)\tan \epsilon }{h+2R}})$$

(7)

Through the calculation above, the angle between plane $PSL$ and the horizontal plane is determined as $\alpha$. In the imaging plane of the visible light sensor, the image height is denoted as $w$, and the image width as $l$. The equation of the sea–sky line in the imaging plane can be expressed as follows:

$$y={\displaystyle \frac{w(\tan \beta -\tan \alpha )}{2\tan \beta }}$$

(8)

After obtaining the Equation of the sea–sky line in the imaging plane under the initial state, we proceed to investigate the influence of six-degree-of-freedom motion on the position of the sea–sky line in the image. If the electro-optical device rotates counterclockwise, the pitch angle is defined as positive; if it rotates clockwise, the pitch angle is defined as negative. As shown in Figure 8, this is a schematic diagram of the electro-optical imaging relationship when the device undergoes only pitch motion from its initial state.

In this schematic, the rotation angle of the electro-optical device is $\theta$, which causes a change in the position of the sea–sky line $SL$, resulting in a new sea–sky line $S^{\prime }{L}^{\prime }$. Analysis shows that the angle between plane $PEF$ and plane ${PE}^{\prime }{F}^{\prime }$ corresponds to the pitch angle $\theta$, while the angle between plane ${PS}^{\prime }{L}^{\prime }$ and the horizontal plane is denoted as $\alpha$. Therefore, after the pitch motion of the electro-optical equipment, the equation of the sea–sky line in the imaging plane can be expressed as follows.

$$y={\displaystyle \frac{w[\tan \beta -\tan (\alpha +\theta )]}{2\tan \beta }}$$

(9)

In the initial state, when the electro-optical equipment experiences only heave motion, the heave displacement is defined as positive for upward movement and negative for downward movement. Figure 9 illustrates the relationship of the electro-optical imaging during heave motion. In this schematic, the heave displacement of the electro-optical equipment is denoted as $h^{\prime }$, which causes the sea–sky line $SL$ to shift to a new position, $S^{\prime }{L}^{\prime }$. The angle between the plane ${PS}^{\prime }{L}^{\prime }$ and the horizontal plane is represented as ${\alpha }^{\prime }$. The new position of the visible light sensor is $P^{\prime }$, located at a height of $h+h^{\prime }$ above the sea surface, and the terrestrial refraction is denoted as ${\epsilon }^{\prime }$. Since the ratio of $h^{\prime }$ to $R+h$ is negligible, the effect of $h^{\prime }$ on terrestrial refraction can be ignored. Therefore, the terrestrial refraction $\epsilon$ remains unchanged. After the heave motion, the angle ${\alpha }^{\prime }$ between the plane ${PS}^{\prime }{L}^{\prime }$ and the horizontal plane can be expressed as follows.

$${\alpha }^{\prime }=2\arctan ({\displaystyle \frac{\sqrt{{\tan }^2\epsilon {(h+h^{\prime }+R)}^2+{(h+h^{\prime })}^2+2(h+h^{\prime })R}-(h+h^{\prime }+R)\tan \epsilon }{h+h^{\prime }+2R}})$$

(10)

The equation of the sea–sky line in the imaging plane can be expressed as follows:

$$y={\displaystyle \frac{w(\tan \beta -{\tan \alpha }^{\prime })}{2\tan \beta }}$$

(11)

4. Sea–Sky Line Detection Algorithm Based on Radar–Electro-Optical System

This algorithm is designed based on the operational mechanism of radar–electro-optical systems, offering the advantages of high precision and efficiency. In sea–sky background images, the sea–sky line represents the boundary between the sky and the sea surface. Significant brightness differences exist between the sky and sea regions in grayscale images, resulting in substantial image gradients around the sea–sky line. Building upon this gradient characteristic, the first step involves performing edge detection [32,33,34,35] on the sea–sky background image to extract edge information. Due to the strong image gradient in the sea–sky line region, its edge information can be effectively preserved.

The Canny edge detector [36,37] is a multi-stage edge detection algorithm that defines three fundamental criteria for edge detection: the signal-to-noise ratio criterion, the localization accuracy criterion, and the single-edge response criterion. The detailed definitions of these three criteria are as follows.

Signal-to-Noise Ratio Criterion: The edge detection algorithm must accurately identify as many edges as possible in the image while minimizing false detections and missed detections. To achieve optimal edge detection performance, the algorithm needs to minimize both the false alarm rate and the missed detection rate. Let the impulse response of the filter be $k(x)$, and the edge to be detected be $e(x)$. Assuming the edge point is located at $M[i,j]$, the response of the filter at this point can be expressed by the following formula:

$$H_G(x)={\displaystyle {\int }_{-w}^{+w}e(-x)k(x)dx}$$

(12)

Assuming the impulse response is defined on the interval $[-w,+w]$ and the noise is denoted as $n(x)$, the square root of the noise response can be expressed as:

$$H_n(x)=n_0\sqrt{{\displaystyle {\int }_{-w}^{+w}{k}^2(x)dx}}$$

(13)

Let $n_0$ denote the mean square of the noise amplitude per unit length. The following formula can express the output signal-to-noise ratio of edge detection:

$$SNR={\displaystyle \frac{\left|{\displaystyle {\int }_{-w}^{+w}e(-x)k(x)dx}\right|}{n_0\sqrt{{\displaystyle {\int }_{-w}^{+w}{k}^2(x)dx}}}}$$

(14)

Localization Accuracy Criterion: The detected edge points should be as close as possible to the actual edge positions, or the deviation of the detected edges from the true edges of the object caused by noise should be minimized. It is expressed explicitly as follows:

$$Localization={\displaystyle \frac{\left|{\displaystyle {\int }_{-w}^{+w}{e}^{\prime }(-x)k^{\prime }(x)dx}\right|}{n_0\sqrt{{\displaystyle {\int }_{-w}^{+w}{k}^{\prime 2}(x)dx}}}}$$

(15)

$e^{\prime }(-x)$ and $k^{\prime }(x)$ represent the first derivatives of $e(-x)$ and $k(x)$, respectively. The closer the detected edge is to the true edge position, the higher the value of $Localization$.

Single-Edge Response Criterion: The edge points detected by the operator should have a one-to-one correspondence with the actual edge points in the image. The detected edges should be only one pixel wide, which can eliminate the occurrence of false edges and facilitate subsequent edge feature extraction and image analysis.

In the specific process of using the Canny algorithm for edge detection on the grayscale image $f(x,y)$ of a sea–sky background, a Gaussian filter is first convolved with the input image to reduce the influence of noise on the image gradient. Since the image gradient near noisy pixels is often large and can be easily mistaken for edge information, Gaussian filtering is applied to suppress noise. Here, $\sigma$ represents the standard deviation. The specific procedure is as follows:

$$g(x,y)={\displaystyle \frac{1}{2\pi {\sigma }^2}}{e}^{-{\displaystyle \frac{x^2+y^2}{2{\sigma }^2}}}$$

(16)

Through the calculations above, the smoothed image $g(x,y)$ can be obtained. Subsequently, the Sobel operator is used to compute the first-order derivatives in the horizontal and vertical directions, i.e., the image gradients $G_x$ and $G_y$. The gradient magnitude and direction of the image can be expressed as follows:

$$G=\sqrt{G_x^2+G_y^2}$$

(17)

$$\theta =\arctan ({\displaystyle \frac{G_x}{G_y}})$$

(18)

After obtaining the gradient magnitude and direction of the image, non-maximum suppression is performed. This process involves traversing each pixel in the gradient image and calculating the gradient magnitudes of the two adjacent pixels along the gradient direction. If the gradient of the current pixel is greater than or equal to the gradient values of the neighboring pixels, the current pixel is confirmed as an edge point; otherwise, it is considered a non-edge point. In this way, the image edges are refined to a width of one pixel, resulting in the image $h(x,y)$.

Although the image after non-maximum suppression still contains noise, the Canny algorithm further processes it using the double threshold detection method. This involves setting an upper threshold $T_{\max }$, and a lower threshold $T_{\min }$. Based on these two thresholds, pixels in the image are classified as follows: if a pixel’s value is greater than the upper threshold $T_{\max }$, it is identified as a strong edge point; if it is less than the lower threshold $T_{\min }$, it is considered a non-edge point; values between the two thresholds are regarded as weak edge points. Weak edge pixels caused by genuine edges will connect to strong edge pixels, while responses caused by noise will not. To track edge connectivity, each weak edge pixel and its eight neighboring pixels are examined. If any of these pixels is a strong edge pixel, the weak edge pixel is retained as a genuine edge. Figure 10 shows the edge detection results obtained using the Canny edge detection algorithm.

After obtaining the edge information of the sea–sky background image, the results of edge detection need to be analyzed and processed. Since the edge of the sea–sky line appears as a horizontal segment in the image, it can be inferred that segments containing more edge pixels than a predefined threshold in the edge image are likely candidate lines for the sea–sky line. However, selecting these candidate lines requires significant computational effort in the current coordinate system. To address this, the Hough transform method can be employed to extract candidate sea–sky lines more efficiently.

The Hough transform is a key technique in digital image processing for line detection. It converts the image coordinate space into a parameter space by leveraging the duality between points and lines. Specifically, a line in the original image, defined by its Equation, corresponds to a point in the parameter space. This transformation converts the problem of detecting lines in the original image into identifying peak values in the parameter space, thereby shifting the focus from global feature detection to local feature analysis.

The core principle of the Hough transform lies in the duality of points and lines: a point in the original coordinate system corresponds to a line in the parameter space, and vice versa. In the original coordinate system, all points lying on the same line share identical slope and intercept values, meaning they map to the same point in the parameter space. Thus, after projecting each point from the original space to the parameter space, the presence of clustered points in the parameter space indicates the existence of a corresponding line in the original image.

In the plane rectangular coordinate system, the Equation of a line passing through the point $(x,y)$ can be expressed as follows:

$$y=kx+b$$

(19)

As shown in Figure 11, $k$ represents the slope of the straight line, and $b$ denotes the intercept of the line. There are infinitely many lines passing through the point $(x,y)$, each corresponding to different values of $k$ and $b$. Here, $\rho$ represents the distance from the line to the origin $O$, and $\theta$ denotes the angle between the polar radius and the polar axi. The following relationship holds:

$$k=-{\displaystyle \frac{\cos \theta }{\sin \theta }}$$

(20)

$$b={\displaystyle \frac{\rho }{\sin \theta }}$$

(21)

Therefore, if we regard $x$ and $y$ as constants, and treat $\rho$ and $\theta$ as variables, the following expression can be obtained:

$$\rho =x\cos \theta +y\sin \theta$$

(22)

Two points exist at positions (x₁, y₁) and (x₂, y₂) within the image plane. The lines passing through point (x₁, y₁) and point (x₂, y₂) can be represented as follows:

$$\rho =x_1\cos \theta +y_1\sin \theta$$

(23)

$$\rho =x_2\cos \theta +y_2\sin \theta$$

(24)

As shown in Figure 12, the line passing through point $(x_1,y_1)$ and the line passing through point $(x_2,y_2)$ can be represented in polar coordinates as follows. Here, the horizontal coordinate is the polar angle, and the vertical coordinate is the polar radius, with a clockwise direction being negative and a counterclockwise direction being positive.

In polar coordinates, two curves intersect at a single point. This intersection point represents the line in rectangular coordinates that passes through both points $(x_1,y_1)$ and $(x_2,y_2)$. Conversely, in rectangular coordinates, any two points on a line correspond to two curves in polar coordinates that share a single intersection point. Furthermore, $n$ distinct points on a single line in the Cartesian coordinate system correspond to $n$ curves intersecting at a single point in the polar coordinate system. Leveraging this property, the Hough transform can be used to determine the Equation of the line connecting these points. Therefore, edge information is first extracted from visible light images using the Canny edge detection algorithm. Then the position of the sea–sky line in the edge image is determined via the method above.

Figure 13 illustrates the edge detection results for a sea surface image, demonstrating the edge detection outcomes obtained using the Canny edge detection algorithm when a USV tracks a ship as its target.

Analysis of the edge detection results reveals that edge information primarily consists of sea–sky boundary edges, target edges, and edges caused by sea clutter. Applying the Hough transform to the edge information in the image yields corresponding curves in polar coordinates. Each pixel in the edge-detected image corresponds to a curve in the polar coordinate system. In this system, the more curves a point traverses, the higher its brightness. Conversely, fewer traversed curves result in lower brightness. Thus, the brightness of a point in the polar coordinate system reflects the number of edge information points on the corresponding line in the Cartesian coordinate system. Analysis of the edge detection image reveals that the horizon line’s edge information traverses the entire image, making it the longest horizontal segment. Consequently, the brightest point in the polar coordinate system corresponds to the horizon line in the sea surface image. Figure 14 shows the schematic diagram in polar coordinates and the horizon detection results, where the green line represents the horizon detected by the Hough transform-based horizon detection algorithm.

However, beyond the edge information of the sea–sky line, other edge details, such as target contours, also exist. These details interfere with the final sea–sky line detection, resulting in false positives and a reduction in accuracy. Among these, target edge information exerts the most significant influence. Given the guidance principle of radar-optoelectronic systems, sea surface targets typically reside in the central region of the image. Therefore, this paper proposes a sea–sky line detection algorithm tailored for radar-optoelectronic systems. This algorithm first processes the region near the target to reduce the gradient differences between pixels. Edge detection is then performed on the processed image. By diminishing gradient differences near the target, the algorithm effectively filters out target edge information, thereby enhancing the accuracy of sea–sky line detection. Figure 15 illustrates the processing results for the target region and the corresponding edge detection results obtained using the proposed algorithm.

After processing the area near the target, significant gradient differences emerge at the boundaries, resulting in two perpendicular edge signals in the edge detection image. During task execution, the sea–sky line in visible light images is not perpendicular to the horizon. Therefore, during sea–sky line selection, these perpendicular edge signals can be filtered out by imposing angular constraints. Figure 16 illustrates the edge detection information in polar coordinates obtained using the radar-optoelectronic system-based sea–sky line detection algorithm, along with the final result. The red line represents the sea–sky line obtained through the improved algorithm. Due to the suppression of target edge information, the number of curves in the figure is significantly reduced compared to Figure 14. This improved algorithm not only enhances the robustness and accuracy of sea–sky line detection but also improves computational efficiency, ensuring the real-time requirements of the perception system. The algorithm is not merely applied at the image level but is a perception method specifically designed for electro-optical radar systems. This method can operate without relying on image integrity, functioning even when images are occluded or blurred.

As shown in Figure 17, the horizontal field of view of the visible light sensor is $\beta$, and the field of view occupied by the target is $\gamma$. The actual image width of the plane where the target is located in the visible light image is $l$, and the pixel width of the visible light image is $l_{image}$. The resolution of the photoelectric visible light sensor is 1920 × 1080, and the field of view is 1.97 to 40 degrees. The resolution of the infrared sensor is 640 × 512, and the field of view is 2 and 10 degrees.

Additionally, the actual height of the target in the visible light image is $b$, while the distance $d_{radar}$ is obtained via marine radar. The target’s pixel width is $b_{image}$. The width of the sea–sky line suppression region in the visible light image is $k$, and the pixel width of this region is $k_{image}$. The target occupies a field of view size of $\mu$, with a sea–sky line suppression coefficient of ${\lambda }_b$. The calculation process for the pixel width $k_{image}$ of the sea–sky line suppression region is as follows.

$$b_{image}={\displaystyle \frac{b\cdot l_{image}}{2d_{radar}\tan (\frac{\beta }{2})}}$$

(25)

$${\lambda }_b={\displaystyle \frac{k_{image}}{b_{image}}}={\displaystyle \frac{\tan (\frac{\mu }{2})}{\tan (\frac{\gamma }{2})}}$$

(26)

$$k_{image}={\displaystyle \frac{{\lambda }_{b}b\cdot l_{image}}{2d_{radar}\tan (\frac{\beta }{2})}}$$

(27)

This paper compares and analyzes the detection accuracy of the radar-optoelectronic system’s sea–sky line detection algorithm with that of the Hough transform-based sea–sky line detection algorithm in various maritime test results. The specific calculation method for sea–sky line detection accuracy is as follows.

$$P=1-{\displaystyle \frac{1}{1+{\left\{\frac{1}{c_{p}n}{\displaystyle \sum _{i=1}^n{\left[f(x_i)-g(x_i)\right]}^2}\right\}}^{-\frac{1}{2}}}}$$

(28)

Here, $n$ represents the number of pixels in the horizontal direction of the image, $c_p$ denotes the sea–sky line detection accuracy coefficient, $f(x_i)$ indicates the accurate vertical coordinate of the sea–sky line in the image, and $g(x_i)$ represents the vertical coordinate of the fitted sea–sky line in the image. The sea–sky line detection accuracy ranges from $(0,1]$. A higher $P$ value indicates greater detection accuracy, while a lower $P$ value signifies greater detection deviation. The sea-line detection is only used for the first frame image. Once the position of the sea-line in the first frame image is determined, the position of the sea-line in subsequent image sequences is solved using the sea-line Equation. As shown in

Table 1. Comparison of sea–sky line detection accuracy.

	Test 1	Test 2	Test 3
Detection Accuracy Based on Hough Transform	0.791931	0.883545	0.816548
Detection Accuracy Based on Radar–Electro-Optical System	0.955562	0.986458	0.975463
Improvement Rate of Detection Accuracy	20.66%	11.65%	19.46%

, sea trials are conducted in three distinct sea areas, with each set of test results analyzed accordingly. The algorithm achieves satisfactory performance across all test scenarios. However, its effectiveness is inherently constrained by its dependency on the first-frame image characteristics provided by the radar–electro-optical system.

Comparative testing revealed that, due to extensive filtering of target edge information, the radar-optical system-based sea–sky line detection algorithm achieved higher detection accuracy while reducing computational load compared to the Hough transform-based approach. In the first test group, the improvement rate in sea–sky line detection accuracy is 20.66%; in the second test group, it is 11.65%; and in the third test group, it is 19.46%. Combining the results from all three test groups, the overall improvement in sea–sky line detection accuracy reached 17.3%. The algorithm achieves satisfactory performance across all test scenarios. However, its effectiveness is inherently constrained by its dependency on the first-frame image characteristics based on the radar–electro-optical system.

5. Target Detection Algorithm Based on the Sea–Sky Line

The sea–sky line-based target detection algorithm is a method that performs target detection by leveraging the position of the sea–sky line. Compared to conventional target detection algorithms, this approach removes interfering information from non-target regions, effectively improving both the accuracy and computational efficiency of target detection. The sea–sky line detection algorithm based on the radar–electro-optical system can accurately determine the position of the sea–sky line within visible light images. In USVs, the electro-optical equipment and the inertial navigation system adopt an integrated design, thereby reducing errors caused by hull deformation during navigation. Based on the principle of how six-degree-of-freedom motion alters the position of the sea–sky line in the image, the current sea–sky line position can be rapidly acquired. According to the guidance mechanism of the radar–electro-optical system, the designated target in the visible light image is located within the sea–sky line region. Consequently, the sea–sky line-based target detection algorithm can swiftly locate sea surface targets by referencing the sea–sky line’s position. By narrowing down the target’s ROI, it reduces interference from sea clutter and noise in the background, thus decreasing the computational load during target extraction and enhancing the efficiency and precision of sea surface target extraction. Figure 18 displays an optical image acquired by a USV during an actual mission execution, clearly showing the positional relationship between the sea–sky line and the designated target within the optical image.

A comparative analysis of optical images reveals that, owing to the perception mechanism where the marine radar cues the electro-optical equipment on the USV, the guide target within the acquired optical images is typically located in the vicinity of the sea–sky line. To investigate the relationship between the size of the sea–sky line region and the target size, the following research is conducted. Figure 19 illustrates a schematic diagram of the USV’s vertical field of view.

The field of view of the electro-optical visible light sensor is defined as $\theta$, and the angular subtense occupied by the target within the field of view is $\alpha$. The actual image height of the target in the visible light image plane is $w$, while the pixel height of the entire visible light image is $w_{image}$. Furthermore, the actual physical height of the target is denoted as $h$, and the distance to the target, $d_{radar}$, is acquired via marine radar. The pixel height $h_{image}$ of the target can then be calculated through the following formula.

$${\displaystyle \frac{w}{2}}=d_{radar}\cdot \tan ({\displaystyle \frac{\theta }{2}})$$

(29)

$${\displaystyle \frac{h}{w}}={\displaystyle \frac{h_{image}}{w_{image}}}$$

(30)

$$h_{image}={\displaystyle \frac{h{w}_{image}}{2d_{radar}\tan (\frac{\theta }{2})}}$$

(31)

The product of the calculated target pixel height $h_{image}$ and the sea–sky line region coefficient ${\lambda }_{ssl}$ determines the range of the sea–sky line-based target detection region. Sea conditions and navigation state influence the sea–sky line region coefficient ${\lambda }_{ssl}$. Under harsh sea conditions and significant navigation turbulence, the value of ${\lambda }_{ssl}$ is larger; conversely, under calm sea conditions and stable navigation, the value of ${\lambda }_{ssl}$ is smaller. The introduction of ${\lambda }_{ssl}$ aims to strike a balance between algorithmic robustness and computational efficiency. The detection region width $s$ can be expressed as follows:

$$s={\lambda }_{ssl}{h}_{image}$$

(32)

As shown in Figure 20, the flowchart illustrates the target detection algorithm based on the sea–sky line. The system first performs preliminary target designation via the marine radar, which then drives the electro-optical device to point toward the designated direction. After the process is initialized, sea–sky line detection and position calculation are executed on the first image frame. Subsequently, the system checks two key conditions in parallel: whether the update count has reached its limit, and whether the detection error exceeds a preset threshold. If either condition is met, the process returns to the initialization step and restarts the iteration. This mechanism embodies the algorithm’s adaptive reset capability, designed to prevent error accumulation or ineffective continuous operation. If neither condition is met, the process proceeds to the determination of the precise target area and finally executes target detection based on the sea–sky line. Based on the inherent advantages of the proposed method, it can support real-time processing even under conditions of limited hardware resources.

The primary targets for the USV’s sea trials are buoys and vessels. This study involved gathering data under various weather conditions, maritime states, and time periods, resulting in the creation of a dataset. Throughout testing, multiple data collection campaigns took place, and data screening is carried out based on the efficiency of the collection and the specific circumstances of the test environment. Figure 21 shows the detection results of the buoy-guided target situation for USVs. Figure 22 shows the detection results of the ship-guided target situation for USVs.

During the navigation of a USV, minimizing the area of non-target regions in visible light images not only improves the efficiency of image processing but also reduces the false detection rate in target identification. However, under conditions of high navigation speed and harsh maritime environments, an excessively small sea–sky line region coefficient ${\lambda }_{ssl}$ may cause the designated target to move outside the field of view, resulting in the inability to acquire the target’s center coordinates accurately. As shown in Figure 23, to investigate the relationship between the magnitude of the sea–sky line region coefficient and the target information integrity rate within the field of view, the following three sets of sea trials are conducted in this study.

As shown in Figure 24, the results of the target information integrity rate corresponding to different sea–sky line region coefficients from three sets of sea trials are presented. During the sea trials, the sea–sky line region coefficient ${\lambda }_{ssl}$ ranged from 1.1 to 1.9, with the aim of investigating the critical value of the sea–sky line region coefficient that achieves a 100% target information integrity rate under different maritime operational environments. Analysis of the test results revealed that the critical value of the sea–sky line region coefficient is smaller under favorable sea conditions and larger under harsh sea conditions. As the sea–sky line region coefficient ${\lambda }_{ssl}$ increases, the coverage of the sea–sky line region expands, consequently leading to a rising trend in the target information integrity rate.

In the first set of tests, when the sea–sky line region coefficient ${\lambda }_{ssl}$ is set to 1.3, the target information integrity rate reached 100%. In the second set of tests, when ${\lambda }_{ssl}$ is 1.4, the target information integrity rate also achieved 100%. In the third set of tests, with ${\lambda }_{ssl}$ at 1.6, the target information integrity rate remained at 100%. Under the premise of ensuring the target information integrity rate, the effective information ratio of the target should be improved. During these three sets of tests, this study investigated the relationship between the sea–sky line region coefficient ${\lambda }_{ssl}$ and the effective information ratio of the target. The larger the ratio of effective target information to global information in the image, the higher the detection efficiency and the lower the false detection rate; conversely, the smaller this ratio, the lower the detection efficiency and the higher the false detection rate. Figure 25 presents the effective information ratio results of the target for the three sets of tests.

Analysis of the test results indicates that, within the value range of the sea–sky line region coefficient ${\lambda }_{ssl}$, the target information effectiveness ratio of the proposed algorithm demonstrates significant improvement. However, as the sea–sky line region coefficient ${\lambda }_{ssl}$ increases, the target information effectiveness ratio gradually decreases. The first set of tests yielded a maximum target information effectiveness ratio of 8.28%, corresponding to an increase of 4.74%. The second set produced a maximum ratio of 7.56%, an increase of 5.51%. The third set achieved a maximum ratio of 10.57%, reflecting an increase of 6.77%. Enhancing the target information effectiveness ratio requires maintaining the target information integrity rate. Under this condition, the first set of tests achieved a maximum effectiveness ratio of 7%, representing an increase of 3.47%. The second set reached 5.94%, an increase of 3.89%. The third set attained 7.27%, an increase of 3.48%. Overall, the comprehensive results from the three test sets indicate an average improvement of 3.61% in the target information effectiveness ratio.

6. Conclusions

The static and dynamic characteristics of the sea–sky line are analyzed, and the impact of six-degree-of-freedom motion on its position in images is investigated. To overcome challenges of low detection accuracy and high computational costs in environments with significant interference from target contour information, a sea–sky line detection algorithm using a radar–electro-optical system is introduced. By employing the guidance mechanism of the radar–electro-optical system, the algorithm minimizes gradient differences between pixels in the target region. It suppresses edge information from interfering targets, thereby enhancing both detection accuracy and computational efficiency. The detection performance of the proposed algorithm is assessed through multiple sea trials.

To overcome the limitations of traditional target detection algorithms in fulfilling the robustness needs of marine perception systems, a target detection algorithm based on sea–sky line detection is proposed. This algorithm uses the position of the sea–sky line to quickly and precisely locate marine targets, which helps narrow down the ROI for detection. As a result, interference from sea clutter and background noise is reduced, leading to improved computational efficiency and detection accuracy. The effectiveness and superiority of the sea–sky line-based target detection algorithm are demonstrated through multiple maritime tests. Sea trials have shown that this method improves the effectiveness of target information by 3.61%. Subsequently, this method requires further refinement and integration with target tracking algorithms to achieve stable tracking of targets by USVs under adverse sea conditions.