Research on Environmental Perception Strategies and Sea Trial Methods for Unmanned Surface Vehicles in Typical Mission Scenarios

Abstract

This paper presents research on a complete, closed-loop environmental perception strategy at the system level and on standardized sea trial verification methods for unmanned surface vehicles (USVs) in typical maritime mission scenarios. Existing research on USV perception mostly focuses on optimizing discrete functional algorithms, such as object detection and tracking, with verification performed only in specific scenarios. These works generally lack task-matched perception schemes covering the full operation cycle and corresponding standardized, reproducible verification systems, making them difficult to adapt to the full-cycle execution requirements of real, complex maritime tasks. To address the above issues, this paper proposes a systematic environmental perception strategy for typical USV operation tasks, establishes a complete workflow from object detection to obstacle avoidance perception and decision-making, and designs two sets of standardized sea trial schemes for core tasks. These schemes provide unified specifications and an evaluation benchmark for verifying the real-world performance of USV environmental perception systems. Finally, full-cycle sea trials on a real ship are completed, and the test results verify the engineering practicability of the proposed perception strategy, as well as the reproducibility, standardization, and extensibility of the established test system, with the laser positioning hit rate improved by 82% and 54% under green-water and foggy conditions compared with the conventional miss distance-based method.

1. Introduction

As a type of intelligent marine equipment with autonomous navigation, intelligent decision-making, and operation capabilities, the USV [1,2,3] has been widely used in many maritime mission scenarios, including offshore patrol, maritime search and rescue, waterway inspection, and marine environment monitoring, with core advantages such as flexible maneuverability, convenient deployment, and no risk of personnel casualties. It has become a core piece of equipment for marine resource development and for protecting maritime rights and interests. As shown in Figure 1, the environmental perception system [4,5] is the core foundation for USVs to achieve fully autonomous operation. At the same time, surface object detection [6,7] and tracking technology [8,9] are the core modules of the environmental perception system. Their perception accuracy, response speed, and robustness under complex sea states directly determine the autonomous obstacle avoidance, trajectory tracking, and task execution capabilities of USVs, which represent the current research core and hotspot in the field of USV intellectualization. At present, domestic and foreign scholars have conducted extensive research on USV surface object detection and tracking technology and have proposed numerous deep learning-based optimization algorithms for multi-sensor fusion, achieving excellent performance on public maritime datasets and in digital simulation environments.

However, the real marine environment exhibits typical characteristics of strong nonlinearity and high uncertainty [10]. During actual operation, USVs are inevitably affected by complex working conditions, including disturbances from wind, waves, and currents; drastic changes in illumination; sea surface reflections and clutter interference; sharp changes in target scale; and the random occurrence of dynamic obstacles [11,12,13]. Effective disturbance rejection control is critical for marine vehicles to operate stably in complex sea conditions [14]. As a result, algorithms verified in simulation environments and on ideal datasets often suffer from sharp degradation in generalization performance and perception failures when deployed on real ships. There is a significant performance gap between simulation-verified results and actual performance under real sea conditions, which has become a bottleneck hindering the transition of USV intelligent perception algorithms from laboratory research to engineering implementation.

In response to the above-mentioned difficulties in real-ship verification, existing studies have gradually conducted related work on USV sea trials. Nevertheless, two core deficiencies exist widely in current sea trials [15,16,17]. First, most existing trials are functional demonstration tests for algorithms, which only involve the scattered verification of perception functions under single working conditions. They lack a systematic, end-to-end trial design tailored to typical USV operational tasks and thus cannot comprehensively verify the adaptability of perception algorithms in real mission scenarios. Second, existing trial schemes lack unified design specifications and implementation standards. The trial scenarios, test procedures, and evaluation dimensions across different studies vary greatly, making it impossible to conduct a scientific cross-algorithm comparison of real ship performance. Thus, there is no reusable, extensible verification basis for the engineering implementation of USV perception algorithms. Especially for the core USV operation tasks of autonomous obstacle avoidance and trajectory tracking, there is still no standardized sea trial system that tightly links the performance of perception algorithms with task execution capabilities, making it impossible to verify perception algorithms under full operational conditions and processes systematically.

To address the above industry problems and research gaps, this study uses USV object detection and tracking algorithms tailored to typical mission scenarios as verification targets. It researches sea trial methods for USV environmental perception strategies. The core work described in this paper is as follows. First, a full-function USV sea trial platform integrating unmanned autonomous systems and shore-based interactive systems is built, with the platform’s hardware architecture and software logic clarified, providing reliable software and hardware support for the full-process implementation of sea tests. Second, based on the core operational requirements of USVs, systematic sea trial schemes covering two typical mission scenarios are designed: an adaptability and obstacle avoidance capability test and a typical trajectory tracking and obstacle avoidance capability test. The core trial contents, the scheme design logic, and the full-process implementation specifications for the two test types are defined. Finally, based on the self-developed USV surface object detection and tracking algorithm, full-process sea tests are conducted under two typical mission scenarios. The test procedure of the perception algorithm in the standardized trial system is fully presented, and the feasibility and engineering practicability of the proposed trial method are verified. The research described in this paper can provide a standardized, reproducible technical framework for the verification of real ship performance via USV intelligent perception algorithms and offer trial support for the engineering implementation of USV intelligent technologies.

2. USV Sea Trial Platform

To realize the full-process, standardized sea trial verification of the USV environmental perception strategy for typical maritime mission scenarios, this paper constructs a USV sea trial platform with fully autonomous navigation, multi-source environmental perception, and shore-based low-latency interaction capabilities. This platform serves as the core software and hardware platform for the design of all sea trial schemes and the implementation of the test procedures described in this paper. It can fully support the full-process standardized tests for the two core mission scenarios, namely the adaptability and obstacle avoidance capability test and the typical trajectory tracking and obstacle avoidance capability test. The platform is divided into two core modules: the unmanned autonomous system and the shore-based interactive system.

2.1. Unmanned Autonomous System

The unmanned autonomous system is deployed on the USV’s onboard end, serving as the execution core for the sea trials. It undertakes four core functions throughout the entire test process—the determination of the platform pose reference, the real-time acquisition of environmental information, the execution of navigation control instructions, and the two-way transmission of ship–shore data—providing onboard software and hardware support for the stable and repeatable implementation of the trials. This system adopts an integrated navigation scheme combining the inertial navigation system and the BeiDou navigation satellite system as the core of the spatiotemporal reference for the entire test platform. It can provide accurate core-state information for the hull, including the longitude and latitude, heading, speed over ground, and real-time attitude angles, throughout the entire sea trial process. Meanwhile, it provides a unified ground truth reference for the spatial coordinate calculation of perceived targets, the benchmarking of trajectory-tracking accuracy, and the calibration of the execution effectiveness of obstacle avoidance maneuvers, which represent the core foundation for ensuring the validity, comparability, and reproducibility of the test data. Meanwhile, the motion control unit in the system serves as the execution core for the test tasks. It undertakes core functions during the test process, including the navigation mode control of the USV, the execution of preset trajectory tracking, and response to autonomous obstacle avoidance maneuvers. It can accurately reproduce the navigation instructions preset in the test scheme, thus ensuring the controllability and repeatability of the test process. This unit uses a hybrid diesel–electric power system, is equipped with a PC104 core controller, and supports four working modes: manual, remote control, semi-autonomous, and fully autonomous. It can perform sea trials stably under a level-4 sea state to meet the control requirements for different test scenarios. At the same time, it is integrated with multi-beam sonar equipment, which enables the real-time detection of the seabed topography and geomorphology, ensures the platform’s navigation safety during testing, and provides underwater environmental safety support for trajectory-tracking tests in complex inshore scenarios. In Figure 2, the interrelationships among the core components of the unmanned system are illustrated.

The multi-source environmental perception unit of this system serves as the hardware carrier for the object detection and tracking perception strategy verified in this trial; meanwhile, it fulfils the function of the real-time acquisition of surface target information and marine environment information in the test scenario, providing input data sources under full working conditions and full scenarios for the perception algorithm. This unit is mainly composed of marine radar [18,19] and electro-optical equipment. Among them, the marine radar uses a patch-array antenna, which enables the wide-area detection of long-range surface targets and outputs core target information, including distance, azimuth, speed over ground, and heading. Its detection range spans 40 m to 18 km, with angular measurement accuracy of 0.2°, enabling the pre-detection of large-scale targets and situational awareness in complex sea states. The electro-optical equipment integrates a visible light sensor, a cooled infrared sensor, a laser ranging sensor, and a stabilized pan-tilt platform. The stabilized pan-tilt platform can implement multi-stage zoom and complete dynamic image stabilization in conjunction with real-time hull attitude data, ensuring imaging stability under severe sea states. The visible light and infrared sensors can achieve the high-definition imaging of targets in both day and night scenarios, and the laser ranging sensor has a ranging error of no more than 5 m, providing accurate distance information for target tracking. The meteorological sensing equipment can collect sea-state information, including the wind speed, wind direction, and wave height, in the test sea area in real time, providing environmental reference data for the standardized calibration of test conditions and analysis of test scenario adaptability.

The onboard computing and communication unit of this system performs the real-time execution of perception algorithms and two-way ship–shore data transmission during testing, serving as the core link for the online testing of perception strategies and real-time monitoring of shore-based trials. The onboard perception computer is equipped with an object detection and tracking module, a data acquisition module, and a decision control module. It can enable the real-time online operation of the self-developed object detection and tracking algorithm and simultaneously perform the real-time processing and encapsulation of perception data and hull-state data. The onboard wireless radio station has a waterproof rating of IP67 and a power consumption range of 8 W to 43 W; it supports channel bandwidths of 5 MHz, 10 MHz, and 20 MHz, with a data transmission rate of up to 100 Mbps, a transmission delay as low as 7 ms, and an operating frequency band covering 400 MHz to 6 GHz. It enables low-latency, high-reliability two-way data transmission between the onboard and shore-based ends, ensuring the real-time monitoring of the onboard status by the shore-based platform and the real-time issuance of test instructions during the trial. This USV platform adopts a modular design, enabling rapid load replacement to meet the requirements of test tasks. It can independently complete a variety of tasks, such as marine hydrometeorological monitoring and scanning of the seabed topography and geomorphology, with high environmental adaptability and task adaptability, fully meeting the sea trial requirements for the two typical mission scenarios considered in this paper.

2.2. Human–Computer Interaction System

The human–computer interaction system is deployed at the shore-based test management and control terminal, serving as the full-process management and control core of the sea trials. It undertakes the functions of state monitoring, process management and control, instruction issuance, and data visualization throughout the entire test process, enabling testers to achieve the full-process controllable management of the USV test state and ensuring the standardized implementation of the test scheme, as well as the safety and controllability of the test process.

This system is mainly divided into two core modules, the environmental perception interactive system and the integrated test monitoring and control system [20,21], which correspond to the monitoring of perception algorithm testing and the comprehensive management and control of the whole test process, respectively.

2.2.1. Environmental Perception Interactive System

The environmental perception interactive system is a monitoring module for the environmental perception strategy test described in this paper, composed of five functional sections: state display, data display, video monitoring, real-time electronic chart, and control interface. As shown in Figure 3, the state display section can present the operational states of onboard perception and motion execution equipment, as well as the platform’s current working mode in real time, ensuring the continuous monitoring of equipment operating conditions during the test. The data display section can synchronously present core perception data, including the tracking azimuth, tracking pitch, miss distance, and laser ranging, enabling the real-time monitoring of the perception algorithm’s operating state.

The video monitoring section can transmit back the visible light and infrared video data collected by the electro-optical equipment in real time and overlay and display the output results of the self-developed object detection and tracking algorithm on the frame, including core information such as the target category, longitude and latitude, distance, and color, to realize the full-time visual monitoring of the perception algorithm test process. The real-time electronic chart section can simultaneously display the USV’s real-time position on the electronic chart and the target coordinate information acquired by the system, enabling the visualization of the global spatial situation in the test scenario [22,23,24,25]. The control interface section can issue task instructions to the perception system, including core functions such as mode switching, servo control, laser control, zoom control, and image tracking. It can adjust the perception system’s operating parameters in real time to ensure the standardized execution of the test process. This interface can intuitively and centrally display the full-dimensional state information of the USV perception system and, at the same time, support testers in issuing perception control instructions conveniently, effectively improving the management and control efficiency of the test process.

2.2.2. Integrated Test Monitoring and Control System

The integrated test monitoring and control system is the comprehensive management and control core of the sea trials described in this paper. It provides operators with manipulation and state-monitoring functions throughout the test process. It can realize core functions, including setting the desired path, the real-time adjustment of the control parameters, the monitoring of navigation attitude data, and test task planning, ensuring the controllable execution of tests across the two typical mission scenarios throughout the process. As shown in Figure 4, this system consists of seven core functional sections.

The pose display interface presents core navigation state information for the USV, including the heading, speed over ground, longitude, and latitude, enabling the real-time monitoring of the execution results of the trajectory-tracking test. The communication interface is responsible for managing the communication link between the ship and the shore, including data transmission, network communication, and BeiDou navigation satellite system communication [26,27,28,29]. It can select the appropriate communication port and baud rate based on the test scenario, ensuring stable ship-to-shore data transmission. The remote control interface displays the remote control joystick’s real-time position. When the platform is in remote control mode, it can execute navigation commands in real time in response to joystick inputs, meeting emergency control requirements during testing. The IO control interface is responsible for the IO control of each onboard piece of equipment. It can realize the uplink and downlink transmission of data transmission communication, BDS communication, and PC104 instructions, as well as complete equipment control operations, such as engine ignition, clutch engagement, clutch disengagement, BDS power on/power off, and inertial navigation system power on/power off, ensuring the standardized start–stop management and control of the equipment before and after the test. The chart interface displays the desired trajectory and the USV’s actual navigation path in real time, based on the electronic chart, enabling testers to adjust navigation tasks and providing visual trajectory monitoring for typical trajectory-tracking tests. The control parameter interface is responsible for setting navigation control parameters such as the speed and heading. It can implement the navigation functions of the USV, including constant speed, constant heading, constant speed and heading, and trajectory tracking, which is the core control input for typical trajectory-tracking tests. The planning parameter interface is responsible for setting parameters such as navigation tasks and paths. It can complete the setting of tasks such as surface target recognition and tracking and meanwhile realize the automatic collision avoidance function by setting parameters including the collision avoidance distance, cruise speed, collision avoidance speed, expansion radius, and planning point radius, which represent the core parameter setting input of the adaptability and obstacle avoidance capability test [30,31].

3. Adaptability and Obstacle Avoidance Capability Test

Aiming at the typical mission scenarios of autonomous obstacle avoidance and wide-area cruise for USVs, this paper designs a set of standardized sea trial methods to enhance the adaptability and obstacle avoidance capabilities. It clarifies the core content, scenario design specifications, and full-process implementation logic of the trial. Meanwhile, based on the self-developed surface object detection and tracking perception strategy, a full-process test is conducted in this trial scenario. This verifies the operability of the trial method and the adaptability of the perception strategy to tasks.

3.1. Test Content

The core orientation of this adaptability and obstacle avoidance capability test is to establish a set of reproducible, standardized, implementable, real-ship test schemes for the USV environmental perception strategy. This scheme is used to verify the full-process execution capabilities, environmental adaptability, and long-term operation stability of the perception system in typical mission scenarios, including long-term cruise, dynamic target detection [32,33], and autonomous obstacle avoidance. Meanwhile, it provides a unified test benchmark for the horizontal comparison of real-ship performance among different USV perception strategies. As shown in Figure 5, the test sea area of this trial is selected as a rectangular offshore area with a length of 3000 m and a width of 2000 m.

This area has a stable hydrological environment and no significant navigation interference, meeting the test requirements for the long-term autonomous cruise of the USV. Before the test, four boundary buoys numbered B01, B02, B03, and B04 are deployed at the four vertex positions of the test area, and the longitude and latitude information of the four buoys is pre-calibrated as the boundary reference of the test area. Meanwhile, four obstacle buoys, numbered F01, F02, F03, and F04, are deployed within the test area, and their general positions are pre-calibrated as the core detection and obstacle avoidance targets for this trial. The position information accuracy of the above targets is 100 m, which is used as the virtual point reference during the test. In the scenario layout for this adaptability and obstacle avoidance capability test, the USV’s initial position is Point A. After the test starts, the USV will perform an autonomous cruise mission in the rectangular area in a counterclockwise direction and complete the detection, positioning, and obstacle avoidance detour of the four obstacle buoys in sequence.

The core test content of this trial consists of three key parts. First, it seeks to verify the long-distance target detection and screening capabilities of the USV perception system in the wide-area cruise scenario and test the closed-loop execution capabilities of the perception system for the full-process detection, identification, and pointing of the preset obstacle targets. Second, it seeks to verify the field-of-view adaptation and target stable tracking capabilities of the perception system during the dynamic navigation of the hull and test the environmental adaptability of the perception strategy under hull motion and sea state interference. Third, it seeks to verify the operational stability of the perception system in long-term navigation missions and to test the continuous execution capabilities of the target-tracking link of the perception strategy in continuous-cruise missions.

Standardized constraint specifications are established during the test process to ensure trial repeatability and process consistency. After completing equipment self-inspection and test preparation at the initial position, Point A, the USV receives the test start instruction and performs the autonomous cruise mission in the rectangular area in a counterclockwise direction. During navigation, the USV should not exceed the test area enclosed by the four boundary buoys. When performing the autonomous obstacle avoidance mission, the USV should stably navigate through the annular area with a target radius of 30 m to 60 m and complete the positioning and detour around the obstacle buoys. To ensure consistent test conditions across all trials, a minimum test duration of 2 h and a minimum sailing speed of 18 kn are specified, in accordance with performance metrics commonly used in major USV competitions and standard practices adopted by most leading maritime robotics contests.

3.2. Test Scheme and Implementation

A full-process, closed-loop, standardized implementation scheme is designed for this trial, clarifying the implementation logic and operational specifications from test initiation to task completion and fully covering the entire operational chain of the typical autonomous obstacle avoidance mission of the USV. As shown in Figure 6, the implementation of this adaptability and obstacle avoidance capability test is based on the full-link working process of the USV perception system, and the specific implementation steps are as follows.

During the test initiation phase, the USV conducts a comprehensive self-inspection of the onboard equipment, perception system, and ship-to-shore communication link at the initial position [34]. After confirming that all equipment is in normal working condition and the navigation and obstacle avoidance parameters have been configured, the shore-based interactive system issues a test start instruction. The USV then switches to the fully autonomous navigation mode and initiates an autonomous cruise along the preset route. In the wide-area environment detection and target screening phase, during the USV’s cruise, the marine radar continuously performs omnidirectional scanning and detection of the surrounding sea areas, collects target information in the test area in real time, and simultaneously records the detected targets in the radar target list to complete the pre-detection of targets within the wide area. To ensure the validity of the test targets, the trial includes standardized target screening rules, allowing only targets that meet the preset constraint conditions to enter the subsequent perception link. The constraint conditions are as follows: the target should be within the fan-shaped range of ±60° of the USV’s bow, the distance from the USV should be in the range of 150 m to 2000 m, and a preset virtual point reference exists within a 100 m radius of the target. If there are no eligible targets in the radar detection list, the perception system will traverse the electro-optical equipment target list and the virtual point list in sequence to complete supplementary target screening. If there are no valid targets in all lists, the electro-optical equipment will perform a zeroing operation, resetting the horizontal azimuth to zero to keep its optical axis aligned with the sea surface, and then wait for the next detection cycle.

In the target pointing and field-of-view adaptation test phase, after screening for valid targets, the perception system calculates the target’s relative azimuth. It guides the electro-optical equipment to complete target pointing, ensuring that the target enters the visible light sensor’s field of view. To adapt to the hull’s attitude changes during dynamic navigation and ensure the stability of target imaging, the trial includes a standardized test process for field-of-view adaptation [35]. After the electro-optical equipment completes target pointing, the perception system performs the field-of-view adaptation calculation. It adjusts the focal length and attitude of the electro-optical equipment so that the target appears in the visible-light field of view at a standardized size. As shown in Figure 7, the field-of-view adaptation calculation keeps the target at the center of the field of view by adjusting the pitch angle and focal length of the electro-optical equipment. It simultaneously adapts the target imaging size at different distances, ensuring the validity of subsequent target extraction.

After completing field-of-view adaptation, the perception system performs scene recognition on the first frame to complete the standardized determination of the test scenario, providing a scene-adaptive reference for subsequent target extraction.

The scene recognition in this trial primarily targets two typical interference scenarios: the mast occlusion scenario and the deck green water scenario. It completes scene determination using the bright-spot statistics of the image segmentation areas. As shown in Figure 8, during the scene recognition process, the number of bright spots in the mast occlusion segmentation area and the deck green water segmentation area is counted, respectively, and the scene determination ratio is calculated based on the statistical results.

$r_m$ denotes the ratio between the count of mast obscuration regions and the total number of partitioned image regions, $c_{mast}$ represents the final determination result for the mast obscuration scene, and $r_{mast}$ is the pre-defined determination threshold for mast obscuration judgment. When the ratio $r_m$ falls below the determination threshold $r_{mast}$, the determination result $c_{mast}$ is assigned a value of 0, which corresponds to a non-mast obscuration scene. In contrast, when the ratio $r_m$ is greater than or equal to the determination threshold $r_{mast}$, $c_{mast}$ is set to 1, indicating that the current scene is classified as a mast obscuration scene. The corresponding mathematical expressions are given as

$$r_m={\displaystyle \frac{n_{mast}}{n_{total}}}$$

(1)

$$c_{mast}=\left\{\begin{array}{l}0,r_m<r_{mast}\\ 1,r_m\ge r_{mast}\end{array}\right.$$

(2)

For the green water on deck judgment, $r_w$ is defined as the ratio of the number of green water-covered regions to the total number of partitioned image regions, $c_{water}$ stands for the determination result of the green water scene, and $r_{water}$ is the corresponding preset determination threshold. If the ratio $r_w$ is lower than the determination threshold $r_{water}$, the value of $c_{water}$ is set to 0, meaning that the current scene is identified as a non-green water scene. In contrast, if the ratio $r_w$ is greater than or equal to the determination threshold $r_{water}$, $c_{water}$ is assigned a value of 1, which indicates that the current scene is classified as a green water on deck scene. The specific expressions are given as

$$r_w={\displaystyle \frac{n_{water}}{n_{total}}}$$

(3)

$$c_{water}=\left\{\begin{array}{l}0,r_w<r_{water}\\ 1,r_w\ge r_{water}\end{array}\right.$$

(4)

In the above scene recognition process, the proportional result $r_m$ obtained from mast occlusion scenario recognition is equal to 0, and the proportional result $r_w$ obtained from deck green water scenario recognition is equal to 0.67. In this trial, the decision threshold $r_{mast}$ of the mast occlusion scenario recognition algorithm is set to 0.25, and the decision threshold $r_{water}$ of the deck green water scenario recognition algorithm is set to 0.77. Based on the above thresholds, it can be determined that the first frame image does not belong to the mast occlusion or deck green water scenario and can proceed to the subsequent sea–sky line detection process. After completing scene recognition and determination, the perception system determines the position of the sea–sky line in the image using a radar–electro-optical sea–sky line detection algorithm. As shown in Figure 9, the sea–sky line detection algorithm performs region segmentation on the image, extracts the segmentation boundary between the sea surface and the sky, and completes the accurate calculation of the sea–sky line position in combination with the hull attitude data.

After obtaining the position of the sea–sky line in the current image, the six-degrees-of-freedom variation of the USV relative to the initial state at the current moment can be obtained through the inertial navigation system, allowing one to calculate and acquire the sea–sky line position in the image at the current moment.

After completing sea–sky line detection in the initial state, the angular coordinates $(x_{corner}^{t_0},y_{corner}^{t_0})$ of the guidance target area in the image, as well as the corresponding area width $b_{image}^{t_0}$ and area height $h_{image}^{t_0}$, are obtained through the guidance target extraction algorithm on the first frame image. The specific expressions are given as

$$x_{center}^{t_0}=x_{corner}^{t_0}+{\displaystyle \frac{b_{image}^{t_0}}{2}}$$

(5)

$$y_{center}^{t_0}=y_{corner}^{t_0}+{\displaystyle \frac{h_{image}^{t_0}}{2}}$$

(6)

After obtaining the initial coordinates $(x_{center}^{t_0},y_{center}^{t_0})$ for region of interest (ROI) prediction, the center coordinates of the predicted target region at the current time instant $(x_{center}^t,y_{center}^t)$, together with the corresponding width $b_{image}^t$ and height $h_{image}^t$ of the predicted target region, can be calculated via the ROI prediction algorithm. The detailed calculation procedure for the corner coordinates of the predicted ROI in the image $(x_{corner}^t,y_{corner}^t)$, as well as the matching width $b_{ROI}^t$ and height $h_{ROI}^t$ of the predicted ROI, is presented as follows:

$$b_{ROI}^t={\lambda }_{ROI}{b}_{image}^{t-1}$$

(7)

$$h_{ROI}^t={\lambda }_{ROI}{h}_{image}^{t-1}$$

(8)

$$\Delta x^t=x_{center}^t-{\displaystyle \frac{b_{ROI}^t}{2}}$$

(9)

$$\Delta y^t=y_{center}^t-{\displaystyle \frac{h_{ROI}^t}{2}}$$

(10)

$$\left\{\begin{array}{ll}{x}_{corner}^t=x_{center}^t-{\displaystyle \frac{b_{ROI}^t}{2}}& \Delta x^t\ge 0\hfill \\ x_{corner}^t=0& \Delta x^t<0\hfill \end{array}\right.$$

(11)

$$\left\{\begin{array}{ll}{y}_{corner}^t=y_{center}^t-{\displaystyle \frac{h_{ROI}^t}{2}}& \Delta y^t\ge 0\hfill \\ y_{corner}^t=0& \Delta y^t<0\hfill \end{array}\right.$$

(12)

The target in the image is tracked using a region of interest-based target tracking algorithm, and the miss distance information is obtained. As shown in Figure 10, after the target is extracted from the first frame, the tracking process continuously locks the target position, ensuring the continuity and stability of target tracking.

In the target extraction and stable tracking test phase, after completing field-of-view adaptation and scene recognition, the perception system performs real-time target extraction from the visible-light image and assesses its stability. If the target extraction result is unstable, the perception system will re-execute the target pointing and field-of-view adaptation processes. If the target extraction is stable, the perception system will activate the target-tracking algorithm to track the target continuously. During the tracking process, if the target fails to meet the tracking constraint conditions, or if target loss or other conditions occur, the perception system will restart the target-screening process to complete the selection and tracking of a new target, ensuring the continuous closed loop of the perception link.

In the target positioning and task execution phase, when the system determines that the target tracking state is stable, it will activate the laser ranging sensor and emit a laser to the target to complete accurate positioning. After target tracking stabilizes, the laser ranging sensor begins emitting a laser at the target. Once the laser hits the target, the distance to the USV can be determined, and the target’s longitude and latitude can be calculated. As shown in Figure 11, during the stable tracking of the obstacle buoy, the USV continuously emits a laser to obtain multiple positions of the obstacle buoy. Due to the inherent ranging error of the laser ranging sensor, to improve the positioning accuracy of the obstacle buoy, the center point of the minimum circular area formed by the target positions measured in each test is output as the final determined position of the obstacle buoy.

After the laser hits the target, the perception system checks whether a preset virtual point reference is within a 100 m radius of the target. If there is a valid virtual point reference, it will update the target information that has been positioned by the laser; if there is no valid virtual point reference, it will add the target to the electro-optical equipment target list and complete the standardized archiving of the target information. After obtaining the accurate position information of the obstacle target, the USV will complete the detour within the annular area with a target radius of 30 m to 60 m, in accordance with the preset obstacle avoidance rules, and sequentially complete the detection, tracking, positioning, and obstacle avoidance detour of the four obstacle buoys, forming the full-process closed loop of the test task. Throughout the test process, the shore-based interactive system monitors the USV’s navigation state, the perception system’s working state, and the execution of the target-tracking link in real time. It synchronously records full-process test data to ensure test process controllability and traceability.

4. Typical Trajectory Tracking and Obstacle Avoidance Capability Test

Aiming at the composite typical mission scenario of preset trajectory tracking and dynamic obstacle avoidance for USVs, this paper designs a set of standardized sea trial methods for trajectory tracking and obstacle avoidance capabilities. It clarifies the core test content, trajectory planning specifications, and full-process implementation logic of the trial. Meanwhile, based on the self-developed surface object detection and tracking perception strategy, a full-process test is conducted in this composite mission scenario. This verifies the operability of the proposed trial method and the environmental adaptability of the perception strategy in trajectory-tracking tasks.

4.1. Test Content

The core objective of the typical trajectory tracking and obstacle avoidance capability test is to establish a set of reproducible, standardized, and implementable real-ship test schemes for the composite mission of USV trajectory tracking and obstacle avoidance. This scheme is used to verify the full-process execution capabilities of USVs to synchronously complete dynamic obstacle target detection, stable tracking, accurate positioning, and autonomous avoidance during the continuous tracking of the preset trajectory; meanwhile, it provides a unified test benchmark for the horizontal comparison of the integrated performance of USV trajectory tracking and autonomous obstacle avoidance.

The test sea area for this trial is selected as a rectangular offshore area measuring 3000 m by 2000 m, to ensure uniform test working conditions and comparable test results. Before the test, four boundary buoys numbered B01, B02, B03, and B04 are deployed at the four vertex positions of the test area, and the longitude and latitude information of the four buoys is pre-calibrated as the boundary reference of the test area. Meanwhile, three obstacle buoys, numbered F01, F02, and F03, are deployed within the test area, serving as the core targets for detection and obstacle avoidance in this trial. Seven trajectory virtual points, namely T01, T02, T03, T04, T05, T06, and T07, are set in the test area as the key waypoints of the USV preset trajectory, and the USV is required to pass through the seven virtual points in sequence during the test. In addition, a virtual point T08 is set in the test area as the center reference for the USV circular motion tracking task. As shown in Figure 12, in the scenario layout for this typical trajectory tracking and obstacle avoidance capability test, the USV’s initial position is Point A. After the test starts, the USV will pass through the virtual points T01 to T07 in sequence along the preset trajectory; synchronously complete the detection and obstacle avoidance detour of the three obstacle buoys F01, F02, and F03; and finally, after reaching Point T07, complete a circular motion around the virtual point T08 with radius R, forming a complete closed loop for the test task.

Through this trial, the full-process completion capabilities of the USV when performing typical trajectory tracking and obstacle avoidance tasks using the perception algorithm can be comprehensively investigated, and the algorithm’s execution performance in the typical trajectory tracking and obstacle avoidance mission scenario can be verified. The core test content of this trial consists of three key parts. First, it seeks to verify the continuous and stable tracking capabilities of the USV for the preset complex trajectory and test the execution accuracy and stability of the USV when passing through all trajectory virtual points in sequence. Second, it seeks to verify the closed-loop execution capabilities of the USV for the synchronous detection, stable tracking, and autonomous avoidance of obstacle targets during trajectory tracking and to test the task adaptability of the perception system during dynamic navigation. Third, it seeks to verify the USV’s tracking capabilities for special trajectories, such as circular motion, and to test the target tracking stability of the perception system during course-changing navigation.

Standardized constraint specifications are established during the test process to ensure trial repeatability and process consistency. Throughout the entire test process, the USV must remain within the test area at all times and must not sail into the area within 30 m of the target radius; during autonomous obstacle avoidance, the USV must pass through the area within 60 m of the target radius. The completion of the final circular motion by the USV marks the end of the test. During the test, except during the obstacle avoidance process, the navigation speed of the USV should not be lower than 18 kn. In addition, the USV must record the longitude and latitude of the designated obstacles to ensure the traceability of the test data.

4.2. Test Scheme and Implementation

As shown in Figure 13, the implementation process for this typical trajectory tracking and obstacle avoidance capability test includes the synchronous closed-loop testing of both aspects, with the following steps.

A full-process, closed-loop, standardized implementation scheme integrating trajectory tracking and obstacle avoidance is designed for this trial, clarifying the implementation logic and operational specifications from test initiation to task completion and fully covering the full working link of the composite typical trajectory tracking mission of the USV. During the test initiation phase, the USV is located at the initial position A before the test starts, and it completes a comprehensive self-inspection of the onboard equipment, environmental perception system, trajectory planning module, and ship-to-shore communication link. After confirming that all equipment is in normal working order and that the preset trajectory and obstacle avoidance parameters have been configured, the tester issues an instruction via the wireless radio station to start the USV’s autonomous navigation. The USV then switches to trajectory-tracking mode and sails towards the first virtual point, T01, along the preset trajectory.

In the wide-area environment detection and target screening phase, the USV begins autonomous navigation immediately upon receiving the start instruction for the typical trajectory tracking and obstacle avoidance capability test. During this process, the marine radar continuously detects the surrounding environment, acquires and records all target information in the test area, and integrates such information into the radar target list. To ensure the validity of the test targets, standardized target-screening rules are designed for the trial, and the electro-optical equipment points only to targets that meet the specific conditions. These conditions are as follows: the target should be within the ±60° fan-shaped range of the USV’s bow, covering a total fan angle of 120°; the distance from the USV should be greater than 100 m and less than 3000 m; and the target should be located within the test area. If no targets meeting the above conditions are detected by the marine radar, the electro-optical equipment will maintain orientation locking; when the marine radar detects eligible targets, the electro-optical equipment will execute target pointing.

In the target pointing and field-of-view adaptation test phase, after the multi-surface target avoidance capability test is officially launched, the USV independently assesses the current operating foggy environment using the surface optical image fog detection algorithm, providing an environmental reference for subsequent field-of-view adaptation and target extraction. As shown in Figure 14, the visible-light image and its grayscale histogram, acquired by the USV at this moment, are presented, and the foggy scene determination of the operating environment can be completed based on these results [36].

For the designed fog detection module, $c_{fog}$ is defined as the binary discrimination result for the existence of fog in the visible-light imagery collected by the USV, and $r_{fog}$ is the preset threshold for judging the foggy state of the sea surface images. When the proportion ratio $r$ is no more than the threshold $r_{fog}$, the output $c_{fog}$ equals 0, which denotes that the input sea surface image is a fog-free image. In contrast, when the proportion ratio $r$ is higher than the threshold $r_{fog}$, the output $c_{fog}$ equals 1, which demonstrates that the sea surface image is confirmed to be foggy. The detailed mathematical expression is described as follows:

$$r={\displaystyle \frac{n_{fog}}{n_{total}}}$$

(13)

$$c_{fog}=\left\{\begin{array}{l}0,r\le r_{fog}\\ 1,r>r_{fog}\end{array}\right.$$

(14)

First, statistics are performed on the total pixel count $n_{total}$ with gray levels ranging from 0 to 255 in the grayscale histogram, as well as the pixel count $n_{fog}$ with gray levels ranging from $a$ to $b$. The proportion of pixels with gray levels between $a$ and $b$, relative to all pixels in the grayscale histogram, is 0.32. The decision threshold $r_{fog}$ for foggy scene determination is set to 0.8, which is greater than the calculated proportion value of 0.32. Based on this threshold, the current operating environment is fog-free.

When the marine radar successfully detects the obstacle buoy, the environmental perception system guides the electro-optical equipment to perform target pointing, so that the obstacle buoy appears in the visible-light image. As shown in Figure 15, after target pointing is complete, field-of-view adaptation calculation is performed on the target immediately, so that the target appears in the image at an appropriate size.

After the USV completes the secondary field-of-view adaptation adjustment, the target will appear in the image acquired by the electro-optical equipment with an appropriate size. Compared with images acquired without the field-of-view adaptation algorithm, this image contains more information from the target area, thereby improving the validity of subsequent target extraction. After completing the field-of-view adaptation calculation for the first frame, scene recognition is performed on this frame to finalize the standardized determination of the test scenario, providing a scene-adaptive reference for subsequent target extraction. As shown in Figure 16, the bright-spot statistics results for segmentation areas under mast occlusion scenario recognition and deck green water scenario recognition are presented, respectively, and scenario determination can be completed based on these statistics.

In the above scene recognition process, the proportional result $r_m$ obtained from mast occlusion scenario recognition is equal to 0, and the proportional result $r_w$ obtained from deck green water scenario recognition is equal to 0.56. In this trial, the decision threshold $r_{mast}$ of the mast occlusion scenario recognition algorithm is set to 0.25, and the decision threshold $r_{water}$ of the deck green water scenario recognition algorithm is set to 0.77. Based on the above thresholds, it can be determined that the first frame image does not belong to the mast occlusion or deck green water scenario, and it can proceed to the subsequent sea–sky line detection process. After completing scene recognition and determination, the position of the sea–sky line in the image is acquired through the sea–sky line detection algorithm based on the radar–electro-optical system. As shown in Figure 17, this algorithm can accurately extract the sea–sky line position in the image, providing a reference for the subsequent calculation of the target area.

After determining the position of the sea–sky line in the image at the current moment, the six degrees-of-freedom variation of the USV relative to the initial state at this moment can be obtained from the inertial navigation system to calculate the sea–sky line position in the image at the current moment. After completing sea–sky line detection in the initial state, the angular coordinates $(x_{corner}^{t_0},y_{corner}^{t_0})$ of the guidance target area in the image, as well as the corresponding area width $b_{image}^{t_0}$ and area height $h_{image}^{t_0}$, are obtained through the guidance target extraction algorithm of the first frame image. Target tracking in the image is performed using a region-of-interest prediction-based algorithm, and the target’s miss distance is computed. In Figure 18, the first frame and the sequence of images from the tracking process are presented, which intuitively demonstrate the continuity and stability of the tracking process.

During the target extraction and stable tracking test phase, after target pointing is complete, the perception system executes the field-of-view adaptation algorithm and the scene recognition algorithm. When the target is within the field of view of the visible-light sensor and is of an appropriate size, the perception system begins extracting the target from the image. Thereafter, the perception system judges whether the pointed target has been stably extracted. If the extraction is unstable, the perception system will continuously perform target pointing; if the target extraction is stable, it will switch to the target tracking mode.

During the target pointing process, if the pointed target fails to meet the constraint conditions, the perception system will begin reselecting the target. During the target tracking process, if the pointed target fails to meet the constraint conditions or target loss occurs, the perception system will also reselect the pointed target to maintain the continuous closed-loop operation of the perception link and the uninterrupted execution of the trajectory tracking process.

In the target positioning and obstacle avoidance execution phase, after the radar-guided target tracking is stabilized, the laser ranging sensor will begin emitting laser pulses to achieve the accurate positioning of the radar-guided target. After the laser hits the target, the perception system will determine whether the deviation between the laser position and the radar-guided position is less than 100 m. If the deviation is less than 100 m, the existing target information of laser positioning will be updated. Otherwise, the perception system will add the new target to the electro-optical equipment target list. Figure 19 presents a schematic diagram of the laser positioning process. Fusing multiple laser positioning results can effectively improve the target positioning accuracy.

During the stable tracking of the obstacle buoy, the USV continuously emits a laser and acquires multiple sets of position information for the buoy. Due to the inherent ranging error of the laser ranging sensor, to improve the positioning accuracy of the obstacle buoy, the center point of the minimum circular area formed by the target positions measured in each test is output as the final determined position of the obstacle buoy.

After acquiring accurate position information for the obstacle buoy, the USV successfully sails through the annular area with a target radius of 30 m to 60 m and continues to perform autonomous navigation along the typical trajectory. In this manner, the USV passes through the annular areas of other obstacle buoys in sequence and finally sails to the virtual point T07. It then completes a circular motion around the virtual point T08 with radius R, thereby ending the typical trajectory tracking and obstacle avoidance capability test. Throughout the entire test process, the shore-based interactive system monitors the trajectory tracking state of the USV, the working state of the environmental perception system, and the execution of obstacle avoidance maneuvers in real time. It synchronously records full-process test data to ensure test process controllability and traceability.

In this experiment, multiple test groups were conducted targeting specific marine scenarios, including green water on deck and foggy conditions. In Figure 20, a schematic diagram of the tests under green-water and foggy conditions is presented.

As shown in

Table 1. Comparison of laser hit rates.

Method	Green-Water Scene	Foggy Scene
Miss distance-based method laser hit rate	9%	28%
Proposed perception strategy laser hit rate	91%	82%

, the laser sensor has an extremely narrow divergence angle, and the laser can only accurately hit the target with stable target tracking realized by the perception system. The miss distance-based method yields a poor laser hit rate, or even a complete target miss, under harsh conditions, including green-water and foggy scenes. In comparison, the laser positioning method using the proposed environmental perception strategy achieves better performance in the above scenes, with the laser hit rate improved by 82% and 54% for green-water and foggy conditions, respectively.

For the composite mission scenario of USV preset trajectory tracking and dynamic obstacle avoidance, this work designs a standardized sea trial scheme and conducts full-process real-ship verification, and it verifies the engineering practicability of the proposed environmental perception strategy and the operability of the trial system. Nevertheless, there are still two limitations in this study regarding full working condition operation requirements. First, the coverage of test scenarios and target types is limited: the test is conducted in an open offshore sea area, covering typical sea-state interference, including deck green water and foggy conditions, but excluding extreme optical environments such as inshore dense navigation, heavy rain, and strong backlight. Second, the fault tolerance of the perception strategy under extreme working conditions is insufficient: the proposed strategy relies on radar-guided electro-optical fusion logic, without establishing a pure electro-optical autonomous detection and tracking mechanism for radar failure conditions. Future research will conduct an in-depth exploration of the limitations outlined above.

5. Conclusions

Aiming at the core limitations in the current research on USV perception methods, namely the difficulty of single-function algorithms to adapt to the full process of real and complex maritime missions and the lack of a standardized and reproducible verification system for real-ship sea trials, this paper researches system-level environmental perception strategies and sea trial methods for USVs in typical mission scenarios and completes the full research work, including the construction of a perception strategy system, the establishment of a trial platform, the design of standardized trial schemes, and full-process real-ship verification.

First, this paper constructs a full-function USV sea trial platform integrating the unmanned autonomous system and the shore-based interactive system, clarifies the hardware architecture and software logic of the platform, and provides a stable and reliable software and hardware carrier, as well as full-process trial management and control support for the real-ship testing of the perception strategy. Second, this paper proposes a systematic environmental perception strategy for typical USV operation tasks and constructs a full-process working closed loop covering wide-area target detection, scene-adaptive pre-processing, stable target tracking, high-precision positioning, and perception and decision-making for autonomous obstacle avoidance, which effectively alleviates the problem whereby existing single algorithms are disconnected from actual mission scenarios. On this basis, this paper designs two sets of standardized sea trial schemes for core tasks, namely the adaptability and obstacle avoidance capability test and the typical trajectory tracking and obstacle avoidance capability test; establishes reproducible and extensible real-ship verification specifications for USV perception performance; and solves the problem whereby existing sea trials are fragmented and lack unified design standards. Finally, through full-process real-ship sea trials, this paper verifies the engineering practicability of the proposed perception strategy, as well as the reproducibility, standardization, and extensibility of the established test system, with the laser positioning hit rate improved by 82% and 54% under green-water and foggy conditions compared with the conventional miss distance-based method.

The research described in this paper effectively addresses the deficiencies of existing research on mission-level and system-level USV perception schemes and standardized real-ship verification systems. It can provide a unified benchmark for the horizontal comparison of the real-ship performance of different USV environmental perception schemes and offer complete technical and trial support for the engineering implementation of USV intelligent perception technologies. Future research will further expand the perception strategies and trial specifications to scenarios such as complex sea states and multi-USV cooperative operations and construct a standardized evaluation system for USV perception performance that covers the full working conditions.