Skip to Content
You are currently on the new version of our website. Access the old version .
  • Article
  • Open Access

5 August 2023

25 Pages

Autonomous Navigation Decision-Making Method for a Smart Marine Surface Vessel Based on an Improved Soft Actor–Critic Algorithm

,
,
and
Navigation College, Dalian Maritime University, Dalian 116026, China
*
Author to whom correspondence should be addressed.

Abstract

In this study, an intelligent hybrid algorithm based on deep-reinforcement learning (DRL) is proposed to achieve autonomous navigation and intelligent collision avoidance for a smart autonomous marine surface vessel (SMASV). First, the kinematic model of the SMASV is used, and clauses 13 to 17 of the Convention on the International Regulations for Preventing Collisions at Sea (COLREGs) are introduced. Then, the electronic chart is rasterized and used for path planning. Next, states, actions, and reward functions are designed, and collision avoidance strategies are formulated. In addition, a temperature factor and a constrained loss function are used to improve the soft actor–critic (SAC) algorithm. This improvement reduces the challenges of hyperparameter adjustment and improves sampling efficiency. By comparing the improved SAC algorithm with other deep-reinforcement learning (DRL) algorithms based on strategy learning, it is proved that the improved SAC algorithm converges faster than the other algorithms. During the experiment, some unknown obstacles are added to the simulation environment to verify the collision-avoidance ability of the trained SMASV. Moreover, eight sea areas are randomly selected to verify the generalization ability of the intelligent-navigation system. The results show that the proposed method can plan a path for the SMASV accurately and effectively, and the SMASV decision-making behavior in the collision-avoidance process conforms to the COLREGs in both unknown and dynamic environments.

Article Metrics

Citations

Article Access Statistics

Multiple requests from the same IP address are counted as one view.