Decision-Making for the Autonomous Navigation of Maritime Autonomous Surface Ships Based on Scene Division and Deep Reinforcement Learning
1
Key Laboratory of Maritime Dynamic Simulation and Control of Ministry of Transportation, Dalian Maritime University, Dalian 116026, China
2
Marine Engineering College, Dalian Maritime University, Dalian 116026, China
3
Department of Mechanical Engineering, University College London, Torrington Place, London WC1E 7JE, UK
4
Department of Civil Environmental and Geomatic Engineering, London WC1E 6BT, UK
*
Authors to whom correspondence should be addressed.
Sensors 2019, 19(18), 4055; https://doi.org/10.3390/s19184055
Received: 19 August 2019
/
Revised: 16 September 2019
/
Accepted: 17 September 2019
/
Published: 19 September 2019
(This article belongs to the Special Issue Internet of Things and Artificial Intelligence in Transportation Revolution)
This research focuses on the adaptive navigation of maritime autonomous surface ships (MASSs) in an uncertain environment. To achieve intelligent obstacle avoidance of MASSs in a port, an autonomous navigation decision-making model based on hierarchical deep reinforcement learning is proposed. The model is mainly composed of two layers: the scene division layer and an autonomous navigation decision-making layer. The scene division layer mainly quantifies the sub-scenarios according to the International Regulations for Preventing Collisions at Sea (COLREG). This research divides the navigational situation of a ship into entities and attributes based on the ontology model and Protégé language. In the decision-making layer, we designed a deep Q-learning algorithm utilizing the environmental model, ship motion space, reward function, and search strategy to learn the environmental state in a quantized sub-scenario to train the navigation strategy. Finally, two sets of verification experiments of the deep reinforcement learning (DRL) and improved DRL algorithms were designed with Rizhao port as a study case. Moreover, the experimental data were analyzed in terms of the convergence trend, iterative path, and collision avoidance effect. The results indicate that the improved DRL algorithm could effectively improve the navigation safety and collision avoidance.
View Full-Text
Keywords:
decision-making; autonomous navigation; collision avoidance; scene division; deep reinforcement learning; maritime autonomous surface ships
▼
Show Figures
Figure 1
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited
MDPI and ACS Style
Zhang, X.; Wang, C.; Liu, Y.; Chen, X. Decision-Making for the Autonomous Navigation of Maritime Autonomous Surface Ships Based on Scene Division and Deep Reinforcement Learning. Sensors 2019, 19, 4055. https://doi.org/10.3390/s19184055
AMA Style
Zhang X, Wang C, Liu Y, Chen X. Decision-Making for the Autonomous Navigation of Maritime Autonomous Surface Ships Based on Scene Division and Deep Reinforcement Learning. Sensors. 2019; 19(18):4055. https://doi.org/10.3390/s19184055
Chicago/Turabian StyleZhang, Xinyu, Chengbo Wang, Yuanchang Liu, and Xiang Chen. 2019. "Decision-Making for the Autonomous Navigation of Maritime Autonomous Surface Ships Based on Scene Division and Deep Reinforcement Learning" Sensors 19, no. 18: 4055. https://doi.org/10.3390/s19184055
Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.
