Reinforcement Learning-Based Control of Single-Track Two-Wheeled Robots in Narrow Terrain

Zheng, Qingyuan; Tian, Yu; Deng, Yang; Zhu, Xianjin; Chen, Zhang; Liang, Bing

doi:10.3390/act12030109

Open AccessArticle

Reinforcement Learning-Based Control of Single-Track Two-Wheeled Robots in Narrow Terrain

by

Qingyuan Zheng

¹,

Yu Tian

¹,

Yang Deng

¹

,

Xianjin Zhu

²

,

Zhang Chen

^1,* and

Bing Liang

¹

Department of Automation, Tsinghua University, Beijing 100084, China

²

School of Mechatronics Engineering, Harbin Institute of Technology, Harbin 150006, China

^*

Author to whom correspondence should be addressed.

Actuators 2023, 12(3), 109; https://doi.org/10.3390/act12030109

Submission received: 30 January 2023 / Revised: 24 February 2023 / Accepted: 25 February 2023 / Published: 28 February 2023

(This article belongs to the Special Issue Advanced Technologies and Applications in Robotics)

Download

Browse Figures

Versions Notes

Abstract

The single-track two-wheeled (STTW) robot has the advantages of small size and flexibility, and it is suitable for traveling in narrow terrains of mountains and jungles. In this article, a reinforcement learning control method for STTW robots is proposed for driving fast in narrow terrain with limited visibility and line-of-sight occlusions. The proposed control scheme integrates path planning, trajectory tracking, and balancing control in a single framework. Based on this method, the state, action, and reward function are defined for narrow terrain passing tasks. At the same time, we design the actor network and the critic network structures and use the twin delayed deep deterministic policy gradient (TD3) to train these neural networks to construct a controller. Next, a simulation platform is formulated to test the performances of the proposed control method. The simulation results show that the obtained controller allows the STTW robot to effectively pass the training terrain, as well as the four test terrains. In addition, this article conducts a simulation comparison to prove the advantages of the integrated framework over traditional methods and the effectiveness of the reward function.

Keywords: single-track two-wheeled robot; reinforcement learning; narrow terrain

Share and Cite

MDPI and ACS Style

Zheng, Q.; Tian, Y.; Deng, Y.; Zhu, X.; Chen, Z.; Liang, B. Reinforcement Learning-Based Control of Single-Track Two-Wheeled Robots in Narrow Terrain. Actuators 2023, 12, 109. https://doi.org/10.3390/act12030109

AMA Style

Zheng Q, Tian Y, Deng Y, Zhu X, Chen Z, Liang B. Reinforcement Learning-Based Control of Single-Track Two-Wheeled Robots in Narrow Terrain. Actuators. 2023; 12(3):109. https://doi.org/10.3390/act12030109

Chicago/Turabian Style

Zheng, Qingyuan, Yu Tian, Yang Deng, Xianjin Zhu, Zhang Chen, and Bing Liang. 2023. "Reinforcement Learning-Based Control of Single-Track Two-Wheeled Robots in Narrow Terrain" Actuators 12, no. 3: 109. https://doi.org/10.3390/act12030109

APA Style

Zheng, Q., Tian, Y., Deng, Y., Zhu, X., Chen, Z., & Liang, B. (2023). Reinforcement Learning-Based Control of Single-Track Two-Wheeled Robots in Narrow Terrain. Actuators, 12(3), 109. https://doi.org/10.3390/act12030109

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Menu

Reinforcement Learning-Based Control of Single-Track Two-Wheeled Robots in Narrow Terrain

Abstract

Share and Cite

Article Metrics

Article Access Statistics

Further Information

Guidelines

MDPI Initiatives

Follow MDPI