Next Article in Journal
Complete Lung Ultrasound Using Liquid Filling: A Review of Methods Regarding Sonographic Findings and Clinical Relevance
Previous Article in Journal
Depression and Microbiome—Study on the Relation and Contiguity between Dogs and Humans
Previous Article in Special Issue
Semantic Information for Robot Navigation: A Survey
Open AccessArticle

Motion Planning of Robot Manipulators for a Smoother Path Using a Twin Delayed Deep Deterministic Policy Gradient with Hindsight Experience Replay

by MyeongSeop Kim 1,†, Dong-Ki Han 1,†, Jae-Han Park 2 and Jung-Su Kim 1,*
1
Department of Electrical and Information Engineering, Seoul National University of Science and Technology, Seoul 01811, Korea
2
Robotics R&D Group, Korea Institute of Industrial Technology (KITECH), Ansan 15588, Korea
*
Author to whom correspondence should be addressed.
These authors contributed equally to this work.
Appl. Sci. 2020, 10(2), 575; https://doi.org/10.3390/app10020575
Received: 2 December 2019 / Revised: 7 January 2020 / Accepted: 7 January 2020 / Published: 13 January 2020
(This article belongs to the Collection Advances in Automation and Robotics)
In order to enhance performance of robot systems in the manufacturing industry, it is essential to develop motion and task planning algorithms. Especially, it is important for the motion plan to be generated automatically in order to deal with various working environments. Although PRM (Probabilistic Roadmap) provides feasible paths when the starting and goal positions of a robot manipulator are given, the path might not be smooth enough, which can lead to inefficient performance of the robot system. This paper proposes a motion planning algorithm for robot manipulators using a twin delayed deep deterministic policy gradient (TD3) which is a reinforcement learning algorithm tailored to MDP with continuous action. Besides, hindsight experience replay (HER) is employed in the TD3 to enhance sample efficiency. Since path planning for a robot manipulator is an MDP (Markov Decision Process) with sparse reward and HER can deal with such a problem, this paper proposes a motion planning algorithm using TD3 with HER. The proposed algorithm is applied to 2-DOF and 3-DOF manipulators and it is shown that the designed paths are smoother and shorter than those designed by PRM. View Full-Text
Keywords: motion planning; Probabilistic Roadmap (PRM); Reinforcement learning; policy gradient; Hindsight Experience Replay (HER) motion planning; Probabilistic Roadmap (PRM); Reinforcement learning; policy gradient; Hindsight Experience Replay (HER)
Show Figures

Figure 1

MDPI and ACS Style

Kim, M.; Han, D.-K.; Park, J.-H.; Kim, J.-S. Motion Planning of Robot Manipulators for a Smoother Path Using a Twin Delayed Deep Deterministic Policy Gradient with Hindsight Experience Replay. Appl. Sci. 2020, 10, 575.

Show more citation formats Show less citations formats
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop