Next Article in Journal
Cell Shape and Surface Colonisation in the Diatom Genus Cocconeis—An Opportunity to Explore Bio-Inspired Shape Packing?
Previous Article in Journal
Fluid–Structure Interaction for Biomimetic Design of an Innovative Lightweight Turboexpander
Previous Article in Special Issue
Neuromechanical Model of Rat Hindlimb Walking with Two-Layer CPGs
Article Menu
Issue 1 (March) cover image

Export Article

Open AccessArticle
Biomimetics 2019, 4(1), 28;

Implementation of Deep Deterministic Policy Gradients for Controlling Dynamic Bipedal Walking

Department of Mechanical and Aerospace Engineering, Case Western Reserve University, Cleveland, OH 44106, USA
Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH 44106, USA
Author to whom correspondence should be addressed.
Received: 12 November 2018 / Revised: 21 February 2019 / Accepted: 11 March 2019 / Published: 22 March 2019
(This article belongs to the Special Issue Selected Papers from Living Machines 2018)
PDF [2415 KB, uploaded 28 March 2019]
  |     |  


A control system for bipedal walking in the sagittal plane was developed in simulation. The biped model was built based on anthropometric data for a 1.8 m tall male of average build. At the core of the controller is a deep deterministic policy gradient (DDPG) neural network that was trained in GAZEBO, a physics simulator, to predict the ideal foot placement to maintain stable walking despite external disturbances. The complexity of the DDPG network was decreased through carefully selected state variables and a distributed control system. Additional controllers for the hip joints during their stance phases and the ankle joint during toe-off phase help to stabilize the biped during walking. The simulated biped can walk at a steady pace of approximately 1 m/s, and during locomotion it can maintain stability with a 30 kg·m/s impulse applied forward on the torso or a 40 kg·m/s impulse applied rearward. It also maintains stable walking with a 10 kg backpack or a 25 kg front pack. The controller was trained on a 1.8 m tall model, but also stabilizes models 1.4–2.3 m tall with no changes. View Full-Text
Keywords: biped; DDPG neural network; gait; stability biped; DDPG neural network; gait; stability

Figure 1

This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0).

Supplementary material


Share & Cite This Article

MDPI and ACS Style

Liu, C.; Lonsberry, A.G.; Nandor, M.J.; Audu, M.L.; Lonsberry, A.J.; Quinn, R.D. Implementation of Deep Deterministic Policy Gradients for Controlling Dynamic Bipedal Walking. Biomimetics 2019, 4, 28.

Show more citation formats Show less citations formats

Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Metrics

Article Access Statistics



[Return to top]
Biomimetics EISSN 2313-7673 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert
Back to Top