# One-Step Deadbeat Control of a 5-Link Biped Using Data-Driven Nonlinear Approximation of the Step-to-Step Dynamics

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

- The use of computed torque control to reduce continuous dynamics to low degree of freedom system. Here, we reduce the state space in the single stance (continuous phase) from 10D to 2D.
- The use of Monte Carlo sampling followed by a low-order polynomial model and a high order error model to approximate the step-to-step map with relatively high accuracy. We represent the step-to-step map using a low dimensional control affine part comprising of a quadratic polynomial and high dimensional error term using Gaussian process model; the approximated model has about $98\%$ accuracy.
- Development of a computationally efficient method to find a one-step deadbeat controller. We use the control affine part to find the control inputs analytically, but then fine tune these control inputs using the Gaussian process error model using iterative learning that converges in less than 10 function evaluations.

## 2. Robot Model

#### 2.1. Single Stance Equations

#### 2.2. Foot-Strike Equations

#### 2.3. Simulating a Single Step

## 3. Methods

#### 3.1. Overview

#### 3.2. Control in the Single Stance Phase Using Computed Torque Control

#### 3.3. Controlling the Step-to-Step Dynamics

#### 3.4. Approximating the Step-to-Step Dynamics

#### 3.5. Feedback Linearization

#### 3.6. Stochastic Gradient Descent

## 4. Results

#### 4.1. Periodic Gait and Optimization Parameters

#### 4.2. Data Generation for the Step-to-Step Map and Curve Fitting

#### 4.3. Stability

#### 4.4. Agility

#### 4.5. Versatility

## 5. Discussion

## 6. Conclusions and Future Work

- Reduce the dimensionality of the continuous dynamics using partial feedback linearization. This reduced the 10 dimensional state space to 2 dimensions.
- Reduce the dimensionality of the step-to-step dynamics using a Poincaré map. This further reduced the 2 dimensional state space to 1 dimension.
- Approximate the step-to-step dynamics using a control affine model and remaining error term with a Gaussian process model. The control affine model enables analytical computation of the control inputs which we improve in 2–9 iterations using the Gaussian process model to enable one-step deadbeat control.

## Author Contributions

## Funding

## Conflicts of Interest

## References

- Gregory, J.; Olivares, A.; Staffetti, E. Energy-optimal trajectory planning for the Pendubot and the Acrobot. Optim. Control. Appl. Methods
**2013**, 34, 275–295. [Google Scholar] [CrossRef] - Hobbelen, D.; Wisse, M. Limit cycle walking. In Humanoid Robots Human-Like Machines; Hackel, M., Ed.; Itech: Vienna, Austria, 2007; pp. 277–294. [Google Scholar]
- Takanishi, A.; Takeya, T.; Karaki, H.; Kato, I. A control method for dynamic biped walking under unknown external force. In Proceedings of the IEEE International Workshop on Intelligent Robots and Systems, Towards a New Frontier of Applications, Ibaraki, Japan, 3–6 July 1990; pp. 795–801. [Google Scholar]
- Stephens, B. Push Recovery Control for Force-Controlled Humanoid Robots. Ph.D. Thesis, Carnegie Mellon University, Pittsburgh, PA, USA, 2011. [Google Scholar]
- Pratt, J.; Chew, C.M.; Torres, A.; Dilworth, P.; Pratt, G. Virtual model control: An intuitive approach for bipedal locomotion. Int. J. Robot. Res.
**2001**, 20, 129–143. [Google Scholar] [CrossRef] - Westervelt, E.R.; Grizzle, J.W. Design of asymptotically stable walking for a 5-link planar biped walker via optimization. In Proceedings of the 2002 International Conference on Robotics and Automation, Washington, DC, USA, 10–17 May 2002. [Google Scholar]
- Grizzle, J.W.; Chevallereau, C. Virtual constraints and hybrid zero dynamics for realizing underactuated bipedal locomotion. arXiv
**2017**, arXiv:1706.01127. [Google Scholar] - Raibert, M.; Tzafestas, S.; Tzafestas, C. Comparative simulation study of three control techniques applied to a biped robot. In Proceedings of the IEEE Systems Man and Cybernetics Conference-SMC, Le Touquet, France, 17–20 October 1993; pp. 494–502. [Google Scholar]
- Saglam, C.O.; Byl, K. Stability and gait transition of the five-link biped on stochastically rough terrain using a discrete set of sliding mode controllers. In Proceedings of the 2013 IEEE International Conference on Robotics and Automation, Karlsruhe, Germany, 6–10 May 2013; pp. 5675–5682. [Google Scholar]
- Pratt, J.; Carff, J.; Drakunov, S.; Goswami, A. Capture point: A step toward humanoid push recovery. In Proceedings of the 2006 6th IEEE-RAS International Conference on Humanoid Robots, Genova, Italy, 4–6 December 2006; pp. 200–207. [Google Scholar]
- Bhounsule, P.A. Control of a compass gait walker based on energy regulation using ankle push-off and foot placement. Robotica
**2015**, 33, 1314–1324. [Google Scholar] [CrossRef] [Green Version] - Zamani, A.; Bhounsule, P.A. Foot Placement and Ankle Push-off Control for the Orbital Stabilization of Bipedal Robots. In Proceedings of the 2017 IEEE International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada, 24–28 September 2017. [Google Scholar]
- Grizzle, J.; Abba, G.; Plestan, F. Asymptotically stable walking for biped robots: Analysis via systems with impulse effects. IEEE Trans. Autom. Control.
**2001**, 46, 51–64. [Google Scholar] [CrossRef] [Green Version] - Dingwell, J.B.; Kang, H.G. Differences between local and orbital dynamic stability during human walking. J. Biomech. Eng.
**2007**, 129, 586–593. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Ames, A.D.; Galloway, K.; Sreenath, K.; Grizzle, J.W. Rapidly exponentially stabilizing control lyapunov functions and hybrid zero dynamics. IEEE Trans. Autom. Control.
**2014**, 59, 876–891. [Google Scholar] [CrossRef] - Chevallereau, C.; Grizzle, J.; Shih, C. Asymptotically stable walking of a five-link underactuated 3-D bipedal robot. IEEE Trans. Robot.
**2009**, 25, 37–50. [Google Scholar] [CrossRef] [Green Version] - Bhounsule, P.A.; Ruina, A.; Stiesberg, G. Discrete-decision continuous-actuation control: Balance of an inverted pendulum and pumping a pendulum swing. J. Dyn. Syst. Meas. Control.
**2015**, 137, 051012. [Google Scholar] [CrossRef] - Bauby, C.E.; Kuo, A.D. Active control of lateral balance in human walking. J. Biomech.
**2000**, 33, 1433–1440. [Google Scholar] [CrossRef] - Tedrake, R.; Manchester, I.R.; Tobenkin, M.; Roberts, J.W. LQR-trees: Feedback motion planning via sums-of-squares verification. Int. J. Robot. Res.
**2010**, 29, 1038–1052. [Google Scholar] [CrossRef] [Green Version] - Sidorov, E.; Zacksenhouse, M. Lyapunov based estimation of the basin of attraction of Poincare maps with applications to limit cycle walking. Nonlinear Anal. Hybrid Syst.
**2019**, 33, 179–194. [Google Scholar] [CrossRef] - Sirichotiyakul, W.; Satici, A.C.; Sanchez, S.; Bhounsule, P.A. Energetically-optimal Discrete and Continuous Stabilization of the Rimless Wheel With Torso. In Proceedings of the ASME 2019 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, Anaheim, CA, USA, 18–21 August 2019. [Google Scholar]
- Bhounsule, P.A.; Zamani, A. A discrete control lyapunov function for exponential orbital stabilization of the simplest walker. J. Mech. Robot.
**2017**, 9, 051011. [Google Scholar] [CrossRef] [Green Version] - Antsaklis, P.; Michel, A. Linear Systems; Birkhauser: Boston, MA, USA, 2006. [Google Scholar]
- Ogata, K. Discrete-Time Control Systems; Prentice Hall: London, UK, 1995. [Google Scholar]
- Bhounsule, P.A.; Ameperosa, E.; Miller, S.; Seay, K.; Ulep, R. Dead-beat control of walking for a torso-actuated rimless wheel using an event-based, discrete, linear controller. In Proceedings of the ASME 2016 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, Charlotte, NC, USA, 21–24 August 2016. [Google Scholar] [CrossRef] [Green Version]
- Carver, S.; Cowan, N.; Guckenheimer, J. Lateral stability of the spring-mass hopper suggests a two-step control strategy for running. Chaos Interdiscip. J. Nonlinear Sci.
**2009**, 19, 26106–26114. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Martin, W.C.; Wu, A.; Geyer, H. Experimental evaluation of deadbeat running on the atrias biped. IEEE Robot. Autom. Lett.
**2017**, 2, 1085–1092. [Google Scholar] [CrossRef] - Bhounsule, P.A.; Kim, M.; Alaeddini, A. Approximation of the Step-to-step Dynamics Enables Computationally Efficient and Fast Optimal Control of Legged Robots. In Proceedings of the ASME 2020 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, Virtual Conference, 17–19 August 2020. [Google Scholar]
- Zamani, A.; Bhounsule, P.A. Nonlinear model predictive control of hopping model using approximate step-to-step models for navigation on complex terrain. In Proceedings of the 2017 IEEE International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada, 24–28 September 2017. [Google Scholar]
- Sharbafi, M.A.; Seyfarth, A. Bioinspired Legged Locomotion: Models, Concepts, Control and Applications; Butterworth-Heinemann: Oxford, UK, 2017. [Google Scholar]
- Rodriguez, N.N.; Carbone, G.; Ceccarelli, M. Antropomorphic design and operation of a new low-cost humanoid robot. In Proceedings of the First IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics, Pisa, Italy, 20–22 February 2006; pp. 933–938. [Google Scholar]
- Kuo, A. Energetics of actively powered locomotion using the simplest walking model. J. Biomech. Eng.
**2002**, 124, 113–120. [Google Scholar] [CrossRef] [Green Version] - Strogatz, S. Nonlinear Dynamics and Chaos; Addison-Wesley: Reading, MA, USA, 1994. [Google Scholar]
- Bhounsule, P.A. Numerical accuracy of two benchmark models of walking: The rimless spoked wheel and the simplest walker. Dyn. Contin. Discret. Impuls. Syst. Ser. Appl. Algorithms
**2014**, 21, 137–148. [Google Scholar] - Ralston, H.J. Energy-speed relation and optimal speed during level walking. Int. Z. Angew. Physiol. Einschl. Arbeitsphysiol.
**1958**, 17, 277–283. [Google Scholar] [CrossRef] - Alexander, R.M. Stride length and speed for adults, children, and fossil hominids. Am. J. Phys. Anthropol.
**1984**, 63, 23–27. [Google Scholar] [CrossRef] - Bowling, A. Impact forces and agility in legged robot locomotion. J. Vib. Control.
**2011**, 17, 335–346. [Google Scholar] [CrossRef] - Kuo, A.D. Choosing your steps carefully. IEEE Robot. Autom. Mag.
**2007**, 14, 18–29. [Google Scholar] [CrossRef] - Hauser, J.; Murray, R.M. Nonlinear controllers for non-integrable systems: The Acrobot example. In Proceedings of the 1990 American Control Conference, San Diego, CA, USA, 23–25 May 1990; pp. 669–671. [Google Scholar]
- Bhounsule, P.A. Foot placement in the simplest slope walker reveals a wide range of walking solutions. IEEE Trans. Robot.
**2014**, 30, 1255–1260. [Google Scholar] [CrossRef] - Zamani, A.; Bhounsule, P. Control Synergies for Rapid Stabilization and Enlarged Region of Attraction for a Model of Hopping. Biomimetics
**2018**, 3, 25. [Google Scholar] [CrossRef] [Green Version] - McGeer, T. Dynamics and control of bipedal locomotion. J. Theor. Biol.
**1993**, 163, 277–314. [Google Scholar] [CrossRef] - Kuo, A.D. Stabilization of lateral motion in passive dynamic walking. Int. J. Robot. Res.
**1999**, 18, 917–930. [Google Scholar] [CrossRef] - Buss, B.G.; Hamed, K.A.; Griffin, B.A.; Grizzle, J.W. Experimental results for 3D bipedal robot walking based on systematic optimization of virtual constraints. In Proceedings of the 2016 American Control Conference (ACC), Boston, MA, USA, 6–8 July 2016; pp. 4785–4792. [Google Scholar]

**Figure 1.**Humanoid model: (

**a**) configuration variables describing the degrees of freedom, (

**b**) mass, center of mass, inertia about center of mass, and length parameters.

**Figure 2.**Overview of the approach: (

**a**) Computed torque control reduces the stance phase dynamics from $\mathsf{\Theta}=[{\mathsf{\Theta}}_{u},\phantom{\rule{0.28453pt}{0ex}}{\mathsf{\Theta}}_{c}]$ (10 dimensions) to $\mathsf{\Theta}={\mathsf{\Theta}}_{u}$ (2 dimensions). (

**b**) Monte Carlo simulations generates data for the step-to-step map ${\mathsf{\Theta}}_{u}^{i+1}=\mathbf{F}({\mathsf{\Theta}}_{u}^{i},{\mathbf{U}}^{i})$. (

**c**) The step-to-step map is curve fitted ${\mathsf{\Theta}}_{u}^{i+1}=\overline{\mathbf{F}}({\mathsf{\Theta}}_{u}^{i},{\mathbf{U}}^{i})$, where $\overline{F}$ consists of two parts, a control affine model and a Gaussian process error model. (

**d**) To enable one-step deadbeat control, the control affine model used to find control ${\mathbf{U}}^{i}$ using feedback linearization and the error model is used to refine the control using iterative learning control.

**Figure 4.**Stability: Stabilizing against state perturbation (

**a**) and necessary controls (

**b**). The system is perturbed at step 1 and is able to do a complete cancellation of disturbances in a single step (one-step deadbeat control).

**Figure 6.**Versatility: Negotiating random stepping stones, which is same as specified foot placement angle (

**a**) and necessary controls (

**b**).

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Bhounsule, P.A.; Hernandez-Hinojosa, E.; Alaeddini, A.
One-Step Deadbeat Control of a 5-Link Biped Using Data-Driven Nonlinear Approximation of the Step-to-Step Dynamics. *Robotics* **2020**, *9*, 90.
https://doi.org/10.3390/robotics9040090

**AMA Style**

Bhounsule PA, Hernandez-Hinojosa E, Alaeddini A.
One-Step Deadbeat Control of a 5-Link Biped Using Data-Driven Nonlinear Approximation of the Step-to-Step Dynamics. *Robotics*. 2020; 9(4):90.
https://doi.org/10.3390/robotics9040090

**Chicago/Turabian Style**

Bhounsule, Pranav A., Ernesto Hernandez-Hinojosa, and Adel Alaeddini.
2020. "One-Step Deadbeat Control of a 5-Link Biped Using Data-Driven Nonlinear Approximation of the Step-to-Step Dynamics" *Robotics* 9, no. 4: 90.
https://doi.org/10.3390/robotics9040090