# Using Deep Learning Models to Predict Prosthetic Ankle Torque

^{1}

^{2}

^{3}

^{4}

^{*}

## Abstract

**:**

## 1. Introduction

#### 1.1. The Role of System Models for Robotic Controllers and the Need for Better Modeling Approaches

#### 1.2. Current Data-Driven Approaches in Prosthetic Control

#### 1.3. Deep Learning Modeling Approach

## 2. Materials and Methods

#### 2.1. Prototype Ankle–Foot Prosthesis and Experimental Setup

#### 2.2. Baseline: Analytical Regression Model

#### 2.3. Neural Network Architectures

#### 2.3.1. FFN

#### 2.3.2. GRU

#### 2.3.3. DA-GRU

#### 2.4. Data Processing

#### 2.5. Loss Function and Network Parameter Optimization

#### 2.6. Network Hyperparameter Optimization Procedure and Outcomes

#### 2.7. Analysis

## 3. Results

## 4. Discussion

## 5. Conclusions

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Conflicts of Interest

## Appendix A. Derivation of the Analytical Regression Model

**Figure A1.**The prototype PAFP represented as two linkages connecting the drivetrain to the ankle joint. Pin joint A is where the ball screw connects to the shank link, pin joint B is where the ball screw nut housing connects to the ankle link, and pin joint O is where the ankle joint is connected to the drivetrain. Lowercase letters represent lengths and Greek letters represent angles. The variable $\omega $ represents the direction of positive angular velocity about the ankle joint.

## References

- Nguyen-Tuong, D.; Peters, J. Model learning for robot control: A survey. Cogn. Process.
**2011**, 12, 319–340. [Google Scholar] [CrossRef] [PubMed] - Teramae, T.; Noda, T.; Morimoto, J. EMG-Based Model Predictive Control for Physical Human–Robot Interaction: Application for Assist-As-Needed Control. IEEE Robot. Autom. Lett.
**2018**, 3, 210–217. [Google Scholar] [CrossRef] - Özen, O.; Traversa, F.; Gadi, S.; Buetler, K.A.; Nef, T.; Marchal-Crespo, L. Multi-purpose Robotic Training Strategies for Neurorehabilitation with Model Predictive Controllers. In Proceedings of the 2019 IEEE 16th International Conference on Rehabilitation Robotics (ICORR), Toroto, ON, Canada, 24–28 June 2019; pp. 754–759. [Google Scholar] [CrossRef]
- Winter, D.A. Kinematic and kinetic patterns in human gait: Variability and compensating effects. Hum. Mov. Sci.
**1984**, 3, 51–76. [Google Scholar] [CrossRef] - Tucker, M.R.; Olivier, J.; Pagel, A.; Bleuler, H.; Bouri, M.; Lambercy, O.; Del Millán, J.R.; Riener, R.; Vallery, H.; Gassert, R. Control strategies for active lower extremity prosthetics and orthotics: A review. J. Neuroeng. Rehabil.
**2015**, 12, 1–30. [Google Scholar] [CrossRef] - Boyan, J.A.; Moore, A.W. Generalization in Reinforcement Learning: Safely Approximating the Value Function. In Proceedings of the 7th International Conference on Neural Information Processing Systems, Denver, CO, USA, 1 January 1994; MIT Press: Cambridge, MA, USA, 1994; Volume 7, pp. 369–376. [Google Scholar]
- Atkeson, C. Using Local Trajectory Optimizers to Speed Up Global Optimization in Dynamic Programming. In Proceedings of the Advances in Neural Information Processing Systems, Denver, CO, USA, 30 November–3 December 1993; Volume 6, pp. 663–670. [Google Scholar]
- Rawlings, J.B.; Mayne, D.Q.; Diehl, M.M. Model Predictive Control: Theory, Computation, and Design, 2nd ed.; Nob Hill Publishing: San Francisco, CA, USA, 2017. [Google Scholar]
- Sutton, R.S.; Mcallester, D.; Singh, S.; Mansour, Y. Policy gradient methods for reinforcement learning with function approximation. Adv. Neural Inf. Process. Syst.
**2000**, 12, 1057–1063. [Google Scholar] - Goldberg, Y. A primer on neural network models for natural language processing. J. Artif. Intell. Res.
**2016**, 57, 345–420. [Google Scholar] [CrossRef] - Redmon, J.; Divvala, S.; Girshick, R.; Farhadi, A. You only look once: Unified, real-time object detection. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016; IEEE Computer Society: Washington, DC, USA, 2016; pp. 779–788. [Google Scholar] [CrossRef]
- Gomez-Gonzalez, S.; Prokudin, S.; Scholkopf, B.; Peters, J. Real time trajectory prediction using deep conditional generative models. IEEE Robot. Autom. Lett.
**2020**, 5, 970–976. [Google Scholar] [CrossRef] - Almeida, A.; Azkune, G. Predicting human behaviour with recurrent neural networks. Appl. Sci.
**2018**, 8, 305. [Google Scholar] [CrossRef] - Varol, H.A.; Sup, F.; Goldfarb, M. Multiclass real-time intent recognition of a powered lower limb prosthesis. IEEE Trans. Biomed. Eng.
**2010**, 57, 542–551. [Google Scholar] [CrossRef] - Young, A.J.; Simon, A.M.; Fey, N.P.; Hargrove, L.J. Classifying the intent of novel users during human locomotion using powered lower limb prostheses. In Proceedings of the International IEEE/EMBS Conference on Neural Engineering, NER, San Diego, CA, USA, 6–8 November 2013; pp. 311–314. [Google Scholar] [CrossRef]
- Huang, H.; Zhang, F.; Hargrove, L.J.; Dou, Z.; Rogers, D.R.; Englehart, K.B. Continuous locomotion-mode identification for prosthetic legs based on neuromuscular-mechanical fusion. IEEE Trans. Biomed. Eng.
**2011**, 58, 2867–2875. [Google Scholar] [CrossRef] - Vu, H.T.T.; Dong, D.; Cao, H.L.; Verstraten, T.; Lefeber, D.; Vanderborght, B.; Geeroms, J. A review of gait phase detection algorithms for lower limb prostheses. Sensors
**2020**, 20, 3972. [Google Scholar] [CrossRef] [PubMed] - Au, S.K.; Bonato, P.; Herr, H. An EMG-position controlled system for an active ankle-foot prosthesis: An initial experimental study. In Proceedings of the 9th International Conference on Rehabilitation Robotics, Chicago, IL, USA, 28 June–1July 2005; pp. 375–379. [Google Scholar] [CrossRef]
- Au, S.; Berniker, M.; Herr, H. Powered ankle-foot prosthesis to assist level-ground and stair-descent gaits. Neural Netw.
**2008**, 21, 654–666. [Google Scholar] [CrossRef] [PubMed] - Dedić, R.; Dindo, H. SmartLeg: An intelligent active robotic prosthesis for lower-limb amputees. In Proceedings of the 2011 23rd International Symposium on Information, Communication and Automation Technologies, ICAT, Sarajevo, Bosnia and Herzegovina, 27–29 October 2011; pp. 1–7. [Google Scholar] [CrossRef]
- Rai, V.; Sharma, A.; Preechayasomboon, P.; Rombokas, E. Coordinated movement for prosthesis reference trajectory generation: Temporal factors and attention. In Proceedings of the IEEE RAS and EMBS International Conference on Biomedical Robotics and Biomechatronics, New York, NY, USA, 29 November–1 December 2020; pp. 939–945. [Google Scholar] [CrossRef]
- Sharma, A.; Rombokas, E. Improving IMU-Based Prediction of Lower Limb Kinematics in Natural Environments Using Egocentric Optical Flow. IEEE Trans. Neural Syst. Rehabil. Eng.
**2022**, 30, 699–708. [Google Scholar] [CrossRef] [PubMed] - Camargo, J.; Molinaro, D.; Young, A. Predicting biological joint moment during multiple ambulation tasks. J. Biomech.
**2022**, 134, 111020. [Google Scholar] [CrossRef] - Dillen, A.; Steckelmacher, D.; Efthymiadis, K.; Langlois, K.; De Beir, A.; Marusic, U.; Vanderborght, B.; Nowe, A.; Meeusen, R.; Ghaffari, F.; et al. Deep learning for biosignal control: Insights from basic to real-time methods with recommendations. J. Neural Eng.
**2022**, 19, 011003. [Google Scholar] [CrossRef] - Kim, M.; Simon, A.M.; Hargrove, L.J. Seamless and intuitive control of a powered prosthetic leg using deep neural network for transfemoral amputees. Wearable Technol.
**2022**, 3, E24. [Google Scholar] [CrossRef] - Hossain, M.S.B.; Guo, Z.; Choi, H. Estimation of Lower Extremity Joint Moments and 3D Ground Reaction Forces Using IMU Sensors in Multiple Walking Conditions: A Deep Learning Approach. IEEE J. Biomed. Health Inform.
**2023**, 27, 2829–2840. [Google Scholar] [CrossRef] - Molinaro, D.D.; Kang, I.; Camargo, J.; Gombolay, M.C.; Young, A.J. Subject-Independent, Biological Hip Moment Estimation During Multimodal Overground Ambulation Using Deep Learning. IEEE Trans. Med. Robot. Bionics
**2022**, 4, 219–229. [Google Scholar] [CrossRef] - Schmidhuber, J. Deep learning in neural networks: An overview. Neural Netw.
**2015**, 61, 85–117. [Google Scholar] [CrossRef] - Cho, K.; Van Merriënboer, B.; Gulcehre, C.; Bahdanau, D.; Bougares, F.; Schwenk, H.; Bengio, Y.; van Merrienboer, B.; Gulcehre, C.; Bahdanau, D.; et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In Proceedings of the EMNLP 2014—2014 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics (ACL), Doha, Qatar, 25–29 October 2014; pp. 1724–1734. [Google Scholar] [CrossRef]
- Qin, Y.; Song, D.; Chen, H.; Cheng, W.; Jiang, G.; Cottrell, G. A dual-stage attention-based recurrent neural network for time series prediction. In Proceedings of the IJCAI International Joint Conference on Artificial Intelligence, Melbourne, Australia, 19–25 August 2017; pp. 2627–2633. [Google Scholar]
- Realmuto, J.; Klute, G.; Devasia, S. Nonlinear passive cam-based springs for powered ankle prostheses. J. Med. Devices Trans. ASME
**2015**, 9, 011007. [Google Scholar] [CrossRef] - Realmuto, J.; Klute, G.; Devasia, S. Preliminary investigation of symmetry learning control for powered ankle-foot prostheses. In Proceedings of the 2019 Wearable Robotics Association Conference (WearRAcon), Scottsdale, AZ, USA, 25–27 March 2019; pp. 40–45. [Google Scholar] [CrossRef]
- Paszke, A.; Gross, S.; Massa, F.; Lerer, A.; Bradbury, J.; Chanan, G.; Killeen, T.; Lin, Z.; Gimelshein, N.; Antiga, L.; et al. PyTorch: An imperative style, high-performance deep learning library. In Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada, 8–14 December 2019; Volume 32, pp. 8026–8037. [Google Scholar]
- Courville, I.G.; Bengio, Y.; Aaron. Deep Learning; MIT Press: Cambridge, MA, USA, 2016. [Google Scholar]
- Ravanelli, M.; Brakel, P.; Omologo, M.; Bengio, Y. Light gated recurrent units for speech recognition. IEEE Trans. Emerg. Top. Comput. Intell.
**2018**, 2, 92–102. [Google Scholar] [CrossRef] - Gruber, N.; Jockisch, A. Are GRU cells more specific and LSTM cells more sensitive in motive classification of text? Front. Artif. Intell.
**2020**, 3, 40. [Google Scholar] [CrossRef] [PubMed] - Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, Ł.; Polosukhin, I. Attention is all you need. In Proceedings of the Advances in Neural Information Processing Systems, Neural Information Processing Systems Foundation, Long Beach, CA, USA, 4–9 December 2017; Volume 30, pp. 5999–6009. [Google Scholar]
- Sun, M.; Hill, J.O. A method for measuring mechanical work and work efficiency during human activities. J. Biomech.
**1993**, 26, 229–241. [Google Scholar] [CrossRef] [PubMed] - Loshchilov, I.; Hutter, F. Decoupled weight decay regularization. In Proceedings of the 7th International Conference on Learning Representations, ICLR 2019, OpenReview.net, New Orleans, LA, USA, 6–9 May 2019. [Google Scholar]
- Patterson, J.; Gibson, A. Understanding Learning Rates. In Deep Learning: A Practitioner’s Approach; O’Reilly: Springfield, MO, USA, 2017; pp. 258–263. [Google Scholar]
- Akiba, T.; Sano, S.; Yanase, T.; Ohta, T.; Koyama, M. Optuna: A next-generation hyperparameter optimization framework. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Anchorage, AK, USA, 4–8 August 2019; pp. 2623–2631. [Google Scholar]
- Gabert, L.; Lenzi, T. Instrumented pyramid adapter for amputee gait analysis and powered prosthesis control. IEEE Sens. J.
**2019**, 19, 8272–8282. [Google Scholar] [CrossRef]

**Figure 1.**Block diagram of the architecture of a generic model-based prosthesis control system. A trajectory generator outputs a desired ankle torque ${y}_{d}$, which is then fed into an optimizer. The optimizer samples different possible control commands ${u}_{k}$ and conducts forward simulations based on system model predictions. The optimizer uses the results to determine which control command would achieve the closest one-sample ahead ankle torque response ${y}_{k+1}$ to the desired behavior. This control command is then sent to the human–robot system and the measurements of the system ${x}_{k}$ (e.g., loading and motion information from wearable sensors) are fed back to the optimizer for the next control sample.

**Figure 2.**Illustrative rendering of the prototype PAFP with major components labeled (note that some components are transparent for ease in visualization).

**Figure 3.**Visual illustration of the deep neural network architectures. The time history of input features are concatenated and fed into each network. Each DNN is trained to output a predicted PAFP ankle torque one timestep ahead or twenty timesteps ahead of the current timestep. Note that only one FFN and GRU are displayed in this diagram, but multiple layers were tested during hyperparameter optimization.

**Figure 4.**One-sample ahead model predictions of total PAFP torques across gait cycles. The periodic time series are time-normalized across the gait cycle for better visualization. The black time series data labeled as “MoCap” represents the PAFP ankle torque calculated using inverse dynamics. This data serve as the ground truth for model validation. The thin solid lines represent the mean and the corresponding shaded areas represent ±1 standard deviation.

**Figure 5.**RMSE (

**top**row) and RMSE percent error (

**bottom**row) for each model class for both one-sample ahead (

**left**column) and twenty-sample ahead (

**right**column) predictions. RMSE percent error was calculated by dividing the RMSE value by the range of ankle torque values within the walking trial. Errors are shown for stance and swing phases individually as well as the full gait cycle. The error bars represent the standard error.

Hyperparameter | Range/Values | Optimal Value | |||||
---|---|---|---|---|---|---|---|

1-Sample | 20-Sample | ||||||

FFN | GRU | DA-GRU | FFN | GRU | DA-GRU | ||

Sequence Length | [2, 3, 4, … , 18, 19, 20] | 16 | 15 | 18 | 20 | 18 | 20 |

Number of Layers | [1, 2, 3] | 3 | 2 | - | 3 | 2 | - |

Number of Hidden Units | [16, 32, 64, 128, 256, 512] | 512 | 512 | - | 512 | 64 | - |

Number of Encoder Hidden Units | [16, 32, 64, 128, 256, 512] | - | - | 128 | - | - | 512 |

Number of Decoder Hidden Units | [16, 32, 64, 128] | - | - | 16 | - | - | 64 |

Dropout Probability | [0.1:0.5] | 0.114 | 0.199 | - | 0.100 | 0.121 | - |

Batch Size | [16, 32, 64, 128, 256] | 16 | 16 | 16 | 16 | 16 | 16 |

Initial Learning Rate | [${10}^{-5}$:${10}^{-1}$] | $5.1\times {10}^{-5}$ | $1.3\times {10}^{-4}$ | $2.8\times {10}^{-3}$ | $4.6\times {10}^{-5}$ | $4.5\times {10}^{-4}$ | $1.8\times {10}^{-3}$ |

Weight Decay Coefficient | [${10}^{-5}$:${10}^{-1}$] | 0.062 | 0.019 | $3.1\times {10}^{-3}$ | 0.073 | 0.013 | 0.070 |

Learning Rate Reduction Factor | [0.1:0.9] | 0.168 | 0.559 | 0.132 | 0.373 | 0.728 | 0.230 |

RMSE ${}^{1}$ (Nm/kg) ${}^{3}$ | % RMSE ${}^{1}$ | PCC ${}^{2}$ | ||||
---|---|---|---|---|---|---|

1-Sample | 20-Sample | 1-Sample | 20-Sample | 1-Sample | 20-Sample | |

Analytical | 0.347 ± 0.534 | - | 26.6 ± 40.9 | - | 0.822 ± 0.202 | - |

FFN | 0.036 ± 0.024 | 0.058 ± 0.041 | 2.7 ± 1.6 | 4.3 ± 2.8 | 0.996 ± 0.006 | 0.988 ± 0.019 |

GRU | 0.042 ± 0.025 | 0.068 ± 0.042 | 3.2 ± 1.7 | 5.1 ± 3.0 | 0.995 ± 0.007 | 0.985 ± 0.024 |

DA-GRU | 0.037 ± 0.024 | 0.058 ± 0.051 | 2.8 ± 1.5 | 4.3 ± 3.5 | 0.996 ± 0.006 | 0.985 ± 0.030 |

^{1}RMSE = Root Mean Squared Error.

^{2}PCC = Pearson Correlation Coefficient.

^{3}Nm/kg = Newton-meter per kilogram bodyweight.

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Prasanna, C.; Realmuto, J.; Anderson, A.; Rombokas, E.; Klute, G.
Using Deep Learning Models to Predict Prosthetic Ankle Torque. *Sensors* **2023**, *23*, 7712.
https://doi.org/10.3390/s23187712

**AMA Style**

Prasanna C, Realmuto J, Anderson A, Rombokas E, Klute G.
Using Deep Learning Models to Predict Prosthetic Ankle Torque. *Sensors*. 2023; 23(18):7712.
https://doi.org/10.3390/s23187712

**Chicago/Turabian Style**

Prasanna, Christopher, Jonathan Realmuto, Anthony Anderson, Eric Rombokas, and Glenn Klute.
2023. "Using Deep Learning Models to Predict Prosthetic Ankle Torque" *Sensors* 23, no. 18: 7712.
https://doi.org/10.3390/s23187712