# Learning Underwater Intervention Skills Based on Dynamic Movement Primitives

^{1}

^{2}

^{3}

^{4}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Related Work

## 3. ROV Teleoperation System

#### 3.1. System Constitution

#### 3.2. Mapping Algorithm

## 4. Methods

#### 4.1. GMM–GMR Preprocessing

#### 4.1.1. Gaussian Mixture Model

#### 4.1.2. Gaussian Mixture Regression

#### 4.2. Cartesian Space Dynamic Movement Primitive

#### 4.2.1. DMP for Position

#### 4.2.2. DMP for Orientation

## 5. Simulation

^{3}.

#### 5.1. Collection of Multiple Demonstration Trajectories

#### 5.2. Learning from Multiple Demonstrations

#### 5.3. Replication and Generalization of Skill

## 6. Conclusions

## Author Contributions

## Funding

## Data Availability Statement

## Conflicts of Interest

## References

- Sivčev, S.; Coleman, J.; Omerdić, E.; Dooly, G.; Toal, D. Underwater manipulators: A review. Ocean Eng.
**2018**, 163, 431–450. [Google Scholar] [CrossRef] - Schjølberg, I.; Gjersvik, T.B.; Transeth, A.A.; Utne, I.B. Next generation subsea inspection, maintenance and repair operations. IFAC-PapersOnLine
**2016**, 49, 434–439. [Google Scholar] [CrossRef] - Shim, H.; Jun, B.H.; Lee, P.M.; Baek, H.; Lee, J. Workspace control system of underwater tele-operated manipulators on an ROV. Ocean Eng.
**2010**, 37, 1036–1047. [Google Scholar] [CrossRef] - Sivčev, S.; Rossi, M.; Coleman, J.; Dooly, G.; Omerdić, E.; Toal, D. Fully automatic visual servoing control for work-class marine intervention ROVs. Control Eng. Pract.
**2018**, 74, 153–167. [Google Scholar] [CrossRef] - Gilmour, B.; Niccum, G.; O’Donnell, T. Field resident AUV systems—Chevron’s long-term goal for AUV development. In Proceedings of the 2012 IEEE/OES Autonomous Underwater Vehicles (AUV), Southampton, UK, 24–27 September 2012; pp. 1–5. [Google Scholar]
- Teigland, H.; Hassani, V.; Møller, M.T. Operator focused automation of ROV operations. In Proceedings of the 2020 IEEE/OES Autonomous Underwater Vehicles Symposium (AUV), St. Johns, NL, Canada, 30 September–2 October 2020; pp. 1–7. [Google Scholar]
- Marani, G.; Choi, S.K.; Yuh, J. Underwater autonomous manipulation for intervention missions AUVs. Ocean Eng.
**2009**, 36, 15–23. [Google Scholar] [CrossRef] - Palomeras, N.; Nagappa, S.; Ribas, D.; Gracias, N.; Carreras, M. Vision-based localization and mapping system for AUV intervention. In Proceedings of the 2013 MTS/IEEE OCEANS-Bergen, Bergen, Norway, 10–14 June 2013; pp. 1–7. [Google Scholar]
- Carrera, A.; Palomeras, N.; Hurtós, N.; Kormushev, P.; Carreras, M. Cognitive system for autonomous underwater intervention. Pattern Recognit. Lett.
**2015**, 67, 91–99. [Google Scholar] [CrossRef] - Havoutis, I.; Calinon, S. Learning from demonstration for semi-autonomous teleoperation. Auton. Robot.
**2019**, 43, 713–726. [Google Scholar] [CrossRef] - Gancet, J.; Weiss, P.; Antonelli, G.; Pfingsthorn, M.F.; Calinon, S.; Turetta, A.; Walen, C.; Urbina, D.; Govindaraj, S.; Letier, P.; et al. Dexterous undersea interventions with far distance onshore supervision: The DexROV project. IFAC-PapersOnLine
**2016**, 49, 414–419. [Google Scholar] [CrossRef] - Yang, C.; Chen, C.; He, W.; Cui, R.; Li, Z. Robot learning system based on adaptive neural control and dynamic movement primitives. IEEE Trans. Neural Networks Learn. Syst.
**2018**, 30, 777–787. [Google Scholar] [CrossRef] [PubMed] - Ijspeert, A.J.; Nakanishi, J.; Schaal, S. Movement imitation with nonlinear dynamical systems in humanoid robots. In Proceedings of the Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292), Washington, DC, USA, 11–15 May 2002; Volume 2, pp. 1398–1403. [Google Scholar]
- Khansari-Zadeh, S.M.; Billard, A. Learning stable nonlinear dynamical systems with gaussian mixture models. IEEE Trans. Robot.
**2011**, 27, 943–957. [Google Scholar] [CrossRef] - Rabiner, L.R. A tutorial on hidden Markov models and selected applications in speech recognition. Proc. IEEE
**1989**, 77, 257–286. [Google Scholar] [CrossRef] - Paraschos, A.; Daniel, C.; Peters, J.R.; Neumann, G. Probabilistic movement primitives. Adv. Neural Inf. Process. Syst.
**2013**, 26, 2616–2624. [Google Scholar] - Huang, Y.; Rozo, L.; Silvério, J.; Caldwell, D.G. Kernelized movement primitives. Int. J. Robot. Res.
**2019**, 38, 833–852. [Google Scholar] [CrossRef] - Liao, Z.; Jiang, G.; Zhao, F.; Wu, Y.; Yue, Y.; Mei, X. Dynamic skill learning from human demonstration based on the human arm stiffness estimation model and Riemannian DMP. IEEE/ASME Trans. Mechatronics
**2022**, 28, 1149–1160. [Google Scholar] [CrossRef] - Lu, Z.; Wang, N.; Li, Q.; Yang, C. A trajectory and force dual-incremental robot skill learning and generalization framework using improved dynamical movement primitives and adaptive neural network control. Neurocomputing
**2023**, 521, 146–159. [Google Scholar] [CrossRef] - Sidiropoulos, A.; Papageorgiou, D.; Doulgeri, Z. A novel framework for generalizing dynamic movement primitives under kinematic constraints. Auton. Robot.
**2023**, 47, 37–50. [Google Scholar] [CrossRef] - Yu, X.; Liu, P.; He, W.; Liu, Y.; Chen, Q.; Ding, L. Human-robot variable impedance skills transfer learning based on dynamic movement primitives. IEEE Robot. Autom. Lett.
**2022**, 7, 6463–6470. [Google Scholar] [CrossRef] - Liao, Z.; Lorenzini, M.; Leonori, M.; Zhao, F.; Jiang, G.; Ajoudani, A. An Ergo-Interactive Framework for Human-Robot Collaboration Via Learning From Demonstration. IEEE Robot. Autom. Lett.
**2023**, 9, 359–366. [Google Scholar] [CrossRef] - Chen, C.; Yang, C.; Zeng, C.; Wang, N.; Li, Z. Robot learning from multiple demonstrations with dynamic movement primitive. In Proceedings of the 2017 2nd International Conference on Advanced Robotics and Mechatronics (ICARM), Hefei and Tai’an, China, 27–31 August 2017; pp. 523–528. [Google Scholar]
- Ginesi, M.; Sansonetto, N.; Fiorini, P. Overcoming some drawbacks of dynamic movement primitives. Robot. Auton. Syst.
**2021**, 144, 103844. [Google Scholar] [CrossRef] - Yang, X.; Zhang, Q.; Wang, C.; Liu, X.; Zhang, Y.; Li, D. Development and Construction of a Simulation Platform for a New R-ROV. In Proceedings of the 2022 12th International Conference on CYBER Technology in Automation, Control, and Intelligent Systems (CYBER), Baishan, China, 27–31 July 2022; pp. 1293–1298. [Google Scholar]
- Sung, H.G. Gaussian Mixture Regression and Classification; Rice University: Houston, TX, USA, 2004. [Google Scholar]
- Burnham, K.P.; Anderson, D.R. Multimodel inference: Understanding AIC and BIC in model selection. Sociol. Methods Res.
**2004**, 33, 261–304. [Google Scholar] [CrossRef] - Dempster, A.P.; Laird, N.M.; Rubin, D.B. Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. Ser. B
**1977**, 39, 1–22. [Google Scholar] [CrossRef] - Hoffmann, H.; Pastor, P.; Park, D.H.; Schaal, S. Biologically-inspired dynamical systems for movement generation: Automatic real-time goal adaptation and obstacle avoidance. In Proceedings of the 2009 IEEE International Conference on Robotics and Automation, Kobe, Japan, 12–17 May 2009; pp. 2587–2592. [Google Scholar]
- Saveriano, M.; Abu-Dakka, F.J.; Kramberger, A.; Peternel, L. Dynamic movement primitives in robotics: A tutorial survey. Int. J. Robot. Res.
**2023**, 42, 1133–1184. [Google Scholar] [CrossRef] - Ude, A.; Nemec, B.; Petrić, T.; Morimoto, J. Orientation in cartesian space dynamic movement primitives. In Proceedings of the 2014 IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, China, 31 May–7 June 2014; pp. 2997–3004. [Google Scholar]
- Abu-Dakka, F.J.; Nemec, B.; Jørgensen, J.A.; Savarimuthu, T.R.; Krüger, N.; Ude, A. Adaptation of manipulation skills in physical contact with the environment to reference force profiles. Auton. Robot.
**2015**, 39, 199–217. [Google Scholar] [CrossRef]

**Figure 6.**GMM–GMR preprocessed demonstration trajectories used to obtain the t-$a\left(t\right)$ of the position.

**Figure 7.**GMM–GMR preprocessed demonstration trajectories used to obtain the t-$a\left(t\right)$ of the orientation.

**Figure 8.**Nonlinear term s-$f\left(s\right)$ in DMP modeling [12] of demonstration trajectories (position).

**Figure 9.**Nonlinear term s-$f\left(s\right)$ in DMP modeling [12] of demonstration trajectories (orientation).

**Figure 11.**Orientation trajectories and errors reproduced by DMP and UDMP methods (expressed as RPY).

Item | Value | |
---|---|---|

ROV | Design depth | 11,000 m |

Size (L × H × W) | 2.3 m × 1.3 m × 1.5 m | |

Mass | 1470 kg | |

Thrusters | 7 | |

Manipulator | Maximum reach | 1.6 m |

Function | 7 | |

Lift at full extension | 20 kg | |

CLAF_mini | Workspace | 0.2 m × 0.2 m × 0.13 m |

Force | 8.5 N |

Position | Orientation | |||||
---|---|---|---|---|---|---|

${\mathit{\alpha}}_{\mathit{p}}$ | ${\mathit{K}}_{\mathit{p}}$ | ${\mathit{N}}_{\mathit{p}}$ | ${\mathit{\alpha}}_{\mathit{p}}$ | ${\mathit{K}}_{\mathit{p}}$ | ${\mathit{N}}_{\mathit{p}}$ | |

DMP | 0.05 | 0.25 | 1 | 900 | ||

UDMP | 0.05 | 0.25 | 50 | 1 | 900 | 50 |

Position | Orientation | ||||||
---|---|---|---|---|---|---|---|

x (m) | y (m) | z (m) | Roll (rad) | Pitch (rad) | Yaw (rad) | ||

DMP | rmse | 0.0037 | 0.0024 | 0.0026 | 0.0073 | 0.0084 | 0.0081 |

max. error | 0.0099 | 0.0053 | 0.0066 | 0.0162 | 0.0178 | 0.0168 | |

UDMP | rmse | 0.0016 | 0.0007 | 0.0007 | 0.003 | 0.0057 | 0.0072 |

max. error | 0.0043 | 0.0021 | 0.0023 | 0.0072 | 0.0139 | 0.0134 |

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Yang, X.; Zhang, Y.; Li, R.; Zheng, X.; Zhang, Q.
Learning Underwater Intervention Skills Based on Dynamic Movement Primitives. *Electronics* **2024**, *13*, 3860.
https://doi.org/10.3390/electronics13193860

**AMA Style**

Yang X, Zhang Y, Li R, Zheng X, Zhang Q.
Learning Underwater Intervention Skills Based on Dynamic Movement Primitives. *Electronics*. 2024; 13(19):3860.
https://doi.org/10.3390/electronics13193860

**Chicago/Turabian Style**

Yang, Xuejiao, Yunxiu Zhang, Rongrong Li, Xinhui Zheng, and Qifeng Zhang.
2024. "Learning Underwater Intervention Skills Based on Dynamic Movement Primitives" *Electronics* 13, no. 19: 3860.
https://doi.org/10.3390/electronics13193860