A Hybrid Control Approach Integrating Model-Predictive Control and Fractional-Order Admittance Control for Automatic Internal Limiting Membrane Peeling Surgery
Abstract
1. Introduction
- A multivision task surgical scene perception method based on microscopic binocular vision combined with target detection, key point recognition, and sparse 3D reconstruction is proposed.
- An automatic surgical trajectory planning method for initiating breaks in the ILM during surgery is proposed.
- An automatic control approach integrating MPC and FOAC is proposed to achieve more precise position and force control.
2. Methods
2.1. Control Framework for Automatic Break Initiation in ILM Peeling
2.2. Hybrid MPC–FOAC Control Method
2.3. Surgical Scene Perception Method Based on Multivision Tasks
Algorithm 1 Multivision-task-based Surgical Scene Perception |
Input: Left image IL, Right image IR. Output: Position of macular center Pmc, Position of the micro-forceps Pmf, sparse point cloud of ILM PointCloudILM, Pose of the micro-forceps Posemf.
|
2.4. Automatic Surgical Trajectory Planning Method
3. Simulation
3.1. Simulation and Analysis of MPC
3.2. Impact of Fractional Order on the Dynamic Characteristics of the Admittance Model
4. Experimental Validation and Results
4.1. Experimental Setup
4.2. Surgical Scene Perception Network Training
4.3. Break Initiation Experiment in ILM Peeling
5. Conclusions
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
Abbreviations
AMD | Age-related macular degeneration |
DR | Diabetic retinopathy |
ILM | Internal limiting membrane |
3D | Three-dimensional |
OCT | Optical coherence tomography |
PID | Proportional–integral–derivative |
MPC | Model=predictive control |
FOAC | Fractional-order admittance control |
Appendix A
References
- Zhou, Y.; Jia, W.; Song, J.; Li, M.; Dai, W.; Zou, J.; Zhou, J.; Chen, X.; Li, X. Burdens and trends of age-related macular degeneration at global, regional, and national levels, 1990–2021: Findings from the 2021 global burden of disease study. Eye 2025, 39, 1517–1525. [Google Scholar] [CrossRef]
- Teo, Z.L.; Tham, Y.; Yu, M.; Chee, M.L.; Rim, T.H.; Cheung, N.; Bikbov, M.M.; Wang, Y.X.; Tang, Y.; Lu, Y. Global prevalence of diabetic retinopathy and projection of burden through 2045: Systematic review and meta-analysis. Ophthalmology 2021, 128, 1580–1591. [Google Scholar] [CrossRef]
- Akhtar, S.M.M.; Saleem, S.Z.; Rizvi, S.A.A.; Fareed, A.; Mumtaz, M.; Saleem, S.; Bai, A.; Shaik, A.A.; Kirchoff, R.; Asghar, M.S. Critical analysis of macular hole repair techniques: A comprehensive systematic review and meta-analysis comparing internal limiting membrane flap and internal limiting membrane peeling for any size of macular hole. BMC Ophthalmol. 2025, 25, 174. [Google Scholar] [CrossRef] [PubMed]
- Michalewska, Z.; Bednarski, M.; Michalewski, J.; Jerzy, N. The role of ILM peeling in vitreous surgery for proliferative diabetic retinopathy complications. Ophthalmic Surg. Lasers Imaging Retin. 2013, 44, 238–242. [Google Scholar] [CrossRef] [PubMed]
- Asencio-Duran, M.; Manzano-Muñoz, B.; Vallejo-García, J.L.; García-Martínez, J. Complications of macular peeling. J. Ophthalmol. 2015, 2015, 467814. [Google Scholar] [CrossRef]
- Zhou, M.; Hao, X.; Eslami, A.; Huang, K.; Cai, C.; Lohmann, C.P.; Navab, N.; Knoll, A.; Nasseri, M.A. 6DOF needle pose estimation for robot-assisted vitreoretinal surgery. IEEE Access 2019, 7, 63113–63122. [Google Scholar] [CrossRef]
- El-Haddad, M.T.; Tao, Y.K. Automated stereo vision instrument tracking for intraoperative OCT guided anterior segment ophthalmic surgical maneuvers. Biomed. Opt. Express 2015, 6, 3014–3031. [Google Scholar] [CrossRef]
- Yang, S.; Martel, J.N.; Lobes Jr, L.A.; Riviere, C.N. Techniques for robot-aided intraocular surgery using monocular vision. Int. J. Robot. Res. 2018, 37, 931–952. [Google Scholar] [CrossRef] [PubMed]
- Zhou, M.; Hennerkes, F.; Liu, J.; Jiang, Z.; Wendler, T.; Nasseri, M.A.; Iordachita, I.; Navab, N. Theoretical error analysis of spotlight-based instrument localization for retinal surgery. Robotica 2023, 41, 1536–1549. [Google Scholar] [CrossRef]
- Zhou, M.; Wu, J.; Ebrahimi, A.; Patel, N.; He, C.; Gehlbach, P.; Taylor, R.H.; Knoll, A.; Nasseri, M.A.; Iordachita, I. Spotlight-based 3D instrument guidance for retinal surgery. In Proceedings of the 2020 International Symposium on Medical Robotics (ISMR), Atlanta, GA, USA, 18–20 November 2020; pp. 69–75. [Google Scholar]
- Koyama, Y.; Marinho, M.M.; Mitsuishi, M.; Harada, K. Autonomous coordinated control of the light guide for positioning in vitreoretinal surgery. IEEE Trans. Med. Robot. Bionics 2022, 4, 156–171. [Google Scholar] [CrossRef]
- Ebrahimi, A.; Alambeigi, F.; Sefati, S.; Patel, N.; He, C.; Gehlbach, P.; Iordachita, I. Stochastic force-based insertion depth and tip position estimations of flexible FBG-equipped instruments in robotic retinal surgery. IEEE/ASME Trans. Mechatron. 2020, 26, 1512–1523. [Google Scholar] [CrossRef] [PubMed]
- Probst, T.; Maninis, K.; Chhatkuli, A.; Ourak, M.; Vander Poorten, E.; Van Gool, L. Automatic tool landmark detection for stereo vision in robot-assisted retinal surgery. IEEE Robot. Autom. Let. 2017, 3, 612–619. [Google Scholar] [CrossRef]
- Richa, R.; Balicki, M.; Sznitman, R.; Meisner, E.; Taylor, R.; Hager, G. Vision-based proximity detection in retinal surgery. IEEE Trans. Bio-Med. Eng. 2012, 59, 2291–2301. [Google Scholar] [CrossRef]
- Laina, I.; Rieke, N.; Rupprecht, C.; Vizcaíno, J.P.; Eslami, A.; Tombari, F.; Navab, N. Concurrent segmentation and localization for tracking of surgical instruments. In Medical Image Computing and Computer-Assisted Intervention–MICCAI 2017, Proceedings of the 20th International Conference, Quebec City, QC, Canada, 11–13 September 2017; Proceedings, Part II 20, 2017; Springer: Berlin/Heidelberg, Germany, 2017; pp. 664–672. [Google Scholar]
- Havlena, M.; Maninis, K.; Bouget, D.; Vander Poorten, E.; Van Gool, L. 3D reconstruction of the retinal surface for robot-assisted eye surgery. In Proceedings of the Computer Assisted Radiology and Surgery, Heidelberg, Germany, 21–25 June 2016; pp. 112–113. [Google Scholar]
- Martinez-Perez, M.E.; Espinosa-Romero, A. Three-dimensional reconstruction of blood vessels extracted from retinal fundus images. Opt. Express 2012, 20, 11451–11465. [Google Scholar] [CrossRef]
- MacLachlan, R.A.; Becker, B.C.; Tabares, J.C.; Podnar, G.W.; Lobes, L.A.; Riviere, C.N. Micron: An actively stabilized handheld tool for microsurgery. IEEE Trans. Robot. 2011, 28, 195–212. [Google Scholar] [CrossRef] [PubMed]
- He, X.; Roppenecker, D.; Gierlach, D.; Balicki, M.; Olds, K.; Gehlbach, P.; Handa, J.; Taylor, R.; Iordachita, I. Toward clinically applicable steady-hand eye robot for vitreoretinal surgery. In Proceedings of the ASME International Mechanical Engineering Congress and Exposition, Houston, TX, USA, 9–15 November 2012; pp. 145–153. [Google Scholar]
- Edwards, T.L.; Xue, K.; Meenink, H.; Beelen, M.J.; Naus, G.; Simunovic, M.P.; Latasiewicz, M.; Farmery, A.D.; De Smet, M.D.; MacLaren, R.E. First-in-human study of the safety and viability of intraocular robotic surgery. Nat. Biomed. Eng. 2018, 2, 649–656. [Google Scholar] [CrossRef] [PubMed]
- Nasseri, M.A.; Eder, M.; Nair, S.; Dean, E.C.; Maier, M.; Zapp, D.; Lohmann, C.P.; Knoll, A. The introduction of a new robot for assistance in ophthalmic surgery. In Proceedings of the 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Osaka, Japan, 3–7 July 2013; pp. 5682–5685. [Google Scholar]
- Kummer, M.P.; Abbott, J.J.; Kratochvil, B.E.; Borer, R.; Sengul, A.; Nelson, B.J. OctoMag: An electromagnetic system for 5-DOF wireless micromanipulation. IEEE Trans. Robot. 2010, 26, 1006–1017. [Google Scholar] [CrossRef]
- Maberley, D.A.; Beelen, M.; Smit, J.; Meenink, T.; Naus, G.; Wagner, C.; de Smet, M.D. A comparison of robotic and manual surgery for internal limiting membrane peeling. Graefe’s Arch. Clin. Exp. Ophthalmol. 2020, 258, 773–778. [Google Scholar] [CrossRef]
- Üneri, A.; Balicki, M.A.; Handa, J.; Gehlbach, P.; Taylor, R.H.; Iordachita, I. New steady-hand eye robot with micro-force sensing for vitreoretinal surgery. In Proceedings of the 2010 3rd IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics, Tokyo, Japan, 26–29 September 2010; pp. 814–819. [Google Scholar]
- Abedloo, E.; Gholami, S.; Taghirad, H.D. Eye-rhas manipulator: From kinematics to trajectory control. In Proceedings of the 2015 3rd RSI International Conference on Robotics and Mechatronics (ICROM), Tehran, Iran, 7–9 October 2015; pp. 61–66. [Google Scholar]
- Wilson, J.T.; Gerber, M.J.; Prince, S.W.; Chen, C.W.; Schwartz, S.D.; Hubschman, J.P.; Tsao, T.C. Intraocular robotic interventional surgical system (iriss): Mechanical design, evaluation, and master–slave manipulation. Int. J. Med. Robot. Comput. Assist. Surg. 2018, 14, e1842. [Google Scholar] [CrossRef]
- Gholami, S.; Arjmandi, A.; Taghirad, H.D. An observer-based adaptive impedance-control for robotic arms: Case study in smos robot. In Proceedings of the 2016 4th International Conference on Robotics and Mechatronics (ICROM), Tehran, Iran, 26–28 October 2016; pp. 49–54. [Google Scholar]
- Becker, B.C.; MacLachlan, R.A.; Lobes, L.A.; Riviere, C.N. Vision-based retinal membrane peeling with a handheld robot. In Proceedings of the 2012 IEEE International Conference on Robotics and Automation, St. Paul, MN, USA, 14–18 May 2012; pp. 1075–1080. [Google Scholar]
- Han, Y.; Routray, A.; Adeghate, J.O.; MacLachlan, R.A.; Martel, J.N.; Riviere, C.N. Monocular vision-based retinal membrane peeling with a handheld robot. J. Med. Devices 2021, 15, 031014. [Google Scholar] [CrossRef] [PubMed]
- Wells, T.S.; Yang, S.; MacLachlan, R.A.; Lobes Jr, L.A.; Martel, J.N.; Riviere, C.N. Hybrid position/force control of an active handheld micromanipulator for membrane peeling. Int. J. Med. Robot. Comput. Assist. Surg. 2016, 12, 85–95. [Google Scholar] [CrossRef] [PubMed]
- Qin, W.; Yi, H.; Fan, Z.; Zhao, J. Haptic Shared Control Framework with Interaction Force Constraint Based on Control Barrier Function for Teleoperation. Sensors 2025, 25, 405. [Google Scholar] [CrossRef] [PubMed]
- Qin, W.; Zhang, H.; Fan, Z.; Zhu, Y.; Zhao, J. A Shared Control Method for Teleoperated Robot Using Artificial Potential Field. Adv. Intell. Syst-Ger. 2024, 6, 2300814. [Google Scholar] [CrossRef]
- Ladoiye, J.S.; Necsulescu, D.S.; Sasiadek, J.Z. Force Control of Surgical Robot with Time Delay Using Model Predictive Control. In Proceedings of the 15th International Conference on Informatics in Control, Automation and Robotics, Portu, Portugal, 29–31 July 2018; pp. 202–210. [Google Scholar]
- Zhang, P.; Kim, J.W.; Gehlbach, P.; Iordachita, I.; Kobilarov, M. Autonomous needle navigation in retinal microsurgery: Evaluation in ex vivo porcine eyes. In Proceedings of the 2023 IEEE International Conference on Robotics and Automation (ICRA), London, UK, 29 May–2 June 2023; pp. 4661–4667. [Google Scholar]
- Dominici, M.; Cortesao, R. Model predictive control architectures with force feedback for robotic-assisted beating heart surgery. In Proceedings of the 2014 IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, China, 31 May–7 June 2014; pp. 2276–2282. [Google Scholar]
- Gangloff, J.; Ginhoux, R.; de Mathelin, M.; Soler, L.; Marescaux, J. Model predictive control for compensation of cyclic organ motions in teleoperated laparoscopic surgery. IEEE Trans. Contr Syst. Trans. 2006, 14, 235–246. [Google Scholar] [CrossRef]
- Matschek, J.; Gonschorek, T.; Hanses, M.; Elkmann, N.; Ortmeier, F.; Findeisen, R. Learning references with Gaussian processes in model predictive control applied to robot assisted surgery. In Proceedings of the 2020 European Control Conference (ECC), Petersburg, Russia, 12–15 May 2020; pp. 362–367. [Google Scholar]
- Wang, N.; Zhang, X.; Li, M.; Zhang, H.; Stoyanov, D.; Stilli, A. A 5-DOFs robot for posterior segment eye microsurgery. IEEE Robot. Autom. Lett. 2022, 7, 10128–10135. [Google Scholar] [CrossRef]
- Jocher, G.; Qiu, J. Ultralytics YOLO11; GitHub: San Francisco, CA, USA, 2024. [Google Scholar]
- Shen, X.; Cai, Z.; Yin, W.; Müller, M.; Li, Z.; Wang, K.; Chen, X.; Wang, C. GIM: Learning Generalizable Image Matcher from Internet Videos. arXiv 2024, arXiv:2402.11095. [Google Scholar]
- Lee, J.E.; Byon, I.S.; Park, S.W. Internal Limiting Membrane Surgery; Springer: Berlin/Heidelberg, Germany, 2021. [Google Scholar]
- Zhang, X.; Liu, H.; Wang, Y.; Xiong, Y.; Niu, H. FBG-based three-dimensional micro-force sensor with axial force sensitivity-enhancing and temperature compensation for micro-forceps. Opt. Express 2023, 31, 40538–40556. [Google Scholar] [CrossRef]
- Ellabban, A. Retinal Surgery: Internal Limiting Membrane (ILM) Peel in Macular Hole Surgery. Available online: https://www.youtube.com/watch?v=7r1ES8YrlPk (accessed on 1 May 2025).
- Semeraro, F.; Morescalchi, F.; Duse, S.; Gambicorti, E.; Russo, A.; Costagliola, C. Current trends about inner limiting membrane peeling in surgery for epiretinal membranes. J. Ophthalmol. 2015, 2015, 671905. [Google Scholar] [CrossRef]
- Zhou, M.; Huang, K.; Eslami, A.; Roodaki, H.; Zapp, D.; Maier, M.; Lohmann, C.P.; Knoll, A.; Nasseri, M.A. Precision Needle Tip Localization using Optical Coherence Tomography Images for Subretinal Injection, to be appear. In Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia, 21–25 May 2018. [Google Scholar]
Error | Eye Phantom 1 | Eye Phantom 2 | Eye Phantom 3 | Eye Phantom 4 |
---|---|---|---|---|
x/μm | 18.275 | 21.345 | 18.332 | 17.532 |
y/μm | 9.880 | 10.626 | 10.547 | 8.876 |
z/μm | 21.920 | 23.598 | 19.249 | 21.823 |
total/μm | 35.115 | 37.588 | 32.962 | 33.762 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Liu, H.; Zhang, X.; Wang, Y.; Zhao, Z.; Wang, N. A Hybrid Control Approach Integrating Model-Predictive Control and Fractional-Order Admittance Control for Automatic Internal Limiting Membrane Peeling Surgery. Actuators 2025, 14, 328. https://doi.org/10.3390/act14070328
Liu H, Zhang X, Wang Y, Zhao Z, Wang N. A Hybrid Control Approach Integrating Model-Predictive Control and Fractional-Order Admittance Control for Automatic Internal Limiting Membrane Peeling Surgery. Actuators. 2025; 14(7):328. https://doi.org/10.3390/act14070328
Chicago/Turabian StyleLiu, Hongcheng, Xiaodong Zhang, Yachun Wang, Zirui Zhao, and Ning Wang. 2025. "A Hybrid Control Approach Integrating Model-Predictive Control and Fractional-Order Admittance Control for Automatic Internal Limiting Membrane Peeling Surgery" Actuators 14, no. 7: 328. https://doi.org/10.3390/act14070328
APA StyleLiu, H., Zhang, X., Wang, Y., Zhao, Z., & Wang, N. (2025). A Hybrid Control Approach Integrating Model-Predictive Control and Fractional-Order Admittance Control for Automatic Internal Limiting Membrane Peeling Surgery. Actuators, 14(7), 328. https://doi.org/10.3390/act14070328