Discrete-Time Visual Servoing Control with Adaptive Image Feature Prediction Based on Manipulator Dynamics
Abstract
:1. Introduction
- (1)
- We present a visual servoing method from the perspective of discrete-time analysis. Based on the visual servo inner–outer loop control architecture, the impact of low-frequency visual feedback on system performance is analyzed.
- (2)
- A linear dynamic model is investigated to describe the movement of the robot under the velocity control mode. By using this model, the actual velocity of the robot end effector can be more accurately estimated.
- (3)
- An adaptive image feature prediction method is proposed to predict the positions of image features during the image sampling and processing time. Past image feature data and real robot velocity data are employed to adjust the predictor parameters.
2. System Description and Control Architecture
2.1. System Description
2.2. Control Architecture
3. Adaptive Image Feature Prediction Based on Robot Velocity Linear Dynamic Model
3.1. Linear Dynamic Model of Robot and Camera Motion
3.2. Adaptive Image Feature Prediction
Algorithm 1 Visual Servoing Controller With Adaptive Image Feature Prediction |
Input: the robot end-effcetor velocity and the image features Output: the robot end-effector velocity command ;
|
3.3. Stability Analysis
4. Experiments
- (1)
- Method 1 utilized the classical visual servoing scheme (VSC) [18]. In VSC, the four points are employed as image features. Due to the easy recognition of image features, the simple interaction matrix of point features, and high computational efficiency, this scheme has been widely applied in practice.
- (2)
- Method 2 (VSC-OB) employed the image prediction method based on the observer, which only uses Equation (6) and does not consider the robot dynamic model.
- (3)
- Method 3 (VSC-P) utilized the proposed adaptive image prediction based on the input-mapping method in combination with the robot’s dynamic model.
4.1. Controller with Smaller Proportional Gain
4.2. Controller with Large Proportional Gain
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Qiu, Y.; Li, B.; Shi, W.; Zhang, X. Visual Servo Tracking of Wheeled Mobile Robots with Unknown Extrinsic Parameters. IEEE Trans. Ind. Electron. 2019, 66, 8600–8609. [Google Scholar] [CrossRef]
- Zhang, X.; Fang, Y.; Li, B.; Wang, J. Visual Servoing of Nonholonomic Mobile Robots with Uncalibrated Camera-to-Robot Parameters. IEEE Trans. Ind. Electron. 2017, 64, 390–400. [Google Scholar] [CrossRef]
- Lin, Y.; Xing, K. Visual servo optimization stabilization of nonholonomic mobile robots based on control Lyapunov functions. Meas. Control 2020, 53, 1825–1831. [Google Scholar] [CrossRef]
- Albekairi, M.; Mekki, H.; Kaaniche, K.; Yousef, A. An Innovative Collision-Free Image-Based Visual Servoing Method for Mobile Robot Navigation Based on the Path Planning in the Image Plan. Sensors 2023, 23, 9667. [Google Scholar] [CrossRef] [PubMed]
- Al Arabi, A.; Tipu, R.S.; Bashar, M.R.; Barman, B.; Monicay, S.A.; Amin, M.A. Implementation of Low Cost Stereo Humanoid Adaptive Vision for 3D Positioning and Distance Measurement for Robotics Application with Self-Calibration. In Proceedings of the 2017 Asia Modelling Symposium (AMS), Kota Kinabalu, Malaysia, 4–6 December 2017; pp. 83–88. [Google Scholar] [CrossRef]
- Pandey, S.K.; Kumar, L.; Kumar, G.; Kumar, A.; Singh, K.U.; Singh, T. Vision-Based Locomotion Control for Humanoid Robots: A Study on Vision-Guided Walking Strategies. In Proceedings of the 2023 International Conference on Computational Intelligence and Sustainable Engineering Solutions (CISES), Greater Noida, India, 28–30 April 2023; pp. 1036–1042. [Google Scholar] [CrossRef]
- Zhang, X.; Fang, Y.; Zhang, X.; Jiang, J.; Chen, X. A Novel Geometric Hierarchical Approach for Dynamic Visual Servoing of Quadrotors. IEEE Trans. Ind. Electron. 2020, 67, 3840–3849. [Google Scholar] [CrossRef]
- Lin, J.; Wang, Y.; Miao, Z.; Fan, S.; Wang, H. Robust Observer-Based Visual Servo Control for Quadrotors Tracking Unknown Moving Targets. IEEE/ASME Trans. Mechatron. 2023, 28, 1268–1279. [Google Scholar] [CrossRef]
- Xie, W.F.; Li, Z.; Tu, X.W.; Perron, C. Switching Control of Image-Based Visual Servoing With Laser Pointer in Robotic Manufacturing Systems. IEEE Trans. Ind. Electron. 2008, 56, 520–529. [Google Scholar] [CrossRef]
- Nguyen, D.T.H.; Nguyen, V.H. Application of Visual Servo in Tracking and Grasping Moving Target. In Proceedings of the 2022 International Conference on Control, Robotics and Informatics (ICCRI), Danang, Vietnam, 2–4 April 2022; pp. 83–87. [Google Scholar] [CrossRef]
- Jiang, J.; Wang, Y.; Jiang, Y.; Xie, H.; Tan, H.; Zhang, H. A Robust Visual Servoing Controller for Anthropomorphic Manipulators with Field-of-View Constraints and Swivel-Angle Motion: Overcoming System Uncertainty and Improving Control Performance. IEEE Robot. Autom. Mag. 2022, 29, 104–114. [Google Scholar] [CrossRef]
- Shu, T.; Gharaaty, S.; Xie, W.; Joubair, A.; Bonev, I.A. Dynamic Path Tracking of Industrial Robots with High Accuracy Using Photogrammetry Sensor. IEEE/ASME Trans. Mechatron. 2018, 23, 1159–1170. [Google Scholar] [CrossRef]
- Lin, W.; Liu, C.; Guo, H.; Gao, H. Hybrid Visual-Ranging Servoing for Positioning Based on Image and Measurement Features. IEEE Trans. Cybern. 2023, 53, 4270–4279. [Google Scholar] [CrossRef]
- Li, T.; Yu, J.; Qiu, Q.; Zhao, C. Hybrid Uncalibrated Visual Servoing Control of Harvesting Robots With RGB-D Cameras. IEEE Trans. Ind. Electron. 2023, 70, 2729–2738. [Google Scholar] [CrossRef]
- He, S.; Xu, Y.; Guan, Y.; Li, D.; Xi, Y. Synthetic Robust Model Predictive Control With Input Mapping for Constrained Visual Servoing. IEEE Trans. Ind. Electron. 2023, 70, 9270–9280. [Google Scholar] [CrossRef]
- Espiau, B.; Chaumette, F.; Rives, P. A New Approach to Visual Servoing in Robotics. In Proceedings of the Geometric Reasoning for Perception and Action, Workshop, Grenoble, France, 16–17 September 1991. Selected Papers. [Google Scholar]
- Hutchinson, S.; Hager, G.D.; Corke, P.I. A tutorial on visual servo control. IEEE Trans. Robot. Autom. 1996, 12, 651–670. [Google Scholar] [CrossRef]
- Chaumette, F.; Hutchinson, S. Visual servo control. I. Basic approaches. IEEE Robot. Autom. Mag. 2006, 13, 82–90. [Google Scholar] [CrossRef]
- Chaumette, F. Potential problems of stability and convergence in image-based and position-based visual servoing. In The Confluence of Vision and Control; Springer: London, UK, 1998; pp. 66–78. [Google Scholar]
- Chaumette, F. Image moments: A general and useful set of features for visual servoing. IEEE Trans. Robot. 2004, 20, 713–723. [Google Scholar] [CrossRef]
- Tahri, O.; Chaumette, F. Point-based and region-based image moments for visual servoing of planar objects. IEEE Trans. Robot. 2005, 21, 1116–1127. [Google Scholar] [CrossRef]
- Bateux, Q.; Marchand, E. Histograms-based visual servoing. IEEE Robot. Autom. Lett. 2016, 2, 80–87. [Google Scholar] [CrossRef]
- Hafez, A.A.; Achar, S.; Jawahar, C. Visual servoing based on gaussian mixture models. In Proceedings of the 2008 IEEE International Conference on Robotics and Automation, Pasadena, CA, USA, 19–23 May 2008; IEEE: Piscataway, NJ, USA, 2008; pp. 3225–3230. [Google Scholar]
- Crombez, N.; Mouaddib, E.M.; Caron, G.; Chaumette, F. Visual servoing with photometric gaussian mixtures as dense features. IEEE Trans. Robot. 2018, 35, 49–63. [Google Scholar] [CrossRef]
- Ourak, M.; Tamadazte, B.; Lehmann, O.; Andreff, N. Wavelets-based 6 DOF visual servoing. In Proceedings of the 2016 IEEE International Conference on Robotics and Automation (ICRA), Stockholm, Sweden, 16–21 May 2016; IEEE: Piscataway, NJ, USA, 2016; pp. 3414–3419. [Google Scholar]
- Liu, Y.H.; Wang, H.; Wang, C.; Lam, K.K. Uncalibrated visual servoing of robots using a depth-independent interaction matrix. IEEE Trans. Robot. 2006, 22, 804–817. [Google Scholar]
- Anwar, A.; Lin, W.; Deng, X.; Qiu, J.; Gao, H. Quality Inspection of Remote Radio Units Using Depth-Free Image-Based Visual Servo with Acceleration Command. IEEE Trans. Ind. Electron. 2018, 66, 8214–8223. [Google Scholar] [CrossRef]
- He, Z.; Wu, C.; Zhang, S.; Zhao, X. Moment-Based 2.5-D Visual Servoing for Textureless Planar Part Grasping. IEEE Trans. Ind. Electron. 2019, 66, 7821–7830. [Google Scholar] [CrossRef]
- Bakthavatchalam, M.; Tahri, O.; Chaumette, F. A Direct Dense Visual Servoing Approach Using Photometric Moments. IEEE Trans. Robot. 2018, 34, 1226–1239. [Google Scholar] [CrossRef]
- Li, S.T.; Ahmad, G.; Xie, W.F.; Gao, Y.B. An enhanced IBVS controller of a 6DOF manipulator using hybrid PD-SMC method. International Journal of Control, Automation and Systems 2018, 16, 844–855. [Google Scholar] [CrossRef]
- Lazar, C.; Burlacu, A. Visual Servoing of Robot Manipulators Using Model-based Predictive Control. In Proceedings of the 2009 7th IEEE International Conference on Industrial Informatics, Cardiff, UK, 23–26 June 2009; pp. 690–695. [Google Scholar] [CrossRef]
- Allibert, G.; Courtial, E.; Chaumette, F. Predictive Control for Constrained Image-Based Visual Servoing. IEEE Trans. Robot. 2010, 26, 933–939. [Google Scholar] [CrossRef]
- Hajiloo, A.; Keshmiri, M.; Xie, W.F.; Wang, T.T. Robust Online Model Predictive Control for a Constrained Image-Based Visual Servoing. IEEE Trans. Ind. Electron. 2016, 63, 2242–2250. [Google Scholar] [CrossRef]
- Bjerkeng, M.; Falco, P.; Natale, C.; Pettersen, K.Y. Discrete-time stability analysis of a control architecture for heterogeneous robotic systems. In Proceedings of the 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, Tokyo, Japan, 3–7 November 2013; pp. 4778–4783. [Google Scholar] [CrossRef]
- Conticelli, F.; Allotta, B. Discrete-time robot visual feedback in 3D positioning tasks with depth adaptation. IEEE/ASME Trans. Mechatron. 2001, 6, 356–363. [Google Scholar] [CrossRef]
- Bjerkeng, M.; Falco, P.; Natale, C. Stability Analysis of a Hierarchical Architecture for Discrete-Time Sensor-Based Control of Robotic Systems. IEEE Trans. Robot. 2014, 30, 745–753. [Google Scholar] [CrossRef]
- He, S.; Xu, Y.; Li, D.; Xi, Y. Eye-in-Hand Visual Servoing Control of Robot Manipulators Based on an Input Mapping Method. IEEE Trans. Control Syst. Technol. 2023, 31, 402–409. [Google Scholar] [CrossRef]
- Aicardi, M.; Caiti, A.; Cannata, G.; Casalino, G. Stability and robustness analysis of a two layered hierarchical architecture for the closed loop control of robots in the operational space. In Proceedings of the 1995 IEEE International Conference on Robotics and Automation, Nagoya, Japan, 21–27 May 1995; Volume 3, pp. 2771–2778. [Google Scholar] [CrossRef]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Liu, C.; Ye, C.; Shi, H.; Lin, W. Discrete-Time Visual Servoing Control with Adaptive Image Feature Prediction Based on Manipulator Dynamics. Sensors 2024, 24, 4626. https://doi.org/10.3390/s24144626
Liu C, Ye C, Shi H, Lin W. Discrete-Time Visual Servoing Control with Adaptive Image Feature Prediction Based on Manipulator Dynamics. Sensors. 2024; 24(14):4626. https://doi.org/10.3390/s24144626
Chicago/Turabian StyleLiu, Chenlu, Chao Ye, Hongzhe Shi, and Weiyang Lin. 2024. "Discrete-Time Visual Servoing Control with Adaptive Image Feature Prediction Based on Manipulator Dynamics" Sensors 24, no. 14: 4626. https://doi.org/10.3390/s24144626
APA StyleLiu, C., Ye, C., Shi, H., & Lin, W. (2024). Discrete-Time Visual Servoing Control with Adaptive Image Feature Prediction Based on Manipulator Dynamics. Sensors, 24(14), 4626. https://doi.org/10.3390/s24144626