Robot Intelligent Grasp of Unknown Objects Based on Multi-Sensor Information
Abstract
:1. Introduction
- We present a method that combines three-dimensional (3D) vision and robot hand action to achieve reliable and accurate segmentation given unknown cluttered objects on a table.
- We propose an efficient algorithm for collision-free grasping pose generation; the grasp pose will be further checked for its grasp quality. Experiments show the efficiency and feasibility of our method.
- We addressed the problem associated with grasping unknown objects and how to set an appropriate grasp force. To fulfill this requirement, we adopted a multi-sensor approach to identify the stiffness of the object and detect slippage, which can promote a more lifelike functionality.
- By fusing all available sensor data appropriately, an intelligent grasp system was achieved that is reliable and able to handle various objects with unknown weights, friction, and stiffness.
2. Preliminary Knowledge
2.1. Point Cloud Processing
2.1.1. Pass-Through Filter
2.1.2. Down-Sampling
2.1.3. Random Sample Consensus (RANSAC)
2.1.4. Euclidean Segmentation
2.2. Oriented Bounding Box and Decomposition
- Compute the centroid (c0, c1, c2) and the normalized covariance.
- Find the eigenvectors for the covariance matrix of the point cloud (i.e., PCA).
- These eigenvectors are used to transform the point cloud to the origin point such that the eigenvectors correspond to the principal axes of the space.
- Compute the max, min, and center of the diagonal.
2.3. Object Segmentation in a Cluttered Scene
3. Grasp Planning
3.1. Grasp Strategy
3.2. Path Planning
3.3. Grasp Analysis
4. Real-Time Grip Force Selection and Control
4.1. Stiffness Measurement
- A positive integer k is specified along with a new sample.
- The k entries in the database that are closest to the new sample are selected.
- The most common classification of these entries is determined.
- This is the classification given to the new sample.
4.2. Slipping Detection and Avoidance
4.3. Real-Time Grasping Control
5. Experiment Results
5.1. Experiment 1: Slipping Detection and Avoidance
5.2. Experiment 2: Grasping in a Cluttered Scene
6. Conclusions
Author Contributions
Funding
Acknowledgments
Conflicts of Interest
References
- Huang, H.-P.; Yan, J.-L.; Huang, T.-H.; Huang, M.-B. IoT-based networking for humanoid robots. J. Chin. Inst. Eng. 2017, 40, 603–613. [Google Scholar] [CrossRef]
- Saudabayev, A.; Varol, H.A. Sensors for robotic hands: A survey of state of the art. IEEE Access 2015, 3, 1765–1782. [Google Scholar] [CrossRef]
- Huang, H.-P.; Yan, J.-L.; Cheng, T.-H. Development and fuzzy control of a pipe inspection robot. IEEE Trans. Ind. Electron. 2010, 57, 1088–1095. [Google Scholar] [CrossRef]
- Huang, M.-B.; Huang, H.-P.; Cheng, C.-C.; Cheng, C.-A. Efficient grasp synthesis and control strategy for robot hand-arm system. In Proceedings of the 11th IEEE International Conference on Automation Science and Engineering (CASE), Gothenburg, Sweden, 24–28 August 2015. [Google Scholar]
- Lee, W.-Y.; Huang, M.-B.; Huang, H.-P. Learning robot tactile sensing of object for shape recognition using multi-fingered robot hands. In Proceedings of the 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), Lisbon, Portugal, 28 August–1 September 2017. [Google Scholar]
- Liu, Y.-R.; Huang, M.-B.; Huang, H.-P. Automated grasp planning and path planning for a robot hand-arm system. In Proceedings of the 11th IEEE/SICE International Symposium on System Integration (SII), Paris, France, 14–16 January 2019. [Google Scholar]
- Huang, M.-B.; Huang, H.-P. Innovative human-like dual robotic hand mechatronic design and its chess-playing experiment. IEEE Access 2019, 7, 7872–7888. [Google Scholar] [CrossRef]
- Rusu, R.B. Semantic 3D object maps for everyday manipulation in human living environments. Künstl. Intell. 2010, 24, 345–348. [Google Scholar] [CrossRef]
- Huebner, K.; Ruthotto, S.; Kragic, D. Minimum volume bounding box decomposition for shape approximation in robot grasping. In Proceedings of the 2008 IEEE International Conference on Robotics and Automation (ICRA), Pasadena, CA, USA, 19–23 May 2008. [Google Scholar]
- Lei, Q.; Chen, G.; Wisse, M. Fast grasping of unknown objects using principal component analysis. AIP Adv. 2017, 7, 095126. [Google Scholar] [CrossRef]
- Liu, Z.; Kamogawa, H.; Ota, J. Fast grasping of unknown objects through automatic determination of the required number of mobile robots. Adv. Robot. 2013, 27, 445–458. [Google Scholar] [CrossRef]
- Faria, D.R.; Dias, J. 3D hand trajectory segmentation by curvatures and hand orientation for classification through a probabilistic approach. In Proceedings of the 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), St. Louis, MO, USA, 10–15 October 2009. [Google Scholar]
- Hirano, Y.; Kitahama, K.-I.; Yoshizawa, S. Image-based object recognition and dexterous hand/arm motion planning using RRTs for grasping in cluttered scene. In Proceedings of the 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Edmonton, AB, Canada, 2–6 August 2005. [Google Scholar]
- Yang, K.; Sukkarieh, S. 3D smooth path planning for a UAV in cluttered natural environments. In Proceedings of the 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Nice, France, 22–26 September 2008. [Google Scholar]
- Bicchi, A.; Kumar, V. Robotic grasping and contact: A review. In Proceedings of the 2000 IEEE International Conference on Robotics and Automation (ICRA), San Francisco, CA, USA, 24–28 April 2000. [Google Scholar]
- Murray, R.M.; Li, Z.; Sastry, S.S. Multifingered hand kinematics. In A Mathematical Introduction to Robotic Manipulation, 1st ed.; CRC Press: Boca Raton, FL, USA, 1994; pp. 211–240. [Google Scholar]
- Roa, M.A.; Suárez, R. Grasp quality measures: Review and performance. Auton. Robot. 2015, 38, 65–88. [Google Scholar] [CrossRef] [PubMed]
- Zheng, Y. An efficient algorithm for a grasp quality measure. IEEE Trans. Robot. 2013, 29, 579–585. [Google Scholar] [CrossRef]
- Ferrari, C.; Canny, J. Planning optimal grasps. In Proceedings of the 1992 IEEE International Conference on Robotics and Automation (ICRA), Nice, France, 12–14 May 1992. [Google Scholar]
- Boyd, S.P.; Wegbreit, B. Fast computation of optimal contact forces. IEEE Trans. Robot. 2007, 23, 1117–1132. [Google Scholar] [CrossRef]
- WACOH-TECH Inc. Available online: https://wacoh-tech.com/products/mudynpick/maf-3.html (accessed on 3 September 1988).
- Altman, N.S. An introduction to kernel and nearest-neighbor nonparametric regression. Am. Stat. 1992, 46, 175–185. [Google Scholar]
- Kozma, L. k Nearest Neighbors algorithm (kNN). Helsinki University of Technology, Special Course in Computer and Information Science. Available online: http://www.lkozma.net/knn2.pdf (accessed on 20 February 2008).
- Shaw-Cortez, W.; Oetomo, D.; Manzie, C.; Choong, P. Tactile-based blind grasping: A discrete-time object manipulation controller for robotic hands. IEEE Robot. Autom. Lett. 2018, 3, 1064–1071. [Google Scholar] [CrossRef]
- Johansson, R.S.; Westling, G. Roles of glabrous skin receptors and sensorimotor memory in automatic control of precision grip when lifting rougher or more slippery objects. Exp. Brain Res. 1984, 56, 550–564. [Google Scholar] [CrossRef] [PubMed]
- Deng, H.; Zhong, G.; Li, X.; Nie, W. Slippage and deformation preventive control of bionic prosthetic hands. IEEE/ASME Trans. Mech. 2017, 22, 888–897. [Google Scholar] [CrossRef]
- Teshigawara, S.; Tadakuma, K.; Ming, A.; Ishikawa, M.; Shimojo, M. High sensitivity initial slip sensor for dexterous grasp. In Proceedings of the 2010 IEEE International Conference on Robotics and Automation (ICRA), Anchorage, AK, USA, 3–7 May 2010. [Google Scholar]
- Heil, C.; Colella, D. Dilation equations and the smoothness of compactly supported wavelets. In Wavelets: Mathematics and Applications, 1st ed.; CRC Press: Boca Raton, FL, USA, 1993; pp. 163–202. [Google Scholar]
- Boggess, A.; Narcowich, F.J. Other wavelet topics. In A First Course in Wavelets with Fourier Analysis, 2nd ed.; John Wiley & Sons: Hoboken, NJ, USA, 2009; pp. 250–272. [Google Scholar]
- Tremblay, M.R.; Cutkosky, M.R. Estimating friction using incipient slip sensing during a manipulation task. In Proceedings of the 1993 IEEE International Conference on Robotics and Automation (ICRA), Atlanta, GA, USA, 2–6 May 1993. [Google Scholar]
- Kappassov, Z.; Corrales, J.-A.; Perdereau, V. Tactile sensing in dexterous robot hands—Review. Robot. Auton. Syst. 2015, 74, 195–220. [Google Scholar] [CrossRef]
- Jähne, B. Image representation. In Digital Image Processing, 6th ed.; Springer: Berlin, Germany, 2005; pp. 41–62. [Google Scholar]
- Alcazar, J.A.; Barajas, L.G. Estimating object grasp sliding via pressure array sensing. In Proceedings of the 2012 IEEE International Conference on Robotics and Automation (ICRA), Saint Paul, MN, USA, 14–18 May 2012. [Google Scholar]
- Vapnik, V.N. Constructing learning algorithms. In The Nature of Statistical Learning Theory, 1st ed.; Springer: New York, NY, USA, 1995; pp. 119–156. [Google Scholar]
- Cheng, C.-A.; Huang, H.-P. Learn the Lagrangian: A vector-valued RKHS approach to identifying Lagrangian systems. IEEE Trans. Cybern. 2016, 46, 3247–3258. [Google Scholar] [CrossRef] [PubMed]
- Cheng, C.-A.; Huang, H.-P.; Hsu, H.-K.; Lai, W.-Z.; Cheng, C.-C. Learning the inverse dynamics of robotic manipulators in structured reproducing kernel Hilbert space. IEEE Trans. Cybern. 2016, 46, 1691–1703. [Google Scholar] [CrossRef] [PubMed]
- Lo, S.-Y.; Cheng, C.-A.; Huang, H.-P. Virtual impedance control for safe human-robot interaction. J. Intell. Robot. Syst. 2016, 82, 3–19. [Google Scholar] [CrossRef]
- LIBSVM: A Library for Support Vector Machines. 2001. Available online: https://www.csie.ntu.edu.tw/~cjlin/libsvm (accessed on 1 September 2001).
Pre-States | Actions | Current States |
---|---|---|
PickUp(object 1) | ||
PutDown(object 2, object 1) | ||
PutDown(object 1, object 2) |
Action | Previous State | Current State | Detect Point Cloud in Previous Position | Detect Point Cloud in Current Position | ||
---|---|---|---|---|---|---|
Yes | No | Yes | No | |||
PickUp(object 1) | covered object | normal | object 1 | err | ||
PickUp(object 1) | object 2 | err | object 1 | err | ||
PutDown(ground) | err | normal | object 1 | err | ||
PutDown(object 1, object 2) | err | normal | object 1 and 2 | err |
|
Sensor Type | Count/Finger |
---|---|
joint torque {current (torque) control loop in BLDC motor} | 3 |
joint position | 3 |
motor speed | 3 |
distributed tactile sensor arrays (376 detecting points) (Tekscan, Inc., South Boston, MA, USA) | 1 |
three-axis force/torque (each fingertip) | 1 |
six-axis force/torque (the wrist joint) (Mini 40, ATI Industrial Automation, Apex, NC, USA) | 1 |
Total Weight | About 5.2 kg (exclude the shoulder) |
Max. Payload | Over 2 kg |
Max. Joint Speed | Exceed 110 deg./sec. |
Max. Reachable distance | 510 mm |
Max. width | 110 mm |
Motor | Brushed DC X6 |
Reduction device | Harmonic drive |
Transmission | Timing Belt, Gear |
© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Ji, S.-Q.; Huang, M.-B.; Huang, H.-P. Robot Intelligent Grasp of Unknown Objects Based on Multi-Sensor Information. Sensors 2019, 19, 1595. https://doi.org/10.3390/s19071595
Ji S-Q, Huang M-B, Huang H-P. Robot Intelligent Grasp of Unknown Objects Based on Multi-Sensor Information. Sensors. 2019; 19(7):1595. https://doi.org/10.3390/s19071595
Chicago/Turabian StyleJi, Shan-Qian, Ming-Bao Huang, and Han-Pang Huang. 2019. "Robot Intelligent Grasp of Unknown Objects Based on Multi-Sensor Information" Sensors 19, no. 7: 1595. https://doi.org/10.3390/s19071595
APA StyleJi, S.-Q., Huang, M.-B., & Huang, H.-P. (2019). Robot Intelligent Grasp of Unknown Objects Based on Multi-Sensor Information. Sensors, 19(7), 1595. https://doi.org/10.3390/s19071595