Vision-Guided Six-Legged Walking of Little Crabster Using a Kinect Sensor
Abstract
:1. Introduction
2. Little Crabster, LCR 200 with Kinect Sensor
3. Camera–Robot Coordinate Transformation
3.1. Depth Camera—CCD Camera Calibration
3.2. Derivation of Transformation Matrix Using Tsai Algorithm
3.2.1. Image Acquisition
3.2.2. Coordinate Transformation
3.2.3. Error Evaluation
3.3. Image Post-Processing
4. Vision-Guided Walking Algorithm
4.1. Wave Gait Pattern Generation
4.2. Landing Position Modification Algorithm
4.3. Ground Merging Algorithm
5. Experiment
6. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Conflicts of Interest
References
- Kohlbrecher, S.; Conner, D.C.; Romay, A.; Bacim, F.; Bowman, D.A.; von Stryk, O. Overview of team vigir’s approach to the virtual robotics challenge. In Proceedings of the IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), Linkoping, Sweden, 21–26 October 2013; pp. 1–2. [Google Scholar]
- Feng, S.; Whitman, E.; Xinjilefu, X.; Atkeson, C.G. Optimization-based full body control for the DARPA robotics challenge. J. Field Robot. 2015, 32, 293–312. [Google Scholar] [CrossRef] [Green Version]
- Wang, H.; Zheng, Y.F.; Jun, Y.; Oh, P. DRC-Hubo walking on rough terrains. In Proceedings of the IEEE International Conference on Technologies for Practical Robot Applications (TePRA), Woburn, MA, USA, 14–15 April 2014; pp. 1–6. [Google Scholar]
- Huang, Y.; Vanderborght, B.; Ham, R.V.; Wang, Q.; Damme, M.V.; Xie, G.; Lefeber, D. Step length and velocity control of a dynamic bipedal walking robot with adaptable compliant joints. IEEE/ASME Trans. Mechatron. 2013, 18, 598–611. [Google Scholar] [CrossRef]
- Li, T.H.S.; Su, Y.T.; Liu, S.H.; Hu, J.J.; Chen, C.C. Dynamic balance control for biped robot walking using sensor fusion, kalman filter, and fuzzy logic. IEEE Trans. Ind. Electron. 2012, 59, 4394–4408. [Google Scholar] [CrossRef]
- Luo, R.C.; Chen, C.C. Biped walking trajectory generator based on three-mass with angular momentum model using model predictive control. IEEE Trans. Ind. Electron. 2016, 63, 268–276. [Google Scholar] [CrossRef]
- Roennau, A.; Heppner, G.; Nowicki, M.; Dillmann, R. LAURON V: A versatile six-legged walking robot with advanced maneuverability. In Proceedings of the IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM), Besancon, France, 8–11 July 2014; pp. 82–87. [Google Scholar]
- Sridharan, M.; Kuhlmann, G.; Stone, P. Practical vision-based monte carlo localization on a legged robot. In Proceedings of the IEEE International Conference on Robotics and Automation, Barcelona, Spain, 18–22 April 2005; pp. 3366–3371. [Google Scholar]
- Michel, P.; Chestnutt, J.; Kuffner, J.; Kanade, T. Vision-guided humanoid footstep planning for dynamic environments. In Proceedings of the IEEE-RAS International Conference on Humanoid Robots, Tsukuba, Japan, 5–7 December 2005; pp. 13–18. [Google Scholar]
- Thompson, S.; Kagami, S. Humanoid robot localisation using stereo vision. In Proceedings of the IEEE-RAS International Conference on Humanoid Robots, Tsukuba, Japan, 5–7 December 2005; pp. 19–25. [Google Scholar]
- Chilian, A.; Hirschmüller, H. Stereo camera-based navigation of mobile robots on rough terrain. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, St. Louis, MO, USA, 10–15 October 2009; pp. 4571–4576. [Google Scholar]
- Belter, D.; Skrzypczynski, P. Precise self-localization of a walking robot on rough terrain using parallel tracking and mapping. Ind. Robot 2013, 40, 229–237. [Google Scholar] [CrossRef]
- Bajracharya, M.; Ma, J.; Malchano, M.; Perkins, A.; Rizzi, A.A.; Matthies, L. High fidelity day/night stereo mapping with vegetation and negative obstacle detection for vision-in-the-loop walking. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Tokyo, Japan, 3–7 November 2013; pp. 3663–3670. [Google Scholar]
- Ramos, O.E.; Garcia, M.; Mansard, N.; Stasse, O.; Hayet, J.B.; Soueres, P. Toward reactive vision-guided walking on rough terrain: An inverse-dynamics based approach. Int. J. Hum. Robot. 2014, 11, 1441004. [Google Scholar] [CrossRef]
- Kanoulas, D.; Zhou, C.; Nguyen, A.; Kanoulas, G.; Caldwell, D.G.; Tsagarakis, N.G. Vision-based foothold contact reasoning using curved surface patches. In Proceedings of the IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids), Birmingham, UK, 15–17 November 2017; pp. 121–128. [Google Scholar]
- Omori, Y.; Kojio, Y.; Ishikawa, T.; Kojima, K.; Sugai, F.; Kakiuchi, Y.; Okada, K.; Inaba, M. Autonomous safe locomotion system for bipedal robot applying vision and sole reaction force to footstep planning. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Macau SAR, China, 3–8 November 2019; pp. 4891–4898. [Google Scholar]
- Xu, J.; Wu, X.; Li, R.; Wang, X. Obstacle Overcoming Gait Design for Quadruped Robot with Vision and Tactile Sensing Feedback. In Proceedings of the 4th International Conference on Robotics, Control and Automation Engineering (RCAE), Wuhan, China, 4–6 November 2021; pp. 272–277. [Google Scholar] [CrossRef]
- Lee, M.; Kwon, Y.; Lee, S.; Choe, J.; Park, J.; Jeong, H.; Heo, Y.; Kim, M.S.; Sungho, J.; Yoon, S.E.; et al. Dynamic Humanoid Locomotion Over Rough Terrain With Streamlined Perception-Control Pipeline. In Proceedings of the 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Prague, Czech Republic, 27 September–1 October 2021; pp. 4111–4117. [Google Scholar] [CrossRef]
- Kim, J.Y. Dynamic balance control algorithm of a six-legged walking robot, little crabster. J. Intell. Robot. Syst. 2015, 18, 47–64. [Google Scholar] [CrossRef]
- Kim, J.Y.; Jun, B.H. Design of six-legged walking robot, little crabster for underwater walking and operation. Adv. Robot. 2014, 28, 77–89. [Google Scholar] [CrossRef]
- Tsai, R.Y. A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses. IEEE J. Robot. Autom. 1987, 3, 323–344. [Google Scholar] [CrossRef] [Green Version]
- Kim, J.Y.; Jun, B.H.; Park, I.W. Six-legged walking of “Little Crabster” on uneven terrain. Int. J. Precis. Eng. Manuf. 2017, 18, 509–518. [Google Scholar] [CrossRef]
- Kim, J.Y. Vision-Guided Six-Legged Walking of Little Crabster Using a Kinect Sensor. Available online: https://www.youtube.com/watch?time_continue=4&v=Svk6n43J4DE (accessed on 22 January 2022).
Specification | LCR 200 |
---|---|
Dimensions | 1000 (L) × 900 (W) × 500 (H) mm |
Weight | 54 kgf |
DOF | 30 (7 for front two legs, 4 for rear four legs) |
Actuators | 48 V Maxon BLDC motors with harmonic gears |
Sensors | 3-axis force/torque sensor at each hip 6-axis inertial sensor at body center. Compressive loadcell at each foot, Kinect sensor |
Power supply | Li-Polymer battery (48 V, 360 Wh) |
Operating system | Robot PC: Windows XP with RTX for robot body control Vision PC: Windows 7 for Kinect sensor |
Motor servo controllers | 2-Ch 200 W BL4804DID (Robocubetech Co., Seoul, Korea) |
Control system | Distributed control system using CAN communication (control frequency: 100 HZ) |
Camera-Fixed Coord. (mm) | Actual Robot-Fixed Coord. (mm) | Calculated Robot-Fixed Coord. (mm) | Error (mm) | ||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
X | Y | Z | X | Y | Z | X | Y | Z | X | Y | Z |
1168.92 | 211.39 | 455.32 | 1200.00 | 200.00 | 300.00 | 1198.11 | 196.97 | 293.45 | 1.89 | 3.03 | 6.55 |
1189.03 | −191.32 | 449.28 | 1200.00 | −200.00 | 250.00 | 1197.13 | −205.03 | 255.51 | 2.87 | 5.03 | 5.51 |
1017.83 | 12.58 | 339.53 | 1000.00 | 0.00 | 300.00 | 1002.41 | 0.93 | 296.39 | 2.41 | 0.93 | 3.61 |
1343.23 | 11.99 | 567.23 | 1400.00 | 0.00 | 250.00 | 1696.72 | −4.54 | 252.37 | 3.28 | 4.54 | 2.37 |
1057.88 | 159.83 | 372.49 | 1068.40 | 141.40 | 300.00 | 1058.62 | 147.33 | 302.04 | 9.98 | 5.93 | 2.06 |
1304.08 | −128.39 | 549.29 | 1341.40 | −141.40 | 250.00 | 1351.31 | −144.80 | 257.74 | 9.91 | 3.40 | 7.74 |
1059.32 | −129.58 | 398.53 | 1068.60 | −141.40 | 300.00 | 1068.20 | −143.35 | 306.15 | 0.40 | 1.95 | 6.15 |
1295.87 | 159.99 | 519.23 | 1341.40 | 141.40 | 250.00 | 1333.96 | 144.92 | 255.03 | 7.44 | 3.52 | 5.03 |
1262.80 | −123.11 | 571.88 | 1341.40 | −141.40 | 300.00 | 1335.42 | −141.48 | 302.19 | 5.98 | 0.08 | 2.19 |
1101.85 | 153.52 | 332.01 | 1068.60 | 141.40 | 250.00 | 1064.69 | 143.93 | 242.43 | 3.91 | 2.53 | 7.66 |
1279.34 | 153.20 | 559.23 | 1341.40 | 141.40 | 300.00 | 1347.76 | 135.80 | 295.60 | 6.36 | 5.60 | 4.40 |
1106.23 | −139.76 | 361.39 | 1068.60 | −141.40 | 250.00 | 1078.57 | −150.73 | 246.82 | 9.97 | 9.33 | 3.18 |
Max | 9.98 | 9.33 | 7.74 | ||||||||
Avg | 6.28 | 4.51 | 5.10 |
Walking Parameters | Description |
---|---|
(1) Swing Time (Tsw) | Time duration of foot in air |
(2) Delay Time (Td) | Time interval between foot landing and swing = Delay Ratio (κd) × Tsw |
(3) Step Time (Tst) | Tst = Tsw + Td |
(4) Walking Cycle Time (Twc) | Twc = 6 × Tst |
(5) Swing Height (Hsw) | Maximum foot swing height |
(6) Body Height (Hb) | Averaged body height from the six feet |
(7) Step Length (Ls) | Longitudinal step length |
(8) Side Step Length (Lss) | Lateral step length |
(9) Rotation Angle (BCθ) | Body rotational angle |
X (mm) | Y (mm) | Z (mm) | Depth Data Collection Rate | Standard Deviation (mm) | |
---|---|---|---|---|---|
Original landing position | 1036.35 | −336.35 | 45.12 | 57% | 13.14 |
1st Alternative landing position | 1066.35 | −336.35 | 8.37 | 71% | 6.35 |
2nd Alternative landing position | 1057.56 | −315.14 | 10.36 | 54% | 17.32 |
3rd Alternative landing position | 1057.56 | −357.56 | 11.97 | 63% | 7.84 |
4th Alternative landing position | 1036.35 | −306.35 | 45.21 | 63% | 11.97 |
5th Alternative landing position | 1036.35 | −366.35 | 45.33 | 58% | 13.29 |
6th Alternative landing position | 1015.14 | −315.14 | 44.84 | 87% | 4.23 |
7th Alternative landing position | 1015.14 | −357.56 | 44.89 | 81% | 2.19 |
8th Alternative landing position | 1006.35 | −336.35 | 44.97 | 79% | 5.57 |
X (mm) | Y (mm) | Z (mm) | Depth Data Collection Rate | Standard Deviation (mm) | |
---|---|---|---|---|---|
Original landing position | 936.35 | −336.35 | 42.74 | 81% | 17.64 |
1st Alternative landing position | 966.35 | −336.35 | 43.94 | 76% | 5.67 |
2nd Alternative landing position | 957.56 | −315.14 | 42.31 | 67% | 6.33 |
3rd Alternative landing position | 957.56 | −357.56 | 44.31 | 81% | 6.18 |
4th Alternative landing position | 936.35 | −306.35 | 39.97 | 68% | 10.92 |
5th Alternative landing position | 936.35 | −366.35 | 44.04 | 75% | 11.24 |
6th Alternative landing position | 915.14 | −315.14 | 31.75 | 78% | 11.84 |
7th Alternative landing position | 915.14 | −357.56 | 29.36 | 73% | 14.49 |
8th Alternative landing position | 906.35 | −336.35 | 28.94 | 71% | 13.61 |
Original | Modified | |||||||||
---|---|---|---|---|---|---|---|---|---|---|
X (mm) | Y (mm) | Z (mm) | Collection Rate (%) | SD (mm) | X (mm) | Y (mm) | Z (mm) | Collection Rate (%) | SD (mm) | |
3rd Scanning | 1536.35 | −336.35 | 47.50 | 99 | 3.33 | |||||
1100.00 | 442.83 | 40.52 | 53 | 4.57 | 1078.79 | 421.62 | 39.05 | 73 | 2.09 | |
800.00 | −392.82 | 29.74 | 94 | 12.46 | 830.00 | −392.83 | 41.67 | 97 | 1.81 | |
1636.35 | 336.35 | 3.00 | 97 | 2.01 | ||||||
1200.00 | −442.82 | 1.57 | 48 | 8.13 | 1200.00 | −412.83 | 2.12 | 74 | 2.65 | |
900.00 | 392.83 | 0.05 | 97 | 2.27 | ||||||
1736.35 | −336.35 | 3.14 | 98 | 2.09 | ||||||
1300.00 | 442.83 | 1.95 | 38 | 1.48 | 1330.00 | 442.83 | 2.04 | 76 | 2.7 | |
1000.00 | −392.83 | 44.23 | 81 | 3.25 | ||||||
1836.35 | 336.35 | 39.87 | 95 | 2.22 | ||||||
1400.00 | −442.83 | 40.45 | 38 | 4.65 | 1421.21 | −421.62 | 42.61 | 79 | 2.25 | |
1100.00 | 392.83 | 39.49 | 86 | 2.14 | ||||||
1936.35 | −336.35 | 4.02 | 94 | 1.75 | ||||||
1500.00 | 442.83 | 3.49 | 87 | 2.37 | ||||||
1200.00 | −392.83 | 1.62 | 94 | 2.55 | ||||||
1936.35 | 336.35 | 40.80 | 96 | 3.43 | ||||||
1500.00 | −442.83 | 44.24 | 85 | 2.65 | ||||||
1200.00 | 392.83 | 39.85 | 100 | 1.93 | ||||||
4th Scanning | 2036.00 | −336.35 | 3.48 | 86 | 1.82 | |||||
1600.00 | 442.83 | 2.57 | 51 | 8.22 | 1621.21 | 421.62 | 1.66 | 74 | 2.73 | |
1300.00 | −392.83 | 5.46 | 97 | 11.41 | 1275.79 | −371.62 | 0.80 | 97 | 1.99 | |
2136.35 | 336.35 | 3.12 | 97 | 2.24 | ||||||
1700.00 | −442.83 | 2.49 | 47 | 9.15 | 1721.21 | −421.62 | 3.69 | 72 | 2.86 | |
1400.00 | 392.83 | 2.82 | 100 | 1.91 | ||||||
2236.35 | −336.35 | 3.30 | 99 | 1.70 | ||||||
1800.00 | 42.83 | 38.80 | 75 | 3.04 | ||||||
1500.00 | −392.83 | 45.27 | 68 | 2.68 | ||||||
2336.35 | 336.35 | 4.25 | 100 | 2.29 | ||||||
1900.00 | −442.83 | 1.45 | 56 | 1.56 | 1921.21 | −421.62 | 1.18 | 80 | 1.81 | |
1600.00 | 392.83 | 392.00 | 71 | 2.31 | ||||||
2436.35 | −336.35 | 4.94 | 92 | 2.10 | ||||||
2000.00 | 442.83 | 0.05 | 28 | 0.01 | 2000.00 | 442.83 | 0.00 | 53 | 2.91 | |
1700.00 | −392.83 | 3.44 | 98 | 2.59 | ||||||
2436.35 | 336.35 | 3.74 | 97 | 1.99 | ||||||
2000.00 | −442.83 | 0.15 | 58 | 0.83 | 2030.00 | −442.83 | 1.05 | 72 | 2.26 | |
1700.00 | 392.83 | 34.24 | 97 | 17.48 | 1730.00 | 392.83 | 31.42 | 100 | 15.87 | |
5th Scanning | 2536.35 | −336.30 | 4.97 | 82 | 2.63 | |||||
2100.00 | 442.83 | 1.22 | 54 | 1.57 | 2121.21 | 421.62 | 1.70 | 71 | 2.42 | |
1800.00 | −392.83 | 2.05 | 100 | 1.72 | ||||||
2636.35 | 336.35 | −1.28 | 99 | 2.88 | ||||||
2200.00 | −442.83 | 2.90 | 73 | 2.40 | ||||||
1900.00 | 392.83 | 40.05 | 89 | 1.80 | ||||||
2736.35 | −336.35 | −2.68 | 97 | 3.29 | ||||||
2300.00 | 442.83 | 3.78 | 30 | 6.24 | 2300.00 | 421.83 | 4.40 | 71 | 2.25 | |
2000.00 | −392.83 | 2.28 | 82 | 2.34 | ||||||
2936.35 | 336.35 | −3.76 | 97 | 2.44 | ||||||
2400.00 | −442.83 | 2.63 | 95 | 1.86 | ||||||
2100.00 | 392.83 | 1.36 | 71 | 2.51 | ||||||
2936.35 | −336.35 | −2.63 | 97 | 2.37 | ||||||
2500.00 | 442.83 | 4.86 | 91 | 2.16 | ||||||
2200.00 | −392.83 | 3.19 | 94 | 1.85 | ||||||
2936.35 | 336.35 | −3.46 | 97 | 2.12 | ||||||
2500.00 | −442.83 | 3.73 | 78 | 2.25 | ||||||
2200.00 | 392.83 | 3.94 | 97 | 2.71 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Kim, J.-Y.; Park, M.-J.; Kim, S.; Shin, D. Vision-Guided Six-Legged Walking of Little Crabster Using a Kinect Sensor. Appl. Sci. 2022, 12, 2140. https://doi.org/10.3390/app12042140
Kim J-Y, Park M-J, Kim S, Shin D. Vision-Guided Six-Legged Walking of Little Crabster Using a Kinect Sensor. Applied Sciences. 2022; 12(4):2140. https://doi.org/10.3390/app12042140
Chicago/Turabian StyleKim, Jung-Yup, Min-Jong Park, Sungjun Kim, and Dongjun Shin. 2022. "Vision-Guided Six-Legged Walking of Little Crabster Using a Kinect Sensor" Applied Sciences 12, no. 4: 2140. https://doi.org/10.3390/app12042140
APA StyleKim, J.-Y., Park, M.-J., Kim, S., & Shin, D. (2022). Vision-Guided Six-Legged Walking of Little Crabster Using a Kinect Sensor. Applied Sciences, 12(4), 2140. https://doi.org/10.3390/app12042140