Design and Analysis of a Single—Camera Omnistereo Sensor for Quadrotor Micro Aerial Vehicles (MAVs) †
Abstract
:1. Introduction
1.1. Sensor Motivation
- Size and payload—In MAV applications, the sensor’s physical dimensions and weight are always a great concern due to payload constraints. Generally, MAVs require fewer and lighter sensors that are compactly designed, while larger robots (including high-payload UAVs) have greater freedom of sensor choice.
- Field-of-view (FOV)—Due to their omnidirectional motion model, MAVs require a simultaneous observation of the 3D surroundings. Conversely, most ground robots can safely rely upon narrow vision as their motion control on the plane is more stable.
1.2. Existing Range Sensors for MAVs
1.3. Related Work
1.4. Proposed Sensor
2. Sensor Design
2.1. Model Parameters
2.2. Single Viewpoint (SVP) Configuration for OmniStereo
2.3. Rig Size
3. Projective Geometry
3.1. Analytical Solutions to Projection (Forward)
3.2. Analytical Solutions to Back Projection
3.3. Field-of-View
- (a)
- (b)
- (c)
3.4. Spatial Resolution
4. Parameter Optimization and Prototyping
4.1. Optimization Constraints
- (a)
- geometrical constraints — including SVP and reflex constraints described by Equations (11), (12) and (2);
- (b)
- physical constraints — the rig’s dimensions, which include the mirrors radii as well as by-product parameters such as system height and mass ;
- (c)
- performance constraints — the spatial resolution and range from triangulation determined by parameters , , and ; the desired viewing angles for an optimal SROI field of view, .
4.1.1. Linear Constraints
- g1:
- In order to set the position of below the origin of the pinhole camera frame , the focal distance of mirror 2 must be larger than d (distance between and ),
- g2:
- Because the hyperboloidal mirror should reflect light towards its effective viewpoint without being occluded by the reflex mirror, mirror 1’s focal distance, , needs to exceed the placement of the reflex mirror,
- g3:
- The empirical constraint
4.1.2. Non-Linear Constraints
- g4:
- The AscTec Pelican quadrotor has a maximum payload of 650 (according to the manufacturer specifications [28]). Therefore, we must satisfy the system mass computed via Equation (16), such that
- g5:
- Similarly, we limit the system’s height obtained with Equation (15) by a height limit ,
- g6:
- The origin of coordinates for the camera frame is set at its viewpoint, . In order to fit the camera enclosure under mirror 2, it is realistic to position the focus on the vertical transverse axis at more than 5 away from :
- g7:
- Let be an acceptable upper-bound for angle , such that
- g8:
- Because we desire a larger view towards the ground from mirror 1, we empirically set as a lower-bound for the minimum elevation ,
- g9:
- In order to avoid occlusions with the MAV’s propellers while being capable to image objects located about 5 under the rig’s base and 20 away (horizontally) from the central axis, we limit mirror 2’s lowest angle by a lower-bound ,
- h1:
- With functions and defined in Equation (14), we set
4.2. Optimal Results
4.2.1. Optimality of Parameters and
4.2.2. Spatial Resolution Optimality
4.3. Prototypes
4.3.1. Synthetic Prototype (Simulation)
- (1)
- to acquire ground-truth 3D-scene information in order to evaluate the computed range by the omnistereo system (as explained in Section 5); and
- (2)
- to provide an almost accurate geometrical representation of the model by discounting some real-life computer vision artifacts such as assembly misalignments, glare from the support tube (motivating the use of standoffs on the real prototype), as well as the camera’s shallow depth-of-field. All of these artifacts can affect the quality of the real-life results shown in Section 6.
4.3.2. Real-Life Prototypes
5. 3D Sensing from Omnistereo Images
5.1. Panoramic Images
Stereo Matching on Panoramas
5.2. Range from Triangulation
5.2.1. Common Perpendicular Midpoint Triangulation Method
5.2.2. Range Variation
5.3. Triangulation Uncertainty Model
6. Experiment Results
6.1. Dense 3D Point Clouds
6.2. Sparse 3D Points from Features
6.3. Triangulation Evaluation
6.3.1. Evaluation of Synthetic Rig
6.3.2. Evaluation of Real-Life Rig
7. Discussion and Future Work
Acknowledgments
Author Contributions
Conflicts of Interest
Appendix A. Symbolic Notation
a point where post-subscript i as a unique identifier. | |
a reference frame or image space with origin . | |
The position vector of with respect to reference frame . | |
for homogeneous coordinates. | |
a 2D point or pixel position on image frame . | |
the magnitude (Euclidean norm) of . | |
A unit vector so . | |
a matrix, or in homogeneous coordinates. | |
a scalar-valued function that outputs some s. | |
a vector-valued function for the computation of . |
References
- Marani, R.; Renò, V.; Nitti, M.; D’Orazio, T.; Stella, E. A Compact 3D Omnidirectional Range Sensor of High Resolution for Robust Reconstruction of Environments. Sensors 2015, 15, 2283–2308. [Google Scholar] [CrossRef]
- Valenti, R.G.; Dryanovski, I.; Jaramillo, C.; Strom, D.P.; Xiao, J. Autonomous quadrotor flight using onboard RGB-D visual odometry. In Proceedings of the International Conference on Robotics and Automation (ICRA 2014), Hong Kong, China, 31 May–7 June 2014; pp. 5233–5238.
- Khoshelham, K.; Elberink, S.O. Accuracy and resolution of Kinect depth data for indoor mapping applications. Sensors 2012, 12, 1437–1454. [Google Scholar] [CrossRef] [PubMed]
- Payá, L.; Fernández, L.; Gil, A.; Reinoso, O. Map building and monte carlo localization using global appearance of omnidirectional images. Sensors 2010, 10, 11468–11497. [Google Scholar] [CrossRef] [PubMed]
- Berenguer, Y.; Payá, L.; Ballesta, M.; Reinoso, O. Position Estimation and Local Mapping Using Omnidirectional Images and Global Appearance Descriptors. Sensors 2015, 15, 26368–26395. [Google Scholar] [CrossRef] [PubMed]
- Hrabar, S.; Sukhatme, G. Omnidirectional vision for an autonomous helicopter. In Proceedings of the International Conference on Robotics and Automation (ICRA), Taipei, Taiwan, 14–19 September 2003; pp. 3602–3609.
- Hrabar, S. 3D path planning and stereo-based obstacle avoidance for rotorcraft UAVs. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Nice, France, 22–26 September 2008; pp. 807–814.
- Orghidan, R.; Mouaddib, E.M.; Salvi, J. Omnidirectional depth computation from a single image. In Proceedings of the IEEE International Conference on Robotics and Automation, Barcelona, Spain, 18–22 April 2005; pp. 1222–1227.
- Paniagua, C.; Puig, L.; Guerrero, J.J. Omnidirectional structured light in a flexible configuration. Sensors 2013, 13, 13903–13916. [Google Scholar] [CrossRef] [PubMed]
- Byrne, J.; Cosgrove, M.; Mehra, R. Stereo based obstacle detection for an unmanned air vehicle. In Proceedings of the International Conference on Robotics and Automation, Orlando, FL, USA, 15–19 May 2006.
- Smadja, L.; Benosman, R.; Devars, J. Hybrid stereo configurations through a cylindrical sensor calibration. Mach. Vis. Appl. 2006, 17, 251–264. [Google Scholar] [CrossRef]
- Hartley, R.; Zisserman, A. Multiple View Geometry in Computer Vision, 2nd ed.; Volume 2, Cambridge University Press: Cambridge, UK, 2004. [Google Scholar]
- Sturm, P.; Ramalingam, S.; Tardif, J.P.; Gasparini, S.; Barreto, J.P. Camera models and fundamental concepts used in geometric computer vision. Found. Trends®Comput. Graph. Vis. 2010, 6, 1–183. [Google Scholar] [CrossRef]
- Gluckman, J.; Nayar, S.K.; Thoresz, K.J. Real-Time Omnidirectional and Panoramic Stereo. Comput. Vis. Image Underst. 1998. [Google Scholar]
- Koyasu, H.; Miura, J.; Shirai, Y. Realtime omnidirectional stereo for obstacle detection and tracking in dynamic environments. In Proceedings of the International Conference on Intelligent Robots and Systems (IROS), Maui, HI, USA, 29 October–3 November 2001; Volume 1, pp. 31–36.
- Bajcsy, R.; Lin, S.S. High resolution catadioptric omni-directional stereo sensor for robot vision. In Proceedings of the 2003 IEEE International Conference on Robotics and Automation, Taipei, Taiwan, 14–19 September 2003; pp. 1694–1699.
- Cabral, E.E.; de Souza, J.C.J.; Hunold, M.C. Omnidirectional stereo vision with a hyperbolic double lobed mirror. In Proceedings of the 17th International Conference on Pattern Recognition (ICPR), Cambridge, UK, 23–26 August 2004; pp. 0–3.
- Su, L.; Zhu, F. Design of a novel stereo vision navigation system for mobile robots. In Proceedings of the IEEE Robotics and Biomimetics (ROBIO), Hong Kong, China, 5–9 July 2005; pp. 611–614.
- Mouaddib, E.M.; Sagawa, R. Stereovision with a single camera and multiple mirrors. In Proceedings of the International Conference on Robotics and Automation, Barcelona, Spain, 18–22 April 2005; pp. 800–805.
- Schönbein, M.; Kitt, B.; Lauer, M. Environmental Perception for Intelligent Vehicles Using Catadioptric Stereo Vision Systems. In Proceedings of the European Conference on Mobile Robots (ECMR), Örebro, Sweden, 7–9 September 2011; pp. 1–6.
- Yi, S.; Ahuja, N. An Omnidirectional Stereo Vision System Using a Single Camera. In Proceedings of the 18th International Conference on Pattern Recognition (ICPR’06), Hong Kong, China, 20–24 August 2006; pp. 861–865.
- Nayar, S.K.; Peri, V. Folded catadioptric cameras. In Proceedings of the 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Fort Collins, CO, USA, 23–25 June 1999; pp. 217–223.
- He, L.; Luo, C.; Zhu, F.; Hao, Y. Stereo Matching and 3D Reconstruction via an Omnidirectional Stereo Sensor. In Motion Planning; Number 60575024; In-Tech Education and Publishing: Vienna, Austria, 2008; pp. 123–142. [Google Scholar]
- Labutov, I.; Jaramillo, C.; Xiao, J. Generating near-spherical range panoramas by fusing optical flow and stereo from a single-camera folded catadioptric rig. Mach. Vis. Appl. 2011, 24, 1–12. [Google Scholar] [CrossRef]
- Swaminathan, R.; Grossberg, M.D.; Nayar, S.K. Caustics of catadioptric cameras. In Proceedings of the Eighth IEEE International Conference on Computer Vision (ICCV 2001), Vancouver, BC, Canada, 7–14 July 2001; Volume 2, pp. 2–9.
- Jang, G.; Kim, S.; Kweon, I. Single camera catadioptric stereo system. In Proceedings of the Workshop on Omnidirectional Vision, Camera Networks and Nonclassical Cameras (OMNIVIS2005), Beijing, China, 21 October 2005.
- Jaramillo, C.; Guo, L.; Xiao, J. A Single-Camera Omni-Stereo Vision System for 3D Perception of Micro Aerial Vehicles (MAVs). In Proceedings of the IEEE Conference on Industrial Electronics and Applications (ICIEA), Melbourne, Australia, 19–21 June 2013; Volume 10016.
- Ascending Technologies (AscTec). Available online: http://www.asctec.de/en/uav-uas-drones-rpas-roav/ (accessed on 23 May 2014).
- Baker, S.; Nayar, S.K. A theory of single-viewpoint catadioptric image formation. Int. J. Comput. Vis. 1999, 35, 175–196. [Google Scholar] [CrossRef]
- Nayar, S.K.; Baker, S. Catadioptric Image Formation. In Proceedings of the 1997 DARPA Image Understanding Workshop, New Orleans, LA, USA, May 1997; pp. 1431–1437.
- Gaspar, J.; Deccó, C.; Okamoto, J.J.; Santos-Victor, J.; Sistemas, I.D.; Pais, A.R.; Brazil, S.P. Constant resolution omnidirectional cameras. In Proceedings of the OMNIVIS’02 Workshop on Omni-directional Vision, Copenhagen, Denmark, 2 June 2002.
- Forsgren, A.; Gill, P.; Wright, M. Interior Methods for Nonlinear Optimization. Soc. Ind. Appl. Math. (SIAM Rev.) 2002, 44, 525–597. [Google Scholar] [CrossRef]
- Tuytelaars, T.T.; Mikolajczyk, K. Local Invariant Feature Detectors- A Survey. Found. Trends® in Comput. Graph. Vis. 2008, 3, 177–280. [Google Scholar] [CrossRef] [Green Version]
- Spacek, L. Coaxial Omnidirectional Stereopsis. Computer Vision-ECCV 2004; Springer Berlin Heidelberg: Berlin, Heidelberg, 2004; pp. 354–365. [Google Scholar]
- Bradski, G.; Kaehler, A. Learning OpenCV: Computer vision with the OpenCV library; O’Reilly Media, Inc.: Sebastopol, California, 2008. [Google Scholar]
- Hirschmüller, H. Stereo processing by semiglobal matching and mutual information. IEEE Trans. Pattern Anal. Mach. Intell. 2008, 30, 328–341. [Google Scholar] [CrossRef] [PubMed]
- Xiang, Z.; Dai, X.; Gong, X. Noncentral catadioptric camera calibration using a generalized unified model. Opt. Lett. 2013, 38, 1367–1369. [Google Scholar] [CrossRef]
- Bay, H.; Ess, A.; Tuytelaars, T.; Vangool, L. Speeded-Up Robust Features (SURF). Comput. Vis. Image Underst. 2008, 110, 346–359. [Google Scholar] [CrossRef]
- Scaramuzza, D.; Fraundorfer, F. Visual Odometry Part 1: The First 30 Years and Fundamentals. IEEE Robot. Autom. Mag. 2011, 18, 80–92. [Google Scholar] [CrossRef]
- Source Code Repository. Available online: https://github.com/ubuntuslave/omnistereo_sensor_design (accessed on 5 February 2016).
Parameter | Big Rig | Small Rig |
---|---|---|
131.61 | 108.92 | |
37.0 | 28.0 | |
123.49 | 104.59 | |
241.80 | 204.34 | |
233.68 | 200.00 | |
5.73 | 6.88 | |
9.74 | 11.47 |
Parameter | Big Rig | Small Rig |
---|---|---|
17.23 | 11.74 | |
7 | 7 | |
150.00 | 120.00 |
Vertex | [] | [] |
---|---|---|
93.5 | 144.4 | |
65.2 | 98.4 | |
763.4 | -170.3 |
[] | RMSE [] | SD [] |
---|---|---|
0.25 | 0.46 | 0.31 |
0.50 | 1.20 | 0.71 |
1.0 | 4.62 | 2.55 |
2.0 | 14.85 | 9.06 |
4.0 | 57.67 | 31.34 |
8.0 | 219.09 | 129.92 |
© 2016 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons by Attribution (CC-BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Jaramillo, C.; Valenti, R.G.; Guo, L.; Xiao, J. Design and Analysis of a Single—Camera Omnistereo Sensor for Quadrotor Micro Aerial Vehicles (MAVs). Sensors 2016, 16, 217. https://doi.org/10.3390/s16020217
Jaramillo C, Valenti RG, Guo L, Xiao J. Design and Analysis of a Single—Camera Omnistereo Sensor for Quadrotor Micro Aerial Vehicles (MAVs). Sensors. 2016; 16(2):217. https://doi.org/10.3390/s16020217
Chicago/Turabian StyleJaramillo, Carlos, Roberto G. Valenti, Ling Guo, and Jizhong Xiao. 2016. "Design and Analysis of a Single—Camera Omnistereo Sensor for Quadrotor Micro Aerial Vehicles (MAVs)" Sensors 16, no. 2: 217. https://doi.org/10.3390/s16020217
APA StyleJaramillo, C., Valenti, R. G., Guo, L., & Xiao, J. (2016). Design and Analysis of a Single—Camera Omnistereo Sensor for Quadrotor Micro Aerial Vehicles (MAVs). Sensors, 16(2), 217. https://doi.org/10.3390/s16020217