Fisheye Image Detection of Trees Using Improved YOLOX for Tree Height Estimation
Abstract
:1. Introduction
2. Materials and Methods
2.1. Establishment of the Measurement Model
- Set up measuring equipment. A smartphone with a fisheye lens is required to set up the measuring equipment.
- Acquire images. After training is complete, only one image of the tree under test needs to be collected.
- Extract extreme points. Deep learning methods can perform this step quickly and accurately.
- Build a tree height calculation model. This step only needs to be done once during the initial calculation.
- Calculate tree height. Obtain results and perform error analysis.
2.2. Improved Target Detection Network
3. Experimental Results and Analysis
3.1. Validation of Fisheye Lens Measurement Model
3.2. Tree Detection and Extreme Point Extraction
3.3. Tree Height Calculation
3.4. Wind Interference Experiment
4. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
Appendix A
Symbol or Abbreviation | Explanation |
---|---|
P | The target point in the world coordinate system. |
P’ | The imaging point corresponding to P in the camera coordinate system. |
r’ | The distance from the point P’ to the optical axis. |
f | The object square focal length of the optical system. |
w | The incident angle of the point P relative to the optical axis. |
L | The horizontal distance between the point in the world coordinate system and the center of the fisheye lens. |
λ | The distortion coefficient. |
θ | The azimuth of point P and the azimuth of point P’ in the camera coordinate system. |
mx and my | The amplification factors. |
kx and ky | The distortion coefficients of the fisheye image. |
h | The horizontal distance in the world coordinate system. |
H | The result was obtained by the measurement system model. |
CBMA | Convolutional Block Attention Module. |
DS | Source domain. |
TS | Source task. |
DT | Target domain. |
TT | Target task. |
fT | Target function. |
Fisheye Lens | Smartphone | ||||
---|---|---|---|---|---|
Attributes | Value | Unit | Attributes | Value | Unit |
Thread diameter | 17 | mm | Size | 148.9 × 71.1 × 8.5 | mm |
Angle | 180 | Pixel | 50 | million | |
Weight | 36 | g | Weight | 175 | g |
Resolution | 4096 × 4096 | dpi | Photo resolution | 8192 × 6144 | dpi |
References
- Huang, Y.D.; Li, M.Z.; Ren, S.Q.; Wang, M.J.; Cui, P.Y.J.B. Impacts of tree-planting pattern and trunk height on the airflow and pollutant dispersion inside a street canyon. Build. Environ. 2019, 165, 106385. [Google Scholar] [CrossRef]
- Calvo-Alvarado, J.C.; Mcdowell, N.G.; Waring, R.H.; Physiology, R.H.J.T. Allometric relationships predicting foliar biomass and leaf area:sapwood area ratio from tree height in five Costa Rican rain forest species. Tree Physiol. 2008, 28, 1601–1608. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Wang, Y.; Lehtomäki, M.; Liang, X.; Pyörälä, J.; Kukko, A.; Jaakkola, A.; Liu, J.; Feng, Z.; Chen, R.; Hyyppä, J. Is field-measured tree height as reliable as believed-A comparison study of tree height estimates from field measurement, airborne laser scanning and terrestrial laser scanning in a boreal forest. ISPRS J. Photogramm. Remote Sens. 2019, 147, 132–145. [Google Scholar] [CrossRef]
- Parent, J.R.; Volin, J.C. Assessing species-level biases in tree heights estimated from terrain-optimized leaf-off airborne laser scanner (ALS) data. Int. J. Remote Sens. 2015, 36, 2697–2712. [Google Scholar] [CrossRef]
- Calders, K.; Adams, J.; Armston, J.; Bartholomeus, H.; Bauwens, S.; Bentley, L.P.; Chave, J.; Danson, F.M.; Demol, M.; Disney, M.; et al. Terrestrial laser scanning in forest ecology: Expanding the horizon. Remote Sens. Environ. 2020, 251, 112102. [Google Scholar] [CrossRef]
- Kędra, K.; Barbeito, I.; Dassot, M.; Vallet, P.; Gazda, A.J. Single-image photogrammetry for deriving tree architectural traits in mature forest stands: A comparison with terrestrial laser scanning. Ann. For. Sci. 2019, 76, 5. [Google Scholar] [CrossRef] [Green Version]
- Eliopoulos, N.J.; Shen, Y.; Luong, N.M.; Vaastav, A.; Zhang, Y.; Shao, G.; Keith, W.; Lu, Y.-H. Rapid Tree Diameter Computation with Terrestrial Stereoscopic Photogrammetry. J. For. 2020, 118, 355–361. [Google Scholar] [CrossRef]
- Zagalikis, G.; Cameron, A.D.; Miller, D.R. The application of digital photogrammetry and image analysis techniques to derive tree and stand characteristics. Can. J. For. Res. 2005, 35, 1224–1237. [Google Scholar] [CrossRef]
- Zhang, Z. A Flexible New Technique for Camera Calibration. IEEE Trans. Pattern Anal. Mach. Intell. 2000, 22, 1330–1334. [Google Scholar] [CrossRef] [Green Version]
- Scaramuzza, D.; Martinelli, A.; Siegwart, R. A Toolbox for Easily Calibrating Omnidirectional Cameras. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots & Systems, Benjing, China, 9 February 2009. [Google Scholar]
- Shen, F.; Qin, F.; Zhang, Z.; Xu, D.; Wu, W. Automated Pose Measurement Method Based on Multivision and Sensor Collaboration for Slice Micro Device. IEEE Trans. Ind. Electron. 2020, 68, 498. [Google Scholar] [CrossRef]
- Zhou, Y.; Li, Q.; Wu, Y.; Ma, Y.; Wang, C.J. Trinocular vision and spatial prior based method for ground clearance measurement of transmission lines. Appl. Opt. 2021, 60, 2422–2433. [Google Scholar] [CrossRef] [PubMed]
- Mao, J.; Huang, W.; Sheng, W.J. Target distance measurement method using monocular vision. IET Image Process. 2020, 14, 3181–3187. [Google Scholar]
- Isa, N.A.M.; Mat, N.A.; Salamah, S.A.; Samy, A.; Ngah, U.K.; Kaithum, U.J. Adaptive Fuzzy Moving K-means Clustering Algorithm for Image Segmentation. IEEE Trans. Consum. Electron. 2009, 55, 2145–2153. [Google Scholar]
- Jaisakthi, S.M.; Murugaiyan, S.; Mirunalini, P.; Aravindan, C. Automated skin lesion segmentation of dermoscopic images using GrabCut and k-means algorithms. IET Comput. Vis. 2018, 12, 1088–1095. [Google Scholar] [CrossRef]
- Liu, Z.-y.; Ding, F.; Xu, Y.; Han, X. Background dominant colors extraction method based on color image quick fuzzy c-means clustering algorithm. Def. Technol. 2020, 17, 1782–1790. [Google Scholar] [CrossRef]
- Dhal, K.G.; Das, A.; Ray, S.; Gálvez, J. Randomly Attracted Rough Firefly Algorithm for histogram based fuzzy image clustering. Knowl.-Based Syst. 2021, 216, 106814. [Google Scholar] [CrossRef]
- Redmon, J.; Divvala, S.; Girshick, R.; Farhadi, A.J.I. You Only Look Once: Unified, Real-Time Object Detection. arXiv 2016, arXiv:1506.02640. [Google Scholar]
- Xu, H.; Guo, M.; Nedjah, N.; Zhang, J.; Li, P. Vehicle and Pedestrian Detection Algorithm Based on Lightweight YOLOv3-Promote and Semi-Precision Acceleration. IEEE Trans. Intell. Transp. Syst. 2022, 1–12. [Google Scholar] [CrossRef]
- Wang, D.; He, D. Channel pruned YOLO V5s-based deep learning approach for rapid and accurate apple fruitlet detection before fruit thinning. Biosyst. Eng. 2021, 210, 271–281. [Google Scholar] [CrossRef]
- Mohamadipanah, H.; Kearse, L.D.; Witt, A.; Wise, B.; Pugh, C. Can Deep Learning Algorithms Help Identify Surgical Workflow and Techniques? J. Surg. Res. 2021, 268, 318–325. [Google Scholar] [CrossRef]
- Bochkovskiy, A.; Wang, C.Y.; Liao, H.Y.M. YOLOv4: Optimal Speed and Accuracy of Object Detection. arXiv 2020, arXiv:2004.10934. [Google Scholar]
- Ge, Z.; Liu, S.; Wang, F.; Li, Z.; Sun, J. YOLOX: Exceeding YOLO Series in 2021. arXiv 2021, arXiv:2107.08430. [Google Scholar]
- Wang, N.; Ri, K.; Liu, H.; Zhao, X. Structural Displacement Monitoring Using Smartphone Camera and Digital Image Correlation. IEEE Sens. J. 2018, 18, 4664–4672. [Google Scholar] [CrossRef]
- Yu, L.; Tao, R.; Lubineau, G. Accurate 3D Shape, Displacement and Deformation Measurement Using a Smartphone. Sensors 2019, 19, 719. [Google Scholar] [CrossRef] [Green Version]
- Yu, L.; Lubineau, G. A smartphone camera and built-in gyroscope based application for non-contact yet accurate off-axis structural displacement measurements. Measurement 2020, 167, 108449. [Google Scholar] [CrossRef]
- Groote, F.D.; Vandevyvere, S.; Vanhevel, F.; Xivry, J.J.O.D.J.G. Validation of a smartphone embedded inertial measurement unit for measuring postural stability in older adults. Gait. Posture 2021, 84, 17–23. [Google Scholar] [CrossRef] [PubMed]
- Song, J.; Zhao, Y.; Chi, Z.; Ma, Q.; Yin, T.; Zhang, X. Improved FCM algorithm for fisheye image cluster analysis for tree height calculation. Math. Biosci. Eng. 2021, 18, 7806–7836. [Google Scholar] [CrossRef] [PubMed]
- Woo, S.; Park, J.; Lee, J.Y.; Kweon, I.S.J.S. Cham. CBAM: Convolutional Block Attention Module. arXiv 2018, arXiv:1807.06521. [Google Scholar]
- Opbroek, A.V.; Ikram, M.A.; Vernooij, M.W.; de Bruijne, M. Transfer learning improves supervised image segmentation across imaging protocols. IEEE Trans. Med. Imaging 2015, 34, 1018. [Google Scholar] [CrossRef] [Green Version]
Measuring Distance (mm) | Corner Point | Pixel Coordinate | Global Coordinates | Calculated Value (mm) | Measured Value (mm) | Relative Error (%) |
---|---|---|---|---|---|---|
320 | A | (898, 687) | (−285.0330, −375.9156) | 656.3162 | 650 | 0.9717 |
B | (810, 2290) | (−300.6751, 280.2142) | ||||
612 | A | (1091, 966) | (−307.0069, −388.4370) | 656.2374 | 650 | 0.9896 |
B | (1062, 2021) | (−310.4134, 267.7916) | ||||
993 | A | (1179, 1160) | (−375.2495, −395.9568) | 657.1972 | 650 | 1.1073 |
B | (1150, 1860) | (−394.4027, 260.9612) | ||||
1502 | A | (1374, 1282) | (−274.3518, −401.1987) | 652.1023 | 650 | 0.3234 |
B | (1358, 1766) | (−292.3145, 250.6562) | ||||
2041 | A | (1405, 1346) | (−312.3874, −422.4221) | 654.4923 | 650 | 0.6911 |
B | (1389, 1707) | (−338.4833, 231.5497) | ||||
320 | A | (898, 687) | (−285.0330, −375.9156) | 647.8777 | 650 | 0.3265 |
C | (2414, 782) | (362.2036, −347.1019) | ||||
612 | A | (1091, 966) | (−307.0069, −388.4370) | 655.2693 | 650 | 0.8107 |
C | (2126, 1000) | (347.9883, −369.4865) | ||||
993 | A | (1179, 1160) | (−375.2495, −395.9568) | 644.3722 | 650 | 0.8658 |
C | (1870, 1195) | (267.0121, −352.6874) | ||||
1502 | A | (1374, 1282) | (−274.3518, −401.1987) | 642.8080 | 650 | 1.1065 |
C | (1850, 1301) | (368.0232, −377.6071) | ||||
2041 | A | (1405, 1346) | (−312.3874, −422.4221) | 641.7176 | 650 | 1.2742 |
C | (1760, 1365) | (328.3721, −387.3684) | ||||
320 | A | (898, 687) | (−285.0330, −375.9156) | 928.6904 | 919.238 | 1.0282 |
D | (2410, 2314) | (344.7874, 306.5740) | ||||
612 | A | (1091, 966) | (−307.0069, −388.4370) | 921.6590 | 919.238 | 0.2633 |
D | (2086, 2065) | (308.7236, 297.3699) | ||||
993 | A | (1179, 1160) | (−375.2495, −395.9568) | 928.6176 | 919.238 | 1.0203 |
D | (1850, 1890) | (244.6171, 281.7389) | ||||
1502 | A | (1374, 1282) | (−274.3518, −401.1987) | 912.5988 | 919.238 | 0.7223 |
D | (1829, 1785) | (335.5699, 277.6474) | ||||
2041 | A | (1405, 1346) | (−312.3874, −422.4221) | 911.4613 | 919.238 | 0.8461 |
D | (1741, 1723) | (291.7372, 260.0696) | ||||
Mean value | 0.8231 |
Parameters | Value |
---|---|
Input size | × 640 |
Output size | × 640 |
Learning rate | adaptive |
Batch size | 8 |
Epoch | 300 |
Model | Epoch | P (%) | R (%) | F1 | AP (%) |
---|---|---|---|---|---|
YOLOX-s | 300 | 92.57 | 95.90 | 0.94 | 96.27 |
YOLOX-tiny | 300 | 93.03 | 95.90 | 0.94 | 97.26 |
Attention-YOLOX-tiny | 300 | 92.27 | 97.95 | 0.95 | 97.80 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Song, J.; Zhao, Y.; Song, W.; Zhou, H.; Zhu, D.; Huang, Q.; Fan, Y.; Lu, C. Fisheye Image Detection of Trees Using Improved YOLOX for Tree Height Estimation. Sensors 2022, 22, 3636. https://doi.org/10.3390/s22103636
Song J, Zhao Y, Song W, Zhou H, Zhu D, Huang Q, Fan Y, Lu C. Fisheye Image Detection of Trees Using Improved YOLOX for Tree Height Estimation. Sensors. 2022; 22(10):3636. https://doi.org/10.3390/s22103636
Chicago/Turabian StyleSong, Jiayin, Yue Zhao, Wenlong Song, Hongwei Zhou, Di Zhu, Qiqi Huang, Yiming Fan, and Chao Lu. 2022. "Fisheye Image Detection of Trees Using Improved YOLOX for Tree Height Estimation" Sensors 22, no. 10: 3636. https://doi.org/10.3390/s22103636
APA StyleSong, J., Zhao, Y., Song, W., Zhou, H., Zhu, D., Huang, Q., Fan, Y., & Lu, C. (2022). Fisheye Image Detection of Trees Using Improved YOLOX for Tree Height Estimation. Sensors, 22(10), 3636. https://doi.org/10.3390/s22103636