Orthographic Video Map Generation Considering 3D GIS View Matching
Abstract
1. Introduction
2. Related Work
2.1. Homography Method
2.2. Camera Parameter Method
2.3. Image Matching
3. Research Methodology
3.1. Video and 3D GIS Synchronizing
3.1.1. Camera Parameter Estimation
3.1.2. Video Synchronization with 3D GIS View
3.2. View Matching
3.2.1. Feature Point Detection and Matching
3.2.2. Homography Matrix Calculation
3.3. Video Orthographic Generation
3.3.1. Video Frame Homography Transformation and Semantic Segmentation
3.3.2. Pixel Coordinates to 3D Coordinates
3.3.3. Generate Color Point Cloud
3.3.4. Orthorectified Video Generation
4. Experiments and Analysis
4.1. Experimental Environment and Data
4.2. Experimental Analysis
4.2.1. Synchronization of Video and 3D GIS View
4.2.2. View Matching Analysis
4.2.3. Video Orthography
- (1)
- Video frame homography transformation and semantic segmentation
- (2)
- Orthophoto generation
4.2.4. Comparative Analysis
- (1)
- Comparative analysis of visualization effects
- (2)
- Comparative analysis of control point deviations
4.2.5. Generation of Orthographic Video Map Based on Real-Scene 3D Model
5. Conclusions and Discussion
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Li, D.; Xu, X.; Shao, Z. On geospatial information science in the era of loE. Acta Geod. Cartogr. Sin. 2022, 51, 1–8. [Google Scholar] [CrossRef]
- Milosavljević, A.; Dimitrijević, A.; Rančić, D. GIS-augmented video surveillance. Int. J. Geogr. Inf. Sci. 2010, 24, 1415–1433. [Google Scholar] [CrossRef]
- Sourimant, G.; Morin, L.; Bouatouch, K. GPS, GIS and video registration for building reconstruction. In Proceedings of the 2007 IEEE International Conference on Image Processing, San Antonio, TX, USA, 16 September–19 October 2007; IEEE: Piscataway, NJ, USA, 2007; Volume 6, pp. VI-401–VI-404. [Google Scholar] [CrossRef]
- Chen, H.; Qi, Z.; Han, X.; Feng, N.; Li, C. Research and application on key technologies of natural resources intelligent monitoring with tower-based video. Nat. Resour. Informatiz. 2023, 2023, 1–6. [Google Scholar] [CrossRef]
- Sankaranarayanan, K.; Davis, J.W. A fast linear registration framework for multi-camera GIS coordination. In Proceedings of the 2008 IEEE Fifth International Conference on Advanced Video and Signal Based Surveillance, Santa Fe, NM, USA, 1–3 September 2008; IEEE: Piscataway, NJ, USA, 2008; pp. 245–251. [Google Scholar] [CrossRef]
- Milosavljević, A.; Rančić, D.; Dimitrijević, A.; Predić, B.; Mihajlović, V. Integration of GIS and video surveillance. Int. J. Geogr. Inf. Sci. 2016, 30, 2089–2107. [Google Scholar] [CrossRef]
- Xie, Y.; Wang, M.; Liu, X.; Wang, Z.; Mao, B.; Wang, F.; Wang, X. Spatiotemporal retrieval of dynamic video object trajectories in geographical scenes. Trans. GIS 2021, 25, 450–467. [Google Scholar] [CrossRef]
- Luo, X.; Wang, Y.; Dong, J.; Li, Z.; Yang, Y.; Tang, K.; Huang, T. Complete trajectory extraction for moving targets in traffic scenes that considers multi-level semantic features. Int. J. Geogr. Inf. Sci. 2023, 37, 913–937. [Google Scholar] [CrossRef]
- Zhang, X.; Shi, X.; Luo, X.; Sun, Y.; Zhou, Y. Real-time web map construction based on multiple cameras and GIS. ISPRS Int. J. Geo Inf. 2021, 10, 803. [Google Scholar] [CrossRef]
- Shao, Z.; Li, C.; Li, D.; Altan, O.; Zhang, L.; Ding, L. An accurate matching method for projecting vector data into surveillance video to monitor and protect cultivated land. ISPRS Int. J. Geo Inf. 2020, 9, 448. [Google Scholar] [CrossRef]
- Zhang, X.; Liu, X.; Wang, S.; Liu, Y. Mutual Mapping Between Surveillance Video and 2D Geospatial Data. Geomat. Inf. Sci. Wuhan Univ. 2015, 40, 1130–1136. [Google Scholar] [CrossRef]
- Zhang, Z. A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 2002, 22, 1330–1334. [Google Scholar] [CrossRef]
- Tsai, R. A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses. IEEE J. Robot. Autom. 2003, 3, 323–344. [Google Scholar] [CrossRef]
- Hartley, R.I. Self-calibration of stationary cameras. Int. J. Comput. Vis. 1997, 22, 5–23. [Google Scholar] [CrossRef]
- Yang, C.; Wang, W.; Hu, Z. An active vision based camera intrinsic parameters self-calibration technique. Chin. J. Comput. 1998, 21, 428–435. [Google Scholar] [CrossRef]
- Hou, W.; Shang, T.; Ding, M. Self-Calibration of a Camera with a Non-Linear Model. Chin. J. Comput. 2002, 25, 276–283. [Google Scholar] [CrossRef]
- Hartley, R.I. Projective reconstruction and invariants from multiple images. IEEE Trans. Pattern Anal. Mach. Intell. 1994, 16, 1036–1041. [Google Scholar] [CrossRef]
- Pollefeys, M.; Van Gool, L.; Oosterlinck, A. The modulus constraint: A new constraint self-calibration. In Proceedings of the 13th International Conference on Pattern Recognition, Vienna, Austria, 25–29 August 1996; IEEE: Piscataway, NJ, USA, 1996; Volume 1, pp. 349–353. [Google Scholar] [CrossRef]
- Agapito, L.; Hayman, E.; Reid, I. Self-calibration of rotating and zooming cameras. Int. J. Comput. Vis. 2001, 45, 107–127. [Google Scholar] [CrossRef]
- Hemayed, E.E. A survey of camera self-calibration. In Proceedings of the IEEE Conference on Advanced Video and Signal Based Surveillance, Miami, FL, USA, 22 July 2003; IEEE: Piscataway, NJ, USA, 2003; pp. 351–357. [Google Scholar] [CrossRef]
- Triggs, B. Autocalibration and the absolute quadric. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Juan, PR, USA, 17–19 June 1997; IEEE: Piscataway, NJ, USA, 1997; pp. 609–614. [Google Scholar] [CrossRef]
- Heyden, A.; Astrom, K. Flexible calibration: Minimal cases for auto-calibration. In Proceedings of the Seventh IEEE International Conference on Computer Vision, Kerkyra, Greece, 20–27 September 1999; IEEE: Piscataway, NJ, USA, 1999; Volume 1, pp. 350–355. [Google Scholar] [CrossRef]
- Hartley, R.I.; Hayman, E.; de Agapito, L.; Reid, I. Camera calibration and the search for infinity. In Proceedings of the Seventh IEEE International Conference on Computer Vision, Kerkyra, Greece, 20–27 September 1999; IEEE: Piscataway, NJ, USA, 1999; Volume 1, pp. 510–517. [Google Scholar] [CrossRef]
- Ge, D.; Yao, X.; Hu, C.; Lian, Z. Nonlinear camera model calibrated by neural network and adaptive genetic-annealing algorithm. J. Intell. Fuzzy Syst. 2014, 27, 2243–2255. [Google Scholar] [CrossRef]
- Woo, D.M.; Park, D.C. An efficient method for camera calibration using multilayer perceptron type neural network. In Proceedings of the 2009 International Conference on Future Computer and Communication, Kuala Lumpar, Malaysia, 3–5 April 2009; IEEE: Piscataway, NJ, USA, 2009; pp. 358–362. [Google Scholar] [CrossRef]
- Woo, D.M.; Park, D.C. Implicit camera calibration using MultiLayer perceptron type neural network. In Proceedings of the 2009 First Asian Conference on Intelligent Information and Database Systems, Dong hoi, Vietnam, 1–3 April 2009; IEEE: Piscataway, NJ, USA, 2009; pp. 313–317. [Google Scholar] [CrossRef]
- Duan, Q.; Wang, Z.; Huang, J.; Xing, C.; Li, Z.; Qi, M.; Gao, J.; Ai, S. A deep-learning based high-accuracy camera calibration method for large-scale scene. Precis. Eng. 2024, 88, 464–474. [Google Scholar] [CrossRef]
- Li, B.; Wang, X.; Gao, Q.; Song, Z.; Zou, C.; Liu, S. A 3D scene information enhancement method applied in augmented reality. Electronics 2022, 11, 4123. [Google Scholar] [CrossRef]
- Li, C.; Liu, Z.; Zhao, Z.; Dai, Z. A fast fusion object determination method for multi-path video and three-dimensional GIS scene. Acta Geod. Cartogr. Sin. 2020, 49, 632. [Google Scholar] [CrossRef]
- Wang, Y.; Yuan, Y.; Lei, Z.; Wang, Y.; Yuan, Y.; Lei, Z. Fast SIFT feature matching algorithm based on geometric transformation. IEEE Access 2020, 8, 88133–88140. [Google Scholar] [CrossRef]
- Qi, F.; Weihong, X.; Qiang, L. Research of image matching based on improved SURF algorithm. TELKOMNIKA Indones. J. Electr. Eng. 2014, 12, 1395–1402. [Google Scholar] [CrossRef]
- Li, S.; Wang, Q.; Li, J. Improved ORB matching algorithm based on adaptive threshold. J. Phys. Conf. Ser. 2021, 1871, 012151. [Google Scholar] [CrossRef]
- Zhang, J.; Zhou, H.; Niu, Y.; Lv, J.; Chen, J.; Cheng, Y. CNN and multi-feature extraction based denoising of CT images. Biomed. Signal Process. Control 2021, 67, 102545. [Google Scholar] [CrossRef]
- Hong, Y.; Li, D.; Luo, S.; Chen, X.; Yang, Y.; Wang, M. An improved end-to-end multi-target tracking method based on transformer self-attention. Remote Sens. 2022, 14, 6354. [Google Scholar] [CrossRef]
- Fujimoto, S.; Matsunaga, N. Deep feature-based RGB-D odometry using SuperPoint and SuperGlue. Procedia Comput. Sci. 2023, 227, 1127–1134. [Google Scholar] [CrossRef]
- Shen, X.; Cai, Z.; Yin, W.; Müller, M.; Li, Z.; Wang, K.; Chen, X.Z.; Wang, C. GIM: Learning generalizable image matcher from internet videos. arXiv 2024, arXiv:2402.11095. [Google Scholar] [CrossRef]
- Vishnu, C.; Khandelwal, J.; Mohan, C.K.; Reddy, C.L. EVAA—Exchange vanishing adversarial attack on LiDAR point clouds in autonomous vehicles. IEEE Trans. Geosci. Remote Sens. 2023, 61, 5703410. [Google Scholar] [CrossRef]
- Chen, L.; Zhu, Y.; Papandreou, G.; Schroff, F.; Adam, H. Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany, 8–14 September 2018; pp. 801–818. [Google Scholar] [CrossRef]
ID | Lon | Lat | Alt | h | Zs | Wvid | Hvid |
---|---|---|---|---|---|---|---|
Camera1 | 112.55 | 25.18 | 408.96 | 11.94 | 420.90 | 752 | 566 |
Camera2 | 114.03 | 32.14 | 77.35 | 30 | 107.35 | 1000 | 750 |
ID | Xo′ | Yo′ | Ho′ | Ageo | Tgeo | f | VFOV |
---|---|---|---|---|---|---|---|
Camera1 | 112.54 | 25.17 | 4292 | −142 | −7 | 101.74 | 60 |
Camera2 | 114.03 | 32.14 | 76.94 | 13 | −30 | 136.62 | 60 |
ID | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 |
---|---|---|---|---|---|---|---|---|---|
dis (m) | 0.30 | 0.17 | 0.18 | 3.46 | 4.79 | 0.23 | 0.64 | 0.28 | 0.49 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Published by MDPI on behalf of the International Society for Photogrammetry and Remote Sensing. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Zhang, X.; Meng, X.; Zhang, L.; Ling, X.; Yang, S. Orthographic Video Map Generation Considering 3D GIS View Matching. ISPRS Int. J. Geo-Inf. 2025, 14, 398. https://doi.org/10.3390/ijgi14100398
Zhang X, Meng X, Zhang L, Ling X, Yang S. Orthographic Video Map Generation Considering 3D GIS View Matching. ISPRS International Journal of Geo-Information. 2025; 14(10):398. https://doi.org/10.3390/ijgi14100398
Chicago/Turabian StyleZhang, Xingguo, Xiangfei Meng, Li Zhang, Xianguo Ling, and Sen Yang. 2025. "Orthographic Video Map Generation Considering 3D GIS View Matching" ISPRS International Journal of Geo-Information 14, no. 10: 398. https://doi.org/10.3390/ijgi14100398
APA StyleZhang, X., Meng, X., Zhang, L., Ling, X., & Yang, S. (2025). Orthographic Video Map Generation Considering 3D GIS View Matching. ISPRS International Journal of Geo-Information, 14(10), 398. https://doi.org/10.3390/ijgi14100398