Search Results (15)

Reflectance anisotropy in remote sensing images can complicate the interpretation of spectral signature, and extracting precise structural information under these pixels is a promising approach. Low-altitude unmanned aerial vehicle (UAV) systems can capture high-resolution imagery even to centimeter-level detail, potentially simplifying the characterization of leaf anisotropic reflectance. We proposed a novel maize point cloud generation method that combines an advanced UAV cross-circling oblique (CCO) photography route with the Structure from the Motion-Multi-View Stereo (SfM-MVS) algorithm. A multi-spectral point cloud was then generated by fusing multi-spectral imagery with the point cloud using a DSM-based approach. The Rahman–Pinty–Verstraete (RPV) model was finally applied to establish maize leaf-level anisotropic reflectance models. Our results indicated a high degree of similarity between measured and estimated maize structural parameters (R² = 0.89 for leaf length and 0.96 for plant height) based on accurate point cloud data obtained from the CCO route. Most data points clustered around the principal plane due to a constant angle between the sun and view vectors, resulting in a limited range of view azimuths. Leaf reflectance anisotropy was characterized by the RPV model with R² ranging from 0.38 to 0.75 for five wavelength bands. These findings hold significant promise for promoting the decoupling of plant structural information and leaf optical characteristics within remote sensing data. Full article

With FaSS-MVS, we present a fast, surface-aware semi-global optimization approach for multi-view stereo that allows for rapid depth and normal map estimation from monocular aerial video data captured by unmanned aerial vehicles (UAVs). The data estimated by FaSS-MVS, in turn, facilitate online 3D mapping, meaning that a 3D map of the scene is immediately and incrementally generated as the image data are acquired or being received. FaSS-MVS is composed of a hierarchical processing scheme in which depth and normal data, as well as corresponding confidence scores, are estimated in a coarse-to-fine manner, allowing efficient processing of large scene depths, such as those inherent in oblique images acquired by UAVs flying at low altitudes. The actual depth estimation uses a plane-sweep algorithm for dense multi-image matching to produce depth hypotheses from which the actual depth map is extracted by means of a surface-aware semi-global optimization, reducing the fronto-parallel bias of Semi-Global Matching (SGM). Given the estimated depth map, the pixel-wise surface normal information is then computed by reprojecting the depth map into a point cloud and computing the normal vectors within a confined local neighborhood. In a thorough quantitative and ablative study, we show that the accuracy of the 3D information computed by FaSS-MVS is close to that of state-of-the-art offline multi-view stereo approaches, with the error not even an order of magnitude higher than that of COLMAP. At the same time, however, the average runtime of FaSS-MVS for estimating a single depth and normal map is less than 14% of that of COLMAP, allowing us to perform online and incremental processing of full HD images at 1–2 Hz. Full article

Deep learning techniques are increasingly applied to point cloud semantic segmentation, where single-modal point cloud often suffers from accuracy-limiting confusion phenomena. Moreover, some networks with image and LiDAR data lack an efficient fusion mechanism, and the occlusion of images may do harm to the segmentation accuracy of a point cloud. To overcome the above issues, we propose the integration of multi-modal data to enhance network performance, addressing the shortcomings of existing feature-fusion strategies that neglect crucial information and struggle with matching modal features effectively. This paper introduces the Multi-View Guided Point Cloud Semantic Segmentation Model (MVG-Net), which extracts multi-scale and multi-level features and contextual data from urban aerial images and LiDAR, and then employs a multi-view image feature-aggregation module to capture highly correlated texture information with the spatial and channel attentions of point-wise image features. Additionally, it incorporates a fusion module that uses image features to instruct point cloud features for stressing key information. We present a new dataset, WK2020, which combines multi-view oblique aerial images with LiDAR point cloud to validate segmentation efficacy. Our method demonstrates superior performance, especially in building segmentation, achieving an F1 score of 94.6% on the Vaihingen Dataset—the highest among the methods evaluated. Furthermore, MVG-Net surpasses other networks tested on the WK2020 Dataset. Compared to backbone network for single point modality, our model achieves overall accuracy improvement of 5.08%, average F1 score advancement of 6.87%, and mean Intersection over Union (mIoU) betterment of 7.9%. Full article

Target geolocation in long-range oblique photography (LOROP) is a challenging study due to the fact that measurement errors become more evident with increasing shooting distance, significantly affecting the calculation results. This paper introduces a novel high-accuracy target geolocation method based on multi-view observations. Unlike the usual target geolocation methods, which heavily depend on the accuracy of GNSS (Global Navigation Satellite System) and INS (Inertial Navigation System), the proposed method overcomes these limitations and demonstrates an enhanced effectiveness by utilizing multiple aerial images captured at different locations without any additional supplementary information. In order to achieve this goal, camera optimization is performed to minimize the errors measured by GNSS and INS sensors. We first use feature matching between the images to acquire the matched keypoints, which determines the pixel coordinates of the landmarks in different images. A map-building process is then performed to obtain the spatial positions of these landmarks. With the initial guesses of landmarks, bundle adjustment is used to optimize the camera parameters and the spatial positions of the landmarks. After the camera optimization, a geolocation method based on line-of-sight (LOS) is used to calculate the target geolocation based on the optimized camera parameters. The proposed method is validated through simulation and an experiment utilizing unmanned aerial vehicle (UAV) images, demonstrating its efficiency, robustness, and ability to achieve high-accuracy target geolocation. Full article

To address the incomplete image data collection of close-to-ground structures, such as bridge piers and local features like the suspension cables in bridges, obtained from single unmanned aerial vehicle (UAV) oblique photography and the difficulty in acquiring point cloud data for the top structures of bridges using single terrestrial laser scanners (TLSs), as well as the lack of textural information in TLS point clouds, this study aims to establish a high-precision, complete, and realistic bridge model by integrating UAV image data and TLS point cloud data. Using a particular large-scale dual-track bridge as a case study, the methodology involves aerial surveys using a DJI Phantom 4 RTK for comprehensive image capture. We obtain 564 images circling the bridge arches, 508 images for orthorectification, and 491 images of close-range side views. Subsequently, all images, POS data, and ground control point information are imported into Context Capture 2023 software for aerial triangulation and multi-view image dense matching to generate dense point clouds of the bridge. Additionally, ground LiDAR scanning, involving the placement of six scanning stations both on and beneath the bridge, was conducted and the point cloud data from each station are registered in Trimble Business Center 5.5.2 software based on identical feature points. Noise point clouds are then removed using statistical filtering techniques. The integration of UAV image point clouds with TLS point clouds is achieved using the iterative closest point (ICP) algorithm, followed by the creation of a TIN model and texture mapping using Context Capture 2023 software. The effectiveness of the integrated modeling is verified by comparing the geometric accuracy and completeness of the images with those obtained from a single UAV image-based model. The integrated model is used to generate cross-sectional profiles of the dual-track bridge, with detailed annotations of boundary dimensions. Structural inspections reveal honeycomb surfaces and seepage in the bridge piers, as well as painted rust and cracks in the arch ribs. The geometric accuracy of the integrated model in the X, Y, and Z directions is 1.2 cm, 0.8 cm, and 0.9 cm, respectively, while the overall 3D model accuracy is 1.70 cm. This method provides technical reference for the reconstruction of three-dimensional point cloud bridge models. Through 3D reconstruction, railway operators can better monitor and assess the condition of bridge structures, promptly identifying potential defects and damages, thus enabling the adoption of necessary maintenance and repair measures to ensure the structural safety of the bridges. Full article

The research and development of an intelligent magnetic levitation transportation system has become an important research branch of the current intelligent transportation system (ITS), which can provide technical support for state-of-the-art fields such as intelligent magnetic levitation digital twin. First, we applied unmanned aerial vehicle oblique photography technology to acquire the magnetic levitation track image data and preprocessed them. Then, we extracted the image features and matched them based on the incremental structure from motion (SFM) algorithm, recovered the camera pose parameters of the image data and the 3D scene structure information of key points, and optimized the bundle adjustment to output 3D magnetic levitation sparse point clouds. Then, we applied multiview stereo (MVS) vision technology to estimate the depth map and normal map information. Finally, we extracted the output of the dense point clouds that can precisely express the physical structure of the magnetic levitation track, such as turnout, turning, linear structures, etc. By comparing the dense point clouds model with the traditional building information model, experiments verified that the magnetic levitation image 3D reconstruction system based on the incremental SFM and MVS algorithm has strong robustness and accuracy and can express a variety of physical structures of magnetic levitation track with high accuracy. Full article

Currently, there is a rapid trend in the production of airborne sensors consisting of multi-view cameras or hybrid sensors, i.e., a LiDAR scanner coupled with one or multiple cameras to enrich the data acquisition in terms of colors, texture, completeness of coverage, accuracy, etc. However, the current UAV hybrid systems are mainly equipped with a single camera that will not be sufficient to view the facades of buildings or other complex objects without having double flight paths with a defined oblique angle. This entails extensive flight planning, acquisition duration, extra costs, and data handling. In this paper, a multi-view camera system which is similar to the conventional Maltese cross configurations used in the standard aerial oblique camera systems is simulated. This proposed camera system is integrated with a multi-beam LiDAR to build an efficient UAV hybrid system. To design the low-cost UAV hybrid system, two types of cameras are investigated and proposed, namely the MAPIR Survey and the SenseFly SODA, integrated with a multi-beam digital Ouster OS1-32 LiDAR sensor. Two simulated UAV flight experiments are created with a dedicated methodology and processed with photogrammetric methods. The results show that with a flight speed of 5 m/s and an image overlap of 80/80, an average density of up to 1500 pts/m² can be achieved with adequate facade coverage in one-pass flight strips. Full article

Unmanned Aerial Vehicles (UAVs) are a novel technology for landform investigations, monitoring, as well as evolution analyses of long−term repeated observation. However, impacted by the sophisticated topographic environment, fluctuating terrain and incomplete field observations, significant differences have been found between 3D measurement accuracy and the Digital Surface Model (DSM). In this study, the DJI Phantom 4 RTK UAV was adopted to capture images of complex pit-rim landforms with significant elevation undulations. A repeated observation data acquisition scheme was proposed for a small amount of oblique-view imaging, while an ortho-view observation was conducted. Subsequently, the 3D scenes and DSMs were formed by employing Structure from Motion (SfM) and Multi-View Stereo (MVS) algorithms. Moreover, a comparison and 3D measurement accuracy analysis were conducted based on the internal and external precision by exploiting checkpoint and DSM of Difference (DoD) error analysis methods. As indicated by the results, the 3D scene plane for two imaging types could reach an accuracy of centimeters, whereas the elevation accuracy of the orthophoto dataset alone could only reach the decimeters (0.3049 m). However, only 6.30% of the total image number of oblique images was required to improve the elevation accuracy by one order of magnitude (0.0942 m). (2) An insignificant variation in internal accuracy was reported in oblique imaging-assisted datasets. In particular, SfM-MVS technology exhibited high reproducibility for repeated observations. By changing the number and position of oblique images, the external precision was able to increase effectively, the elevation error distribution was improved to become more concentrated and stable. Accordingly, a repeated observation method only including a few oblique images has been proposed and demonstrated in this study, which could optimize the elevation and improve the accuracy. The research results could provide practical and effective technology reference strategies for geomorphological surveys and repeated observation analyses in sophisticated mountain environments. Full article

With the progress of photogrammetry and computer vision technology, three-dimensional (3D) reconstruction using aerial oblique images has been widely applied in urban modelling and smart city applications. However, state-of-the-art image-based automatic 3D reconstruction methods cannot effectively handle the unavoidable geometric deformation and incorrect texture mapping problems caused by moving cars in a city. This paper proposes a method to address this situation and prevent the influence of moving cars on 3D modelling by recognizing moving cars and combining the recognition results with a photogrammetric 3D modelling procedure. Through car detection using a deep learning method and multiview geometry constraints, we can analyse the state of a car’s movement and apply a proper preprocessing method to the geometrically model generation and texture mapping steps of 3D reconstruction pipelines. First, we apply the traditional Mask R-CNN object detection method to detect cars from oblique images. Then, a detected car and its corresponding image patch calculated by the geometry constraints in the other view images are used to identify the moving state of the car. Finally, the geometry and texture information corresponding to the moving car will be processed according to its moving state. Experiments on three different urban datasets demonstrate that the proposed method is effective in recognizing and removing moving cars and can repair the geometric deformation and error texture mapping problems caused by moving cars. In addition, the methods proposed in this paper can be applied to eliminate other moving objects in 3D modelling applications. Full article

Extracting damage information of buildings after an earthquake is crucial for emergency rescue and loss assessment. Low-altitude remote sensing by unmanned aerial vehicles (UAVs) for emergency rescue has unique advantages. In this study, we establish a remote sensing information-extraction method that combines ultramicro oblique UAV and infrared thermal imaging technology to automatically detect the structural damage of buildings and cracks in external walls. The method consists of four parts: (1) 3D live-action modeling and building structure analysis based on ultramicro oblique images; (2) extraction of damage information of buildings; (3) detection of cracks in walls based on infrared thermal imaging; and (4) integration of detection systems for information of earthquake-damaged buildings. First, a 3D live-action building model is constructed. A multi-view structure image for segmentation can be obtained based on this method. Second, a method of extracting information on damage to building structures using a 3D live-action building model as the geographic reference is proposed. Damage information of the internal structure of the building can be obtained based on this method. Third, based on analyzing the temperature field distribution on the exterior walls of earthquake-damaged buildings, an automatic method of detecting cracks in the walls by using infrared thermal imaging is proposed. Finally, the damage information detection and assessment system is researched and developed, and the system is integrated. Taking earthquake search-and-rescue simulation as an example, the effectiveness of this method is verified. The damage distribution in the internal structure and external walls of buildings in this area is obtained with an accuracy of 78%. Full article

The unmanned aerial vehicle (UAV) sensors and platforms nowadays are being used in almost every application (e.g., agriculture, forestry, and mining) that needs observed information from the top or oblique views. While they intend to be a general remote sensing (RS) tool, the relevant RS data processing and analysis methods are still largely ad-hoc to applications. Although the obvious advantages of UAV data are their high spatial resolution and flexibility in acquisition and sensor integration, there is in general a lack of systematic analysis on how these characteristics alter solutions for typical RS tasks such as land-cover classification, change detection, and thematic mapping. For instance, the ultra-high-resolution data (less than 10 cm of Ground Sampling Distance (GSD)) bring more unwanted classes of objects (e.g., pedestrian and cars) in land-cover classification; the often available 3D data generated from photogrammetric images call for more advanced techniques for geometric and spectral analysis. In this paper, we perform a critical review on RS tasks that involve UAV data and their derived products as their main sources including raw perspective images, digital surface models, and orthophotos. In particular, we focus on solutions that address the “new” aspects of the UAV data including (1) ultra-high resolution; (2) availability of coherent geometric and spectral data; and (3) capability of simultaneously using multi-sensor data for fusion. Based on these solutions, we provide a brief summary of existing examples of UAV-based RS in agricultural, environmental, urban, and hazards assessment applications, etc., and by discussing their practical potentials, we share our views in their future research directions and draw conclusive remarks. Full article

In the last few years, it has been possible to observe a considerable increase in the use of unmanned aerial vehicles (UAV) equipped with compact digital cameras for environment mapping. The next stage in the development of photogrammetry from low altitudes was the development of the imagery data from UAV oblique images. Imagery data was obtained from side-facing directions. As in professional photogrammetric systems, it is possible to record footprints of tree crowns and other forms of the natural environment. The use of a multi-camera system will significantly reduce one of the main UAV photogrammetry limitations (especially in the case of multirotor UAV) which is a reduction of the ground coverage area, while increasing the number of images, increasing the number of flight lines, and reducing the surface imaged during one flight. The approach proposed in this paper is based on using several head cameras to enhance the imaging geometry during one flight of UAV for mapping. As part of the research work, a multi-camera system consisting of several cameras was designed to increase the total Field of View (FOV). Thanks to this, it will be possible to increase the ground coverage area and to acquire image data effectively. The acquired images will be mosaicked in order to limit the total number of images for the mapped area. As part of the research, a set of cameras was calibrated to determine the interior orientation parameters (IOPs). Next, the method of image alignment using the feature image matching algorithms was presented. In the proposed approach, the images are combined in such a way that the final image has a joint centre of projections of component images. The experimental results showed that the proposed solution was reliable and accurate for the mapping purpose. The paper also presents the effectiveness of existing transformation models for images with a large coverage subjected to initial geometric correction due to the influence of distortion. Full article

Building footprint information is vital for 3D building modeling. Traditionally, in remote sensing, building footprints are extracted and delineated from aerial imagery and/or LiDAR point cloud. Taking a different approach, this paper is dedicated to the optimization of OpenStreetMap (OSM) building footprints exploiting the contour information, which is derived from deep learning-based semantic segmentation of oblique images acquired by the Unmanned Aerial Vehicle (UAV). First, a simplified 3D building model of Level of Detail 1 (LoD 1) is initialized using the footprint information from OSM and the elevation information from Digital Surface Model (DSM). In parallel, a deep neural network for pixel-wise semantic image segmentation is trained in order to extract the building boundaries as contour evidence. Subsequently, an optimization integrating the contour evidence from multi-view images as a constraint results in a refined 3D building model with optimized footprints and height. Our method is leveraged to optimize OSM building footprints for four datasets with different building types, demonstrating robust performance for both individual buildings and multiple buildings regardless of image resolution. Finally, we compare our result with reference data from German Authority Topographic-Cartographic Information System (ATKIS). Quantitative and qualitative evaluations reveal that the original OSM building footprints have large offset, but can be significantly improved from meter level to decimeter level after optimization. Full article

Multi-view dense matching is a crucial process in automatic 3D reconstruction and mapping applications. In this paper, we present a robust and effective multi-view dense matching algorithm for high-resolution aerial images based on a graph network. The overlap ratio and intersection angle between image pairs are used to find candidate stereo pairs and build the graph network. A Coarse-to-Fine strategy based on an improved Semi-Global Matching algorithm is applied for disparity computation across stereo pairs. Based on the constructed graph, point clouds of base views are generated by triangulating all connected image nodes, followed by a fusion process with the average reprojection error as a priority measure. The proposed method was successfully applied in experiments on aerial image test dataset provided by the ISPRS of Vaihingen, Germany and an oblique nadir image block of Zürich, Switzerland, using three kinds of matching configurations. The proposed method was compared to other state-of-art methods, SURE and PhotoScan. The results demonstrate that the proposed method delivers matches at higher completeness, efficiency, and accuracy than the other methods tested; the RMS for average reprojection error reached the sub pixel level and the actual positioning deviation was better than 1.5 GSD. Full article

This study proposes a method to recognize façades from large-scale urban scenes based on multi-level image features utilizing a recently developed oblique aerial photogrammetry technique. The method involves the use of multi-level image features, a bottom-up feature extraction procedure to produce regions of interest through monoscopic analysis, and then a coarse-to-fine feature matching strategy to characterise and match the regions in a stereoscopic model. Feature extraction from typical urban Manhattan scenes is based on line segments. Windows are re-organised based on the spatial constraints of line segments and the homogeneous structure of the spectrum. Façades as regions of interest are successfully constructed with a remarkable single edge and evidence from windows to get rid of occlusion. Feature matching is hierarchically performed beginning from distinctive facades and regularly distributed windows to the sub-pixel point primitives. The proposed strategy can effectively solve ambiguity and multi-solution problems in the complex urban scene matching process, particularly repetitive and poor-texture façades in oblique view. Full article