Search Results (33)

Effective monitoring of surface damage in mining areas is vital for ecological restoration. Unmanned aerial vehicles (UAVs) have been widely used to obtain ground subsidence data owing to their low cost and ease of operation. The images captured by UAVs can generate dense image matching (DIM) point clouds, which, after screening, can be used to create a digital elevation model (DEM) required for deformation analysis. Existing filtering algorithms mainly rely on the spatial geometric features of point clouds and rarely utilize color information, which limits their accuracy in areas with vegetation coverage. To address this issue, this study proposes a H-PTD method that combines visible light with progressive triangulated irregular network densification (PTD). First, initial ground seeds are selected based on the H value in the HSV space. Subsequently, a triangulated irregular network (TIN) is constructed, and iterative densification is performed by evaluating the relationship between the target point and adjacent triangular faces, thereby achieving an accurate distinction between ground and non-ground. Evaluated on three terrain datasets and against five classical methods, the results indicate that the Total error in the H-PTD cross-matrix is controlled between 2.9% and 7.8%, and remains below 8% overall. The standard deviation of the DEM difference is around 0.02 m. Compared to other methods, H-PTD shows higher filtering accuracy and better terrain adaptability, making it more promising for monitoring mining areas and providing a more reliable tool for subsidence detection based on UAVs. Full article

The use of depth cameras in low-texture environments is crucial for ensuring the feasibility of visual simultaneous localization and mapping (SLAM) algorithms. Nevertheless, in underwater scenarios, light propagation through multi-layered media gives rise to refractive distortion. Directly utilizing distorted images acquired by depth cameras for visual SLAM computations inevitably introduces substantial errors in localization and mapping. Additionally, the waterproof glass mounted in front of the depth camera renders traditional air-based camera calibration ineffective, thereby introducing calibration inaccuracies. To mitigate these challenges, we propose a comprehensive SLAM algorithm framework for underwater multi-layered media refraction correction based on RGB-D cameras. Firstly, a multi-layer refraction calibration module is developed to calibrate the depth camera in air. Subsequently, the calibrated parameters are leveraged to construct an underwater multi-layer refraction correction module, which retrieves undistorted color images and aligned depth images. Finally, the corrected color images and depth images are fed into the front-end of the visual SLAM algorithm to generate dense point cloud maps. Both simulation and real-world experiments are conducted to validate the accuracy of the multi-layer refraction calibration results and the precision of the dense point clouds obtained via multi-layer refraction correction. Furthermore, the superiority of the proposed method is demonstrated through both qualitative and quantitative evaluations. Full article

Dense foliage, severe illumination variations, and interference from multiple individuals with similar appearances in complex orchard environments pose significant challenges for vision-based following robots in maintaining persistent target perception and identity consistency, thereby compromising the stability and safety of fruit transportation operations. To address these challenges, we propose a novel framework, DeepDIMP-ReID, which integrates the Deep Implicit Model Prediction (DIMP) tracker with a person re-identification (ReID) module based on EfficientNet. This visual perception and autonomous following framework is designed for differential-drive orchard transportation robots, aiming to achieve robust target perception and reliable identity maintenance in unstructured orchard settings. The proposed framework adopts a hierarchical perception–verification–control architecture. Visual tracking and three-dimensional localization are jointly achieved using synchronized color and depth data acquired from a RealSense camera, where target regions are obtained via the discriminative model prediction (DIMP) method and refined through an elliptical-mask-based depth matching strategy. Front obstacle detection is performed using DBSCAN-based point cloud clustering techniques. To suppress erroneous following caused by occlusion, target switching, or target reappearance after occlusion, an enhanced HOReID person re-identification module with an EfficientNet backbone is integrated for identity verification at critical decision points. Based on the verified perception results, a state-driven motion control strategy is employed to ensure safe and continuous autonomous following. Extensive long-term experiments conducted in real orchard environments demonstrate that the proposed system achieves a correct tracking rate exceeding 94% under varying human walking speeds, with an average localization error of 0.071 m. In scenarios triggering re-identification, a target discrimination success rate of 93.3% is obtained. These results confirm the effectiveness and robustness of the proposed framework for autonomous fruit transportation in complex orchard environments. Full article

Accurate and nondestructive monitoring of tomato growth is essential for large-scale greenhouse production; however, it remains challenging for small-fruited cultivars such as cherry tomatoes. Traditional 2D image analysis often fails to capture precise morphological traits, limiting its usefulness in growth modeling and yield estimation. This study proposes an automated phenotyping framework that integrates deep learning-based instance segmentation with high-resolution 3D point cloud reconstruction and ellipsoid fitting to estimate fruit size and ripeness from daily video recordings. These techniques enable accurate camera pose estimation and dense geometric reconstruction (via SfM and MVS), while Nerfacto enhances surface continuity and photorealistic fidelity, resulting in highly precise and visually consistent 3D representations. The reconstructed models are followed by CIELAB color analysis and logistic curve fitting to characterize the growth dynamics. When applied to real greenhouse conditions, the method achieved an average size estimation error of 8.01% compared to manual caliper measurements. During summer, the maximum growth rate (g_max) of size and ripeness were 24.14%, and 95.24% higher than in winter, respectively. Seasonal analysis revealed that winter-grown tomatoes matured approximately 10 days later than summer-grown fruits, highlighting environmental influences on phenological development. By enabling precise, noninvasive tracking of size and ripeness progression, this approach is a novel tool for smart and sustainable agriculture. Full article

Simultaneous Localization and Mapping (SLAM) is a critical technology for robot intelligence. Compared to cameras, Light Detection and Ranging (LiDAR) sensors achieve higher accuracy and stability in indoor environments. However, LiDAR can only capture the geometric structure of the environment, and LiDAR-based SLAM often fails in scenarios with insufficient geometric features or highly similar structures. Furthermore, low-cost mechanical LiDARs, constrained by sparse point cloud density, are particularly prone to odometry drift along the Z-axis, especially in environments such as tunnels or long corridors. To address the localization issues in such scenarios, we propose a forward-enhanced SLAM algorithm. Utilizing a 16-line LiDAR and a monocular camera, we construct a dense colored point cloud input and apply an efficient multi-modal feature extraction algorithm based on centroid distance to extract a set of feature points with significant geometric and color features. These points are then optimized in the back end based on constraints from points, lines, and planes. We compare our method with several classic SLAM algorithms in terms of feature extraction, localization, and elevation constraint. Experimental results demonstrate that our method achieves high-precision real-time operation and exhibits excellent adaptability to indoor environments with similar structures. Full article

This study proposes a non-contact method for assessing building inclination and damage by integrating 3D point cloud data with image recognition techniques. Conventional approaches, such as plumb bobs, require physical contact, posing safety risks and practical challenges, especially in densely built urban areas. The proposed method utilizes a 3D scanner to capture point cloud data and images, which are processed to extract building surfaces, detect inclination, and assess secondary structural components such as window frames. Experiments were conducted on prefabricated structures, detached houses, and dense residential areas to validate the method’s accuracy. Results show that the proposed approach achieved measurement accuracy comparable to or better than traditional methods, with an error reduction of approximately 19% in prefabricated structures and 21.72% in detached houses. Additionally, the method successfully identified window frame deformations, contributing to a comprehensive assessment of structural integrity. By applying gradient-based color mapping, damage severity was visualized intuitively. The findings demonstrate that this system can replace conventional measurement techniques, enabling safe, efficient, and large-scale post-disaster assessments. Future work will focus on enhancing point cloud interpolation and refining machine learning-based damage classification for broader applicability. Full article

Image-based 3D reconstruction enables robot-assisted interventions and image-guided navigation, which are emerging technologies in laparoscopy. When a robotic arm guides a laparoscope for image acquisition, hand–eye calibration is required to know the transformation between the camera and the robot flange. The calibration procedure is complex and must be conducted after each intervention (when the laparoscope is dismounted for cleaning). In the field, the surgeons and their assistants cannot be expected to do so. Thus, our approach is a procedure for a robot-based multi-view 3D reconstruction without hand–eye calibration, but with pose optimization algorithms instead. In this work, a robotic arm and a stereo laparoscope build the experimental setup. The procedure includes the stereo matching algorithm Semi Global Matching from OpenCV for depth measurement and the multiscale color iterative closest point algorithm from Open3D (v0.19), along with the multiway registration algorithm using a pose graph from Open3D (v0.19) for pose optimization. The procedure is evaluated quantitatively and qualitatively on ex vivo organs. The results are a low root mean squared error (1.1–3.37 mm) and dense point clouds. The proposed procedure leads to a plausible 3D model, and there is no need for complex hand–eye calibration, as this step can be compensated for by pose optimization algorithms. Full article

Precise photogrammetric mapping of preharvest conditions in an apple orchard can help determine the exact position and volume of single apple fruits. This can help estimate upcoming yields and prevent losses through spatially precise cultivation measures. These parameters also are the basis for effective storage management decisions, post-harvest. These spatial orchard characteristics can be determined by low-cost drone technology with a consumer grade red-green-blue (RGB) sensor. Flights were conducted in a specified setting to enhance the signal-to-noise ratio of the orchard imagery. Two different altitudes of 7.5 m and 10 m were tested to estimate the optimum performance. A multi-seasonal field campaign was conducted on an apple orchard in Brandenburg, Germany. The test site consisted of an area of 0.5 ha with 1334 trees, including the varieties ‘Gala’ and ‘Jonaprince’. Four rows of trees were tested each season, consisting of 14 blocks with eight trees each. Ripe apples were detected by their color and structure from a photogrammetrically created three-dimensional point cloud with an automatic algorithm. The detection included the position, number, volume and mass of apples for all blocks over the orchard. Results show that the identification of ripe apple fruit is possible in RGB point clouds. Model coefficients of determination ranged from 0.41 for data captured at an altitude of 7.5 m for 2018 to 0.40 and 0.53 for data from a 10 m altitude, for 2018 and 2020, respectively. Model performance was weaker for the last captured tree rows because data coverage was lower. The model underestimated the number of apples per block, which is reasonable, as leaves cover some of the fruits. However, a good relationship to the yield mass per block was found when the estimated apple volume per block was combined with a mean apple density per variety. Overall, coefficients of determination of 0.56 (for the 7.5 m altitude flight) and 0.76 (for the 10 m flights) were achieved. Therefore, we conclude that mapping at an altitude of 10 m performs better than 7.5 m, in the context of low-altitude UAV flights for the estimation of ripe apple parameters directly from 3D RGB dense point clouds. Full article

The cost of indoor mapping methods based on three-dimensional (3D) LiDAR can be relatively high, and they lack environmental color information, thereby limiting their application scenarios. This study presents an innovative, low-cost, omnidirectional 3D color LiDAR mapping system for indoor environments. The system consists of two two-dimensional (2D) LiDARs, six monocular cameras, and a servo motor. The point clouds are fused with imagery using a pixel-spatial dual-constrained depth gradient adaptive regularization (PS-DGAR) algorithm to produce dense 3D color point clouds. During fusion, the point cloud is reconstructed inversely based on the predicted pixel depth values, compensating for areas of sparse spatial features. For indoor scene reconstruction, a globally consistent alignment algorithm based on particle filter and iterative closest point (PF-ICP) is proposed, which incorporates adjacent frame registration and global pose optimization to reduce mapping errors. Experimental results demonstrate that the proposed density enhancement method achieves an average error of 1.5 cm, significantly improving the density and geometric integrity of sparse point clouds. The registration algorithm achieves a root mean square error (RMSE) of 0.0217 and a runtime of less than 4 s, both of which outperform traditional iterative closest point (ICP) variants. Furthermore, the proposed low-cost omnidirectional 3D color LiDAR mapping system demonstrates superior measurement accuracy in indoor environments. Full article

The integration of technological advancements has significantly impacted artificial intelligence (AI), enhancing the reliability of AI model outputs. This progress has led to the widespread utilization of AI across various sectors, including automotive, robotics, healthcare, space exploration, and defense. Today, air defense operations predominantly rely on laser designation. This process is entirely dependent on the capability and experience of human operators. Considering that UAV systems can have flight durations exceeding 24 h, this process is highly prone to errors due to the human factor. Therefore, the aim of this study is to automate the laser designation process using advanced deep learning algorithms on 3D point clouds obtained from different sources, thereby eliminating operator-related errors. As different data sources, dense 3D point clouds produced with photogrammetric methods containing color information, and point clouds produced with LiDAR systems were identified. The photogrammetric point cloud data were generated from images captured by the Akinci UAV’s multi-axis gimbal camera system within the scope of this study. For the point cloud data obtained from the LiDAR system, the DublinCity LiDAR dataset was used for testing purposes. The segmentation of point cloud data utilized the PointNet++ and RandLA-Net algorithms. Distinct differences were observed between the evaluated algorithms. The RandLA-Net algorithm, relying solely on geometric features, achieved an approximate accuracy of 94%, while integrating color features significantly improved its performance, raising its accuracy to nearly 97%. Similarly, the PointNet++ algorithm, relying solely on geometric features, achieved an accuracy of approximately 94%. Notably, the model developed as a unique contribution in this study involved enriching the PointNet++ algorithm by incorporating color attributes, leading to significant improvements with an approximate accuracy of 96%. The obtained results demonstrate a notable improvement in the PointNet++ algorithm with the proposed approach. Furthermore, it was demonstrated that the methodology proposed in this study can be effectively applied directly to data generated from different sources in aerial scanning systems. Full article

This paper introduces an innovative approach to 3D environmental mapping through the integration of a compact, handheld sensor package with a two-stage sensor fusion pipeline. The sensor package, incorporating LiDAR, IMU, RGB, and thermal cameras, enables comprehensive and robust 3D mapping of various environments. By leveraging Simultaneous Localization and Mapping (SLAM) and thermal imaging, our solution offers good performance in conditions where global positioning is unavailable and in visually degraded environments. The sensor package runs a real-time LiDAR-Inertial SLAM algorithm, generating a dense point cloud map that accurately reconstructs the geometric features of the environment. Following the acquisition of that point cloud, we post-process these data by fusing them with images from the RGB and thermal cameras and produce a detailed, color-enriched 3D map that is useful and adaptable to different mission requirements. We demonstrated our system in a variety of scenarios, from indoor to outdoor conditions, and the results showcased the effectiveness and applicability of our sensor package and fusion pipeline. This system can be applied in a wide range of applications, ranging from autonomous navigation to smart agriculture, and has the potential to make a substantial benefit across diverse fields. Full article

In the construction of large-scale water conservancy and hydropower transportation projects, the rock mass structural information is often used to evaluate and analyze various engineering geological problems such as high and steep slope stability, dam abutment stability, and natural rock landslide geological disasters. The complex shape and extremely irregular distribution of the structural planes make it challenging to identify and extract automatically. This study proposes a method for extracting structural planes from UAV images based on Geo-AINet ensemble learning. The UAV images of the slope are first used to generate a dense point cloud through a pipeline of SfM and PMVS; then, the multiple geological semantics, including color and texture from the image and local geological occurrence and surface roughness from the dense point cloud, are integrated with Geo-AINet for ensemble learning to obtain a set of semantic blocks; finally, the accurate extraction of structural planes is achieved through a multi-semantic hierarchical clustering strategy. Experimental results show that the structural planes extracted by the proposed method perform better integrity and edge adherence than that extracted by the AINet algorithm. In comparison with the results from the laser point cloud, the geological occurrence differences are less than three degrees, which proves the reliability of the results. This study widens the scope for surveying and mapping using remote sensing in engineering geological applications. Full article

This paper presents the CRBeDaSet—a new benchmark dataset designed for evaluating close range, image-based 3D modeling and reconstruction techniques, and the first empirical experiences of its use. The test object is a medium-sized building. Diverse textures characterize the surface of elevations. The dataset contains: the geodetic spatial control network (12 stabilized ground points determined using iterative multi-observation parametric adjustment) and the photogrammetric network (32 artificial signalized and 18 defined natural control points), measured using Leica TS30 total station and 36 terrestrial, mainly convergent photos, acquired from elevated camera standpoints with non-metric digital single-lens reflex Nikon D5100 camera (ground sample distance approx. 3 mm), the complex results of the bundle block adjustment with simultaneous camera calibration performed in the Pictran software package, and the colored point clouds (ca. 250 million points) from terrestrial laser scanning acquired using the Leica ScanStation C10 and post-processed in the Leica Cyclone™ SCAN software (ver. 2022.1.1) which were denoized, filtered, and classified using LoD3 standard (ca. 62 million points). The existing datasets and benchmarks were also described and evaluated in the paper. The proposed photogrammetric dataset was experimentally tested in the open-source application GRAPHOS and the commercial suites ContextCapture, Metashape, PhotoScan, Pix4Dmapper, and RealityCapture. As the first experience in its evaluation, the difficulties and errors that occurred in the software used during dataset digital processing were shown and discussed. The proposed CRBeDaSet benchmark dataset allows obtaining high accuracy (“mm” range) of the photogrammetric 3D object reconstruction in close range, based on a multi-image view uncalibrated imagery, dense image matching techniques, and generated dense point clouds. Full article

Rain spots on green plum are superficial micro-defects. Defect detection based on a two-dimensional image is easily influenced by factors such as placement position and light and is prone to misjudgment and omission, which are the main problems affecting the accuracy of defect screening of green plum. In this paper, using computer vision technology, an improved structure from motion (SFM) and patch-based multi-view stereo (PMVS) algorithm based on similar graph clustering and graph matching is proposed to perform three-dimensional sparse and dense reconstruction of green plums. The results show that, compared with the traditional algorithm, the running time of this algorithm is lower, at only 26.55 s, and the mean values of camera optical center error and pose error are 0.019 and 0.631, respectively. This method obtains a higher reconstruction accuracy to meet the subsequent plum micro-defect detection requirements. Aiming at the dense point cloud model of green plums, through point cloud preprocessing, the improved adaptive segmentation algorithm based on the Lab color space realizes the effective segmentation of the point cloud of green plum micro-defects. The experimental results show that the average running time of the improved adaptive segmentation algorithm is 2.56 s, showing a faster segmentation speed and better effect than the traditional K-means and K-means++ algorithms. After clustering the micro-defect point cloud, the micro-defect information of green plums was extracted on the basis of random sample consensus (RANSAC) plane fitting, which provides a theoretical model for further improving the accuracy of sorting the appearance quality of green plums. Full article

The one-shot structured light method using a color stripe pattern can provide a dense point cloud in a short time. However, the influence of noise and the complex characteristics of scenes still make the task of detecting the color stripe edges in deformed pattern images difficult. To overcome these challenges, a color structured light stripe edge detection method based on generative adversarial networks, which is named horizontal elastomeric attention residual Unet-based GAN (HEAR-GAN), is proposed in this paper. Additionally, a De Bruijn sequence-based color stripe pattern and a multi-slit binary pattern are designed. In our dataset, selecting the multi-slit pattern images as ground-truth images not only reduces the labor of manual annotation but also enhances the quality of the training set. With the proposed network, our method converts the task of detecting edges in color stripe pattern images into detecting centerlines in curved line images. The experimental results show that the proposed method can overcome the above challenges, and thus, most of the edges in the color stripe pattern images are detected. In addition, the comparison results demonstrate that our method can achieve a higher performance of color stripe segmentation with higher pixel location accuracy than other edge detection methods. Full article