Search Results (3,523)

Light Detection and Ranging (LiDAR) is fundamental to autonomous driving and robotics, as it provides reliable 3D geometric information. However, snowfall introduces numerous spurious reflections that corrupt range measurements and severely degrade downstream perception. Existing desnowing techniques either rely on handcrafted filtering rules that fail under varying snow densities, or require paired snowy–clean scans, which are nearly impossible to collect in real-world scenarios. Self-supervised LiDAR desnowing approaches address these challenges by projecting raw 3D point clouds into 2D range images and jointly training a point reconstruction network (PR-Net) and a reconstruction difficulty network (RD-Net). Nevertheless, these methods remain limited by their reliance on the outdated Noise2Void training paradigm, which restricts reconstruction quality. In this paper, we redesign PR-Net with a blind-spot architecture to overcome the limitation. Specifically, we introduce a 3D-KNN encoder that aggregates neighborhood features directly in Euclidean 3D space, ensuring geometrically consistent representations. Additionally, we integrate residual state-space blocks (RSSB) to capture long-range contextual dependencies with linear computational complexity. Extensive experiments on both synthetic and real-world datasets, including SnowyKITTI and WADS, demonstrate that our method outperforms state-of-the-art self-supervised desnowing approaches by up to 0.06 IoU while maintaining high computational efficiency. Full article

The highway is a crucial component of modern transportation systems, and its efficient management is essential for ensuring safety and facilitating communication. The automatic understanding and reconstruction of highway environments are therefore pivotal for advanced traffic management and intelligent transportation systems. This work introduces a methodology for the fine-grained semantic segmentation and reconstruction of highway environments using dense 3D point cloud data acquired via mobile laser scanning. First, a multi-scale, object-based data augmentation and down-sampling method is introduced to address the issue of training sample imbalance. Subsequently, a deep learning approach utilizing the KPConv convolutional network is proposed to achieve fine-grained semantic segmentation. The segmentation results are then used to reconstruct a 3D model of the highway environment. The methodology is validated on a 32 km stretch of highway, achieving semantic segmentation across 27 categories of environmental features. When evaluated against a manually annotated ground truth, the results exhibit a mean Intersection over Union (mIoU) of $87.27\%$. These findings demonstrate that the proposed methodology is effective for fine-grained semantic segmentation and instance-level reconstruction of highways in practical scenarios. Full article

In the context of Industry 4.0, new methods of manufacturing, monitoring, and data generation related to industrial processes have emerged. Over the last decade, a new method of part manufacturing that has been revolutionizing the industry is Additive Manufacturing, which comes in various forms, including the more traditional Fusion Deposition Modeling (FDM) and the more innovative ones, such as Laser Metal Deposition (LMD) and Wire Arc Additive Manufacturing (WAAM). New technologies related to monitoring these processes are also emerging, such as Cyber-Physical Systems (CPSs) or Digital Twins (DTs), which can be used to enable Artificial Intelligence (AI)-powered analysis of generated big data. However, few works have dealt with a comprehensive data analysis, based on Digital Twin systems, to study quality levels of manufactured parts using 3D models. With this background in mind, this current project uses a Digital Twin-enabled dataflow to constitute a basis for a proposed data analysis pipeline. The pipeline consists of analyzing metal AM-manufactured parts’ surface roughness quality levels by the application of a Deep Neural Network (DNN) analytical model and enabling the assessment and tuning of deposition parameters by comparing AM-built models’ 3D representation, obtained by photogrammetry scanning, with the positional data acquired during the deposition process and stored in a cloud database. Stored and analyzed data may be further used to refine the manufacturing of parts, calibration of sensors and refining of the DT model. Also, this work presents a comprehensive study on experiments carried out using the CW-GTAW (Cold Wire Gas Tungsten Arc Welding) process as the means of depositing metal, resulting in hollow parts whose geometries were evaluated by means of both 3D scanned data, obtained via photogrammetry, and positional/deposition process parameters obtained from the Digital Twin architecture pipeline. Finally, an adapted PointNet DNN model was used to evaluate surface roughness quality levels of point clouds into 3 classes (good, fair, and poor), obtaining an overall accuracy of 75.64% on the evaluation of real deposited metal parts. Full article

This study introduces a novel, automated image-to-BIM (Building Information Modeling) workflow designed to generate semantically rich and geometrically useful BIM models directly from RGB images. Conventional scan-to-BIM often relies on specialized, costly, and time-intensive equipment, specifically if LiDAR is used to generate point clouds (PCs). Typical workflows are followed by a separate post-processing step for semantic segmentation recently performed by deep learning models on the generated PCs. Instead, the proposed method integrates vision language object detection (YOLOv8x-World v2) and vision based segmentation (SAM 2.1) with Neural Radiance Fields (NeRF) 3D reconstruction to generate segmented, color-labeled PCs directly from images. The key novelty lies in bypassing post-processing on PCs by embedding semantic information at the pixel level in images, preserving it through reconstruction, and encoding it into the resulting color labeled PC, which allows building elements to be directly identified and geometrically extracted based on color labels. Extracted geometry is serialized into a JSON format and imported into Revit to automate BIM creation for walls, windows, and doors. Experimental validation on BIM models generated from Unmanned Aerial Vehicle (UAV)-based exterior datasets and standard camera-based interior datasets demonstrated high accuracy in detecting windows and doors. Spatial evaluations yielded up to 0.994 precision and 0.992 Intersection over Union (IoU). NeRF and Gaussian Splatting models, Nerfacto, Instant-NGP, and Splatfacto, were assessed. Nerfacto produced the most structured PCs suitable for geometry extraction and Splatfacto achieved the highest image reconstruction quality. The proposed method removes dependency on terrestrial surveying tools and separate segmentation processes on PCs. It provides a low-cost and scalable solution for generating BIM models in aging or undocumented buildings and supports practical applications such as renovation, digital twin, and facility management. Full article

Three-dimensional semantic segmentation plays a crucial role in advancing scene understanding in fields such as autonomous driving, drones, and robotic applications. Existing studies usually improve prediction accuracy by fusing data from vehicle-mounted cameras and vehicle-mounted LiDAR. However, current semantic segmentation methods face two main challenges: first, they often directly fuse 2D and 3D features, leading to the problem of information redundancy in the fusion process; second, there are often issues of image feature loss and missing point cloud geometric information in the feature extraction stage. From the perspective of multimodal fusion, this paper proposes a point cloud semantic segmentation method based on a multimodal gated attention mechanism. The method comprises a feature extraction network and a gated attention fusion and segmentation network. The feature extraction network utilizes a 2D image feature extraction structure and a 3D point cloud feature extraction structure to extract RGB image features and point cloud features, respectively. Through feature extraction and global feature supplementation, it effectively mitigates the issues of fine-grained image feature loss and point cloud geometric structure deficiency. The gated attention fusion and segmentation network increases the network’s attention to important categories such as vehicles and pedestrians through an attention mechanism and then uses a dynamic gated attention mechanism to control the respective weights of 2D and 3D features in the fusion process, enabling it to solve the problem of information redundancy in feature fusion. Finally, a 3D decoder is used for point cloud semantic segmentation. In this paper, tests will be conducted on the SemanticKITTI and nuScenes large-scene point cloud datasets. Full article

The spatial characteristics of the dry beach in tailings ponds are critical indicators for the safety assessment of tailings dams. This study presents a method for dry beach extraction that combines deep learning-based semantic segmentation with 3D reconstruction, overcoming the limitations of 2D methods in spatial analysis. The workflow includes four steps: (1) High-resolution 3D point clouds are reconstructed from UAV images, and the projection matrix of each image is derived to link 2D pixels with 3D points. (2) AlexNet and GoogLeNet are employed to extract image features and automatically select images containing the dry beach boundary. (3) A DeepLabv3+ network is trained on manually labeled samples to perform semantic segmentation of the dry beach, with a lightweight incremental training strategy for enhanced adaptability. (4) Boundary pixels are detected and back-projected into 3D space to generate consistent point cloud boundaries. The method was validated on two-phase UAV datasets from a tailings pond in Yunnan Province, China. In phase I, the model achieved high segmentation performance, with a mean Accuracy and IoU of approximately 0.95 and a BF of 0.8267. When applied to phase II without retraining, the model maintained stable performance on dam boundaries, while slight performance degradation was observed on hillside and water boundaries. The 3D back-projection converted 2D boundary pixels into 3D coordinates, enabling the extraction of dry beach point clouds and supporting reliable dry beach length monitoring and deposition morphology analysis. Full article

Accurate collision detection in off-line robot simulation is essential for ensuring safety in modern manufacturing. However, current simulation environments often neglect flexible components such as wire harnesses, which are attached to articulated robots with irregular slack to accommodate motion. Because these components are rarely modeled in CAD, the absence of accurate 3D harness models leads to discrepancies between simulated and actual robot behavior, which sometimes result in physical interference or damage. This paper addresses this limitation by introducing a fully automated framework for extracting, segmenting, and reconstructing 3D wire-harness models directly from dense, partially occluded point clouds captured by terrestrial laser scanners. The key contribution lies in a motion-aware segmentation strategy that classifies harnesses into static and dynamic parts based on their physical attachment to robot links, enabling realistic motion simulation. To reconstruct complex geometries from incomplete data, we further propose a dual reconstruction scheme: an OBB-tree-based method for robust centerline recovery of unbranched cables and a Reeb-graph-based method for preserving topological consistency in branched structures. The experimental results on multiple industrial robots demonstrate that the proposed approach can generate high-fidelity 3D harness models suitable for collision detection and digital-twin simulation, even under severe data occlusions. These findings close a long-standing gap between geometric sensing and physics-based robot simulation in real factory environments. Full article

Rapid detection and precise segmentation of the weld seam region of interest (ROI) remain a core challenge in robotic intelligent grinding. To address this issue, this paper proposes a method for weld seam ROI detection and segmentation based on the fusion of active and passive vision. The proposed approach primarily consists of two stages: weld seam image instance segmentation and weld seam ROI point cloud segmentation. In the image segmentation stage, an enhanced segmentation network is constructed by integrating a convolutional attention module into YOLOv8n-seg, which effectively improves the localization accuracy and mask extraction quality of the weld seam region. In the point cloud segmentation stage, the 3D point cloud is first mapped onto a 2D pixel plane to achieve spatial alignment. Subsequently, a coarse screening of the projected point cloud is performed based on the bounding boxes output from the instance segmentation, eliminating a large amount of redundant data. Furthermore, a grayscale matrix is constructed based on the segmentation masks, enabling precise extraction of the weld seam ROI point cloud through point-wise discrimination. Experimental results demonstrate that the proposed method achieves high-quality segmentation of the weld seam region, providing a reliable foundation for robotic automated grinding. Full article

This study presents a terrestrial-laser-scanning-based scan-to-BIM workflow that transforms point cloud data into a BIM-based digital twin and analyzes how data collected with LiDAR (Light Detection and Ranging) can be converted into an information-rich model using Autodesk ReCap and Revit. Point clouds provided by laser scanning were processed in the ReCap environment and imported into Revit in an application that took place within an industrial facility of approximately 240 m² in Izmir. The scans were registered and pre-processed in Autodesk ReCap 2022 and modeled in Autodesk Revit 2022, with visualization updates prepared in Autodesk Revit 2023. Geometric quality was evaluated using point-to-model distance checks, since the dataset was imported in a pre-registered form and ReCap did not provide station-level RMSE values. The findings indicate that the ReCap–Revit integration offers high geometric accuracy and visual detail for both building elements and production-line machinery, but that high data density and complex geometry limit processing performance and interactivity. The study highlights both the practical applicability and the current technical limitations of terrestrial-laser-scanning-based scan-to-BIM workflows in an industrial context, offering a replicable reference model for future digital twin implementations in Turkey. Full article

Unmanned Aerial Vehicle (UAV)-based photogrammetric reconstruction is a key step in geometric digital twinning of bridges, but ensuring the quality of the reconstruction data through the planning of measurement configurations is not straightforward. This research investigates an approach for quantitatively evaluating the impact of different methodologies and configurations of UAV-based image collection on the quality of the collected images and 3D reconstruction data in the bridge inspection context. For an industry-grade UAV and a consumer-grade UAV, paths for image collection from different Ground Sampling Distance (GSD) and image overlap ratios are considered, followed by the 3D reconstruction with different algorithm configurations. Then, an approach for evaluating these data collection methodologies and configurations is discussed, focusing on trajectory accuracy, point-cloud reconstruction quality, and accuracy of geometric measurements relevant to inspection tasks. Through a case study on short-span road bridges, errors in different steps of the photogrammetric 3D reconstruction workflow are characterized. The results indicate that, for the global dimensional measurements, the consumer-grade UAV works comparably to the industry-grade UAV with different GSDs. In contrast, the local measurement accuracy changes significantly depending on the selected hardware and path-planning parameters. This research provides practical insights into controlling 3D reconstruction data quality in the context of bridge inspection and geometric digital twinning. Full article

Terrestrial laser scanning (TLS) point clouds require high-precision registration and georeferencing to be used effectively. Only then can data from multiple stations be integrated and transformed from the instrument’s local coordinate system into a common, stable reference frame that ensures temporal consistency for further analyses of displacement and deformation. The article demonstrates the validation of an innovative referencing system devised to improve the reliability and accuracy of registering and georeferencing TLS point clouds. The primary component of the system is openable reference spheres, whose centroids can be directly and precisely determined using surveying methods. It also includes dedicated adapters: tripods and adjustable F-clamps with which the spheres can be securely mounted on various structural components, facilitating the optimal distribution of the reference markers. Laboratory tests with four modern laser scanners (Z+F Imager 5010C, Riegl VZ-400, Leica ScanStation P40, and Trimble TX8) revealed sub-millimetre accuracy of sphere fit and form errors, along with the sphere distance error within the acceptance threshold. This confirms that there are no significant systematic errors and that the system is fully compatible with various TLS technologies. The registration and georeferencing quality parameters demonstrate the system’s stability and repeatability. They were additionally verified with independent control points and geodetic levelling of the centres of the spheres. The system overcomes the critical limitations of traditional reference spheres because their centres can be measured directly using surveying methods. This facilitates registration and georeferencing accuracy on par with, or even better than, that of commercial targets. The proposed system serves as a stable and repeatable reference frame suitable for high-precision engineering applications, deformation monitoring, and longitudinal analyses. Full article

LiDAR is used in autonomous driving for navigation, obstacle avoidance, and environment mapping. However, adverse weather conditions introduce noise into sensor data, potentially degrading the performance of perception algorithms and compromising the safety and reliability of autonomous driving systems. Hence, in this paper, we investigate the limitations of LiDAR perception algorithms in adverse weather conditions, explore ways to mitigate the effects of noise, and propose future research directions to achieve all-weather autonomy with LiDAR sensors. Using real-world datasets and synthetically generated dense fog, we characterize the noise in adverse weather such as snow, rain, and fog; their effect on sensor data; and how to effectively mitigate the noise for tasks like object detection, localization, and SLAM. Specifically, we investigate point cloud filtering methods and compare them based on their ability to denoise point clouds, focusing on processing time, accuracy, and limitations. Additionally, we evaluate the impact of adverse weather on state-of-the-art 3D object detection, localization, and SLAM methods, as well as the effect of point cloud filtering on the algorithms’ performance. We find that point cloud filtering methods are partially successful at removing noise due to adverse weather, but must be fine-tuned for the specific LiDAR, application scenario, and type of adverse weather. 3D object detection was negatively affected by adverse weather, but performance improved with dynamic filtering algorithms. We found that heavy snowfall does not affect localization when using a map constructed in clear weather, but it fails in dense fog due to a low number of feature points. SLAM also failed in thick fog outdoors, but it performed well in heavy snowfall. Filtering algorithms have varied effects on SLAM performance depending on the type of scan-matching algorithm. Full article

The representation of 3D shapes from point clouds remains a fundamental challenge in computer vision. A common approach decomposes 3D objects into interpretable geometric primitives, enabling compact, structured, and efficient representations. Building upon prior frameworks, this study introduces an enhanced unsupervised deep learning approach for 3D shape representation using superquadrics. The proposed framework fits a set of superquadric primitives to 3D objects through a fully integrated, differentiable pipeline that enables efficient optimization and parameter learning, directly extracting geometric structure from 3D point clouds without requiring ground-truth segmentation labels. This work introduces three key advancements that substantially improve representation quality, interpretability, and evaluation rigor: (1) A uniform sampling strategy that enhances training stability compared with random sampling used in earlier models; (2) An overlapping loss that penalizes intersections between primitives, reducing redundancy and improving reconstruction coherence; and (3) A novel evaluation framework comprising Primitive Accuracy, Structural Accuracy, and Overlapping Percentage metrics. This new metric design transitions from point-based to structure-aware assessment, enabling fairer and more interpretable comparison across primitive-based models. Comprehensive evaluations on benchmark 3D shape datasets demonstrate that the proposed modifications yield coherent, compact, and semantically consistent shape representations, establishing a robust foundation for interpretable and quantitative evaluation in primitive-based 3D reconstruction. Full article

Over the past few years, various research has been conducted to utilize 3D point cloud data in construction sites. This is because 3D point cloud data contain a variety of information, such as spatial coordinates (X, Y, Z), intensity, and color (RGB), making them highly applicable to construction environments that require precise operations. Accordingly, this research developed a new terrain surface interpolation method that leverages diverse information embedded in large-scale 3D point cloud data acquired from earthwork sites, as part of a foundational study for construction automation. In particular, the proposed terrain surface interpolation method was designed to be integrated with semantic segmentation based on 3D point cloud data, with a focus on enhancing the accuracy of earthwork volume estimation. Furthermore, field experiments were conducted using heavy construction equipment to compare terrain change and earthwork volume analyses between 3D point cloud data with and without the application of the proposed interpolation method. The analysis results of earthwork volumes indicated that the application of the terrain interpolation method to 3D point cloud data for construction equipment reduced estimation errors by approximately 94% compared to non-interpolated data. These findings demonstrate the effectiveness of the proposed method and are expected to contribute to future research in artificial intelligence and robotics utilizing 3D point cloud data within the construction industry. Full article

For large curved parts with complex surfaces, which often exhibit both symmetry and asymmetry in their geometric features, the multi-view combined with the phase shift method and highlight regions treatment method has been proposed and applied to the online measurement system. The hardware components of the measuring system include a self-designed multi-vision platform and a multi-view three-dimensional measurement platform composed of rotating platform, robot and linear guide rail. The overall calibration of the system was conducted to guarantee the effectiveness of the measurement point cloud splicing of each viewing angle. And the system integrates the three-dimensional measurement technology of multi vision combined with the phase shift method and online measure system to realize full coverage and high-precision measurement of the impeller—addressing both its inherent symmetry (regular blade arrangement) and local asymmetry (irregular edge details)—and controls the relative error of the measured size and the actual size within 1%. In addition, the highlight regions treatment method has also been proposed. By adjusting the camera’s exposure time to change the light intensity of the captured images, images under different exposures and their valid pixels are obtained, thereby facilitating the synthesis of a composite image free of highlight phenomena. Experimental results demonstrate that the proposed method can achieve full-coverage measurement of the measured object and effective measurement of highlight regions. Full article