Search Results (125)

The rapid measurement of three-dimensional bridge geometric shapes is crucial for assessing construction quality and in-service structural conditions. Existing geometric shape measurement methods predominantly rely on traditional surveying instruments, which suffer from low efficiency and are limited to sparse point sampling. This study proposes a novel framework that utilizes an airborne LiDAR–camera fusion system for data acquisition, reconstructs high-precision 3D bridge models through real-time mapping, and automatically extracts structural geometric shapes using deep learning. The main contributions include the following: (1) A synchronized LiDAR–camera fusion system integrated with an unmanned aerial vehicle (UAV) and a microprocessor was developed, enabling the flexible and large-scale acquisition of bridge images and point clouds; (2) A multi-sensor fusion mapping method coupling visual-inertial odometry (VIO) and Li-DAR-inertial odometry (LIO) was implemented to construct 3D bridge point clouds in real time robustly; and (3) An instance segmentation network-based approach was proposed to detect key structural components in images, with detected geometric shapes projected from image coordinates to 3D space using LiDAR–camera calibration parameters, addressing challenges in automated large-scale point cloud analysis. The proposed method was validated through geometric shape measurements on a concrete arch bridge. The results demonstrate that compared to the oblique photogrammetry method, the proposed approach reduces errors by 77.13%, while its detection time accounts for 4.18% of that required by a stationary laser scanner and 0.29% of that needed for oblique photogrammetry. Full article

To enhance the automatic quality monitoring of bolt production, YOLOv10, Intel RealSense D435, and OpenCV were integrated to leverage GPU parallel computing capabilities for defect recognition and size measurement. To improve the model’s effectiveness across various industrial production environments, data augmentation techniques were employed, resulting in a trained model with notable precision, accuracy, and robustness. A high-precision camera calibration method was used, and image processing was accelerated through GPU parallel computing to ensure efficient and real-time target size measurement. In the real-time monitoring system, the average defect prediction time was 0.009241 s, achieving an accuracy of 99% and demonstrating high stability under varying lighting conditions. The average size measurement time was 0.021616 s, and increasing the light intensity could reduce the maximum error rate to 1%. These results demonstrated that the system excelled in real-time performance, accuracy, robustness, and efficiency, effectively addressing the demands of industrial production lines for rapid and precise defect detection and size measurement. In the dynamic and variable context of industrial applications, the system can be optimized and adjusted according to specific production environments and requirements, further enhancing the accuracy of defect detection and size measurement tasks. Full article

This paper describes a proposed method for preserving tangible cultural heritage by reconstructing a 3D model of cultural heritage using 2D captured images. The input data represent a set of multiple 2D images captured using different views around the object. An image registration technique is applied to configure the overlapping images with the depth of images computed to construct the 3D model. The automatic 3D reconstruction system consists of three steps: (1) Image registration for managing the overlapping of 2D input images; (2) Depth computation for managing image orientation and calibration; and (3) 3D reconstruction using point cloud and stereo-dense matching. We collected and recorded 2D images of tangible cultural heritage objects, such as high-relief and round-relief sculptures, using a low-cost digital camera. The performance analysis of the proposed method, in conjunction with the generation of 3D models of tangible cultural heritage, demonstrates significantly improved accuracy in depth information. This process effectively creates point cloud locations, particularly in high-contrast backgrounds. Full article

To date, there is little publicly available scientific data on unidentified aerial phenomena (UAP) whose properties and kinematics purportedly reside outside the performance envelope of known phenomena. To address this deficiency, the Galileo Project is designing, building, and commissioning a multi-modal, multi-spectral ground-based observatory to continuously monitor the sky and collect data for UAP studies via a rigorous long-term aerial census of all aerial phenomena, including natural and human-made. One of the key instruments is an all-sky infrared camera array using eight uncooled long-wave-infrared FLIR Boson 640 cameras. In addition to performing intrinsic and thermal calibrations, we implement a novel extrinsic calibration method using airplane positions from Automatic Dependent Surveillance–Broadcast (ADS-B) data that we collect synchronously on site. Using a You Only Look Once (YOLO) machine learning model for object detection and the Simple Online and Realtime Tracking (SORT) algorithm for trajectory reconstruction, we establish a first baseline for the performance of the system over five months of field operation. Using an automatically generated real-world dataset derived from ADS-B data, a dataset of synthetic 3D trajectories, and a hand-labeled real-world dataset, we find an acceptance rate (fraction of in-range airplanes passing through the effective field of view of at least one camera that are recorded) of 41% for ADS-B-equipped aircraft, and a mean frame-by-frame aircraft detection efficiency (fraction of recorded airplanes in individual frames which are successfully detected) of 36%. The detection efficiency is heavily dependent on weather conditions, range, and aircraft size. Approximately 500,000 trajectories of various aerial objects are reconstructed from this five-month commissioning period. These trajectories are analyzed with a toy outlier search focused on the large sinuosity of apparent 2D reconstructed object trajectories. About 16% of the trajectories are flagged as outliers and manually examined in the IR images. From these ∼80,000 outliers and 144 trajectories remain ambiguous, which are likely mundane objects but cannot be further elucidated at this stage of development without information about distance and kinematics or other sensor modalities. We demonstrate the application of a likelihood-based statistical test to evaluate the significance of this toy outlier analysis. Our observed count of ambiguous outliers combined with systematic uncertainties yields an upper limit of 18,271 outliers for the five-month interval at a 95% confidence level. This test is applicable to all of our future outlier searches. Full article

Multi-modal systems extract information about the environment using specialized sensors that are optimized based on the wavelength of the phenomenology and material interactions. To maximize the entropy, complementary systems operating in regions of non-overlapping wavelengths are optimal. VIS-IR (Visible-Infrared) systems have been at the forefront of multi-modal fusion research and are used extensively to represent information in all-day all-weather applications. Prior to image fusion, the image pairs have to be properly registered and mapped to a common resolution palette. However, due to differences in the device physics of image capture, information from VIS-IR sensors cannot be directly correlated, which is a major bottleneck for this area of research. In the absence of camera metadata, image registration is performed manually, which is not practical for large datasets. Most of the work published in this area assumes calibrated sensors and the availability of camera metadata providing registered image pairs, which limits the generalization capability of these systems. In this work, we propose a novel end-to-end pipeline termed DeepFusion for image registration and fusion. Firstly, we design a recursive crop and scale wavelet spectral decomposition (WSD) algorithm for automatically extracting the patch of visible data representing the thermal information. After data extraction, both the images are registered to a common resolution palette and forwarded to the DNN for image fusion. The fusion performance of the proposed pipeline is compared and quantified with state-of-the-art classical and DNN architectures for open-source and custom datasets demonstrating the efficacy of the pipeline. Furthermore, we also propose a novel keypoint-based metric for quantifying the quality of fused output. Full article

The accurate and timely acquisition of high-frequency three-dimensional (3D) displacement responses of large structures is crucial for evaluating their condition during seismic excitation on shaking tables. This paper presents a distributed high-speed videogrammetric method designed to rapidly measure the 3D displacement of large shaking table structures at high sampling frequencies. The method uses non-coded circular targets affixed to key points on the structure and an automatic correspondence approach to efficiently estimate the extrinsic parameters of multiple cameras with large fields of view. This process eliminates the need for large calibration boards or manual visual adjustments. A distributed computation and reconstruction strategy, employing the alternating direction method of multipliers, enables the global reconstruction of time-sequenced 3D coordinates for all points of interest across multiple devices simultaneously. The accuracy and efficiency of this method were validated through comparisons with total stations, contact sensors, and conventional approaches in shaking table tests involving large structures with RCBs. Additionally, the proposed method demonstrated a speed increase of at least six times compared to the advanced commercial photogrammetric software. It could acquire 3D displacement responses of large structures at high sampling frequencies in real time without requiring a high-performance computing cluster. Full article

Multisensor (MS) data fusion is important for improving the stability of vehicle environmental perception systems. MS joint calibration is a prerequisite for the fusion of multimodality sensors. Traditional calibration methods based on calibration boards require the manual extraction of many features and manual registration, resulting in a cumbersome calibration process and significant errors. A joint calibration algorithm for a Light Laser Detection and Ranging (LiDAR) and camera is proposed based on deep learning without the need for other special calibration objects. A network model constructed based on deep learning can automatically capture object features in the environment and complete the calibration by matching and calculating object features. A mathematical model was constructed for joint LiDAR-camera calibration, and the process of sensor joint calibration was analyzed in detail. By constructing a deep-learning-based network model to determine the parameters of the rotation matrix and translation matrix, the relative spatial positions of the two sensors were determined to complete the joint calibration. The network model consists of three parts: a feature extraction module, a feature-matching module, and a feature aggregation module. The feature extraction module extracts the image features of color and depth images, the feature-matching module calculates the correlation between the two, and the feature aggregation module determines the calibration matrix parameters. The proposed algorithm was validated and tested on the KITTI-odometry dataset and compared with other advanced algorithms. The experimental results show that the average translation error of the calibration algorithm is 0.26 cm, and the average rotation error is 0.02°. The calibration error is lower than those of other advanced algorithms. Full article

Weather radar is an active remote sensing device used to monitor the full lifecycle changes in severe convective weather with high spatial and temporal resolution. Effective radar calibration is a crucial foundation for ensuring the high-quality application of observational data. This paper utilizes a UAV platform equipped with a high-precision RTK system and standard metal spheres to study the principles and methods of metal sphere calibration, constructing a complete calibration process and calibration accuracy evaluation metrics. Additionally, a collocated radar comparison observation experiment was conducted for cross-validation, and metal sphere calibration tests were performed on problematic radars. The experimental results indicate the following: (1) The combined application of a high-precision RTK system and a laser range camera can provide real-time position information on the metal sphere, improving the efficiency of radar target acquisition. (2) The calibration method based on UAV-suspended metal spheres can periodically conduct the quantitative calibration of Z and Z_DR, achieving calibration accuracies within 0.5 dB and 0.2 dB, respectively, and supports the qualitative inspection of key parameters such as beamwidth and pulse width. (3) During field tests, a high success rate “coarse adjustment + fine adjustment + staring” sphere-finding technique was established, based on automatic switching between RHI, PPI, and FIX scanning modes. This method directs the UAV to adjust the metal sphere to the center of the radar distance bin, reducing the impact of uneven beam filling and bin crossing, ensuring the accuracy of scattering characteristic measurements. Full article

Preserving cultural heritage through monitoring, registering, and analyzing damage in historical architectural structures presents significant financial and logistical burdens. Developed approaches for monitoring and registering 4D (4-dimensional)-scanned range and raster images of damaged objects were investigated in a case study of historical Baron Palace in Egypt. In the methodology, we first prepared and observed the damaged historical models. The damaged historical models were scanned using a laser scanner at a predetermined date and time. Simultaneously, digital images of the models were captured (by a calibrated digital camera) and stored on a researcher’s tablet device. By observing and comparing the scanned models with the digital images, geometric defects and their extent are identified. Then, the observed data components were detected on the map. Then, damaged statue materials were investigated using system of energy dispersive (SEM; scanning electron microscope, Gemini Zeiss-Ultra 55) and XRF (X-ray fluorescence) spectroscopic analysis to identify the statue’s marble elements, and the results indicate that SEM-EDX and XRF analyses accurately identify major and minor compositions of the damaged statue. Then, the damaged models were registered in two stages. In the registration stages, the corresponding points were determined automatically by detecting the closest points in the clouds and ICP (iterative closest point) algorithm in RiSCAN. The point clouds (of the Palace and damaged statues) gave very detailed resolutions and more realistic images in RiSCAN, but it is a costly program. Finally, the accuracies of the registration tasks were assessed; the standard deviations are within acceptable limits and tend to increase irregularly as the number of polydata observations used in the registration calculations increase. Full article

The acquisition of the body temperature of animals kept in captivity in biology laboratories is crucial for several studies in the field of animal biology. Traditionally, the acquisition process was carried out manually, which does not guarantee much accuracy or consistency in the acquired data and was painful for the animal. The process was then switched to a semi-manual process using a thermal camera, but it still involved manually clicking on each part of the animal’s body every 20 s of the video to obtain temperature values, making it a time-consuming, non-automatic, and difficult process. This project aims to automate this acquisition process through the automatic recognition of parts of a lizard’s body, reading the temperature in these parts based on a video taken with two cameras simultaneously: an RGB camera and a thermal camera. The first camera detects the location of the lizard’s various body parts using artificial intelligence techniques, and the second camera allows reading of the respective temperature of each part. Due to the lack of lizard datasets, either in the biology laboratory or online, a dataset had to be created from scratch, containing the identification of the lizard and six of its body parts. YOLOv5 was used to detect the lizard and its body parts in RGB images, achieving a precision of 90.00% and a recall of 98.80%. After initial calibration, the RGB and thermal camera images are properly localised, making it possible to know the lizard’s position, even when the lizard is at the same temperature as its surrounding environment, through a coordinate conversion from the RGB image to the thermal image. The thermal image has a colour temperature scale with the respective maximum and minimum temperature values, which is used to read each pixel of the thermal image, thus allowing the correct temperature to be read in each part of the lizard. Full article

The study presents a bioindication complex and a technology of the experiment based on a submersible digital holographic camera with advanced monitoring capabilities for the study of plankton and its behavioral characteristics in situ. Additional mechanical and software options expand the capabilities of the digital holographic camera, thus making it possible to adapt the depth of the holographing scene to the parameters of the plankton habitat, perform automatic registration of the “zero” frame and automatic calibration, and carry out natural experiments with plankton photostimulation. The paper considers the results of a long-term digital holographic experiment on the biotesting of the water area in Arctic latitudes. It shows additional possibilities arising during the spectral processing of long time series of plankton parameters obtained during monitoring measurements by a submersible digital holographic camera. In particular, information on the rhythmic components of the ecosystem and behavioral characteristics of plankton, which can be used as a marker of the ecosystem well-being disturbance, is thus obtained. Full article

In this paper, we present and evaluate a calibration-free mobile eye-traking system. The system’s mobile device consists of three cameras: an IR eye camera, an RGB eye camera, and a front-scene RGB camera. The three cameras build a reliable corneal imaging system that is used to estimate the user’s point of gaze continuously and reliably. The system auto-calibrates the device unobtrusively. Since the user is not required to follow any special instructions to calibrate the system, they can simply put on the eye tracker and start moving around using it. Deep learning algorithms together with 3D geometric computations were used to auto-calibrate the system per user. Once the model is built, a point-to-point transformation from the eye camera to the front camera is computed automatically by matching corneal and scene images, which allows the gaze point in the scene image to be estimated. The system was evaluated by users in real-life scenarios, indoors and outdoors. The average gaze error was 1.6∘ indoors and 1.69∘ outdoors, which is considered very good compared to state-of-the-art approaches. Full article

The rapid evolution of 3D technology in recent years has brought about significant change in the field of agriculture, including precision livestock management. From 3D geometry information, the weight and characteristics of body parts of Korean cattle can be analyzed to improve cow growth. In this paper, a system of cameras is built to synchronously capture 3D data and then reconstruct a 3D mesh representation. In general, to reconstruct non-rigid objects, a system of cameras is synchronized and calibrated, and then the data of each camera are transformed to global coordinates. However, when reconstructing cattle in a real environment, difficulties including fences and the vibration of cameras can lead to the failure of the process of reconstruction. A new scheme is proposed that automatically removes environmental fences and noise. An optimization method is proposed that interweaves camera pose updates, and the distances between the camera pose and the initial camera position are added as part of the objective function. The difference between the camera’s point clouds to the mesh output is reduced from 7.5 mm to 5.5 mm. The experimental results showed that our scheme can automatically generate a high-quality mesh in a real environment. This scheme provides data that can be used for other research on Korean cattle. Full article

This paper focuses on the development of an integrated system that can rapidly and accurately extract the geometrical dimensions of a physical object assisted by a robotic hand and generate a 3D model of an object in a popular commercial Computer-Aided Design (CAD) software using computer vision. Two sets of experiments were performed: one with a simple cubical object and the other with a more complex geometry that needed photogrammetry to redraw it in the CAD system. For the accurate positioning of the object, a robotic hand was used. An Internet of Things (IoT) based camera unit was used for capturing the image and wirelessly transmitting it over the network. Computer vision algorithms such as GrabCut, Canny edge detector, and morphological operations were used for extracting border points of the input. The coordinates of the vertices of the solids were then transferred to the Computer-Aided Design (CAD) software via a macro to clean and generate the border curve. Finally, a 3D solid model is generated by linear extrusion based on the curve generated in CATIA. The results showed excellent regeneration of an object. This research makes two significant contributions. Firstly, it introduces an integrated system designed to achieve precise dimension extraction from solid objects. Secondly, it presents a method for regenerating intricate 3D solids with consistent cross-sections. The proposed system holds promise for a wide range of applications, including automatic 3D object reconstruction and quality assurance of 3D-printed objects, addressing potential defects arising from factors such as shrinkage and calibration, all with minimal user intervention. Full article

Previous camera self-calibration methods have exhibited certain notable shortcomings. On the one hand, they either exclusively emphasized scene cues or solely focused on vehicle-related cues, resulting in a lack of adaptability to diverse scenarios and a limited number of effective features. Furthermore, these methods either solely utilized geometric features within traffic scenes or exclusively extracted semantic information, failing to comprehensively consider both aspects. This limited the comprehensive feature extraction from scenes, ultimately leading to a decrease in calibration accuracy. Additionally, conventional vanishing point-based self-calibration methods often required the design of additional edge-background models and manual parameter tuning, thereby increasing operational complexity and the potential for errors. Given these observed limitations, and in order to address these challenges, we propose an innovative roadside camera self-calibration model based on the Transformer architecture. This model possesses a unique capability to simultaneously learn scene features and vehicle features within traffic scenarios while considering both geometric and semantic information. Through this approach, our model can overcome the constraints of prior methods, enhancing calibration accuracy and robustness while reducing operational complexity and the potential for errors. Our method outperforms existing approaches on both real-world dataset scenarios and publicly available datasets, demonstrating the effectiveness of our approach. Full article