Search Results (42)

Synthetic Aperture Radar (SAR) images and optical imagery have high potential for extracting digital elevation models (DEMs). The two main approaches for deriving elevation models from SAR data are interferometry (InSAR) and radargrammetry. Adapted from photogrammetric principles, radargrammetry relies on disparity model estimation as its core component. Matching strategies in radargrammetry typically follow local, global, or semi-global methodologies. Local methods, while having higher accuracy, especially in low-texture SAR images, require larger kernel sizes, leading to quadratic computational complexity. Conversely, global and semi-global models produce more consistent and higher-quality disparity maps but are computationally more intensive than local methods with small kernels and require more memory (RAM). In this study, inspired by the advantages of local matching algorithms, a computationally efficient and novel model is proposed for extracting corresponding pixels in SAR-intensity stereo images. To enhance accuracy, the proposed two-stage algorithm operates without an image pyramid structure. Notably, unlike traditional local and global models, the computational complexity of the proposed approach remains stable as the input size or kernel dimensions increase while memory consumption stays low. Compared to a pyramid-based local normalized cross-correlation (NCC) algorithm and adaptive semi-global matching (SGM) models, the proposed method maintains good accuracy comparable to adaptive SGM while reducing processing time by up to 50% relative to pyramid SGM and achieving a 35-fold speedup over the local NCC algorithm with an optimal kernel size. Validated on a Sentinel-1 stereo pair with a 10 m ground-pixel size, the proposed algorithm yields a DEM with an average accuracy of 34.1 m. Full article

Improper regulation of sugarcane feed volume can lead to harvester inefficiency or clogging. Accurate recognition of feed volume is therefore critical. However, visual recognition is challenging due to sugarcane stacking during feeding. To address this, we propose YOLO-ASM (YOLO Accurate Stereo Matching), a novel detection method. At the target detection level, we integrate a Convolutional Block Attention Module (CBAM) into the YOLOv5s backbone network. This significantly reduces missed detections and low-confidence predictions in dense stacking scenarios, improving detection speed by 28.04% and increasing mean average precision (mAP) by 5.31%. At the stereo matching level, we enhance the SGBM (Semi-Global Block Matching) algorithm through improved cost calculation and cost aggregation, resulting in Opti-SGBM (Optimized SGBM). This double-cost fusion approach strengthens texture feature extraction in stacked sugarcane, effectively reducing noise in the generated depth maps. The optimized algorithm yields depth maps with smaller errors relative to the original images, significantly improving depth accuracy. Experimental results demonstrate that the fused YOLO-ASM algorithm reduces sugarcane volume error rates across feed volumes of one to six by 3.45%, 3.23%, 6.48%, 5.86%, 9.32%, and 11.09%, respectively, compared to the original stereo matching algorithm. It also accelerates feed volume detection by approximately 100%, providing a high-precision solution for anti-clogging control in sugarcane harvester conveyor systems. Full article

With the continuous advancement of industrialization and intelligentization, stereo-vision-based measurement technology for large-scale components has become a prominent research focus. To address weak-textured regions in large-scale component images and reduce mismatches in stereo matching, we propose a cross-scale multi-feature stereo matching algorithm. In the cost-computation stage, the sum of absolute differences (SAD), census, and modified census cost aggregation are employed as cost-calculation methods. During the cost-aggregation phase, cross-scale theory is introduced to fuse multi-scale cost volumes using distinct aggregation parameters through a cross-scale framework. Experimental results on both benchmark and real-world datasets demonstrate that the enhanced algorithm achieves an average mismatch rate of 12.25%, exhibiting superior robustness compared to conventional census transform and semi-global matching (SGM) algorithms. Full article

Image-based 3D reconstruction enables robot-assisted interventions and image-guided navigation, which are emerging technologies in laparoscopy. When a robotic arm guides a laparoscope for image acquisition, hand–eye calibration is required to know the transformation between the camera and the robot flange. The calibration procedure is complex and must be conducted after each intervention (when the laparoscope is dismounted for cleaning). In the field, the surgeons and their assistants cannot be expected to do so. Thus, our approach is a procedure for a robot-based multi-view 3D reconstruction without hand–eye calibration, but with pose optimization algorithms instead. In this work, a robotic arm and a stereo laparoscope build the experimental setup. The procedure includes the stereo matching algorithm Semi Global Matching from OpenCV for depth measurement and the multiscale color iterative closest point algorithm from Open3D (v0.19), along with the multiway registration algorithm using a pose graph from Open3D (v0.19) for pose optimization. The procedure is evaluated quantitatively and qualitatively on ex vivo organs. The results are a low root mean squared error (1.1–3.37 mm) and dense point clouds. The proposed procedure leads to a plausible 3D model, and there is no need for complex hand–eye calibration, as this step can be compensated for by pose optimization algorithms. Full article

This paper presents a novel method to enhance ground truth disparity maps generated by Semi-Global Matching (SGM) using Maximum a Posteriori (MAP) estimation. SGM, while not producing visually appealing outputs like neural networks, offers high disparity accuracy in valid regions and avoids the generalization issues often encountered with neural network-based disparity estimation. However, SGM struggles with occlusions and textureless areas, leading to invalid disparity values. Our approach, though relatively simple, mitigates these issues by interpolating invalid pixels using surrounding disparity information and Bayesian inference, improving both the visual quality of disparity maps and their usability for training neural network-based commercial depth-sensing devices. Experimental results validate that our enhanced disparity maps preserve SGM’s accuracy in valid regions while improving the overall performance of neural networks on both synthetic and real-world datasets. This method provides a robust framework for advanced stereoscopic camera systems, particularly in autonomous applications. Full article

The advancement of unloading technology in combine harvesting is crucial for the intelligent development of agricultural machinery. Accurately measuring material pile height in transport vehicles is essential, as uneven accumulation can lead to spillage and voids, reducing loading efficiency. Relying solely on manual observation for measuring stack height can decrease harvesting efficiency and pose safety risks due to driver distraction. This research applies binocular vision to agricultural harvesting, proposing a novel method that uses a stereo matching algorithm to measure material pile height during harvesting. By comparing distance measurements taken in both empty and loaded states, the method determines stack height. A linear regression model processes the stack height data, enhancing measurement accuracy. A binocular vision system was established, applying Zhang’s calibration method on the MATLAB (R2019a) platform to correct camera parameters, achieving a calibration error of 0.15 pixels. The study implemented block matching (BM) and semi-global block matching (SGBM) algorithms using the OpenCV (4.8.1) library on the PyCharm (2020.3.5) platform for stereo matching, generating disparity, and pseudo-color maps. Three-dimensional coordinates of key points on the piled material were calculated to measure distances from the vehicle container bottom and material surface to the binocular camera, allowing for the calculation of material pile height. Furthermore, a linear regression model was applied to correct the data, enhancing the accuracy of the measured pile height. The results indicate that by employing binocular stereo vision and stereo matching algorithms, followed by linear regression, this method can accurately calculate material pile height. The average relative error for the BM algorithm was 3.70%, and for the SGBM algorithm, it was 3.35%, both within the acceptable precision range. While the SGBM algorithm was, on average, 46 ms slower than the BM algorithm, both maintained errors under 7% and computation times under 100 ms, meeting the real-time measurement requirements for combine harvesting. In practical operations, this method can effectively measure material pile height in transport vehicles. The choice of matching algorithm should consider container size, material properties, and the balance between measurement time, accuracy, and disparity map completeness. This approach aids in manual adjustment of machinery posture and provides data support for future autonomous master-slave collaborative operations in combine harvesting. Full article

With FaSS-MVS, we present a fast, surface-aware semi-global optimization approach for multi-view stereo that allows for rapid depth and normal map estimation from monocular aerial video data captured by unmanned aerial vehicles (UAVs). The data estimated by FaSS-MVS, in turn, facilitate online 3D mapping, meaning that a 3D map of the scene is immediately and incrementally generated as the image data are acquired or being received. FaSS-MVS is composed of a hierarchical processing scheme in which depth and normal data, as well as corresponding confidence scores, are estimated in a coarse-to-fine manner, allowing efficient processing of large scene depths, such as those inherent in oblique images acquired by UAVs flying at low altitudes. The actual depth estimation uses a plane-sweep algorithm for dense multi-image matching to produce depth hypotheses from which the actual depth map is extracted by means of a surface-aware semi-global optimization, reducing the fronto-parallel bias of Semi-Global Matching (SGM). Given the estimated depth map, the pixel-wise surface normal information is then computed by reprojecting the depth map into a point cloud and computing the normal vectors within a confined local neighborhood. In a thorough quantitative and ablative study, we show that the accuracy of the 3D information computed by FaSS-MVS is close to that of state-of-the-art offline multi-view stereo approaches, with the error not even an order of magnitude higher than that of COLMAP. At the same time, however, the average runtime of FaSS-MVS for estimating a single depth and normal map is less than 14% of that of COLMAP, allowing us to perform online and incremental processing of full HD images at 1–2 Hz. Full article

Monitoring tree growth helps operators better understand the growth mechanism of trees and the health status of trees and to formulate more effective management measures. Computer vision technology can quickly restore the three-dimensional geometric structure of trees from two-dimensional images of trees, playing a huge role in planning and managing tree growth. This study used binocular reconstruction technology to measure the height, canopy width, and ground diameter of Castanopsis hystrix and compared the growth differences under different nitrogen levels. In this research, we proposed a wavelet exponential decay thresholding method for image denoising. At the same time, based on the traditional semi-global matching (SGM) algorithm, a cost search direction is added, and a multi-line scanning semi-global matching (MLC-SGM) algorithm for stereo matching is proposed. The results show that the wavelet exponential attenuation threshold method can effectively remove random noise in red cone images, and the denoising effect is better than the traditional hard-threshold and soft-threshold denoising methods. The disparity images produced by the MLC-SGM algorithm have better disparity continuity and noise suppression than those produced by the SGM algorithm, with more minor measurement errors for C. hystrix growth factors. Medium nitrogen fertilization significantly promotes the height, canopy width, and ground diameter growth of C. hystrix. However, excessive fertilization can diminish this effect. Compared to tree height, excessive fertilization has a more pronounced impact on canopy width and ground diameter growth. Full article

In the realm of ocean engineering and maintenance of subsea structures, accurate underwater distance quantification plays a crucial role. However, the precision of such measurements is often compromised in underwater environments due to backward scattering and feature degradation, adversely affecting the accuracy of visual techniques. Addressing this challenge, our study introduces a groundbreaking method for underwater object measurement, innovatively combining image sonar with stereo vision. This approach aims to supplement the gaps in underwater visual feature detection with sonar data while leveraging the distance information from sonar for enhanced visual matching. Our methodology seamlessly integrates sonar data into the Semi-Global Block Matching (SGBM) algorithm used in stereo vision. This integration involves introducing a novel sonar-based cost term and refining the cost aggregation process, thereby both elevating the precision in depth estimations and enriching the texture details within the depth maps. This represents a substantial enhancement over existing methodologies, particularly in the texture augmentation of depth maps tailored for subaquatic environments. Through extensive comparative analyses, our approach demonstrates a substantial reduction in measurement errors by 1.6%, showing significant promise in challenging underwater scenarios. The adaptability and accuracy of our algorithm in generating detailed depth maps make it particularly relevant for underwater infrastructure maintenance, exploration, and inspection. Full article

Rice lodging is a crucial problem in rice production. Lodging during growing and harvesting periods can decrease rice yields. Practical lodging judgment for rice can provide effective reference information for yield prediction and harvesting. This article proposes a binocular camera-based lodging judgment method for rice in real-time. As a first step, the binocular camera and Inertial Measurement Unit (IMU) were calibrated. Secondly, Census and Grayscale Level cost features are constructed for stereo matching of left and right images. The Cross-Matching Cost Aggregation method is improved to compute the aggregation space in the LAB color space. Then, the Winner-Takes-All algorithm is applied to determine the optimal disparity for each pixel. A disparity map is constructed, and Multi-Step Disparity Refinement is applied to the disparity map to generate the final one. Finally, coordinate transformation obtains 3D world coordinates corresponding to pixels. IMU calculates the real-time pose of the binocular camera. A pose transformation is applied to the 3D world coordinates of the rice to obtain its 3D world coordinates in the horizontal state of the camera (pitch and roll angles are equal to 0). Based on the distance between the rice and the camera level, thresholding was used to determine whether the region to be detected belonged to lodging rice. The disparity map effect of the proposed matching algorithm was tested on the Middlebury Benchmark v3 dataset. The results show that the proposed algorithm is superior to the widely used Semi-Global Block Matching (SGBM) stereo-matching algorithm. Field images of rice were analyzed for lodging judgments. After the threshold judgment, the lodging region results were accurate and could be used to judge rice lodging. By combining the algorithms with binocular cameras, the research results can provide practical technical support for yield estimation and intelligent control of rice harvesters. Full article

In this paper, we propose a grain volume measurement method based on binocular structured light to address the need for fast and high-precision grain volume measurement in grain stocks. Firstly, we utilize speckle structured light imaging to tackle the image matching problem caused by non-uniform illumination in the grain depot environment and the similar texture of the grain pile surface. Secondly, we employ a semi-global stereo matching algorithm with census transformation to obtain disparity maps in grain bins, which are then converted into depth maps using the triangulation principle. Subsequently, each pixel in the depth map is transformed from camera coordinates to world coordinates using the internal and external parameter information of the camera. This allows us to construct 3D cloud data of the grain pile, including the grain warehouse scene. Thirdly, the improved European clustering method is used to achieve the segmentation of the three-dimensional point cloud data of the grain pile and the scene of the grain depot, and the pass-through filtering method is used to eliminate some outliers and poor segmentation points generated by segmentation to obtain more accurate three-dimensional point cloud data of the grain pile. Finally, the improved Delaunay triangulation method was used to construct the optimal topology of the grain surface continuous triangular mesh, and the nodes of the grain surface triangular mesh were projected vertically to the bottom of the grain warehouse to form several irregular triangular prisms; then, the cut and complement method was used to convert these non-plane triangular prisms into regular triangular prisms that could directly calculate the volume. The measured volume of the pile is then obtained by calculating the volume of the triangular prism. The experimental results indicate that the measured volume has a relative error of less than 1.5% and an average relative error of less than 0.5%. By selecting an appropriate threshold, the relative standard deviation can be maintained within 0.6%. The test results obtained from the laboratory test platform meet the requirements for field inspection of the granary. Full article

Thermal infrared imaging is less affected by lighting conditions and smoke compared to visible light imaging. However, thermal infrared images often have lower resolution and lack rich texture details, making them unsuitable for stereo matching and 3D reconstruction. To enhance the quality of infrared stereo imaging, we propose an advanced stereo matching algorithm. Firstly, the images undergo preprocessing using a non-local mean noise reduction algorithm to remove thermal noise and achieve a smoother result. Subsequently, we perform camera calibration using a custom-made chessboard calibration board and Zhang’s camera calibration method to obtain accurate camera parameters. Finally, the disparity map is generated using the SGBM (semi-global block matching) algorithm based on the weighted least squares method, enabling the 3D point cloud reconstruction of the object. The experimental results demonstrate that the proposed algorithm performs well in objects with sufficient thermal contrast and relatively simple scenes. The proposed algorithm reduces the average error value by 10.9 mm and the absolute value of the average error by 1.07% when compared with the traditional SGBM algorithm, resulting in improved stereo matching accuracy for thermal infrared imaging. While ensuring accuracy, our proposed algorithm achieves the stereo reconstruction of the object with a good visual effect, thereby holding high practical value. Full article

The low light conditions, abundant dust, and rocky terrain on the lunar surface pose challenges for scientific research. To effectively perceive the surrounding environment, lunar rovers are equipped with binocular cameras. In this paper, with the aim of accurately detect obstacles on the lunar surface under complex conditions, an Improved Semi-Global Matching (I-SGM) algorithm for the binocular cameras is proposed. The proposed method first carries out a cost calculation based on the improved Census transform and an adaptive window based on a connected component. Then, cost aggregation is performed using cross-based cost aggregation in the AD-Census algorithm and the initial disparity of the image is calculated via the Winner-Takes-All (WTA) strategy. Finally, disparity optimization is performed using left–right consistency detection and disparity padding. Utilizing standard test image pairs provided by the Middleburry website, the results of the test reveal that the algorithm can effectively improve the matching accuracy of the SGM algorithm, while reducing the running time of the program and enhancing noise immunity. Furthermore, when applying the I-SGM algorithm to the simulated lunar environment, the results show that the I-SGM algorithm is applicable in dim conditions on the lunar surface and can better help a lunar rover to detect obstacles during its travel. Full article

Digital Surface Model (DSM) is a three-dimensional model presenting the elevation of the Earth’s surface, which can be obtained by the along-track or cross-track stereo images of optical satellites. This paper investigates the DSM extraction method using Gaofen-6 (GF-6) high-resolution (HR) cross-track images with a wide field of view (WFV). To guarantee the elevation accuracy, the relationship between the intersection angle and the overlap of the cross-track images was analyzed. Cross-track images with 20–40% overlaps could be selected to conduct DSM extraction. First, the rational function model (RFM) based on error compensation was used to realize the accurate orientation of the image. Then, the disparity map was generated based on the semi-global block matching (SGBM) algorithm with epipolar constraint. Finally, the DSM was generated by forward intersection. The GF-6 HR cross-track images with about 30% overlap located in Taian, Shandong Province, China, were used for DSM extraction. The results show that the mountainous surface elevation features were retained completely, and the details, such as houses and roads, were presented in valleys and urban areas. The root mean square error (RMSE) of the extracted DSM could reach 6.303 m, 12.879 m, 14.929 m, and 19.043 m in valley, ridge, urban, and peak areas, respectively. The results indicate that the GF-6 HR cross-track images with a certain overlap can be used to extract a DSM to enhance its application in land cover monitoring. Full article

Semi-global matching (SGM) has been widely used in binocular vision. In spite of its good efficiency, SGM still has difficulties in dealing with low-texture regions. In this paper, an SGM algorithm based on multi-scale information fusion (MSIF), named SGM-MSIF, is proposed by combining multi-path cost aggregation and cross-scale cost aggregation (CSCA). Firstly, the stereo pairs at different scales are obtained by Gaussian pyramid down-sampling. The initial matching cost volumes at different scales are computed by combining census transform and color information. Then, the multi-path cost aggregation in SGM is introduced into the cost aggregation at each scale and the aggregated cost volumes are fused by CSCA. Thirdly, the disparity map is optimized by internal left-right consistency check and median filter. Finally, experiments are conducted on Middlebury datasets to evaluate the proposed algorithm. Experimental results show that the average error matching rate (EMR) of the proposed SGM-MSIF algorithm reduced by 1.96% compared with SGM. Compared with classical cross-scale stereo matching algorithm, the average EMR of SGM-MSIF algorithm reduced by 0.92%, while the processing efficiency increased by 58.7%. In terms of overall performance, the proposed algorithm outperforms the classic SGM and CSCA algorithms. It can achieve high matching accuracy and high processing efficiency for binocular vision applications, especially for those with low-texture regions. Full article