Search Results (16)

Capturing reliable hyperspectral imager data at a meaningful frame rate for explosions and other fast-changing scenes is not possible in the mid-wave infrared region under traditional sensor operating configurations and processing techniques, which typically have frame rates on the order of 0.5–2.0 Hz. To combat these shortcomings, the scene acquisition parameters were tailored for explosions and a new method for processing optical signatures of fast transient scenes with Fourier-transform infrared hyperspectral imagers was developed. For this technique, the instrument was first configured to collect asymmetric interferograms while optimizing the number of measurement points on the short side of the interferogram. Additionally, pixel-wise zero path distance offset and phase corrections were applied to the interferograms, a reduced spectral resolution of 8 cm⁻¹ was selected, and the window size was narrowed to 32 × 64 pixels while using a lens with a wide field of view. The smooth offset correction for scene change artifacts was then applied in post-processing to address any remaining artifacts in the Fourier-transformed spectra. These procedures yielded a 29× increase in frame rate and significant improvements in spectra fidelity. This work makes reliable field calibrations and measurements of explosions with Fourier-transform infrared hyperspectral imagers more achievable than before. Full article

Long-term LiDAR maps support urban remote sensing, infrastructure inventories, change analysis, digital twins, and map-based localization. Most map-cleaning pipelines, however, output a single retained map, forcing transient observed-dynamic artifacts and movable but currently stationary scene content into the same keep/remove decision. This paper formulates layer assignment within long-term LiDAR map maintenance as a layered representation problem. The resulting output comprises a long-term static (LTS) layer, a potentially dynamic (PD) layer, and a removed observed-dynamic (OD) set. For geometry-only accumulated maps with per-frame semantic predictions, the proposed layer-assignment framework combines map-to-scan geometric inconsistency, dual-timescale map-side semantic memory, PD admission, and PD-preserving recovery within the geometric rejection set. Geometric evidence first identifies OD candidates and rejection regions, while semantic memory converts framewise class predictions into map-side movability evidence for LTS/PD assignment. Experiments on SemanticKITTI under a controlled in-sequence LTS/PD/OD protocol show that this framework maintains high LTS-layer retention, improves PD retention, and reduces PD over-cutting while maintaining competitive OD suppression with predicted semantic input. Mechanism analyses further show that the dual-timescale semantic memory helps preserve PD evidence under conflicting or temporally sparse semantic predictions, while PD-preserving recovery provides boundary compensation with an explicit OD-suppression trade-off. Collapsing the output to a single retained map also improves the balance between static and PD retention relative to single-layer cleaning baselines. Qualitative Apollo SouthBay transfer observations further indicate that the same configuration can produce a three-layer map output without retuning, with binary annotations used only to support map-quality checks. Full article

Reliable three-dimensional (3D) reconstruction from endoscopic video is essential for endoscopic digital twins, scene review, and minimally invasive visual analysis. However, endoscopic images are not clean observations of intrinsic tissue appearance. Depth-dependent blur, shallow mucosal color diffusion, wet-surface specular reflection, and frame-wise color variation are often coupled with the captured signal. When such observation-dependent effects are directly optimized as Gaussian colors, conventional 3D Gaussian Splatting may encode transient imaging artifacts as persistent tissue appearance, leading to blurred textures, color drift, specular residues, and unstable novel-view synthesis. This paper presents EndoDGS (Endoscopic Degradation-Decoupled Gaussian Splatting), a degradation-decoupled Gaussian Splatting framework for endoscopic novel-view reconstruction. The core idea is to keep stable geometry and base tissue appearance in the Gaussian representation, while modeling endoscope-induced degradations separately in a bounded render-space compensation pipeline. EndoDGS combines lightweight appearance modulation for frame-wise color stabilization with sequential degradation compensation for optical blur, mucosal color transport, and wet-surface specular response. This design reduces the entanglement between persistent tissue appearance and transient imaging degradations without changing the underlying Gaussian geometry and visibility ordering. Experiments on synthetic colonoscopy and real endoscopic/laparoscopic datasets covering 38 scenes show that EndoDGS consistently improves reconstruction quality over representative implicit and explicit reconstruction baselines. The results demonstrate that separating stable tissue representation from observation-dependent endoscopic degradations provides a more faithful, stable, and interpretable foundation for endoscopic 3D reconstruction. Full article

High-resolution remote sensing image change detection holds significant application value in the fields of urban planning, disaster assessment, and others. However, it faces the dual challenge of pseudo-change interference and loss of detailed information. To address these issues, a frequency-domain-aware Siamese detail recovery network (FSD-Net) is designed in this paper. Firstly, from the perspective of frequency domain analysis, a theory on the dual roles of frequency domain components is introduced to reveal the robustness of low-frequency components to pseudo-changes and the dual semantic noise attributes of high-frequency components. Based on this theory, a frequency-aware context-guided difference (FCGD) module is designed. By explicitly decoupling the difference features into low-frequency global components and high-frequency residual components, it utilizes the prior low-frequency scene as a semantic gate to adaptively modulate the high-frequency differences, which effectively suppress pseudo-change interference. Subsequently, a detail recovery block (DRB), based on sub-pixel convolution, is constructed. This achieves unbiased spatial rearrangement through the semantic redundancy of channel dimensions, which avoids the checkerboard artifacts of traditional upsampling, and by employing a progressive multi-stage upsampling strategy to integrate shallow detail features from the encoder. The experimental results on the three public datasets of LEVIR-CD, WHU-CD, and CDD-CD demonstrate that the FSD-Net outperforms current mainstream methods (e.g., ChangeFormer, BAN, and so on) in core metrics such as F1 score and IoU, with a particularly significant improvement in recall. The ablation experiments validate the effectiveness and complementarity of the FCGD and DRB. Parameter sensitivity analysis indicates that the auxiliary loss weight λ is dataset dependent, with λ = 0.1 serving as a robust default choice. This study provides an efficient and reliable solution for change detection in high-resolution remote sensing imagery. Full article

Fourier-transform spectroscopy is a widely used technique for determining the spectral and thermal properties of a target. However, target temperature variations during measurement can compromise the spectral accuracy. Temperature fluctuations induce oscillations superimposed on the target spectrum. These oscillations, referred to as scene-change artifacts, degrade the spectral accuracy. The literature is divided, with theoretical predictions suggesting negligible artifacts and growing experimental evidence reporting significant artifacts. This paper presents a theory and experimental validation of scene-change artifacts originating from target temperature variations. Traditionally, the interferogram offset is assumed to be constant, an invalid assumption for a changing scene. The error is subsequently Fourier-transformed, producing scene-change artifacts. Accurately estimating the truth spectrum is often challenging. To address this, we propose the signal-to-scene-change-artifact ratio, a metric that quantifies the impact of scene-change artifacts without knowledge of the truth spectrum. The artifacts will be eliminated by estimating the interferogram offset using smooth offset correction. Furthermore, the interferogram offset enables determination of the target’s temperature with a greater accuracy and an increased temporal resolution compared to using the spectra. These results will demonstrate that a smooth offset correction can improve the spectrum and temperature accuracy on thermally variant targets when measured with a Fourier-transform spectrometer. Full article

Mini-LED local dimming systems commonly suffer from luminance discontinuity, halo artifacts, and temporal instability in dynamic scenes. Traditional heuristic-based methods and standard convolutional neural networks often fail to capture long-range spatial dependencies and struggle to balance spatial smoothness, content fidelity, and real-time performance under hardware constraints. To address these challenges, this paper proposes SwinLightNet, an efficient adaptive backlight optimization network tailored for Mini-LED displays. Built upon a Swin Transformer framework tailored for Mini-LED backlight optimization, SwinLightNet integrates five hardware-aware design strategies: (i) a lightweight Swin variant (window size = 8, MLP ratio = 2.0) for efficient global context modeling; (ii) CNN encoder–decoder integration for multi-scale feature extraction; (iii) a partition-level alignment module ensuring spatial consistency; (iv) a backlight constraint module enforcing local luminance consistency and contrast preservation; (v) a change-aware temporal decision framework stabilizing dynamic sequences. These components synergistically resolve core limitations: global modeling suppresses halo artifacts while preserving content fidelity; alignment and constraint modules eliminate luminance discontinuity without compromising contrast; and the temporal framework guarantees flicker-free output under motion. Evaluated on DIV2K (static images) and a custom 2K-resolution video dataset (dynamic scenes), SwinLightNet demonstrates robust reconstruction quality while maintaining only 1.18 million parameters and 0.088 GFLOPs (Computational Cost). The results confirm SwinLightNet’s effectiveness in holistically addressing spatial, temporal, and hardware constraints, demonstrating strong potential for practical deployment in resource-constrained Mini-LED backlight control systems. Full article

Fourier-transform spectrometers are widely used for spectral measurements. Changes in the field of view during measurement introduce oscillations into the measured spectra known as scene-change artifacts. Field-of-view changes also introduce uncertainty about which target the measured spectrum represents. Though scene-change artifacts are often present in dynamic data, their significance is disputed in the current literature. This work presents a theoretical framework and experimental validation for scene-change artifacts. Field-of-view changes introduce variable interferogram offsets, which standard processing techniques assume are constant. The error between the interferogram offset and its estimate is Fourier-transformed, yielding scene-change artifacts, often confused with noise, in the calibrated spectrum. Previous theoretical models ignored the effect of the interferogram offset in generating SCAs, leading to an underestimation of the scene-change artifact significance. Smooth offset correction removes these artifacts by estimating the variable interferogram offset using locally weighted scatter-plot smoothing. Updating the interferogram offset estimate resulted in the same accuracy expected for static conditions. The resulting spectra resemble the zero path difference spectra, similar to earlier theoretical predictions. These results indicate that Fourier-transform spectroscopy accuracy with variable scenes can be significantly improved with minor modifications to data processing. Full article

Conventional High Dynamic Range (HDR) image fusion algorithms generally require two or more original images with different exposure times for synthesis, making them unsuitable for real-time processing scenarios such as video streams. Additionally, the synthesized HDR images have the same bit depth as the original images, which may lead to banding artifacts and limits their applicability in professional fields requiring high fidelity. This paper utilizes a Field Programmable Gate Array (FPGA) to support an image sensor operating in Clear HDR mode, which simultaneously outputs High Conversion Gain (HCG) and Low Conversion Gain (LCG) images. These two images share the same exposure duration and are captured at the same moment, making them well-suited for real-time HDR fusion. This approach provides a feasible solution for real-time processing of video streams. An adaptive adjustment algorithm is employed to address the requirement for high fidelity. First, the initial HCG and LCG images are fused under the initial fusion parameters to generate a preliminary HDR image. Subsequently, the gain of the high-gain images in the video stream is adaptively adjusted according to the brightness of the fused HDR image, enabling stable brightness under dynamic illumination conditions. Finally, by evaluating the read noise of the HCG and LCG images, the fusion parameters are adaptively optimized to synthesize an HDR image with higher bit depth. Experimental results demonstrate that the proposed method achieves a processing rate of 46 frames per second for 2688 × 1520 resolution video streams, enabling real-time processing. The bit depth of the image is enhanced from 12 bits to 16 bits, preserving more scene information and effectively addressing banding artifacts in HDR images. This improvement provides greater flexibility for subsequent image processing tasks. Consequently, the adaptive algorithm is particularly suitable for dynamically changing scenarios such as real-time surveillance and professional applications including industrial inspection. Full article

Semantic segmentation tasks encompass various applications, such as autonomous driving, medical imaging, and robotics. Achieving accurate semantic information retrieval under conditions of high dynamic range and rapid scene changes remains a significant challenge for image-based algorithms. This challenge is primarily attributable to the limitations of conventional image sensors, which can experience motion blur or exposure artifacts. In contrast, event-based vision sensors, which asynchronously report changes in pixel intensity, offer a compelling solution by acquiring visual information at the same rate as the scene dynamics, thereby mitigating these limitations. However, we encounter a significant challenge in event-based semantic segmentation tasks: the need to expend time on converting event data into frame images to align with existing image-based semantic segmentation techniques. This approach squanders the inherently high temporal resolution of event data, compromising the accuracy and real-time performance of semantic segmentation tasks. To address these issues, this work explores a sparse semantic segmentation approach that directly addresses event data. We propose a network named EventSegNet that improves the ability to extract geometric features from event data by combining geometric feature enhancement operations and attention mechanisms. Based on this, we propose a large-scale event-based semantic segmentation dataset that provides labels for each event. Our approach achieved a new F1 score of 84.2% on the dataset. In addition, a lightweight and edge-oriented AI inference deployment technique was implemented for the network model. Compared to the baseline model, the optimized network model reduces the F1 score by 1.1% but is more than twice as fast computationally, enabling real-time inference on the NVIDIA AGX Xavier. Full article

Existing research indicates that detecting near-surface methane point sources using Sentinel-2 satellite imagery can offer crucial data support for mitigating climate change. However, current retrieval methods necessitate the identification of reference images unaffected by methane, which presents certain limitations. This study introduces the use of a matched filter, developing a novel methane detection algorithm for Sentinel-2 imagery. Compared to existing algorithms, this algorithm does not require selecting methane-free images from historical imagery in methane-sensitive bands, but estimates the background spectral information across the entire scene to extract methane gas signals. We tested the algorithm using simulated Sentinel-2 datasets. The results indicated that the newly proposed algorithm effectively reduced artifacts and noise. It was then validated in a known methane emission point source event and a controlled release experiment for its ability to quantify point source emission rates. The average estimated difference between the new algorithm and other algorithms was about 34%. Compared to the actual measured values in the controlled release experiment, the average estimated values ranged from −48% to 42% of the measurements. These estimates had a detection limit ranging from approximately 1.4 to 1.7 t/h and an average error percentage of 19%, with no instances of false positives reported. Finally, in a real case scenario, we demonstrated the algorithm’s ability to precisely locate the source position and identify, as well as quantify, methane point source emissions. Full article

In heterogeneous image fusion problems, different imaging mechanisms have always existed between time-of-flight and visible light heterogeneous images which are collected by binocular acquisition systems in orchard environments. Determining how to enhance the fusion quality is key to the solution. A shortcoming of the pulse coupled neural network model is that parameters are limited by manual experience settings and cannot be terminated adaptively. The limitations are obvious during the ignition process, and include ignoring the impact of image changes and fluctuations on the results, pixel artifacts, area blurring, and the occurrence of unclear edges. Aiming at these problems, an image fusion method in a pulse coupled neural network transform domain guided by a saliency mechanism is proposed. A non-subsampled shearlet transform is used to decompose the accurately registered image; the time-of-flight low-frequency component, after multiple lighting segmentation using a pulse coupled neural network, is simplified to a first-order Markov situation. The significance function is defined as first-order Markov mutual information to measure the termination condition. A new momentum-driven multi-objective artificial bee colony algorithm is used to optimize the parameters of the link channel feedback term, link strength, and dynamic threshold attenuation factor. The low-frequency components of time-of-flight and color images, after multiple lighting segmentation using a pulse coupled neural network, are fused using the weighted average rule. The high-frequency components are fused using improved bilateral filters. The results show that the proposed algorithm has the best fusion effect on the time-of-flight confidence image and the corresponding visible light image collected in the natural scene, according to nine objective image evaluation indicators. It is suitable for the heterogeneous image fusion of complex orchard environments in natural landscapes. Full article

Contact with nature has been used to promote both physical and mental health, and is increasingly used among cancer patients. However, the COVID-19 pandemic created new challenges in both access to nature in public spaces and in cancer care. The purpose of our study was to evaluate the change in active and passive use of nature, places of engaging with nature and associations of nature contact with respect to improvements to perceived stress and symptom experience among breast cancer patients during the pandemic. We conducted a cross-sectional survey of people diagnosed with breast cancer using ResearchMatch (n = 56) in July 2020 (the first wave of COVID-19). In this US-based, predominantly white, affluent, highly educated, female sample, we found that, on average, participants were first diagnosed with breast cancer at 54 years old and at stage 2 or 3. Eighteen percent of participants experienced disruptions in their cancer care due to the pandemic. As expected, activities in public places significantly decreased as well, including use of parks/trails and botanical gardens. In contrast, spending time near home, on the porch or in the backyard significantly increased. Also observed were significant increases in indoor activities involving passive nature contact, such as watching birds through a window, listening to birdsong, and smelling rain or plants. Decreased usage of parks/trails was significantly associated with higher stress (Coef = −2.30, p = 0.030) and increased usage of the backyard/porch was significantly associated with lower stress (Coef = −2.69, p = 0.032), lower symptom distress (Coef = −0.80, p = 0.063) and lower symptom severity (Coef = −0.52, p = 0.009). The most commonly reported alternatives to outdoor engagement with nature were watching nature through a window (84%), followed by looking at images of nature (71%), and listening to nature through a window (66%). The least commonly enjoyed alternative was virtual reality of nature scenes (25%). While outdoor contact with nature away from home decreased, participants still found ways to experience the restorative benefits of nature in and around their home. Of special interest in planning interventions was the fact that actual or real nature was preferred over that experienced through technology. This could be an artifact of our sample, or could represent a desire to be in touch with the “real world” during a health crisis. Nature contact may represent a flexible strategy to decrease stress and improve symptom experience among patients with cancer, particularly during public health crises or disruptions to cancer care. Full article

Estimation of the Digital Surface Model (DSM) and building heights from single-view aerial imagery is a challenging inherently ill-posed problem that we address in this paper by resorting to machine learning. We propose an end-to-end trainable convolutional-deconvolutional deep neural network architecture that enables learning mapping from a single aerial imagery to a DSM for analysis of urban scenes. We perform multisensor fusion of aerial optical and aerial light detection and ranging (Lidar) data to prepare the training data for our pipeline. The dataset quality is key to successful estimation performance. Typically, a substantial amount of misregistration artifacts are present due to georeferencing/projection errors, sensor calibration inaccuracies, and scene changes between acquisitions. To overcome these issues, we propose a registration procedure to improve Lidar and optical data alignment that relies on Mutual Information, followed by Hough transform-based validation step to adjust misregistered image patches. We validate our building height estimation model on a high-resolution dataset captured over central Dublin, Ireland: Lidar point cloud of 2015 and optical aerial images from 2017. These data allow us to validate the proposed registration procedure and perform 3D model reconstruction from single-view aerial imagery. We also report state-of-the-art performance of our proposed architecture on several popular DSM estimation datasets. Full article

Due to sharp changes in local brightness in high dynamic range scenes, fused images obtained by the traditional multi-exposure fusion methods usually have an unnatural appearance resulting from halo artifacts. In this paper, we propose a halo-free multi-exposure fusion method based on sparse representation of gradient features for high dynamic range imaging. First, we analyze the cause of halo artifacts. Since the range of local brightness changes in high dynamic scenes may be far wider than the dynamic range of an ordinary camera, there are some invalid, large-amplitude gradients in the multi-exposure source images, so halo artifacts are produced in the fused image. Subsequently, by analyzing the significance of the local sparse coefficient in a luminance gradient map, we construct a local gradient sparse descriptor to extract local details of source images. Then, as an activity level measurement in the fusion method, the local gradient sparse descriptor is used to extract image features and remove halo artifacts when the source images have sharp local changes in brightness. Experimental results show that the proposed method obtains state-of-the-art performance in subjective and objective evaluation, particularly in terms of effectively eliminating halo artifacts. Full article

This paper presents a parallax-robust video stitching technique for timely synchronized surveillance video. An efficient two-stage video stitching procedure is proposed in this paper to build wide Field-of-View (FOV) videos for surveillance applications. In the stitching model calculation stage, we develop a layered warping algorithm to align the background scenes, which is location-dependent and turned out to be more robust to parallax than the traditional global projective warping methods. On the selective seam updating stage, we propose a change-detection based optimal seam selection approach to avert ghosting and artifacts caused by moving foregrounds. Experimental results demonstrate that our procedure can efficiently stitch multi-view videos into a wide FOV video output without ghosting and noticeable seams. Full article