Search Results (43)

Structured illumination (SI) involves projecting controlled spatial light patterns onto a scene or sample and processing the captured response to recover information such as the phase, surface shape, or optical contrast. Three-Step Phase Shifting (TPS), also known as three-phase demodulation (TPD) in the spatial frequency domain imaging (SFDI) field, is a common SI workflow in which three phase-shifted sinusoidal patterns are projected and captured sequentially for one demodulation. To achieve acceptable demodulation quality, all three phase-shifted patterns must be captured correctly. However, achieving this quality introduces a trade-off since each phase must be captured at the correct time, increasing acquisition time and requiring precise projector–camera synchronization. In real-time TPD-based SI, low pattern-generation throughput, synchronization uncertainty, and often bulky desktop implementations remain major limitations. Therefore, this work investigates, designs, and validates a deterministic, low-latency, and portable projection–capture synchronization system for TPD-based SI. First, a Hyperspectral (HS) Python-based SI (HSPy-SI) system, representative of common state-of-the-art (SOTA) implementations, is evaluated. It uses an HS snapshot camera and a fixed-delay desktop synchronizer. Then, the proposed HyperSI system is introduced as a real-time projection–capture synchronizer implemented as a bench-top prototype on heterogeneous embedded platforms: a Single-Board Computer (SBC) and a System-on-Chip (SoC) board. Its core contribution is a tunable parameter, W (Frame Count to Wait), which counts frame-generation interrupts before capture and reduces the delay-search space. Compared with the SOTA, HyperSI achieves over 8× higher pattern-generation throughput, increases polarized acquisition by 7× to nearly 4 FPS, reaches about 12 FPS without polarizers, and reduces waiting time by 88×. Full article

Hyperspectral image change detection (HSI-CD) is becoming increasingly important in understanding how the Earth’s surface evolves over time, from monitoring ecosystems to tracking urban expansion. Unlike traditional pixel-based or hand-crafted approaches, deep learning models can automatically learn powerful spectral–spatial features, making them especially effective for this task. In this review, we bring together recent advances in deep learning for HSI-CD, combining a meta-analysis of the literature with an overview of the main model families and training strategies. We cover supervised, semi-supervised, and unsupervised methods, as well as newer directions such as transfer learning, self-supervised frameworks, and hybrid designs that blend CNNs, transformers, and graph neural networks. We also discuss benchmark datasets, evaluation protocols, and case studies that show how these methods perform in practice. Beyond summarizing the current progress, the review highlights ongoing gaps, such as limited labeled data, generalization across sensors, computational efficiency, and the need for interpretability, and points to emerging opportunities for future work. Our goal is to provide both a snapshot of the current state of the field and a road map for advancing deep learning-based HSI-CD. Full article

Hyperspectral sensing allows for the capture of spatially resolved spectral data, a capability critical for applications spanning from remote sensing to biomedical diagnostics. Nevertheless, the widespread adoption of this technology is hindered by the bulk and complexity of traditional systems based on diffractive optics. To overcome these hurdles, substantial research efforts have been dedicated to system miniaturization via component scaling and computational imaging. This review outlines the technological progression of compact hyperspectral imaging, ranging from miniaturized dispersive elements and tunable filters to computational snapshot designs using optical multiplexing. Although these approaches decrease system volume, they generally treat the sensor as a passive intensity recorder requiring external encoding. Therefore, we focus here on the rising paradigm of sensor-level integration made possible by nanomanufacturing. We examine optics-free architectures where spectral discrimination is embedded directly into the pixel, distinguishing between pixel-level nanophotonic filtering and intrinsic material-based selectivity. We specifically highlight emerging platforms such as compositionally engineered and cavity-enhanced perovskites, as well as electrically tunable organic or two-dimensional (2D) material heterostructures. To conclude, this review discusses persistent challenges regarding fabrication uniformity and stability, providing an outlook on the future of scalable and fully integrated hyperspectral vision systems. Full article

To address the technical challenge of simultaneously achieving snapshot imaging capability and high spectral resolution in thermal infrared spectral imaging, this paper proposes a computational imaging method based on a snapshot divided-aperture imaging system. In this method, a self-developed divided-aperture snapshot multispectral camera is utilized to simultaneously capture nine low-spectral-resolution images in a single exposure. The precise registration of the sub-channel images is accomplished via a star-point array calibration method. To construct the spectral reconstruction dataset, a Fourier-transform infrared hyperspectral camera (FTIR HCam) is employed to simultaneously acquire hyperspectral data from real-world scenes. Based on this, a neural network model is applied to reconstruct 127-channel hyperspectral information from the low-dimensional multispectral measurements. Experimental results demonstrate that the proposed method effectively achieves hyperspectral reconstruction while maintaining system compactness and snapshot imaging capability, thus providing a viable technical approach for hyperspectral sensing in dynamic thermal infrared scenarios. Full article

This paper investigated the feasibility of snapshot multispectral fluorescence imaging for nondestructive identification of cold stress in pepper plants. Fluorescence spectra were obtained by exciting the plant with a 405 nm ultraviolet LED. The plants were grown under three temperature conditions: 17 °C (control), 10 °C (moderate cold stress), and 5 °C (severe cold stress). Raw fluorescence spectra extracted from the demosaiced snapshot images were used as inputs for a deep-learning pipeline consisting of feature extraction, an encoder–decoder GRU, and a multilayer perceptron (MLP), and the results were compared with conventional machine learning classifiers, including linear discriminant analysis (LDA), quadratic discriminant analysis (QDA), and a Gaussian support vector machine (G-SVM). Tukey’s HSD test indicated that the proposed deep-learning model achieved the highest cross-validation accuracy and consistently produced superior classification metrics (accuracy of 85.7%, precision of 85.3%, recall of 85.3%, F1-score of 85.2). The trained model was further applied to hyperspectral cubes to generate classification maps; however, moderate misclassification was observed, consistent with the overall prediction performance. Full article

Coded Aperture Snapshot Spectral Imaging (CASSI) is a rapid hyperspectral imaging technique with broad application prospects. Due to limitations in three-dimensional compressed data acquisition modes and hardware constraints, the compressed measurements output by actual CASSI systems have a finite dynamic range, leading to degraded hyperspectral reconstruction quality. To address this issue, a high-quality hyperspectral reconstruction method based on multi-exposure fusion is proposed. A multi-exposure data acquisition strategy is established to capture low-, medium-, and high-exposure low-dynamic-range (LDR) measurements. A multi-exposure fusion-based high-dynamic-range (HDR) CASSI measurement reconstruction network (HCNet) is designed to reconstruct physically consistent HDR measurement images. Unlike traditional HDR networks for visual enhancement, HCNet employs a multiscale feature fusion architecture and combines local–global convolutional joint attention with residual enhancement mechanisms to efficiently fuse complementary information from multiple exposures. This makes it more suitable for CASSI systems, ensuring high-fidelity reconstruction of hyperspectral data in both spatial and spectral dimensions. A multi-exposure fusion CASSI mathematical model is constructed, and a CASSI experimental system is established. Simulation and real-world experimental results demonstrate that the proposed method significantly improves hyperspectral image reconstruction quality compared to traditional single-exposure strategies, exhibiting high robustness against multi-exposure interval jitters and shot noise in practical systems. Leveraging the higher-dynamic-range target information acquired through multiple exposures, especially in HDR scenes, the method enables reconstruction with enhanced contrast in both bright and dark details and also demonstrates higher spectral correlation, validating the enhancement of CASSI reconstruction and effective measurement capability in HDR scenarios. Full article

Hyperspectral imaging (HSI) is increasingly used in studies for medical applications as it provides both structural and functional information of biological tissue, enhancing diagnostic accuracy and clinical decision-making. Recently, HSI cameras (HSICs) have been integrated with medical endoscopes (HSIEs), capturing hypercube data beyond conventional white light imaging endoscopes. However, there are currently no cleared or approved HSIEs by the U.S. Food and Drug Administration (FDA). HSI accuracy depends on technologies and experimental parameters, which must be assessed for reliability. Importantly, the reflectance spectrum of a target can vary across different cameras and under different environmental or operational conditions. Thus, before reliable clinical translation can be achieved, a fundamental question must be addressed: can the same target yield consistent spectral measurements across different HSI systems and under varying acquisition conditions? This study investigates the impact of eight parameters—ambient light, exposure time, camera warm-up time, spatial and temporal averaging, camera focus, working distance, illumination angle, and target angle—on spectral measurements using two HSI techniques: interferometer-based spectral scanning and snapshot. Controlled experiments were conducted to evaluate how each parameter affects spectral accuracy and whether normalization can mitigate these effects. Our findings reveal that several parameters significantly influence spectral measurements, with some having a more pronounced impact. While normalization reduced variations for most parameters, it was less effective at mitigating errors caused by ambient light and camera warm-up time. Additionally, normalization did not eliminate spectral noise resulting from low exposure time, small region of interest, or a spectrally non-uniform light source. From these results, we propose practical considerations for optimizing HSI system performance. Implementing these measures can minimize variations in reflectance spectra of identical targets captured by different cameras and under diverse conditions, thereby supporting the reliable translation of HSI techniques to clinical applications. Full article

Coded-Aperture Snapshot Spectral Imaging (CASSI) systems acquire 3D spatial–spectral information on dynamic targets by converting 3D hyperspectral images (HSIs) into 2D compressed measurements. Various end-to-end networks have been proposed for HSI reconstruction from these measurements. However, these methods have not explored the frequency-domain information of HSIs. This research presents the spatial/spectral-frequency adaptive network (SSFAN) for CASSI image reconstruction. A frequency-division transformation (FDT) decomposes HSIs into distinct Fourier frequency components, enabling multiscale feature extraction in the frequency domain. The proposed dual-branch architecture consists of a spatial–spectral module (SSM) to preserve spatial–spectral consistency and a frequency division module (FDM) to model inter-frequency dependencies. Channel compression/expansion modules are integrated into the FDM to balance computational efficiency and reconstruction quality. Frequency-division loss supervises feature learning across divided frequency channels. Ablation experiments validate the contributions of each network module. Furthermore, comparison experiments on synthetic and real CASSI datasets demonstrate that SSFAN outperforms state-of-the-art end-to-end methods in reconstruction performance. Full article

Accurate color measurement is crucial for ensuring reliable sensing performance in vision-based applications. However, existing color measurement methods suffer from illumination variability, operational complexity, and perceptual subjectivity. In this study, dental color measurement, with its strict perceptual and spectral fidelity demands, is adopted to validate the proposed method. Using self-made resin-permeated ceramic teeth, this study proposes a deep-learned end-to-end spectral reflectance prediction framework to achieve snapshot teeth spectral reflectance from RGB images under complex light sources in the fundamental spectral domain through the construction of a physically interpretable network that enables physically informed feature fusion. A dual-attention modular-information fusion neural network is developed to recover the spectral reflectance directly from the RGB image for natural teeth and ceramics across multiple scenarios. A dataset containing 4000 RGB–hyperspectral image pairs is built from a self-designed optical system with complex illumination conditions. Results confirm that the proposed framework demonstrates effective performance in predicting teeth spectral reflectance with an MSE of 0.0024 and an SSIM of 0.8724. This method achieves high-accuracy color measurement while avoiding the color mismatch caused by metamerism, which empowers various advanced applications including optical property characterization, 3D surface reconstruction, and computer-aided restorative design. Full article

Hyperspectral (HS) image reconstruction has become a pivotal research area in computational imaging, facilitating the recovery of high-resolution spectral information from compressive snapshot measurements. With the rapid advancement of deep neural networks, reconstruction techniques have achieved significant improvements in both accuracy and computational efficiency, enabling more precise spectral recovery across a wide range of applications. This survey presents a comprehensive overview of recent progress in HS image reconstruction, systematically categorized into three main paradigms: traditional model-based methods, deep learning-based approaches, and hybrid frameworks that integrate data-driven priors with the mathematical modeling of the degradation process. We examine the foundational principles, strengths, and limitations of each category, with particular attention to developments such as sparsity and low-rank priors in model-based methods, the evolution from convolutional neural networks to Transformer architectures in learning-based approaches, and deep unfolding strategies in hybrid models. Furthermore, we review benchmark datasets, evaluation metrics, and prevailing challenges including spectral distortion, computational cost, and generalizability across diverse conditions. Finally, we outline potential research directions to address current limitations. This survey aims to provide a valuable reference for researchers and practitioners striving to advance the field of HS image reconstruction. Full article

Compressive spectral imaging (CSI) methods aim to reconstruct a three-dimensional hyperspectral image (HSI) from a single or a few two-dimensional compressive measurements. Conventional CSIs use separate optical elements to independently modulate the light field in the spatial and spectral domains, thus increasing the system complexity. In addition, real applications of CSIs require advanced reconstruction algorithms. This paper proposes a low-cost color-coded compressive snapshot spectral imaging method to reduce the system complexity and improve the HSI reconstruction performance. The combination of a color-coded aperture and an RGB detector is exploited to achieve higher degrees of freedom in the spatio-spectral modulations, which also renders a low-cost miniaturization scheme to implement the system. In addition, a deep learning method named Focus-based Mask-guided Spectral-wise Transformer (F-MST) network is developed to further improve the reconstruction efficiency and accuracy of HSIs. The simulations and real experiments demonstrate that the proposed F-MST algorithm achieves superior image quality over commonly used iterative reconstruction algorithms and deep learning algorithms. Full article

Uncrewed aerial vehicles (UAVs) have transformed remote sensing, offering unparalleled flexibility and spatial resolution across diverse applications. Many of these applications rely on mapping flights using snapshot imaging sensors for creating 3D models of the area or for generating orthomosaics from RGB, multispectral, hyperspectral, or thermal cameras. Based on a literature review, this paper provides comprehensive guidelines and best practices for executing such mapping flights. It addresses critical aspects of flight preparation and flight execution. Key considerations in flight preparation covered include sensor selection, flight height and GSD, flight speed, overlap settings, flight pattern, direction, and viewing angle; considerations in flight execution include on-site preparations (GCPs, camera settings, sensor calibration, and reference targets) as well as on-site conditions (weather conditions, time of the flights) to take into account. In all these steps, high-resolution and high-quality data acquisition needs to be balanced with feasibility constraints such as flight time, data volume, and post-flight processing time. For reflectance and thermal measurements, BRDF issues also influence the correct setting. The formulated guidelines are based on literature consensus. However, the paper also identifies knowledge gaps for mapping flight settings, particularly in viewing angle pattern, flight direction, and thermal imaging in general. The guidelines aim to advance the harmonization of UAV mapping practices, promoting reproducibility and enhanced data quality across diverse applications. Full article

Long-wave infrared (LWIR) spectral imaging plays a critical role in various applications such as gas monitoring, mineral exploration, and fire detection. Recent advancements in computational spectral imaging, powered by advanced algorithms, have enabled the acquisition of high-quality spectral images in real time, such as with the Uncooled Snapshot Infrared Spectrometer (USIRS). However, the USIRS system faces challenges, particularly a low spectral resolution and large amount of data noise, which can degrade the image quality. Deep learning has emerged as a promising solution to these challenges, as it is particularly effective at handling noisy data and has demonstrated significant success in hyperspectral imaging tasks. Nevertheless, the application of deep learning in LWIR imaging is hindered by the severe scarcity of long-wave hyperspectral image data, which limits the training of robust models. Moreover, existing networks that rely on convolutional layers or attention mechanisms struggle to effectively capture both local and global spectral correlations. To address these limitations, we propose the pixel-based Hierarchical Spectral Transformer (HST), a novel deep learning architecture that learns from publicly available single-pixel long-wave infrared spectral databases. The HST is designed to achieve a high spectral resolution for LWIR spectral image reconstruction, enhancing both the local and global contextual understanding of the spectral data. We evaluated the performance of the proposed method on both simulated and real-world LWIR data, demonstrating the robustness and effectiveness of the HST in improving the spectral resolution and mitigating noise, even with limited data. Full article

Recently, deep unfolding network methods have significantly progressed in hyperspectral snapshot compressive imaging. Many approaches directly employ Transformer models to boost the feature representation capabilities of algorithms. However, they often fall short of leveraging the full potential of self-attention mechanisms. Additionally, current methods lack adequate consideration of both intra-stage and inter-stage feature fusion, which hampers their overall performance. To tackle these challenges, we introduce a novel approach that hybridizes the sparse Transformer and wavelet fusion-based deep unfolding network for hyperspectral image (HSI) reconstruction. Our method includes the development of a spatial sparse Transformer and a spectral sparse Transformer, designed to capture spatial and spectral attention of HSI data, respectively, thus enhancing the Transformer’s feature representation capabilities. Furthermore, we incorporate wavelet-based methods for both intra-stage and inter-stage feature fusion, which significantly boosts the algorithm’s reconstruction performance. Extensive experiments across various datasets confirm the superiority of our proposed approach. Full article

Skin cancer is the predominant form of cancer worldwide, including 75% of all cancer cases. This study aims to evaluate the effectiveness of the spectrum-aided visual enhancer (SAVE) in detecting skin cancer. This paper presents the development of a novel algorithm for snapshot hyperspectral conversion, capable of converting RGB images into hyperspectral images (HSI). The integration of band selection with HSI has facilitated the identification of a set of narrow band images (NBI) from the RGB images. This study utilizes various iterations of the You Only Look Once (YOLO) machine learning (ML) framework to assess the precision, recall, and mean average precision in the detection of skin cancer. YOLO is commonly preferred in medical diagnostics due to its real-time processing speed and accuracy, which are essential for delivering effective and efficient patient care. The precision, recall, and mean average precision (mAP) of the SAVE images show a notable enhancement in comparison to the RGB images. This work has the potential to greatly enhance the efficiency of skin cancer detection, as well as improve early detection rates and diagnostic accuracy. Consequently, it may lead to a reduction in both morbidity and mortality rates. Full article