Search Results (59)

Cross-modal image registration for unmanned aerial vehicle (UAV) platforms presents significant challenges due to large-scale deformations, distinct imaging mechanisms, and pronounced modality discrepancies. This paper proposes a novel multi-scale cascaded registration network based on style transfer that achieves superior performance: up to 67% reduction in mean squared error (from 0.0106 to 0.0068), 9.27% enhancement in normalized cross-correlation, 26% improvement in local normalized cross-correlation, and 8% increase in mutual information compared to state-of-the-art methods. The architecture integrates a cross-modal style transfer network (CSTNet) that transforms visible images into pseudo-infrared representations to unify modality characteristics, and a multi-scale cascaded registration network (MCRNet) that performs progressive spatial alignment across multiple resolution scales using diffeomorphic deformation modeling to ensure smooth and invertible transformations. A self-supervised learning paradigm based on image reconstruction eliminates reliance on manually annotated data while maintaining registration accuracy through synthetic deformation generation. Extensive experiments on the LLVIP dataset demonstrate the method’s robustness under challenging conditions involving large-scale transformations, with ablation studies confirming that style transfer contributes 28% MSE improvement and diffeomorphic registration prevents 10.6% performance degradation. The proposed approach provides a robust solution for cross-modal image registration in dynamic UAV environments, offering significant implications for downstream applications such as target detection, tracking, and surveillance. Full article

Multimodal sensing is essential in order to reach the robustness required of autonomous vehicle perception systems. Infrared (IR) imaging is of particular interest due to its low cost and complementarity with traditional RGB sensors. However, the lack of IR data in many datasets and simulation tools limits the development and validation of sensor fusion algorithms that exploit this complementarity. To address this, we propose an augmentation method that synthesizes realistic IR data from RGB images using gradient-boosting decision trees. We demonstrate that this method is an effective alternative to traditional deep learning methods for image translation such as CNNs and GANs, particularly in data-scarce situations. The proposed approach generates high-quality synthetic IR, i.e., Near-Infrared (NIR) and thermal images from RGB images, enhancing datasets such as MS2, EPFL, and Freiburg. Our synthetic images exhibit good visual quality when evaluated using metrics such as $R^2$, PSNR, SSIM, and LPIPS, achieving an $R^2$ of 0.98 on the MS2 dataset and a PSNR of $21.3$ dB on the Freiburg dataset. We also discuss the application of this method to synthetic RGB images generated by the CARLA simulator for autonomous driving. Our approach provides richer datasets with a particular focus on IR modalities for sensor fusion along with a framework for generating a wider variety of driving scenarios within urban driving datasets, which can help to enhance the robustness of sensor fusion algorithms. Full article

Citrus sooty mould caused by Capnodium spp. alters the quality of fruits on the tree and affects their productivity. Past laboratory and hand-held spectrometry tests have concluded that sooty mould exhibits a typical spectral response in the near-infrared spectrum region. For this reason, this study aims at developing an automatic method for remote sensing of this disease, combining 10 m spatial resolution Sentinel-2 satellite images and 0.25 m spatial resolution orthophotos to identify sooty mould infestation levels in small orchards, common in Mediterranean conditions. Citrus orchards of the Comunitat Valenciana region (Spain) underwent field inspection in 2022 during two months of minimum (August) and maximum (October) infestation. The inspectors categorised their observations according to three levels of infestation in three representative positions of each orchard. Two synthetic images condensing the monthly information were generated for both periods. A filtering algorithm was created, based on high-resolution images, to select informative pixels in the lower resolution images. The data were used to evaluate the performance of a Random Forest classifier in predicting intensity levels through cross-validation. Combining the information from medium- and high-resolution images improved the overall accuracy from 0.75 to 0.80, with mean producer’s accuracies of above 0.65 and mean user’s accuracies of above 0.78. Bowley–Yule skewness coefficients were +0.50 for the overall accuracy and +0.28 for the kappa index. Full article

High-temperature thermal images enable monitoring and controlling processes in metal, semiconductors, and ceramic manufacturing but also monitor activities of volcanoes or contrasting wildfires. Infrared thermal cameras require knowledge of the emissivity coefficient, while multispectral pyrometers provide fast and accurate temperature measurements with limited spatial resolution. Bayer-pattern cameras offer a compromise by capturing multiple spectral bands with high spatial resolution. However, temperature estimation from color remains challenging due to spectral overlaps among the color filters in the Bayer pattern, and a widely accepted calibration method is still missing. In this paper, the quantum efficiency of an imaging system including the camera sensor, lens, and filters is inferred from a sequence of images acquired by looking at a black body source between 700 °C and 1100 °C. The physical model of the camera, based on the Planck law and the optimized quantum efficiency, allows the calculation of the Planckian locus in the color space of the camera. A regression neural network, trained on a synthetic dataset representing the Planckian locus, predicts temperature pixel by pixel in the 700 °C to 3500 °C range from live images. Experiments done with a color camera, a multispectral camera, and a furnace for heat treatment of metals as ground truth show that our calibration procedure leads to temperature prediction with accuracy and precision of a few tens of Celsius degrees in the calibration temperature range. Tests on a temperature-calibrated halogen bulb prove good generalization capability to a wider temperature range while being robust to noise. Full article

The elevator light curtain is constrained by technical limitations in its infrared detection mechanism; thus, it is difficult to effectively identify the transparent material and elongated form of the object, which has become one of the main causes of abnormal elevator jamming accidents. To mitigate elevator accidents, we propose a novel visual segmentation method, PKNet (Prior Knowledge Network), specifically designed for detecting transparent and slender objects. We observe that the majority of cameras used in elevators are stationary, resulting in an inherently static background, while vision tasks primarily focus on detecting foreground objects. To this end, PKNet enhances the segmentation of dynamic foreground objects by incorporating prior knowledge of the static background and the characteristics of foreground objects. We also introduce ETAS-D, the first dataset designed for the segmentation of transparent and slender anomalous objects in elevator environments. This dataset consists of 4797 image frames, each with meticulously annotated masks of transparent and slender objects, captured from multiple viewpoints of 10 elevators. Extensive experimental results demonstrate that PKNet significantly outperforms existing methods in this domain. Furthermore, we propose a synthetic data generation workflow specifically designed for slender objects to enhance the model’s generalization ability and reliability. Full article

To date, there is little publicly available scientific data on unidentified aerial phenomena (UAP) whose properties and kinematics purportedly reside outside the performance envelope of known phenomena. To address this deficiency, the Galileo Project is designing, building, and commissioning a multi-modal, multi-spectral ground-based observatory to continuously monitor the sky and collect data for UAP studies via a rigorous long-term aerial census of all aerial phenomena, including natural and human-made. One of the key instruments is an all-sky infrared camera array using eight uncooled long-wave-infrared FLIR Boson 640 cameras. In addition to performing intrinsic and thermal calibrations, we implement a novel extrinsic calibration method using airplane positions from Automatic Dependent Surveillance–Broadcast (ADS-B) data that we collect synchronously on site. Using a You Only Look Once (YOLO) machine learning model for object detection and the Simple Online and Realtime Tracking (SORT) algorithm for trajectory reconstruction, we establish a first baseline for the performance of the system over five months of field operation. Using an automatically generated real-world dataset derived from ADS-B data, a dataset of synthetic 3D trajectories, and a hand-labeled real-world dataset, we find an acceptance rate (fraction of in-range airplanes passing through the effective field of view of at least one camera that are recorded) of 41% for ADS-B-equipped aircraft, and a mean frame-by-frame aircraft detection efficiency (fraction of recorded airplanes in individual frames which are successfully detected) of 36%. The detection efficiency is heavily dependent on weather conditions, range, and aircraft size. Approximately 500,000 trajectories of various aerial objects are reconstructed from this five-month commissioning period. These trajectories are analyzed with a toy outlier search focused on the large sinuosity of apparent 2D reconstructed object trajectories. About 16% of the trajectories are flagged as outliers and manually examined in the IR images. From these ∼80,000 outliers and 144 trajectories remain ambiguous, which are likely mundane objects but cannot be further elucidated at this stage of development without information about distance and kinematics or other sensor modalities. We demonstrate the application of a likelihood-based statistical test to evaluate the significance of this toy outlier analysis. Our observed count of ambiguous outliers combined with systematic uncertainties yields an upper limit of 18,271 outliers for the five-month interval at a 95% confidence level. This test is applicable to all of our future outlier searches. Full article

Close-proximity operations play a crucial role in emerging mission concepts, such as Active Debris Removal or small celestial bodies exploration. When approaching a non-cooperative target, the increased risk of collisions and reduced reliance on ground intervention necessitate autonomous on-board relative pose (position and attitude) estimation. Although navigation strategies relying on monocular cameras which operate in the visible (VIS) spectrum have been extensively studied and tested in flight for navigation applications, their accuracy is heavily related to the target’s illumination conditions, thus limiting their applicability range. The novelty of the paper is the introduction of a thermal-infrared (TIR) camera to complement the VIS one to mitigate the aforementioned issues. The primary goal of this work is to evaluate the enhancement in navigation accuracy and robustness by performing VIS-TIR data fusion within an Extended Kalman Filter (EKF) and to assess the performance of such navigation strategy in challenging illumination scenarios. The proposed navigation architecture is tightly coupled, leveraging correspondences between a known uncooperative target and feature points extracted from multispectral images. Furthermore, handover from one camera to the other is introduced to enable seamlessly operations across both spectra while prioritizing the most significant measurement sources. The pipeline is tested on Tango spacecraft synthetically generated VIS and TIR images. A performance assessment is carried out through numerical simulations considering different illumination conditions. Our results demonstrate that a combined VIS-TIR navigation strategy effectively enhances operational robustness and flexibility compared to traditional VIS-only navigation chains. Full article

Ultraviolet solar radiation at the Earth’s surface significantly impacts both human health and ecosystems. A biologically effective daily radiant exposure (BEDRE) model is proposed for various biological processes with an analytical formula for its action spectrum. The following processes are considered: erythema formation, previtamin D3 synthesis, psoriasis clearance, and inactivation of SARS-CoV-2 virions. The BEDRE model is constructed by multiplying the synthetic BEDRE value under cloudless conditions by a cloud modification factor (CMF) parameterizing the attenuation of radiation via clouds. The CMF is an empirical function of the solar zenith angle (SZA) at midday and the daily clearness index from the Spinning Enhanced Visible and Infrared Imager (SEVIRI) measurements on board the second-generation Meteosat satellites. Total column ozone, from MERRA-2 reanalysis, is used in calculations of clear-sky BEDRE values. The proposed model was trained and validated using data from several European ground-based spectrophotometers and biometers for the periods 2014–2023 and 2004–2013, respectively. The model provides reliable estimates of BEDRE for all biological processes considered. Under snow-free conditions and SZA < 45° at midday, bias and standard deviation of observation-model differences are approximately ±5% and 15%, respectively. The BEDRE model can be used as an initial validation tool for ground-based UV data. Full article

Extreme meteorological events hit our planet with increasing frequency, resulting in an ever-increasing number of natural disasters. Flash floods generated by intense and violent rains are among the most dangerous natural disasters that compromise crops and cause serious damage to infrastructure and human lives. In the case of such a kind of disastrous events, timely and accurate information about the location and extent of the affected areas can be crucial to better plan and implement recovery and containment interventions. Satellite systems may efficiently provide such information at different spatial/temporal resolutions. Several authors have developed satellite techniques to detect and map inundated areas using both Synthetic Aperture Radar (SAR) and a new generation of high-resolution optical data but with some accuracy limits, mostly due to the use of fixed thresholds to discriminate between the inundated and unaffected areas. In this paper, the RST-FLOOD fully automatic technique, which does not suffer from the aforementioned limitation, has been exported for the first time to the mid–high-spatial resolution (20 m) optical data provided by the Copernicus Sentinel-2 Multi-Spectral Instrument (MSI). The technique was originally designed for and successfully applied to Advanced Very High Resolution Radiometer (AVHRR), Moderate Resolution Imaging Spectroradiometer (MODIS), and Visible Infrared Imaging Radiometer Suite (VIIRS) satellite data at a mid–low spatial resolution (from 1000 to 375 m). The processing chain was implemented in a completely automatic mode within the Google Earth Engine (GEE) platform to study the recent strong flood event that occurred in May 2023 in Emilia Romagna (Italy). The outgoing results were compared with those obtained through the implementation of an existing independent optical-based technique and the products provided by the official Copernicus Emergency Management Service (CEMS), which is responsible for releasing information during crisis events. The comparisons carried out show that RST-FLOOD is a simple implementation technique able to retrieve more sensitive and effective information than the other optical-based methodology analyzed here and with an accuracy better than the one offered by the CEMS products with a significantly reduced delivery time. Full article

Garlic constitutes a significant small-scale agricultural commodity in China. A key factor influencing garlic prices is the planted area, which can be accurately and efficiently determined using remote sensing technology. However, the spectral characteristics of garlic and winter wheat are easily confused, and the widespread intercropping of these crops in the study area exacerbates this issue, leading to significant challenges in remote sensing image analysis. Additionally, remote sensing data are often affected by weather conditions, spatial resolution, and revisit frequency, which can result in delayed and inaccurate area extraction. In this study, historical data were utilized to restore Sentinel-2 remote sensing images, aimed at mitigating cloud and rain interference. Feature combinations were devised, incorporating two vegetation indices into a comprehensive time series, along with Sentinel-1 synthetic aperture radar (SAR) time series and other temporal datasets. Multiple classification combinations were employed to extract garlic within the study area, and the accuracy of the classification results was systematically analyzed. First, we used passive satellite imagery to extract winter crops (garlic, winter wheat, and others) with high accuracy. Second, we identified garlic by applying various combinations of time series features derived from both active and passive remote sensing data. Third, we evaluated the classification outcomes of various feature combinations to generate an optimal garlic cultivation distribution map for each region. Fourth, we developed a garlic fragmentation index to assess the impact of landscape fragmentation on garlic extraction accuracy. The findings reveal that: (1) Better results in garlic extraction can be achieved using active–passive time series remote sensing. The performance of the classification model can be further enhanced by incorporating short-wave infrared bands or spliced time series data into the classification features. (2) Examination of garlic cultivation fragmentation using the garlic fragmentation index aids in elucidating variations in accuracy across the study area’s six counties. (3) Comparative analysis with validation samples demonstrated superior garlic extraction outcomes from the six primary garlic-producing counties of the North China Plain in 2021, achieving an overall precision exceeding 90%. This study offers a practical exploration of target crop identification using multi-source remote sensing data in mixed cropping areas. The methodology presented here demonstrates the potential for efficient, cost-effective, and accurate garlic classification, which is crucial for improving garlic production management and optimizing agricultural practices. Moreover, this approach holds promise for broader applications, such as nationwide garlic mapping. Full article

Human NAD(P)H-quinone oxidoreductase1 (HNQO1) is a two-electron reductase antioxidant enzyme whose expression is driven by the NRF2 transcription factor highly active in the prooxidant milieu found in human malignancies. The resulting abundance of NQO1 expression (up to 200-fold) in cancers and a barely detectable expression in body tissues makes it a selective marker of neoplasms. NQO1 can catalyze the repeated futile redox cycling of certain natural and synthetic quinones to their hydroxyquinones, consuming NADPH and generating rapid bursts of cytotoxic reactive oxygen species (ROS) and H₂O₂. A greater level of this quinone bioactivation due to elevated NQO1 content has been recognized as a tumor-specific therapeutic strategy, which, however, has not been clinically exploited. We review here the natural and new quinones activated by NQO1, the catalytic inhibitors, and the ensuing cell death mechanisms. Further, the cancer-selective expression of NQO1 has opened excellent opportunities for distinguishing cancer cells/tissues from their normal counterparts. Given this diagnostic, prognostic, and therapeutic importance, we and others have engineered a large number of specific NQO1 turn-on small molecule probes that remain latent but release intense fluorescence groups at near-infrared and other wavelengths, following enzymatic cleavage in cancer cells and tumor masses. This sensitive visualization/quantitation and powerful imaging technology based on NQO1 expression offers promise for guided cancer surgery, and the reagents suggest a theranostic potential for NQO1-targeted chemotherapy. Full article

Infrared images are susceptible to interference from a variety of factors during acquisition and transmission, resulting in the inclusion of mixed noise, which seriously affects the accuracy of subsequent vision tasks. To solve this problem, we designed a mixed noise removal algorithm for infrared images based on improved CycleGAN. First, we proposed a ResNet-E Block that incorporates the EMA (Efficient Multi-Scale Attention Module) and build a generator based on it using the skip-connection structure to improve the network’s ability to remove mixed noise of different strengths. Second, we added the PSNR (Peak Signal-to-Noise Ratio) as an extra calculation item of cycle consistency loss, so that the network can effectively retain the detailed information of infrared images while denoising. Finally, we conducted experimental validation on both synthetic noisy images and real noisy images, which proved that our algorithm can effectively remove the mixed noise in infrared images and the denoising effect is better than other similar methods. Full article

This paper explores a data augmentation approach for images of rigid bodies, particularly focusing on electrical equipment and analogous industrial objects. By leveraging manufacturer-provided datasheets containing precise equipment dimensions, we employed straightforward algorithms to generate synthetic images, permitting the expansion of the training dataset from a potentially unlimited viewpoint. In scenarios lacking genuine target images, we conducted a case study using two well-known detectors, representing two machine-learning paradigms: the Viola–Jones (VJ) and You Only Look Once (YOLO) detectors, trained exclusively on datasets featuring synthetic images as the positive examples of the target equipment, namely lightning rods and potential transformers. Performances of both detectors were assessed using real images in both visible and infrared spectra. YOLO consistently demonstrates $F_1$ scores below 26% in both spectra, while VJ’s scores lie in the interval from 38% to 61%. This performance discrepancy is discussed in view of paradigms’ strengths and weaknesses, whereas the relatively high scores of at least one detector are taken as empirical evidence in favor of the proposed data augmentation approach. Full article

This paper presents the first insight into how Almada Negreiros, a key artist of the first generation of modernism in Portugal, created his mural painting masterpiece in the maritime station of Rocha do Conde de Óbidos in Lisbon. This set of six monumental mural paintings dates from 1946 to 1949 and is considered Almada’s artistic epitome. As part of the ALMADA project: Unveiling the mural painting art of Almada Negreiros, the murals are being analyzed from a technical and material perspective to understand his modus operandi and the material used. This is the first study of this nature carried out on site and in the laboratory using standard and more advanced imaging, non-invasive analysis, and microanalysis techniques. This article reports the results obtained with visual examination, technical photography in visible (Vis), visible raking (Vis-Rak), complemented by 2D and 3D optical microscopy (OM), scanning electron microscopy with energy-dispersive spectrometry (SEM-EDS), and Fourier transform infrared micro-spectroscopy (µ-FTIR) of the paint layers. The results show the similarities, differences, and technical difficulties that the painter may have had when working on the first, third, and presumably last mural to be painted. Vis-Rak light images were particularly useful in providing a clear idea of how the work progressed from top to bottom through large sections of plaster made with lime mortars. It also revealed an innovative pounced technique used by Almada Negreiros to transfer the drawings in full scale to the walls. Other technical characteristics highlighted by the analytical setup are the use of textured, opaque, and transparent paint layers. The structure of the paintings does not follow a rigid build-up from light to dark, showing that the artist freely adapted according to the motif represented. As far as the colour palette is concerned, Almada masterfully uses primary and complementary colours made with Fe-based pigments and with synthetic ultramarine blue, cadmium pigments, and emerald green. Full article

This study presents a novel method for the nighttime detection of waterborne individuals using an enhanced YOLOv5s algorithm tailored for infrared thermal imaging. To address the unique challenges of nighttime water rescue operations, we have constructed a specialized dataset comprising 5736 thermal images collected from diverse aquatic environments. This dataset was further expanded through synthetic image generation using CycleGAN and a newly developed color gamut transformation technique, which significantly improves the data variance and model training effectiveness. Furthermore, we integrated the Convolutional Block Attention Module (CBAM) at the end of the last encoder’s feedforward network. This integration maximizes the utilization of channel and spatial information to capture more intricate details in the feature maps. To decrease the computational demands of the network while maintaining model accuracy, Ghost convolution was employed, thereby boosting the inference speed as much as possible. Additionally, we applied hyperparameter evolution to refine the training parameters. The improved algorithm achieved an average detection accuracy of 85.49% on our proprietary dataset, significantly outperforming its predecessor, with a prediction speed of 23.51 FPS. The experimental outcomes demonstrate the proposed solution’s high recognition capabilities and robustness, fulfilling the demands of intelligent lifesaving missions. Full article