Search Results (255)

In the field of drone-to-drone detection tasks, the issue of fusing temporal information with infrared and visible light data for detection has been rarely studied. This paper presents the first temporal misaligned rgb-thermal dataset for drone-to-drone target detection, named TMRGBT-D2D. The dataset covers various lighting conditions (i.e., high-light scenes captured during the day, medium-light and low-light scenes captured at night, with night scenes accounting for 38.8% of all data), different scenes (sky, forests, buildings, construction sites, playgrounds, roads, etc.), different seasons, and different locations, consisting of a total of 42,624 images organized into sequential frames extracted from 19 RGB-T video pairs. Each frame in the dataset has been meticulously annotated, with a total of 94,323 annotations. Except for drones that cannot be identified under extreme conditions, infrared and visible light annotations are one-to-one corresponding. This dataset presents various challenges, including small object detection (the average size of objects in visible light images is approximately 0.02% of the image area), motion blur caused by fast movement, and detection issues arising from imaging differences between different modalities. To our knowledge, this is the first temporal misaligned rgb-thermal dataset for drone-to-drone target detection, providing convenience for research into rgb-thermal image fusion and the development of drone target detection. Full article

Objectives: Anterior petrosectomy is a challenging neurosurgical procedure requiring precise identification and preservation of multiple critical structures. This technical note explores the feasibility of using real-time near-infrared indocyanine green (NIR-ICG) fluorescence with white light overlay to enhance visualization of the petrous internal carotid artery (ICA) during transpetrosal drilling. We aimed to assess its utility for planning and performing modified Dolenc–Kawase drilling. Methods: We integrated NIR-ICG and white light overlay using a robotic microscope with simultaneous visualization capabilities. This technique was applied to improve neurovascular preservation and skull base landmark identification. Intraoperative video frames and images were captured during an anterior transpetrosal approach for a petroclival meningioma, with technical details, surgical time, and feedback documented. Results: Real-time NIR-ICG with white light overlay successfully identified the posterior genu, horizontal petrosal segment, anterior genu, and superior petrosal sinus. It facilitated precise localization of cochlear landmarks, enabling tailored drilling of the Dolenc–Kawase rhomboid according to patient anatomy and accommodating potential anatomical variants. Conclusions: This approach could enhance intraoperative safety and improve exposure, possibly reducing neurovascular risks without extending operative time. It may serve as a valuable adjunct for complex skull base surgeries. Full article

Using a camera system developed earlier for monitoring the behavior of lemmings under the snow, we are now able to record a large number of short image sequences from this rodent which plays a central role in the Arctic food web. Identifying lemming species in these images manually is wearisome and time-consuming. To perform this task, we present a deep neural network which has several million parameters to configure. Training a network of such an immense size with conventional methods requires a huge amount of data but a sufficiently large labeled dataset of lemming images is currently lacking. Another challenge is that images are obtained in darkness in the near infrared spectrum, causing the loss of some image texture information. We investigate whether these challenges can be tackled by a transfer learning approach in which a network is pretrained on a dataset of visible spectrum images that does not include lemmings. We believe this work provides a basis for moving toward developing intelligent software programs that can facilitate the analysis of videos by biologists. Full article

Gas leaks threaten ecological and social safety. Non-contact infrared imaging enables large-scale, real-time measurements; however, in complex environments, weak signals from small leaks can hinder reliable detection. This study proposes a novel automated methane leak detection method based on infrared imaging and a Gas-Faster Region-based convolutional neural network (Gas R-CNN) to classify leakage amounts (≥30 mL/min). An uncooled infrared imaging system was employed to capture gas leak images containing leak volume features. We developed the Gas R-CNN model for gas leakage detection. This model introduces a multiscale feature network to improve leak feature extraction and enhancement, and it incorporates region-of-interest alignment to address the mismatch caused by double quantization. Feature extraction was enhanced by integrating ResNet50 with an efficient channel attention mechanism. Image enhancement techniques were applied to expand the dataset diversity. Leak detection capabilities were validated using the IOD-Video dataset, while the constructed gas dataset enabled the first quantitative leak assessment. The experimental results demonstrated that the model can accurately detect the leakage area and classify leakage amounts, enabling the quantitative analysis of infrared images. The proposed method achieved average precisions of 0.9599, 0.9647, and 0.9833 for leak rates of 30, 100, and 300 mL/min, respectively. Full article

Bats are ecologically vital mammals, serving as pollinators, seed dispersers, and bioindicators of ecosystem health. Many species inhabit natural caves, which offer optimal conditions for survival but present challenges for direct ecological monitoring due to their dark, complex, and inaccessible environments. Traditional monitoring methods, such as mist-netting, are invasive and limited in scope, highlighting the need for non-intrusive alternatives. In this work, we present a portable multisensor platform designed to operate in underground habitats. The system captures multimodal data, including near-infrared (NIR) imagery, ultrasonic audio, 3D structural data, and RGB video. Focusing on NIR imagery, we evaluate the effectiveness of the YOLO object detection framework for automated bat detection and counting. Experiments were conducted using a dataset of NIR images collected in natural shelters. Three YOLO variants (v10, v11, and v12) were trained and tested on this dataset. The models achieved high detection accuracy, with YOLO v12m reaching a mean average precision (mAP) of 0.981. These results demonstrate that combining NIR imaging with deep learning enables accurate and non-invasive monitoring of bats in challenging environments. The proposed approach offers a scalable tool for ecological research and conservation, supporting population assessment and behavioral studies without disturbing bat colonies. Full article

Infrared video imaging is an cornerstone technology for environmental perception, particularly in drone-based remote sensing applications such as disaster assessment and infrastructure inspection. Conventional systems, however, rely on bulky optical architectures that limit deployment on lightweight aerial platforms. Computational imaging offers a promising alternative by integrating optical encoding with algorithmic reconstruction, enabling compact hardware while maintaining imaging performance comparable to sophisticated multi-lens systems. Nonetheless, achieving real-time video-rate computational image restoration on resource-constrained unmanned aerial vehicles (UAVs) remains a critical challenge. To address this, we propose Mobile Wavelet Restoration-Net (MWR-Net), a lightweight deep learning framework tailored for real-time infrared image restoration. Built on a MobileNetV4 backbone, MWR-Net leverages depthwise separable convolutions and an optimized downsampling scheme to minimize parameters and computational overhead. A novel wavelet-domain loss enhances high-frequency detail recovery, while the modulation transfer function (MTF) is adopted as an optics-aware evaluation metric. With only 666.37 K parameters and 6.17 G MACs, MWR-Net achieves a PSNR of 37.10 dB and an SSIM of 0.964 on a custom dataset, outperforming a pruned U-Net baseline. Deployed on an RK3588 chip, it runs at 42 FPS. These results demonstrate MWR-Net’s potential as an efficient and practical solution for UAV-based infrared sensing applications. Full article

Traditional culture plays a vital role in shaping national identity and emotional belonging, making it imperative to explore innovative strategies for its digital preservation and engagement. This study investigates how interaction design in cultural digital games influences users’ emotional experiences and cultural understanding. Centering on the Chinese intangible cultural heritage puppet manipulation, we developed an interactive cultural game with three modes: gesture-based interaction via Leap Motion, keyboard control, and passive video viewing. A multimodal evaluation framework was employed, integrating subjective questionnaires with physiological indicators, including Functional Near-Infrared Spectroscopy (fNIRS), infrared thermography (IRT), and electrodermal activity (EDA), to assess users’ emotional responses, immersion, and perception of cultural content. Results demonstrated that gesture-based interaction, which aligns closely with the embodied cultural behavior of puppet manipulation, significantly enhanced users’ emotional engagement and cultural comprehension compared to the other two modes. Moreover, fNIRS data revealed broader activation in brain regions associated with emotion regulation and cognitive control during gesture interaction. These findings underscore the importance of culturally congruent interaction design in enhancing user experience and emotional resonance in digital cultural applications. This study provides empirical evidence supporting the integration of cultural context into interaction strategies, offering valuable insights for the development of emotionally immersive systems for intangible cultural heritage preservation. Full article

Optical sensor technologies are reshaping obesity detection by enabling non-invasive, dynamic analysis of biomechanical and morphological biomarkers. This review synthesizes recent advances in three key areas: optical gait analysis, vision-based pose estimation, and depth-sensing voxel modeling. Gait analysis leverages optical sensor arrays and video systems to identify obesity-specific deviations, such as reduced stride length and asymmetric movement patterns. Pose estimation algorithms—including markerless frameworks like OpenPose and MediaPipe—track kinematic patterns indicative of postural imbalance and altered locomotor control. Human voxel modeling reconstructs 3D body composition metrics, such as waist–hip ratio, through infrared-depth sensing, offering precise, contactless anthropometry. Despite their potential, challenges persist in sensor robustness under uncontrolled environments, algorithmic biases in diverse populations, and scalability for widespread deployment in existing health workflows. Emerging solutions such as federated learning and edge computing aim to address these limitations by enabling multimodal data harmonization and portable, real-time analytics. Future priorities involve standardizing validation protocols to ensure reproducibility, optimizing cost-efficacy for scalable deployment, and integrating optical systems with wearable technologies for holistic health monitoring. By shifting obesity diagnostics from static metrics to dynamic, multidimensional profiling, optical sensing paves the way for scalable public health interventions and personalized care strategies. Full article

An underground utility tunnel (UUT) is essential for the efficient use of urban underground space. However, current maintenance systems rely on patrol personnel and professional equipment. This study explores industrial detection methods for identifying and monitoring natural gas leaks in UUTs. Via infrared thermal imaging gas experiments, data were acquired and a dataset established. To address the low-resolution problem of existing imaging devices, video super-resolution (VSR) was used to improve the data quality. Based on a convolutional recurrent neural network (CRNN), the image features at each moment were extracted, and the time series data were modeled to realize the risk-level classification mechanism based on the automatic classification of the leakage rate. The experimental results show that when the sliding window size was set to 10 frames, the classification accuracy of the CRNN was the highest, which could reach 0.98. This method improves early warning precision and response efficiency, offering practical technical support for UUT maintenance management. Full article

Tracking unmanned aerial vehicles (UAVs) in infrared video is an essential technology for the anti-UAV task. Given frequent UAV target disappearances caused by occlusion or moving out of view, global trackers, which have the unique ability to recapture targets, are widely used in infrared UAV tracking. However, global trackers perform poorly when dealing with large target scale variation because they cannot maintain approximate consistency between target sizes in the template and the search region. To enhance the scale adaptation of global trackers, we propose a plug-and-play scale adaptation enhancement module (SAEM). This can generate a scale adaptation enhancement kernel according to the target size in the previous frame, and then perform implicit scale adaptation enhancement on the extracted target template features. To optimize training, we introduce an auxiliary branch to supervise the learning of SAEM and add Gaussian noise to the input size to improve its robustness. In addition, we propose a one-stage anchor-free global tracker (OSGT), which has a more concise structure than other global trackers to meet the real-time requirement. Extensive experiments on three Anti-UAV Challenge datasets and the Anti-UAV410 dataset demonstrate the superior performance of our method and verify that our proposed SAEM can effectively enhance the scale adaptation of existing global trackers. Full article

Effective monitoring of fish passage through river barriers is essential for evaluating fishway performance and supporting adaptive river management. Traditional methods are often invasive, labor-intensive, or too costly to enable widespread implementation across most fishways. Infrared (IR) beam-break counters offer a promising alternative, but their adoption has been limited by high costs and a lack of flexibility. We developed and tested a novel, low-cost infrared beam-break counter—FishTracker—based on open-source Raspberry Pi and Arduino platforms. The system detects fish passages by analyzing interruptions in an IR curtain and reconstructing fish silhouettes to estimate movement, direction, speed, and morphometrics under a wide range of turbidity conditions. It also offers remote access capabilities for easy management. Field validation involved controlled tests with dummy fish, experiments with small-bodied live specimens (bleak) under varying turbidity conditions, and verification against synchronized video of free-swimming fish (koi carp). This first version of FishTracker achieved detection rates of 95–100% under controlled conditions and approximately 70% in semi-natural conditions, comparable to commercial counters. Most errors were due to surface distortion caused by partial submersion during the experimental setup, which could be avoided by fully submerging the device. Body length estimation based on passage speed and beam-interruption duration proved consistent, aligning with published allometric models for carps. FishTracker offers a promising and affordable solution for non-invasive fish monitoring in multispecies contexts. Its design, based primarily on open technology, allows for flexible adaptation and broad deployment, particularly in locations where commercial technologies are economically unfeasible. Full article

Human Activity Recognition (HAR) systems aim to understand human behavior and assign a label to each action, attracting significant attention in computer vision due to their wide range of applications. HAR can leverage various data modalities, such as RGB images and video, skeleton, depth, infrared, point cloud, event stream, audio, acceleration, and radar signals. Each modality provides unique and complementary information suited to different application scenarios. Consequently, numerous studies have investigated diverse approaches for HAR using these modalities. This survey includes only peer-reviewed research papers published in English to ensure linguistic consistency and academic integrity. This paper presents a comprehensive survey of the latest advancements in HAR from 2014 to 2025, focusing on Machine Learning (ML) and Deep Learning (DL) approaches categorized by input data modalities. We review both single-modality and multi-modality techniques, highlighting fusion-based and co-learning frameworks. Additionally, we cover advancements in hand-crafted action features, methods for recognizing human–object interactions, and activity detection. Our survey includes a detailed dataset description for each modality, as well as a summary of the latest HAR systems, accompanied by a mathematical derivation for evaluating the deep learning model for each modality, and it also provides comparative results on benchmark datasets. Finally, we provide insightful observations and propose effective future research directions in HAR. Full article

Infrared video satellites have the characteristics of wide-area long-duration surveillance, enabling continuous operation day and night compared to visible light imaging methods. Therefore, they are widely used for continuous monitoring and tracking of important targets. However, energy attenuation caused by long-distance radiation transmission reduces imaging contrast and leads to the loss of edge contours and texture details, posing significant challenges to target tracking algorithm design. This paper proposes an infrared small-target tracking method, the UIMM-Tracker, based on the tracking-by-detection (TbD) paradigm. First, detection uncertainty is measured and injected into the multi-model observation noise, transferring the distribution knowledge of the detection process to the tracking process. Second, a dynamic modulation mechanism is introduced into the Markov transition process of multi-model fusion, enabling the tracking model to autonomously adapt to targets with varying maneuvering states. Additionally, detection uncertainty is incorporated into the data association method, and a distance cost matrix between trajectories and detections is constructed based on scale and energy invariance assumptions, improving tracking accuracy. Finally, the proposed method achieves average performance scores of 68.5%, 45.6%, 56.2%, and 0.41 in IDF1, MOTA, HOTA, and precision metrics, respectively, across 20 challenging sequences, outperforming classical methods and demonstrating its effectiveness. Full article

The application of intelligent video monitoring for natural resource protection and management has become increasingly common in recent years. To enhance safety monitoring during the grazing prohibition and rest period of grassland, this paper proposes a multi-object segmentation algorithm based on mid-infrared images for all-weather surveillance. The approach integrates the Image Timing Features–Gaussian Mixture Model (ITF-GMM) and Convolutional-UNet (Con-UNet) to improve the accuracy of target detection. First, a robust background modelling, i.e., the ITF-GMM, is proposed. Unlike the basic Gaussian Mixture Model (GMM), the proposed model dynamically adjusts the learning rate according to the content difference between adjacent frames and optimizes the number of Gaussian distributions through time series histogram analysis of pixels. Second, a segmentation framework based on Con-UNet is developed to improve the feature extraction ability of UNet. In this model, the maximum pooling layer is replaced with a convolutional layer, addressing the challenge of limited training data and improving the network’s ability to preserve spatial features. Finally, an integrated computation strategy is designed to combine the outputs of ITF-GMM and Con-UNet at the pixel level, and morphological operations are performed to refine the segmentation results and suppress noises, ensuring clearer object boundaries. The experimental results show the effectiveness of proposed approach, achieving a precision of 96.92%, an accuracy of 99.87%, an intersection over union (IOU) of 94.81%, and a recall of 97.75%. Furthermore, the proposed algorithm meets real-time processing requirements, confirming its capability to enhance small-target detection in complex outdoor environments and supporting the automation of grassland monitoring and enforcement. Full article

Due to the increased age of the global population, the proportion of the elderly population continues to rise. The safety of the elderly living alone is becoming an increasingly prominent area of concern. They often miss timely treatment due to undetected falls or illnesses, which pose risks to their lives. In order to address this challenge, the technology of indoor abnormal behavior detection has become a research hotspot. This paper systematically reviews detection methods based on sensors, video, infrared, WIFI, radar, depth, and multimodal fusion. It analyzes the technical principles, advantages, and limitations of various methods. This paper further explores the characteristics of relevant datasets and their applicable scenarios and summarizes the challenges facing current research, including multimodal data scarcity, risk of privacy leakage, insufficient adaptability of complex environments, and human adoption of wearable devices. Finally, this paper proposes future research directions, such as combining generative models, federated learning to protect privacy, multi-sensor fusion for robustness, and abnormal behavior detection on the Internet of Things environment. This paper aims to provide a systematic reference for academic research and practical application in the field of indoor abnormal behavior detection. Full article