Search Results (27)

UAV-based remote sensing excels in rapid response, high timeliness, simple operation, and high degrees of automation, and has been widely applied for geological hazard monitoring. Deep learning methods based on unitemporal UAV images can only analyze the static appearance of a scene, while bitemporal change detection can capture the dynamic evolution of hazards; however, due to diverse geological landforms and topography, environmental noises such as vegetation cover, and dynamic weather conditions, change detection of geological hazards from UAV images based on traditional deep learning technology is not always effective. Therefore, there is an urgent need to utilize large vision-language models (LVLMs) to further improve the accuracy and robustness of the change detection model. Motivated by this, this paper proposes a novel remote sensing model for geological hazard monitoring, referred to as CLIFF (CLIP-BIT-EfficientNet), based on the multi-modal LVLM Contrastive Language–Image Pre-training (CLIP), the change detection network Bitemporal Image Transformer (BIT), and the classification network EfficientNet, along with corresponding datasets and model fine-tuning strategies. The proposed transfer fusion module bridges the CLIFF and BIT networks by aligning their feature distributions and dimensions, allowing the general knowledge of the LVLM and the task-specific knowledge of the learnable branch to reinforce each other. Furthermore, this integrated pipeline addresses the scarcity of labeled hazard data by allowing the BIT to train on larger public datasets, while fine-tuning EfficientNet on smaller hazard-classification datasets within the change area, making the approach more efficient and reliable than direct classification methods. Experimental results show that the proposed CLIFF algorithm outperforms state-of-the-art deep learning algorithms such as LightCDNet and ChangeFormer, with an IoU of 75.74% and an F1 score of 0.8689 for change detection. Meanwhile, CLIFF has an overall accuracy rate of 86.89% in identifying geological hazards along gas pipelines, such as crude oil spills, collapses, landslides, and floods, with per-class accuracies of 87.32% and 86.17% for crude oil spills and landslides, respectively. Full article

The proliferation of IoT-based surveillance has caused a sharp rise in video data, straining network bandwidth and cloud storage. Conventional video compression exploits pixel-level redundancy but ignores the semantic importance of content, transmitting large volumes of redundant background. This paper proposes a semantic-aware video offloading framework that improves bandwidth efficiency in cloud-based surveillance. DeepLabV3+ with a ResNet-50 backbone performs semantic segmentation at the edge to extract relevant foreground objects (e.g., pedestrians and vehicles) while suppressing static background. A background reference caching mechanism transmits the static scene once and reuses it at the cloud for full-frame reconstruction, minimizing redundant transmission. On a dataset of 12 surveillance sequences (self-captured videos plus sequences from the CDnet 2014 benchmark), the method achieves up to 74.63% reduction in transmitted data, a 33% improvement in storage efficiency, and a compression ratio of 2.88×, while maintaining an average PSNR of 44.92 dB. Paired t-tests ($p<0.001$) and sensitivity analysis across varying scene dynamics and semantic configurations confirm the robustness of the approach, and comparisons indicate clear gains over conventional motion-based offloading in bandwidth efficiency and reconstruction fidelity. Full article

Foreground segmentation and background subtraction are critical components in many computer vision applications, such as intelligent video surveillance, urban security systems, and obstacle detection for autonomous vehicles. Although extensively studied over the past decades, these tasks remain challenging, particularly due to rapid illumination changes, dynamic backgrounds, cast shadows, and camera movements. The emergence of supervised deep learning-based methods has significantly enhanced performance, surpassing traditional approaches on the benchmark dataset CDnet2014. In this context, this paper provides a comprehensive review of recent supervised deep learning techniques applied to background subtraction, along with an in-depth comparative analysis of state-of-the-art approaches available on the official CDnet2014 results platform. Specifically, we examine several key architecture families, including convolutional neural networks (CNN and FCN), encoder–decoder models such as FgSegNet and Motion U-Net, adversarial frameworks (GAN), Transformer-based architectures, and hybrid methods combining intermittent semantic segmentation with rapid detection algorithms such as RT-SBS-v2. Beyond summarizing existing works, this review contributes a structured cross-family comparison under a unified benchmark, a focused analysis of performance behavior across challenging CDnet2014 scenarios, and a critical discussion of the trade-offs between segmentation accuracy, robustness, and computational efficiency for practical deployment. Full article

In this paper, we present a novel method for background estimation and updating in video sequences, utilizing an innovative approach that combines an intelligent truncated mean, the stationary wavelet transform (SWT), and advanced thresholding techniques. This method aims to significantly enhance the accuracy of moving object detection by mitigating the impact of outliers and adapting background estimation to dynamic scene conditions. The proposed approach begins with a robust initial background estimation, followed by moving object detection through frame subtraction and gamma correction. Segmentation is then performed using SWT, coupled with adaptive thresholding methods, including hard and soft thresholding. These techniques work in tandem to effectively reduce noise while preserving critical details. Finally, the background is selectively updated to integrate new information from static regions while excluding moving objects, ensuring a precise and robust detection system. Experimental evaluation on the CDnet 2014 and SBI 2015 datasets demonstrates that the proposed method improves the F1 score by 12.5 percentage points (from 0.7511 to 0.8765), reduces false positives by up to 65%, and achieves higher PSNR values compared to GMM_Zivk, SuBSENSE, and SC_SOBS. These results confirm the robustness of the hybrid approach based on truncated mean and SWT in dynamic and challenging environments. Full article

The high operational velocities of high-speed trains present constraints on their onboard track intrusion detection systems for real-time capture and analysis, encompassing limited computational resources and motion image blurring. This emphasizes the critical necessity of track perimeter intrusion monitoring systems. Consequently, an intelligent monitoring system employing trackside cameras is constructed, integrating weakly supervised video anomaly detection and unsupervised foreground segmentation, which offers a solution for monitoring foreign objects on high-speed train tracks. To address the challenges of complex dataset annotation and unidentified target detection, weakly supervised learning detection is proposed to track foreign object intrusions based on video. The pretraining of Xception3D and the integration of multiple attention mechanisms have markedly enhanced the feature extraction capabilities. The Top-K sample selection alongside the amplitude score/feature loss function effectively discriminates abnormal from normal samples, incorporating time-smoothing constraints to ensure detection consistency across consecutive frames. Once abnormal video frames are identified, a multiscale variational autoencoder is proposed for the positioning of foreign objects. A downsampling/upsampling module is optimized to increase feature extraction efficiency. The pixel-level background weight distribution loss function is engineered to jointly balance background authenticity and noise resistance. Ultimately, the experimental results indicate that the video anomaly detection model achieved an AUC of 0.99 on the track anomaly detection dataset and processes 2 s video segments in 0.41 s. The proposed foreground segmentation algorithm achieved an F1 score of 0.9030 in the track anomaly dataset and 0.8375 on CDnet2014, with 91 Frames per Second, confirming its efficacy. Full article

Building change detection involves identifying areas where buildings have changed by comparing multi-temporal remote sensing imagery of the same geographical region. Recent advances in Transformer-based methods have significantly improved remote sensing change detection. However, current Transformer models still exhibit persistent limitations in effectively extracting multi-scale semantic features within complex scenarios. To more effectively extract multi-scale semantic features in complex scenes, we propose a novel model, which is the Transformer-based Multi-Scale Semantic Extraction Change Detection Network (MSSE-CDNet). The model employs a Siamese network architecture to enable precise change recognition. MSSE-CDNet comprises four parts, which together contain five modules: (1) a CNN feature extraction module, (2) a multi-scale semantic extraction module, (3) a Transformer encoder and decoder module, and (4) a prediction module. Comprehensive experiments on the standard LEVIR-CD benchmark for building change detection demonstrate our approach’s superiority over state-of-the-art methods. Compared to existing models such as FC-Siam-Di, FC-Siam-Conc, DTCTSCN, BIT, and SNUNet, MSSE-CDNet achieves significant and consistent gains in performance metrics, with F1 scores improved by 4.22%, 6.84%, 2.86%, 1.22%, and 2.37%, respectively, and Intersection over Union (IoU) improved by 6.78%, 10.74%, 4.65%, 2.02%, and 3.87%, respectively. These results robustly substantiate the effectiveness of our framework on an established benchmark dataset. Full article

Pedestrian panic behavior is a primary cause of overcrowding and stampede accidents in public micro-road network areas with high pedestrian density. However, reliably detecting such behaviors remains challenging due to their inherent complexity, variability, and stochastic nature. Current detection models often rely on single-modality features, which limits their effectiveness in complex and dynamic crowd scenarios. To overcome these limitations, this study proposes a contour-driven multimodal framework that first employs a CNN (CDNet) to estimate density maps and, by analyzing steep contour gradients, automatically delineates a candidate panic zone. Within these potential panic zones, pedestrian trajectories are analyzed through LSTM networks to capture irregular movements, such as counterflow and nonlinear wandering behaviors. Concurrently, semantic recognition based on Transformer models is utilized to identify verbal distress cues extracted through Baidu AI’s real-time speech-to-text conversion. The three embeddings are fused through a lightweight attention-enhanced MLP, enabling end-to-end inference at 40 FPS on a single GPU. To evaluate branch robustness under streaming conditions, the UCF Crowd dataset (150 videos without panic labels) is processed frame-by-frame at 25 FPS solely for density assessment, whereas full panic detection is validated on 30 real Itaewon-Stampede videos and 160 SUMO/Unity simulated emergencies that include explicit panic annotations. The proposed system achieves 91.7% accuracy and 88.2% F1 on the Itaewon set, outperforming all single- or dual-modality baselines and offering a deployable solution for proactive crowd safety monitoring in transport hubs, festivals, and other high-risk venues. Full article

To address the challenges of accurately segmenting irregular building boundaries in complex urban environments faced by existing remote sensing change detection methods, this paper proposes a building change detection network based on multilevel geometric representation optimization using frame fields called BuildingCDNet. The proposed method employs a multi-scale feature aggregation encoder–decoder architecture, leveraging contextual information to capture the characteristics of buildings of varying sizes in the imagery. Cross-attention mechanisms are incorporated to enhance the feature correlations between the change pairs. Additionally, the frame field is introduced into the network to model the complex geometric structure of the building target. By learning the local orientation information of the building structure, the frame field can effectively capture the geometric features of complex building features. During the training process, a multi-task learning strategy is used to align the predicted frame field with the real building outline, while learning the overall segmentation, edge outline, and corner point features of the building. This improves the accuracy of the building polygon representation. Furthermore, a discriminative loss function is constructed through multi-task learning to optimize the polygonal structured information of the building targets. The proposed method achieves state-of-the-art results on two commonly used datasets. Full article

Moving object detection is a vital research area that plays an essential role in intelligent transportation systems (ITSs) and various applications in computer vision. Recently, researchers have utilized convolutional neural networks (CNNs) to develop new techniques in object detection and recognition. However, with the increasing number of machine learning strategies used for object detection, there has been a growing need for large datasets with accurate ground truth used for the training, usually demanding their manual labeling. Moreover, most of these deep strategies are supervised and only applicable for specific scenes with large computational resources needed. Alternatively, other object detection techniques such as classical background subtraction need low computational resources and can be used with general scenes. In this paper, we propose a new a reliable semi-automatic method that combines a modified version of the detection-based CNN You Only Look Once V4 (YOLOv4) technique and background subtraction technique to perform an unsupervised object detection for surveillance videos. In this proposed strategy, background subtraction-based low-rank decomposition is applied firstly to extract the moving objects. Then, a clustering method is adopted to refine the background subtraction (BS) result. Finally, the refined results are used to fine-tune the modified YOLO v4 before using it in the detection and classification of objects. The main contribution of this work is a new detection framework that overcomes manual labeling and creates an automatic labeler that can replace manual labeling using motion information to supply labeled training data (background and foreground) directly from the detection video. Extensive experiments using real-world object monitoring benchmarks indicate that the suggested framework obtains a considerable increase in mAP compared to state-of-the-art results on both the CDnet 2014 and UA-DETRAC datasets. Full article

The challenge of detecting changes in high-resolution remote sensing imagery often stems from the difficulties in effectively extracting features and constructing appropriate change detection models considering the scale characteristics of ground objects. To solve these issues, we propose a novel UNet 3+ change detection method that considers the scale characteristics inherent in various land-cover change types. Our method includes three key steps: a multi-scale segmentation method, a class-specific UNet 3+ method, and an object-oriented change detection method based on UNet 3+. To verify the effectiveness of this method, we select two datasets for experiments and compare our proposed method with the UNet 3+ single-scale sampling method, the class-specific UNet 3+ single-scale sampling method, and the UNet 3+ multi-scale hierarchical sampling method. Our experimental results show that our proposed method has higher overall accuracy and F1, lower missed detection rate and false detection rate, and can detect more changes in ground features than other methods. To verify the scalability of this method, we compare this method with traditional change detection methods such as PCA-k-means, OCVA, a single-scale sampling method based on random forest, and a class-specific object-based method. Experimental results and accuracy indexes show that our proposed method better considers the scale characteristics of ground objects and achieves higher accuracy. Additionally, we compared our proposed method with other DLCD methods including LamboiseNet, BIT, CDNet, FCSiamConc, and FCSiamDiff. Our results show that our proposed method effectively considers edge information and has an acceptable time consumption. Our approach not only considers the full-scale characteristics of the feature extraction but also the scale characteristics of the change detection model. In addition, it considers a more practical feature extraction unit (object), making it more accurate. Full article

Motion object detection (MOD) with freely moving cameras is a challenging task in computer vision. To extract moving objects, most studies have focused on the difference in motion features between foreground and background, which works well for dynamic scenes with relatively regular movements and variations. However, abrupt illumination changes and occlusions often occur in real-world scenes, and the camera may also pan, tilt, rotate, and jitter, etc., resulting in local irregular variations and global discontinuities in motion features. Such complex and changing scenes bring great difficulty in detecting moving objects. To solve this problem, this paper proposes a new MOD method that effectively leverages local and global visual information for foreground/background segmentation. Specifically, on the global side, to support a wider range of camera motion, the relative inter-frame transformations are optimized to absolute transformations referenced to intermediate frames in a global form after enriching the inter-frame matching pairs. The global transformation is fine-tuned using the spatial transformer network (STN). On the local side, to address the problem of dynamic background scenes, foreground object detection is optimized by utilizing the pixel differences between the current frame and the local background model, as well as the consistency of local spatial variations. Then, the spatial information is combined using optical flow segmentation methods, enhancing the precision of the object information. The experimental results show that our method achieves a detection accuracy improvement of over 1.5% compared with the state-of-the-art methods on the datasets of CDNET2014, FBMS-59, and CBD. It demonstrates significant effectiveness in challenging scenarios such as shadows, abrupt changes in illumination, camera jitter, occlusion, and moving backgrounds. Full article

Deep Learning (DL) has become a popular method for Remote Sensing (RS) Change Detection (CD) due to its superior performance compared to traditional methods. However, generating extensive labeled datasets for DL models is time-consuming and labor-intensive. Additionally, the imbalance between changed and unchanged areas in object CD datasets, such as buildings, poses a critical issue affecting DL model efficacy. To address this issue, this paper proposes a change detection enhancement method using artificial image generation and attention mechanism. Firstly, the content of the imbalanced CD dataset is enhanced using a data augmentation strategy that synthesizes effective building CD samples using artificial RS image generation and building label creation. The created building labels, which serve as new change maps, are fed into a generator model based on a conditional Generative Adversarial Network (c-GAN) to generate high-resolution RS images featuring building changes. The generated images with their corresponding change maps are then added to the CD dataset to create the balance between changed and unchanged samples. Secondly, a channel attention mechanism is added to the proposed Adversarial Change Detection Network (Adv-CDNet) to boost its performance when training on the imbalanced dataset. The study evaluates the Adv-CDNet using WHU-CD and LEVIR-CD datasets, with WHU-CD exhibiting a higher degree of sample imbalance compared to LEVIR-CD. Training the Adv-CDNet on the augmented dataset results in a significant 16.5% F1-Score improvement for the highly imbalanced WHU-CD. Moreover, comparative analysis showcases the superior performance of the Adv-CDNet when complemented with the attention module, achieving a 6.85% F1-Score enhancement. Full article

In recent years, computer vision has witnessed remarkable advancements in image classification, specifically in the domains of fully convolutional neural networks (FCNs) and self-attention mechanisms. Nevertheless, both approaches exhibit certain limitations. FCNs tend to prioritize local information, potentially overlooking crucial global contexts, whereas self-attention mechanisms are computationally intensive despite their adaptability. In order to surmount these challenges, this paper proposes cross-and-diagonal networks (CDNet), innovative network architecture that adeptly captures global information in images while preserving local details in a more computationally efficient manner. CDNet achieves this by establishing long-range relationships between pixels within an image, enabling the indirect acquisition of contextual information. This inventive indirect self-attention mechanism significantly enhances the network’s capacity. In CDNet, a new attention mechanism named “cross and diagonal attention” is proposed. This mechanism adopts an indirect approach by integrating two distinct components, cross attention and diagonal attention. By computing attention in different directions, specifically vertical and diagonal, CDNet effectively establishes remote dependencies among pixels, resulting in improved performance in image classification tasks. Experimental results highlight several advantages of CDNet. Firstly, it introduces an indirect self-attention mechanism that can be effortlessly integrated as a module into any convolutional neural network (CNN). Additionally, the computational cost of the self-attention mechanism has been effectively reduced, resulting in improved overall computational efficiency. Lastly, CDNet attains state-of-the-art performance on three benchmark datasets for similar types of image classification networks. In essence, CDNet addresses the constraints of conventional approaches and provides an efficient and effective solution for capturing global context in image classification tasks. Full article

Dynamic monitoring of cropland using high spatial resolution remote sensing images is a powerful means to protect cropland resources. However, when a change detection method based on a convolutional neural network employs a large number of convolution and pooling operations to mine the deep features of cropland, the accumulation of irrelevant features and the loss of key features will lead to poor detection results. To effectively solve this problem, a novel cropland change detection network (CroplandCDNet) is proposed in this paper; this network combines an adaptive receptive field and multiscale feature transmission fusion to achieve accurate detection of cropland change information. CroplandCDNet first effectively extracts the multiscale features of cropland from bitemporal remote sensing images through the feature extraction module and subsequently embeds the receptive field adaptive SK attention (SKA) module to emphasize cropland change. Moreover, the SKA module effectively uses spatial context information for the dynamic adjustment of the convolution kernel size of cropland features at different scales. Finally, multiscale features and difference features are transmitted and fused layer by layer to obtain the content of cropland change. In the experiments, the proposed method is compared with six advanced change detection methods using the cropland change detection dataset (CLCD). The experimental results show that CroplandCDNet achieves the best F1 and OA at 76.04% and 94.47%, respectively. Its precision and recall are second best of all models at 76.46% and 75.63%, respectively. Moreover, a generalization experiment was carried out using the Jilin-1 dataset, which effectively verified the reliability of CroplandCDNet in cropland change detection. Full article

Change detection is a crucial task in remote sensing that finds broad application in land resource planning, forest resource monitoring, natural disaster monitoring, and evaluation. In this paper, we propose a change detection model for cross-domain recognition, which we call CrossCDNet. Our model significantly improves the modeling ability of the change detection on one dataset and demonstrates good generalization on another dataset without any additional operations. To achieve this, we employ a Siamese neural network for change detection and design an IBNM (Instance Normalization and Batch Normalization Module) that utilizes instance normalization and batch normalization in order to serve as the encoder backbone in the Siamese neural network. The IBNM extracts feature maps for each layer, and the Siamese neural network fuses the feature maps of the two branches using a unique operation. Finally, a simple MLP decoder is used for end-to-end change detection. We train our model on the LEVIR-CD dataset and achieve competitive performance on the test set. In cross-domain dataset testing, CrossCDNet outperforms all the other compared models. Specifically, our model achieves an F1-score of 91.69% on the LEVIR-CD dataset and an F1-score of 77.09% on the WHU-CD dataset, where the training set was LEVIR-CD. Full article