CLIFF: A Multi-Modal Remote Sensing Model for Geological Hazard Monitoring Based on Bitemporal UAV Images

Highlights

What are the main findings?

• A CLIP-based multi-modal framework (CLIFF) is proposed for simultaneous geological hazard classification and change detection from bitemporal UAV images.
• CLIFF achieves an IoU of 75.74% for change detection and an accuracy of 86.89% for hazard classification, outperforming existing deep learning methods.

What are the implications of the main findings?

• The transfer fusion module improves multi-scale feature representation and enhances robustness to complex geological scene variations.
• The proposed framework provides an effective solution for intelligent geological hazard monitoring and rapid post-disaster assessment using UAV imagery.

Abstract

UAV-based remote sensing excels in rapid response, high timeliness, simple operation, and high degrees of automation, and has been widely applied for geological hazard monitoring. Deep learning methods based on unitemporal UAV images can only analyze the static appearance of a scene, while bitemporal change detection can capture the dynamic evolution of hazards; however, due to diverse geological landforms and topography, environmental noises such as vegetation cover, and dynamic weather conditions, change detection of geological hazards from UAV images based on traditional deep learning technology is not always effective. Therefore, there is an urgent need to utilize large vision-language models (LVLMs) to further improve the accuracy and robustness of the change detection model. Motivated by this, this paper proposes a novel remote sensing model for geological hazard monitoring, referred to as CLIFF (CLIP-BIT-EfficientNet), based on the multi-modal LVLM Contrastive Language–Image Pre-training (CLIP), the change detection network Bitemporal Image Transformer (BIT), and the classification network EfficientNet, along with corresponding datasets and model fine-tuning strategies. The proposed transfer fusion module bridges the CLIFF and BIT networks by aligning their feature distributions and dimensions, allowing the general knowledge of the LVLM and the task-specific knowledge of the learnable branch to reinforce each other. Furthermore, this integrated pipeline addresses the scarcity of labeled hazard data by allowing the BIT to train on larger public datasets, while fine-tuning EfficientNet on smaller hazard-classification datasets within the change area, making the approach more efficient and reliable than direct classification methods. Experimental results show that the proposed CLIFF algorithm outperforms state-of-the-art deep learning algorithms such as LightCDNet and ChangeFormer, with an IoU of 75.74% and an F1 score of 0.8689 for change detection. Meanwhile, CLIFF has an overall accuracy rate of 86.89% in identifying geological hazards along gas pipelines, such as crude oil spills, collapses, landslides, and floods, with per-class accuracies of 87.32% and 86.17% for crude oil spills and landslides, respectively.

1. Introduction

Over the past two decades, the frequency of natural hazard incidents has steadily increased on a global scale, resulting in significant human casualties, threats to diverse species, widespread infrastructure damage, and profound social and economic disruptions [1]. As such, the occurrence of disasters, both of natural and human origin, should be monitored in a timely manner to mitigate potential losses.

Currently, the mainstream geological hazard monitoring technologies include geodesy [1], global positioning system (GPS) [2], close-range photography [3], and remote sensing image detection [1]. Remote sensing image detection, especially that based on UAV images, has occupied a core position in the field of geological hazard monitoring due to its advantages of rapid response, high timeliness, simple operation, and high degrees of automation [4]. UAV-based remote sensing overcomes the limitations of traditional aerial remote sensing, including long flight durations, adverse weather conditions, and dangerous environments [5]. Recent studies have further advanced UAV-based hazard monitoring from different perspectives: Fu et al. [6] proposed a reinforcement learning-based low-delay data collection scheme in UAV-assisted IoT for secondary geological hazard monitoring, and Li et al. [7] developed a UAV photogrammetry method with low image overlap for panoramic imaging and river morphology monitoring. Therefore, it is considered an effective solution for dealing with remote areas such as rural and mountainous areas, where conventional hazard monitoring measures become inaccessible.

After acquiring remote sensing image data, pre-processing and analysis are usually required to accurately extract ground object information [8]. With the advancement of machine learning technology, image classification technology based on deep learning methods and convolutional neural networks (CNNs) has been widely studied in the field of geological hazard monitoring. For example, Fang et al. [9] proposed a deep learning method based on Light Detection and Ranging (LiDAR) data for automatic identification of historical landslides. In order to solve the processing difficulties of multi-source remote sensing data fusion, Li et al. [10] proposed a Transformer-based neural network that can identify landslides from multi-modal data, including a digital elevation model (DEM), a hillside shadow map, and an optical image. Compared with traditional visual interpretation methods, deep learning can greatly improve the accuracy and efficiency of detecting and identifying geological hazards.

However, geographical hazards are not only complex but also spatiotemporally varying. More specifically, the types and ranges of hazards change dynamically over places and times. Due to diverse geological landforms and topography, and environmental factors such as vegetation cover and dynamic weather conditions, detecting geological hazards from UAV images using traditional deep learning methods is not always effective. Large vision-language models (LVLMs) (a.k.a. foundation models) are pre-trained on massive unlabeled data, which gives them a higher generalization ability than traditional deep learning models. Thus, there is an urgent need to utilize LVLMs to further improve the accuracy and robustness of change detection in geographical hazard monitoring.

Motivated by this, this paper proposes a novel remote sensing model for geological hazard monitoring, referred to as CLIFF, based on the multi-modal LVLM Contrastive Language–Image Pre-training (CLIP) [11], the change detection network Bitemporal Image Transformer (BIT) [12], and the classification network EfficientNet [13], along with corresponding datasets and model fine-tuning strategies. Firstly, fixed-point aerial images of the same spot taken by a UAV at different times are used as input. The Oriented FAST and Rotated BRIEF (ORB) algorithm, the Brute-Force method, and the affine transformation method are employed to achieve image alignment based on key point matching. Subsequently, the CLIP image feature extractor in the proposed network is used to extract features from the two aligned images. After that, BIT in the proposed network is used to locate the position of geological hazards. Finally, EfficientNet in the proposed network is used to determine the type of geological hazards. In this way, the transferability of CLIFF is significantly improved by combining LVLM with state-of-the-art deep learning models.

In summary, the main problem addressed in this study is the reliable detection and classification of geological hazards along long-distance oil and gas pipelines from bitemporal UAV images, under diverse landforms, environmental noises, and dynamic weather conditions. The research gap is twofold: existing deep learning-based change detection methods suffer from limited transferability in heterogeneous environments, and the application of LVLMs to bitemporal remote sensing change detection, particularly under scarce hazard-labeled data, remains largely unexplored. Accordingly, the main objectives of this study are: (i) to develop a multi-modal framework that transfers the general knowledge of a frozen LVLM to the change detection task through a dedicated transfer fusion module; (ii) to design a fine-tuning strategy and a detect-then-classify pipeline suitable for scarce hazard-labeled UAV data; and (iii) to validate the framework on real-world pipeline monitoring data in the Sichuan Basin.

The main contributions of this paper are listed as follows.

• First, the multi-modal remote sensing model, CLIFF, is proposed. The LVLM, CLIP, is used as the backbone network for image feature extraction, which enhances the accuracy of change detection and the model’s generalization ability. Meanwhile, the transfer fusion module is integrated into the BIT change detection network to enable detailed analysis of image features and accurate description of change features, followed by EfficientNet for geological hazard type classification.
• Second, a model fine-tuning method suitable for the field of remote sensing image change detection is proposed. Experimental results show that the proposed CLIFF algorithm outperforms state-of-the-art deep learning methods, such as CDNet and ChangeFormer, achieving an intersection-over-union (IoU) of 75.74% in change detection. Meanwhile, CLIFF has an accuracy rate of 86.89% in identifying geological hazards along gas pipelines, such as crude oil spills, collapses, landslides, and floods.

The remainder of this work is presented as follows. Section 2 reviews the state-of-the-art literature. Section 3 overviews the exemplary scenario and methodology. Section 4 illustrates the proposed CLIFF model. Section 5 provides the experimental datasets, setups, and results. Section 6 discusses this work from practical and technical perspectives. Section 7 concludes this work.

2. Related Works

UAV-based remote sensing methods have been widely used for geological hazard monitoring. For example, Eker et al. [14] used the structured light method (SFM) to generate a digital elevation model (DEM), orthophoto images, and dense point clouds from high-resolution images captured by UAVs. Subsequently, the M3C2 algorithm was used to compare bitemporal UAV images to analyze landslide deformation and support monitoring and analysis. Yan et al. [15] proposed a set of UAV point cloud filtering methods based on orthophoto images, which can effectively eliminate the influence of non-topographic elements such as vegetation when generating digital terrain models (DTMs). The landform change detection software GCD 7.0 was used to detect surface changes and unstable slope activities in the oil and gas pipeline area, providing scientific support for geological disaster prevention and engineering practice in the oil pipeline area.

The deep learning methods used in geological hazard monitoring are mostly change detection [16]. Change detection can be divided into three stages: the data-driven stage, the model-driven stage, and the data-model-knowledge-driven stage.

The data-driven change detection stage includes algorithms such as interpolation, ratio, and principal component transformation methods, which rely on physical characteristics such as spectrum and reflectance characteristics to analyze image changes. For example, Jong et al. [17] proposed an unsupervised change-detection method based on principal component analysis and k-means clustering. The image is divided into many blocks; each block is reduced to a vector using principal component analysis, and the resulting vectors are classified into changed and unchanged areas in the image using k-means clustering.

Models in the model-driven stage include simple neural networks, decision trees, random forests, and support vector machines. For example, Li et al. [18], Liu et al. [19], Gao et al. [20], and Wang et al. [21], all used simple and basic variants of CNNs to perform change detection on different types of remote sensing satellite images. The above CNN-based methods are limited by the lack of the latest neural network structures, such as residual structure and attention mechanism. Thus, they often lack deep convolutional layers and exhibit poor recognition performance when features change.

In recent years, change detection has been continuously incorporating the latest computer vision network architectures, including but not limited to skip connections, UNet structures, Transformer-based semantic analysis modules, and other architectures, evolving in a more complex direction. The modeling paradigm has shifted from pre-training to self-supervision, and it has also shown continuously improved performance.

Following this trend, the models in the data-model-knowledge-driven stage integrate cutting-edge neural networks, such as UNet and Transformer, which are modified to complete complex change detection tasks. For example, Shi et al. [22] proposed a deep supervised attention neural network, which contains a twin CNN with multiple layers of residual skip connections and deep supervision modules, and conducted experiments on aerial image datasets. Zhang et al. [23] proposed a SwinSUNet network that replaced the convolutional layers of the UNet encoder and decoder with Swin Transformer modules, enabling the network to process two images in parallel and extract multi-scale features. Chen et al. [12] used the Transformer network, together with a CNN backbone, to extract semantic information; then leveraged the Transformer structure to restore the semantic information to obtain fine features; and finally used a convolutional-layer-based prediction module to determine the image differences.

Recently, transferring pre-trained vision-language models (VLMs) to downstream visual tasks has become an active research direction, and various application patterns have been explored. A comprehensive review and evaluation of VLMs for object detection and segmentation is provided in [24]. For domain generalization, VL2V-ADiP [25] distills the aligned embeddings of a VLM teacher into a vision student to improve out-of-distribution image classification, and MFuser [26] employs a Mamba-based fusion framework to bridge vision foundation models and VLMs for domain-generalized semantic segmentation. For parameter-efficient transfer, M2IST [27] and MaPPER [28] keep the pre-trained encoders frozen and update only lightweight side-adapters or prior-guided adapters for referring expression comprehension. In addition, TaskCLIP [29] extends VLMs to task-oriented object detection through a two-stage detect-then-select design, and VLDadaptor [30] performs domain adaptive object detection via VLM distillation. These studies demonstrate the strong generalization ability of pre-trained VLMs across visual tasks. However, they mainly target single-image detection, segmentation, or grounding tasks in natural scenes. In contrast, this paper transfers a frozen CLIP encoder to bitemporal remote sensing change detection through a lightweight transfer fusion module, and further combines it with a cascaded hazard-type classification stage under the condition of scarce hazard-labeled UAV data.

In summary, when processing UAV images, the above deep learning algorithms perform well in homogeneous environments but suffer from low adaptation to heterogeneous environments. In addition, they are greatly affected by environmental noise, such as vegetation cover. The LVLM is pre-trained on massive amounts of unlabeled data and thus has stronger generalization capabilities. Therefore, we propose to combine LVLM with deep learning methods to improve the transferability and robustness of the model.

3. System Overview

This section provides an overview of the exemplary scenario and methodology.

3.1. Exemplary Scenario

In this work, we demonstrate an exemplary scenario for monitoring geological disasters around long-distance oil and gas pipelines in the Sichuan Basin, China. These pipelines transport crude oil and natural gas over long distances. Their safety and stable operation are vital to economic and social development. The spatial distribution of long-distance oil and gas pipelines used in this study, as well as their associated geological hazard risks, is shown in Figure 1.

However, since long-distance pipelines often cross complex and changeable geological environments, such as farmland, rivers, and mountains, they are exposed to significant potential threats from geological disasters, which may cause pipeline damage, oil and gas leakage, fires, explosions, and other safety-critical accidents. Therefore, real-time and effective detection and early warning of geological disasters near long-distance oil and gas pipelines is an important measure to ensure pipeline safety and prevent potential risks.

In this paper, we focus on the use of remote sensing based on UAVs to solve the above scenario. When processing UAV remote sensing images, traditional deep learning algorithms have strong scene recognition capabilities under a single type of environment, but they suffer from low transferability when applied to diverse complex environments. Besides, they are greatly affected by environmental noise, such as vegetation cover. Motivated by the generation capabilities of LVLM, the proposed CLIFF model processes UAV images by cascading a large visual model with a downstream change-detection deep learning model to monitor geological disasters around oil and gas pipelines. With high accuracy, robustness, and transferability, the proposed model facilitates the construction of a secure and sustainable solution across diverse geological landforms and topography.

3.2. Overall Structure

The overall structure of the proposed CLIFF model is depicted in Figure 2. The CLIFF model takes fixed-point aerial images of the same pipeline taken by UAVs at different times as input. First, the ORB algorithm, the Brute-Force method, and the affine transformation method are used to achieve key point-based image alignment. Then, the core modules of the CLIFF model are followed, consisting of the multi-modal LVLM, CLIP, the change detection network, BIT, and the classification network, EfficientNet. Specifically, CLIP is used to extract features, BIT is used to locate geological disasters, and EfficientNet is used to determine the type of geological hazard.

First, considering that there could exist a certain level of position deviation when the UAV collects images at a fixed point in order to facilitate the model to detect changes at different times, the images collected at corresponding fixed points at different times are aligned based on key point matching. Next, the two images are fed into the CLIP-based image feature-extraction backbone network to obtain their feature maps. Then, the BIT network is used to detect the area and range of the changes. Considering that the model inference results may include small burrs and difference areas that are negligibly small, they are excluded through post-processing, and the corresponding change detection box is obtained. Finally, the images of areas within the change detection box are input into the geological disaster type recognition and classification model, EfficientNet, to obtain the specific geological disaster type.

4. CLIFF: The Multi-Modal Remote Sensing Model for Geological Hazard Monitoring

This section details the design of the CLIFF model, including UAV image alignment, CLIP-based feature extraction, BIT-based change detection, and EfficientNet-based classification of geological hazards.

4.1. UAV Image Alignment Based on Key Point Matching

The input of the CLIFF model is a pair of bitemporal UAV remote sensing images, as shown in Figure 3a,b Affected by external factors such as positioning error, wind disturbance, and sunlight intensity, there could be errors in UAV images, such as shooting center position offset, zooming caused by different shooting heights, and shooting angle rotation, which will affect subsequent change detection. Generally, it is assumed that the UAV trajectories, heights, camera positions, and orientations are fixed, and each image pair is captured at identical positions. However, residual errors are inevitable in practice: the positioning error of the onboard navigation system, the instantaneous attitude jitter caused by wind gusts at the moment of capture, and the altitude drift across different weather conditions all translate into pixel-level offsets, rotations, and scale differences between the bitemporal images. Since pixel-level change detection is sensitive to even slight misalignment, the lightweight keypoint-based alignment is retained as a safeguard against residual errors at negligible computational cost. Therefore, it is necessary to perform geometric correction and image alignment to address the problems outlined above.

Figure 3c presents the corrected images after alignment, where images with different timestamps and the same three-dimensional geographic information are averaged to the same spatial position.

More specifically, CLIFF uses key-point matching technology to assist image alignment, and the specific steps are shown in Figure 4. The ORB algorithm is used to calculate the key point feature description of image data at different times. The Brute-Force method based on the Hamming distance is used to match feature points across images at different times, and an affine transformation is applied to perform necessary distortion and rotation to complete bitemporal image alignment.

4.1.1. ORB Algorithm

Among the pre-processing techniques, the ORB algorithm is a feature detection and description method that combines the Features from the Accelerated Segment Test (FAST) feature detector, the Harris corner response function, and the Binary Robust Independent Elementary Features (BRIEF) feature descriptor. Furthermore, the ORB algorithm improves upon them so that it is robust to rotation and scale changes while maintaining high efficiency and speed. In what follows, we explicitly describe these components.

4.1.2. FAST Feature Detector

The ORB algorithm first uses FAST, a fast corner detection algorithm, to quickly identify corner points in the image. Its core idea is to quickly determine whether a pixel is a corner by comparing the pixel values in a circular neighborhood around the pixel. From a mathematical point of view, a corner point is a location in an image where the two-dimensional gradient changes the most. At these locations, the gray-scale values not only vary in the horizontal and vertical directions, but the variation is also significant. This makes the corner points more stable under different viewing angles, rotations, scales, and lighting conditions. The FAST algorithm compares the brightness difference between the central pixel and the 16 pixels around it. If the brightness value of the central pixel is significantly different, then the point is considered a corner point. The specific steps of the FAST corner point selection algorithm are as follows:

First, we select a candidate pixel p (called a key point) and define a circular neighborhood with a radius of 3 around p, containing 16 pixels. Assume that the positions of these pixels on the circumference are denoted as $p_1,p_2,\dots ,p_{16}$.

Then, we compare the gray-scale value $I_p$ of the candidate pixel p with the gray-scale values $I_{p_i}(i=1,2,\dots ,16)$ of the 16 pixels in its circular neighborhood. The following three results may occur:

• $I_{p_i}>I_p+t$ (brighter than the center point);
• $I_{p_i}<I_p-t$ (darker than the center point);
• $I_p-t\le I_{p_i}\le I_p+t$ (darker than the center point);

where t is a threshold used to determine the difference in brightness. where t is a threshold used to determine the difference in brightness. In this work, t is set to 40 to balance the quality and quantity of the detected corners: a smaller threshold produces excessive spurious key points from noise and vegetation textures in UAV images, while a larger one retains too few corners in weakly textured regions, making the affine estimation unstable. This value lies within the commonly adopted range of the FAST detector in practice.

Finally, the statistical analysis is performed. If at least n consecutive pixels on the circumference are brighter (or darker) than the gray-scale value of the central pixel p, the candidate pixel p is considered to be a corner point. Otherwise, it is not a corner point. We set $n=12$ as the threshold for corner point determination.

4.1.3. Harris Corner Response Function

Next, the ORB algorithm uses the Harris corner response function to further screen the corner points and remove those that are not local maxima. This step helps to reduce the number of feature points and retain only the most significant ones, thus obtaining more accurate corner points. The Harris corner response function first calculates the local gray-scale gradient of each pixel in the image and constructs a matrix representing these gradients. Then, it determines which pixels are true corners by evaluating the eigenvalues of these matrices.

4.1.4. BRIEF Feature Descriptor

Finally, the ORB algorithm uses the BRIEF feature descriptor to generate a descriptor for each key point. BRIEF is a binary descriptor based on a simple comparison mechanism (i.e., gray-scale value comparison). It calculates the gray-scale gradient of the local area of the image and selects a series of pixels for gray-scale value comparison based on the size of the gray-scale gradient. These comparison results form a binary string, which serves as the descriptor for the key point. Moreover, ORB improves BRIEF, making it more robust to rotation changes. Specifically, ORB performs rotation normalization on the image before computing the BRIEF descriptor, making it insensitive to image rotation.

4.1.5. Brute-Force Matching

Following ORB, the Brute-Force matching strategy involves selecting key points from the first image and evaluating their (descriptor) similarity with all key points in the second image one by one. This evaluation process is achieved by calculating the distance between descriptors, and the distance metric used in this paper is the Hamming distance. After a one-by-one comparison, the key points in the second image that are most similar to the selected key point descriptor in the first image (i.e., the smallest Hamming distance) will be returned. This method ensures the accuracy and reliability of the matching and provides a solid foundation for subsequent image processing tasks.

4.1.6. Affine Transformation

Affine transformation is the last step in image alignment, which is a geometric transformation method that preserves lines and parallelism. The affine transformation technique is used to correct geometric distortions caused by non-ideal camera angles and to ensure alignment of the bitemporal images.

4.2. CLIP-Based Image Feature Extraction Backbone Network

Next, the LVLM, CLIP, is used as the image feature extraction backbone network. Specifically, we use the image encoder (ViT-L/14@336px structure) of the CLIP model pre-trained on a large-scale dataset, as depicted in Figure 5.

CLIP, or Contrastive Language–Image Pre-training [11], is a pre-training model launched by OpenAI. As shown in Figure 6, the structure of the CLIP network mainly includes two branches: the image encoder and text encoder. The image encoder uses ResNet50 [31] or ViT [32], and the text encoder uses a structure similar to BERT [33]. The CLIP model captures the correspondence between text and image by applying a contrastive learning strategy.

During the training phase, CLIP first converts the image and text features into vector representations using the image encoder and text encoder, respectively. Then, the similarity between these vectors is measured to determine whether the image and text are successfully paired. In order to optimize this process, the model will try to minimize the vector distance between those successfully paired text-image pairs while maximizing the vector distance between those unmatched image-text pairs. Based on this, CLIP can effectively learn the deep semantic association between images and texts.

The advantage of the CLIP model is that it shows incredible performance while maintaining a simple architecture. Without additional training (i.e., zero-shot), CLIP has achieved remarkable results across multiple tasks, including text-image retrieval, image classification, text-guided image generation, and open-field detection and segmentation, demonstrating the model’s powerful feature extraction capabilities.

In the CLIFF model based on CLIP, the general branch of the LVLM adopts the image encoder (ViT-L/14@336px) from CLIP and loads its pre-trained parameters. In this way, CLIFF makes full use of the powerful image feature-extraction capabilities obtained through massive data training, adaptively transfers general knowledge to the change detection task, and captures a wide range of knowledge and understanding of bitemporal images from low-altitude UAV remote sensing imagery. In the training of the change-region detection model based on CLIP, the parameters of this branch model are frozen and do not participate in the gradient backpropagation during the training process.

4.3. BIT-Based Change Detection

Then, the CLIFF model uses BIT, or Bitemporal Image Transformer [12], to efficiently model global semantic information in bitemporal UAV images, thereby improving the performance of change detection. The characteristic of BIT is that it does not directly capture the close relationship between pixels (i.e., extract a feature at the pixel level), but instead uses a more abstract approach. It converts the input image into several high-level semantic tokens, which contain the key information of the image. BIT models context in a compact space-time based on tokens to capture the deep connection inside the bitemporal images. In addition, BIT leverages the relationship between each pixel and these semantic tokens to enhance the feature representation in the original pixel space, further consistently highlighting the information of the region of interest. The bitemporal remote sensing images are fed into the general branch of the LVLM to obtain the semantic features of each image, and these feature maps are input into the BIT module to obtain pixel-level change prediction. The BIT network structure is shown in Figure 7.

The BIT module includes four main components: the transfer fusion module, the semantic tokenizer, the transformer encoder, and the transformer decoder. The transfer fusion module is used to fuse the multi-scale image features extracted by the image feature extraction backbone network. It effectively fuses the multi-scale feature information by capturing and aligning the spatial and semantic features between images, and enhances the local feature representation for subsequent processing; the semantic tokenizer is responsible for dividing the image features into different semantic regions, which helps the model to pay more attention to the changes in different regions when understanding the image content, thereby improving the accuracy of change detection; the transformer encoder processes the fused image features through the self-attention mechanism, captures the long-range dependencies between features, and generates high-dimensional context feature representations, further enhancing the expressiveness of the features; the transformer decoder extracts change information from the context features generated by the encoder. Through the processing of the decoder, the model can accurately locate and identify change areas between images and output the final change detection results.

4.3.1. Transfer Fusion Module

The motivation for combining frozen LVLM features with learnable features lies in their complementarity. The frozen CLIP features, obtained from large-scale image-text contrastive pre-training, provide general semantic representations that generalize well across diverse landforms, vegetation cover, and illumination conditions, but they are not optimized for the change detection task. In contrast, the learnable branch is trained end-to-end on change-detection annotations and captures task-specific fine-grained difference features, such as edge- and texture-level changes, but its generalization is limited by the scale of labeled data. The transfer fusion module bridges the two by aligning their feature distributions and dimensions, allowing the general knowledge of the LVLM and the task-specific knowledge of the learnable branch to reinforce each other.

Specifically, inside the BIT module, there are differences in the dimensions of the feature maps obtained by the general branch of the LVLM and those required by the change detection branch of the BIT module, as well as an inconsistent distribution between the general features extracted by the LVLM branch and the task-specific features extracted by the change detection branch. The transfer fusion module is therefore built to further extract features and unify the features and dimensions to those required by the change detection branch of the BIT module.

The effectiveness of the transfer fusion module in mitigating the feature distribution discrepancy can be explained from three aspects. First, the discrepancy in spatial resolutions and channel dimensions is resolved by the feature pyramid, which resizes the multi-level LIP features to the scales required by the change detection branch while preserving both global semantics and local details. Second, the discrepancy in feature statistics is directly reduced by the LN layer: it standardizes the mean and variance of the fused features, and its learnable affine parameters, optimized by the change-detection loss, map the standardized features to the statistics expected by the change detection branch. Third, since the CLIP encoder is frozen, the transfer fusion module is the only pathway through which the gradients of the task loss can adjust the interface between the two feature spaces; it therefore acts as a learned adapter that projects the general CLIP representation into the task-specific feature space.

Considering that the shallow network mainly focuses on the local and detailed information of the image, and the deep network of the model mainly focuses on the global and abstract features of the image, CLIFF proposes a transfer-fusion module with the structure of “feature pyramid + Windows Attention + LN” as shown in Figure 8. This module receives feature maps output from different levels of the image feature extraction backbone network (the 2nd, 4th, 10th, and 12th ResidualAttentionBlock [34]), and resizes them to different spatial resolutions (4×, 8×, and 16×), and obtains each feature map to represent the feature representation of the input image at different scales. Then, features across layers are aggregated using bottom-up and top-down feature fusion to ensure feature integrity and diversity. It is worth noting that in the bottom-up feature fusion process, Windows Attention [35] is used to process the results of the fusion of feature map 1 and feature map 2, and the processed results are fused with feature map 3. The feature maps of multi-scale output after feature fusion are fused by channel cascading. In order to enable features of different scales to be fused in the same space, the feature maps of each scale are sampled and adjusted so that they have the same spatial resolution. The fused feature map contains information from local to global scales and can be used for subsequent change-detection tasks. On this basis, the LN layer is used to standardize the feature dimension so that it is transformed to be consistent with the feature dimension required by the change detection branch of the BIT module. The transfer fusion module based on Windows Attention effectively captures the dependency of local features through the self-attention mechanism within the local window, and enhances the multi-scale expression ability of features through cross-scale fusion. It not only improves the richness of feature expression, but also reduces the computational complexity. Hence, it is an efficient module in multi-scale feature fusion.

4.3.2. Semantic Tokenizer

In the BIT model, the two feature maps output by the transfer fusion module are fed into the same semantic tokenizer. The structure of the semantic tokenizer [12] draws on the idea of a tokenizer in NLP to divide a sentence into several elements. By dividing the feature graph into several subgraphs and mapping them to tokens, spatial attention is used to extract spatial information and obtain compact semantic tokens.

4.3.3. Transformer-Based Encoder and Decoder

The encoder-decoder structure used in CLIFF is the same as the BIT network [12]. The Transformer encoder [36] is used to model the context within the two token sets. The Transformer’s self-attention mechanism can capture long-range dependencies in the sequence, thereby modeling global semantics. The transformer decoder is then used to optimize the original features, and the generated context-rich semantic tokens are reprojected into the pixel space to enhance the original pixel-level features and produce a refined feature map at each time step. Finally, the two refined features are subtracted to obtain a difference feature to identify the changed part, and after several convolutional layers, a pixel-level change prediction map is generated.

4.4. EfficientNet-Based Geographical Hazard Classification

Finally, the CLIFF model identifies geological hazard types by combining public and proprietary datasets. Since the data samples involving geological disasters collected by UAVs are very limited, resulting in training data scarcity, it is difficult to directly achieve the two tasks of “geological hazard area labeling” and “geological hazard type identification” through an end-to-end model. To overcome this challenge, an anomaly detection model can be added to the change detection model. This method can accurately label the target area while identifying the specific type of geological hazards in the area.

Compared with directly classifying the whole image, this detect-then-classify strategy offers several advantages. First, it decouples the two tasks under scarce hazard-labeled UAV data: the change detection model can be trained on large-scale public change detection datasets, while the classification model can leverage abundant disaster images from public sources, so the two stages exploit complementary data sources without requiring jointly annotated location-and-category labels. Second, change detection filters out static background, such as vegetation and buildings, so the classifier processes only regions where changes actually occurred, reducing false alarms caused by confusion between background textures and hazard features. Third, hazard regions, such as a small crude oil spill, often occupy only a tiny fraction of a high-resolution aerial image; effectively classifying the cropped detection boxes increases the resolution of the target region and improves the recognition of small hazards. Finally, the cascaded scheme naturally outputs the spatial location and extent of hazards, supporting change-area statistics and changed-building counting, shown in Figure 9, whereas direct whole-image classification only provides an image-level label and cannot localize hazard spots along the pipeline. In addition, the detection and classification modules can be upgraded and retrained independently, facilitating the extension of hazard categories.

Geological Hazard Type Recognition Results

The detection box of the target area is obtained as follows. The difference area map output by the BIT change area detection model is divided into instances through connected domain analysis implemented with the OpenCV library [37]. To handle minor variations, connected regions whose areas fall below a predefined pixel threshold are removed. The rationale is that such small regions are typically caused by residual registration errors, vegetation sway, and illumination differences between the bitemporal images, rather than by actual geological hazards; in contrast, the hazards of practical concern along pipelines, such as landslides, collapses, and crude oil spills, occupy ground areas that are considerably larger than the scale of these noise-induced variations. The threshold is therefore set according to the spatial resolution of the UAV images so that it suppresses noise-level changes while preserving all hazard-scale regions. The remaining regions are then expanded and smoothed through morphological operations to restore complete hazard boundaries. After determining the target area, the image within the detection box is input into the geological disaster type recognition model to obtain the specific geological disaster type in the area.

In order to improve the accuracy and efficiency of the model, the geological hazard type identification model in CLIFF adopts the EfficientNet network structure [13]. EfficientNet achieves efficient model design by reasonably scaling the depth, width, and resolution of the network and combining the AutoML MNAS framework [38] for neural network architecture search, thereby significantly improving the recognition performance of the model without significantly increasing the computational cost.

During hazard type identification, the parts of the target area that may change are first identified using a change-detection model (such as the BIT model), and a corresponding difference-area map is generated. These changed regions are then processed as independent instances, and the detection boxes containing the potential changes are extracted. Next, the image fragment within the detection box is fed into the EfficientNet model. EfficientNet uses its efficient CNN structure to extract and classify features from these image fragments and ultimately identify specific types of changes in the area (e.g., different types of geological disasters). The efficiency and accuracy of EfficientNet in image classification tasks significantly improve the performance of CLIFF in geological hazard monitoring.

5. Experimental Results and Analysis

This section presents the experimental datasets, setup, and results. We demonstrate a real-world use case of the CLIFF model for conducting geological hazard monitoring along long-distance pipelines located in the Sichuan Basin, China.

5.1. Bitemporal Change Detection Dataset

To verify the effectiveness of CLIP, we built a bitemporal change detection dataset for geological hazard monitoring. More specifically, we collected multiple public datasets and annotated the remote sensing data. Among them, the public datasets we selected are LEVIR-CD, S2Looking, WHU-CD, BANDON, and Landsat-SCD datasets.

5.1.1. LEVIR-CD

The LEVIR-CD dataset [39] contains 637 pairs of bitemporal remote sensing images collected by aerial cameras, with a time span of 5 to 14 years. The size of each pair of images is 1024 pixels × 1024 pixels, and the image resolution is 0.5 m. There are 31,333 changed building instances as the change information of the dataset, with an average change size of 987 pixels.

5.1.2. S2Looking

The S2Looking dataset [40] is a building change detection dataset that contains large-scale side-view satellite images captured from the lowest point at different angles. It consists of 5000 co-registered bitemporal image pairs of rural areas around the world, with a size of 1024 × 1024, a spatial resolution of 0.5∼0.8 m/pixel, and a total of 65,920 annotated change instances. It provides two annotated maps for each pair of images, indicating new and demolished construction areas.

5.1.3. WHU-CD

The WHU-CD dataset [41] contains a pair of images taken in the same area in 2012 and 2016, each with a size of 32,507 × 15,354 pixels.

5.1.4. BANDON

The BANDON dataset [42] contains a large number of off-ground aerial images with a spatial resolution of 0.6 m/pixel, mainly collected from Google Earth Pro (v7.3.6), Microsoft Virtual Earth (Bing Maps, https://www.bing.com/maps, accessed on 15 January 2025), and ArcGIS Pro (v3.4). The data is divided into 1689/202/392 for training, validation, and testing, with a size of 2048 × 2048.

5.1.5. Landsat-SCD Dataset

The Landsat-SCD dataset [43] consists of 8468 image pairs collected between 1990 and 2020, each with a size of 416 × 416 and a spatial resolution of 30 m/pixel. The dataset is annotated with the “no change” category and ten semantic changes, including four types of land cover (farmland, desert, building, and water). The self-annotated data are derived from daily UAV inspection data for oil and gas pipelines in the Southwest Oil and Gas Field, including UAV images of 10 km of pipelines in the Sichuan Basin across five time periods. Due to the relatively small number of changes, very few geological disasters and illegal building labels were manually annotated. On this basis, the dataset is divided into the training set, validation set, and test set in a ratio of 8:1:1 to construct a bitemporal change detection dataset.

5.2. Geological Hazard Type Classification Dataset

For the geological hazard type classification task, a total of 2100 multi-source remote sensing data were collected. The data came from image data collected by UAV inspections in the southwest oil and gas field area, including UAV aerial images, LiDAR point cloud data, and generated two-dimensional orthophoto images. Such image data, of which 1680 were used for training sets, 210 for validation sets, and 210 for test sets. Since there is limited UAV data on geological disasters, we also captured images of natural disasters from publicly available online datasets. A total of 209,154 geological disaster data samples, including hazards such as hail, floods, and collapses, were collected. We annotated 15 types of geological disasters, such as landslides, mudslides, flooding, collapses, construction, smoke, wildfires, hail, and snow, and divided them into training, validation, and testing in an 8:1:1 manner to construct a geological disaster type identification dataset. The histogram of hazard categories is depicted in Figure 10.

Considering that the above data comes from different sources, it may cover different shooting angles and conditions, varying resolutions, and heterogeneous data formats. Therefore, the following sorting process is implemented: first, we delete duplicate, incomplete, or poor-quality images, and over-sample the images in the dataset to balance their category distribution (approximately 252,000 raw images were initially collected, of which 209,154 valid images were retained after cleaning; the distribution in Figure 10 corresponds to the cleaned dataset before oversampling, and oversampling is applied only to the training split, leaving the validation and test sets untouched; the five public change detection datasets are used with their official splits without modification); second, we convert all image data into the PNG format (lossless compression) and unify the color space to the 8-bit three-channel RGB space, where grayscale images are replicated across channels and alpha channels are discarded, and the annotation masks are saved as single-channel binary PNG files (0 for background/unchanged and 255 for changed/hazard regions); third, we adjust all images to the same resolution so that the model can handle a uniform input size: the bitemporal images are cropped or resized into 256 × 256 patches for change detection, and image regions are resized to 224 × 224 for hazard type recognition; bilinear interpolation is used for RGB images and nearest-neighbor interpolation for binary masks to avoid introducing non-binary label values; fourth, we normalize the pixel values of the image, first scaling them to [0, 1] and then applying channel-wise normalization (mean and std of 0.5 per channel, mapping the input approximately to [−1, 1], applied consistently to both temporal images), which is essential for accelerating model training and improving stability; fifth, we use data augmentation techniques on the training set only: random horizontal and vertical flipping (each with probability 0.5), random rotation by 0/90/180/270 degrees (probability 0.5), random cropping to the training patch size, and, for RGB images only, color jittering (probability 0.3) with brightness, contrast, and saturation perturbations within ±20%; the same geometric transformations are applied synchronously to the two temporal images and the corresponding mask, to increase the diversity of image data and improve the generalization ability of the model.

5.3. Experimental Setup

During the experiment, we used key point matching (i.e., ORB algorithm, Brute-Force, and affine transformation) to align the dataset images, employed the multi-modal LVLM and CLIP to extract image features, used the change detection network, BIT, to locate the geological disasters, and utilized the classification network, EfficientNet, to determine the type of geological disasters.

Furthermore, we apply transfer learning during training, leveraging common feature representations learned by the model on a common dataset to achieve better results. At the beginning of model training, the general branch of the LVLM uses a pre-trained model trained on massive data to initialize its parameters and freeze them to prevent parameter adjustment during training. Random initialization is used for the transfer fusion module. The pre-trained BIT model is used to initialize parameters for the change detection branch.

During training, cross-entropy loss and the AdamW optimizer are used, with a learning rate of 0.0001. In the prediction head of land change detection, the learning rate is set to 0.001. Random cropping, flipping, and photometric distortion are used for data augmentation. All experiments are performed on an NVIDIA A100 GPU, and the batch size is set to 1000.

To prove the change detection performance of the proposed CLIFF model, the following models are selected as baselines for comparative experiments: FC-Siam-diff-PA [44], TinyCD [45], STANet [39], Changer [46], SNUNet [47], HANet [48], BIT [12], LightCDNet [49], and ChangeFormer [50]. These baselines cover both accuracy-oriented architectures (e.g., ChangeFormer, HANet) and efficiency-oriented lightweight designs (e.g., TinyCD, LightCDNet), providing a comprehensive comparison across different design objectives; note that TinyCD and LightCDNet are included as representative lightweight models, and their results should be interpreted with their efficiency-oriented positioning in mind.

It is worth noting that the key point-based image registration and alignment are applied as a dataset-level preprocessing step: all bitemporal image pairs are aligned once before training and testing, and the identical aligned dataset, together with the same training/validation/test split, is used for CLIFF and all comparison methods in

Table 1. Quantitative comparison of change detection results of different models.

Model	IoU	F1	Model	IoU	F1
FC-Siam-diff-PA [44]	45.55	0.6239	TinyCD [45]	70.83	0.8322
STANet [39]	62.49	0.7143	Changer [46]	70.92	0.8339
SNUNet [47]	64.91	0.7871	HANet [48]	71.38	0.8476
BIT [12]	65.90	0.7914	LightCDNet [49]	73.18	0.8583
ChangeFormer [50]	66.77	0.8031	CLIFF	75.74	0.8689

. Therefore, the performance differences reported in Table 1 are attributed to the models themselves rather than to differences in preprocessing.

The engineering characteristics of the main CLIFF components are summarized in

Table 2. Engineering characteristics of the main CLIFF components (NVIDIA A100).

Component	Params	Trainable	FLOPs
CLIP encoder (ViT-L/14@336px)	304.3 M	frozen	190.7 G/image
EfficientNet-B0 [13]	5.3 M	yes	0.39 G/region

. The frozen CLIP encoder dominates the model size, but its features for baseline images at fixed inspection points can be pre-computed and cached, so only a single forward pass is required online per newly captured image; the remaining trainable components (the transfer fusion module and the BIT change detection branch [12]) are lightweight, of the same order of magnitude as the vanilla BIT baseline in Table 1.

5.4. Experimental Results

This section presents the results of bitemporal change detection and geological hazard type recognition.

5.4.1. Bitemporal Change Detection Results

To demonstrate the effectiveness of the proposed CLIFF model, we use the IoU and the F1 score as the performance evaluation indicators for change detection. After 50 rounds of training iterations, the test results of the proposed and baseline change detection models are shown in Table 1.

Experimental results show that, across the selected datasets, the proposed CLIFF model outperforms other state-of-the-art deep learning change detection schemes, such as LightCDNet and ChangeFormer. As can be seen from Table 1, the IoU and F1 score of the CLIFF network are 75.74% and 86.89%, respectively, which are higher than those of other change detection networks.

The CLIFF model can automatically detect environmental changes near the target block to achieve automated monitoring of geological disasters, pipeline safety, and illegal constructions. The experimental results are shown in Figure 9. In addition to the change mask image, this method can also count the number of changed buildings and the percentage of change.

Therefore, the proposed CLIFF model performs well in environmental change detection, both qualitatively and quantitatively.

Figure 11 presents the confusion matrix of geological disaster type recognition based on EfficientNet, with an overall accuracy of $86.89\%$. Events of interest, such as crude oil spills, collapses, landslides, and floods, showed stable performance in accurate classification, especially for crude oil spills and landslides, with accuracy rates of 87.32% and 86.17%, respectively. Figure 12 further visualizes representative hazard recognition results on debris flow cases. As shown in the figure, the identified gully-type and slope-type debris flow regions are accurately overlaid on the corresponding UAV images, demonstrating the capability of CLIFF in recognizing fine-grained hazard types in complex mountainous scenes. Overall, the above performance metrics proved the outstanding performance of the proposed CLIFF model for geological hazard monitoring based on bitemporal UAV images.

5.4.2. Ablation Analysis of the Hybrid Feature Structure

To verify the effectiveness of the hybrid structure, we compare CLIFF with two single-feature schemes: (i) the learnable features only, i.e., the vanilla BIT without the LVLM branch; and (ii) the frozen CLIP features only, where the learnable ResNet18 branch is removed, and the BIT detection head is trained directly on the CLIP features adapted by the transfer fusion module. As shown in

Table 3. Ablation study on the hybrid feature structure.

Scheme	IoU (%)	F1
Learnable features only (vanilla BIT)	65.90	0.7914
Frozen CLIP features only	42.90	0.6005
CLIFF (hybrid)	75.74	0.8689

, the learnable-only scheme achieves an IoU of 65.90%, and the CLIP-only scheme achieves 42.90%, both considerably lower than the 75.74% of the full CLIFF model. This confirms that the general semantic features and the task-specific learnable features are complementary, and that neither alone is sufficient for robust change detection in complex geological scenes.

6. Discussion

From a practical perspective, the proposed CLIFF model can be used to dynamically monitor the environment around oil and gas pipelines, automatically identify geological anomalies, ensure pipeline safety, and detect illegal construction activities. Furthermore, it provides a scientific basis for building early warning systems, thereby effectively responding to various abnormal conditions that may occur in the target area and greatly improving safety and sustainability.

From a technical perspective, the proposed CLIFF model can be applied to a wide range of real-world use cases. For example, it can be used to generate target mask images of the changed area and further count the number of changed buildings and their change ratios. Moreover, it provides quantitative data support for the comprehensive assessment of the impacts of environmental changes. It also enhances the accuracy and practicality of geological hazard monitoring in diverse landforms and topography.

The above practical and technical merits stem from several original designs beyond a simple integration of existing CLIP, BIT, and EfficientNet components. First, we propose a transfer fusion module (Section 4.3.1), specifically designed to bridge the frozen CLIP feature space and the BIT change-detection feature space; this module employs a feature pyramid combined with Windows Attention and Layer Normalization, a structure not present in either the original BIT [12] or CLIP [11] formulations. Second, we introduce a model fine-tuning strategy tailored to remote sensing change detection under conditions of scarce hazard-labeled data, comprising the freezing of the LVLM branch, transfer learning of the BIT branch, and a staged change-detection-then-classification training scheme. Third, we construct a multi-source bitemporal UAV dataset and a geological-hazard classification dataset specific to oil and gas pipeline monitoring in the Sichuan Basin, integrating public change-detection datasets with newly collected and annotated UAV and LiDAR data. Finally, we develop an end-to-end application pipeline that integrates UAV image alignment, change detection, anomaly-based instance extraction, and hazard-type classification, demonstrated on a real-world pipeline safety use case. This transfer pattern is consistent with recent findings on frozen VLM representations in other visual domains, where they improve domain generalization in image classification [25] and semantic segmentation [26]; it also complements upstream optimization of UAV-assisted hazard monitoring, such as the reinforcement learning-based data collection of Fu et al. [6], by addressing the subsequent image interpretation stage.

Despite these merits, a natural concern is the computational cost introduced by the LVLM. Although the CLIP image encoder is large in size, it is frozen throughout training and does not participate in gradient back-propagation. The trainable parameters of CLIFF come only from the lightweight transfer fusion module and the change detection branch, so the training complexity remains close to that of the vanilla BIT [12]. At inference time, the CLIP forward pass introduces additional computation; however, UAV-based pipeline inspection is an offline task, and the CLIP features of baseline images at fixed inspection points can be pre-computed and cached, so the practical overhead is acceptable. Considering the substantial accuracy improvement over BIT [12] and LightCDNet [49] shown in Table 1, this trade-off is reasonable for geological hazard monitoring.

7. Conclusions

This paper focuses on change detection and geological disaster classification using bitemporal UAV remote sensing imagery. The proposed multimodal remote sensing model, CLIFF, comprises the feature-extraction LVLM and CLIP, the change-detection network BIT, and the geological-disaster-type recognition model EfficientNet. The CLIFF model is verified through real-world datasets along oil and gas pipelines for geological disaster monitoring. Experimental results showed that the proposed CLIFF model outperforms deep learning baselines, such as LightCDNet and ChangeFormer, in change detection, as measured by IoU and F1 scores. For example, the IoU of the change-detection model in CLIFF reached 75.74%, demonstrating its effectiveness in interpreting UAV images. At the same time, the geological disaster type recognition model based on EfficientNet in CLIFF has achieved an accuracy of 86.89% in recognizing major hazards along oil and gas pipelines, including crude oil spills, collapses, landslides, and floods.

In the future, we will expand this work towards the optimization of UAV inspection routes based on the results of environmental change detection and geological hazard monitoring.