RSWFormer: A Multi-Scale Fusion Network from Local to Global with Multiple Stages for Regional Geological Mapping

Abstract

Geological mapping involves the identification of elements such as rocks, soils, and surface water, which are fundamental tasks in Geological Environment Remote Sensing (GERS) interpretation. High-precision intelligent interpretation technology can not only reduce labor requirements and significantly improve the efficiency of geological mapping but also assist geological disaster prevention assessment and resource exploration. However, the high interclass similarity, high intraclass variability, gradational boundaries, and complex distributional characteristics of GERS elements coupled with the difficulty of manual labeling and the interference of imaging noise, all limit the accuracy of DL-based methods in wide-area GERS interpretation. We propose a Transformer-based multi-stage and multi-scale fusion network, RSWFormer (Rock–Soil–Water Network with Transformer), for geological mapping of spatially large areas. RSWFormer first uses a Multi-stage Geosemantic Hierarchical Sampling (MGHS) module to extract geological information and high-dimensional features at different scales from local to global, and then uses a Multi-scale Geological Context Enhancement (MGCE) module to fuse geological semantic information at different scales to enhance the understanding of contextual semantics. The cascade of the two modules is designed to enhance the interpretation and performance of GERS elements in geologically complex areas. The high mountainous and hilly areas located in western China were selected as the research area. A multi-source geological remote sensing dataset containing diverse GERS feature categories and complex lithological characteristics, Multi-GL9, is constructed to fill the significant gaps in the datasets required for extensive GERS. Using overall accuracy as the evaluation index, RSWFormer achieves 92.15% and 80.23% on the Gaofen-2 and Landsat-8 datasets, respectively, surpassing existing methods. Experiments show that RSWFormer has excellent performance and wide applicability in geological mapping tasks.

1. Introduction

The geological environment is a complex environmental system mainly composed of the lithosphere, hydrosphere, and atmosphere on the earth’s surface, including rocks, soil, groundwater, and other geological bodies. In recent years, environmental problems, such as geological disasters, mine pollution, and soil erosion have occurred frequently. Therefore, it is particularly important and necessary to carry out geological environment research and other related work.

Research on the geological environment covers many aspects, such as geological survey mapping, geological structure analysis [1], mineral resource exploration [2], environmental management [3], and geological disaster prevention [4] (landslides and debris flows), which effectively reduces the losses caused by natural disasters and is of great importance for the promotion of sustainable development of humankind.

However, in areas with harsh environments, such as high-altitude snowy mountainous areas and arid uninhabited areas, traditional artificial geological surveys face many challenges. Geological surveys in these areas are difficult, have limited coverage, and usually require a large investment of time and human resources. In addition, if geological mapping errors occur in complex areas, they may lead to errors in resource assessment, which in turn may cause serious consequences in disaster management. For example, in soil erosion control and mine pollution control, mapping errors may lead to inefficient environmental restoration and increased restoration costs. In the field of mineral resource exploration, the accuracy of geological mapping is directly related to resource assessment and the formulation of mining strategies. If the error is large, it may cause wastage of resources, increase the cost of exploration and mining, and even miss development opportunities due to assessment errors. Therefore, it is particularly important to adopt advanced techniques to carry out regional geological surveys.

With the rapid advancement of satellite sensor technology (for example, the Gaofen, Sentinel, and Landsat series), we can efficiently and accurately obtain high-precision images of the earth’s surface, which are then used to monitor, analyze, and interpret the geological environment, a process called Geological Environment Remote Sensing [5] (GERS). However, the complexity of the spatial distribution characteristics of GERS elements in remote sensing images still limits the possibility of high-precision interpretation. Specifically,

• High interclass similarity and intraclass variability in GERS elements. For example, diorite and granite often show similar texture and tone characteristics in remote sensing images, while arenaceous rock shows diverse spatial distribution patterns after experiencing long geological processes and changes.
• The boundaries of GERS elements such as rocks and soil are gradational. Interactions between different geological features make it difficult to accurately delineate boundaries.
• Distribution complexity and noise interference. As shown in Figure 1, the distribution space of different GERS elements is complex, and the non-uniform distribution varies greatly. In addition, disturbances in atmospheric conditions such as clouds and fog also significantly reduce the accuracy and reliability of remote sensing image data.

These factors [5,6,7] increase the difficulty of GERS interpretation.

In view of the complex characteristics of GERS elements and intricacies of terrain, it is particularly important to develop effective GERS interpretation methods. GERS interpretation technology is the process of using data from various remote sensing platforms, image processing, and information extraction techniques to build an accurate GERS interpretation model to interpret and analyze the ground surface and geological information. It can carry out geological interpretation of spatially large areas at a low cost for various GERS elements such as rocks, soils, and surface water. To improve the extraction efficiency of GERS elemental features, a variety of machine learning (ML) algorithms (for example, SVM [8], KNN [9], PCA [10], etc.) are currently mainstream. For example, performance comparison among six ML algorithms using Aster spectral data from Fels et al. [11] and Cracknell et al. [12] explored a variety of machine learning classifiers in supervised lithology classification tasks and pointed out that random forest (RF [13]) are easy to use and have stable performance. Zuo et al. [14] proposed a method to combine ML algorithms with important features of sample data to build a high-precision classification model for lunar geological unit identification. Rezaei et al. [15] used band ratio, SAM [16], and SVM methods to classify the main lithological units in the Sanggan area with a classification accuracy of nearly 79%.

However, ML algorithms have limitations in processing high-precision and spatially large area remote sensing data, resulting in limited interpretation accuracy. Deep learning (DL) has quickly become a leading technology in GERS interpretation [17] due to its ability to combine basic features into high-level features. DL outperforms traditional ML methods in tasks such as object detection [18], scene classification [19,20], and semantic segmentation [6,7]. Wang et al. [21] proposed a framework consisting of multi-source data fusion technology (enhanced feature learning) and FCN [22], which improved the classification accuracy of geological mapping and showed superior performance to traditional ML algorithms. Brandmeier et al. [23] combined U-Net [24] with ArcGIS for lithological mapping in the Mount Isa region (Australia) and found that the DL model outperformed RF and SVM in identifying rare lithology categories. Hajaj et al. [25] evaluated the importance of ML and DL techniques in lithological mapping and emphasized the effectiveness of using high-resolution hyperspectral data combined with advanced DL models to achieve accurate mapping results.

Despite the impressive performance of the DL method, it suffers from complex model structure, a large number of parameters, and cumbersome calculations. In addition, DL faces problems such as difficulty in sample labeling and training and creating the GERS dataset and providing ML and DL models for training requires professional knowledge and time as well as money. Currently, publicly available GERS datasets are relatively scarce, especially high-resolution GERS datasets are almost not-existent, and the existing remote sensing scene datasets have problems such as inconsistent spatial resolution and low diversity. To effectively realize intelligent interpretation of GERS elements in spatially large areas, this study constructed a multi-source GERS dataset called Multi-GL9. This dataset is mainly collected by the Landsat-8 and Gaofen-2 remote sensing satellites and covers nine major GERS elements with multiple categories, complex lithology, and high resolution.

In GERS interpretation, faced with challenges such as the diversity of element changes and the complexity of distribution and considering that an attention mechanism can effectively capture the global dependencies of GERS images, we propose a Transformer-based regional GERS interpretation model called RSWFormer. RSWFormer captures the semantic information of remote sensing images from local to global through LTG block to obtain beneficial contextual information. In addition, a multi-stage downsampling method is used to capture high-level semantic features in remote sensing images and integrate the low- and high-level features through a multi-scale fusion module, which realizes the sharing of valuable semantic information. This design greatly enhances the model’s understanding of the details and context of remote sensing images, thereby significantly improving its ability to interpret complex GERS elements. We conduct extensive comparative experiments on the Multi-GL9 dataset, demonstrating that our model outperforms existing methods. We make the Multi-GL9 dataset publicly available for the first time at: https://github.com/WanZhan-lucky/Geological-remote-sensing-data-set (accessed on 24 June 2024). The contributions of this article include the following points:

• To the best of our knowledge, this study is the first to propose a meter-level high-resolution GERS multi-source dataset (Multi-GL9) which is used in scene classification tasks to fill the lack of datasets of complex regional GERS elements, such as rocks, soils, and surface water. It can provide rich regional and large-scale GERS elements information and further use for the interpretation of complex geological environments.
• In order to meet the needs of regional GERS interpretation, a novel RSWFormer is proposed to capture local to global semantic information, extract high-dimensional feature information of GERS elements, and effectively fuse semantic features of different scales.
• Experiments were conducted on the Multi-GL9 dataset. Compared with existing scene classification models, RSWFormer achieved an overall accuracy improvement of 0.61% and 1.7% on the Gaofen-2 and Landsat-8 datasets, respectively.

2. Related Work

2.1. Datasets in the Field of GERS Interpretation

In the past decade, Earth observation technology has developed rapidly, and the emergence of advanced remote sensing platforms, such as Landsat, Sentinel, SPOT, WorldView, Gaofen, and ZY-3, which have not only provided high-definition surface structural data but have also enhanced the visual representation and spectral characteristics of GERS elements, such as the significant improvement of spatial, temporal and spectral resolutions. They play an important role in GERS interpretation [26].

At present, there are few spatially large area public remote sensing scene datasets, such as NWPU-RESISC45 [27], covering 45 scene classes, and the AID and WPU-RESISC45 datasets from Google Earth. However, there are relatively few datasets available for specialized interpretation of GERS elements. In recent years, the number of publications related to the GERS field has surged from 75 to 2510 [5], accompanied by the proposal of geologic remote sensing datasets. For example, Y Zhang, M Li, S Han [28] used rock image datasets (granite, phyllite, and breccia) to establish a transfer learning model for lithology identification and classification. C Kumar, S Chatterjee, T Oommen, A Guha, A Mukherjee [29] proposed a multi-sensing (ASTER, PALSAR, and Sentinel-1) dataset to optimize lithology classification. D Li, J Zhao, Z Liu [30] collected four basic single-type rock datasets of sandstone, shale, monzogranite, and tuff, and used data augmentation to generate a multi-type mixed rock lithology dataset for training and evaluation. H Shi, ZH Xu, P Lin, W Ma [31] constructed a laboratory dataset of 160 kinds of lithology and a field dataset of 13 kinds of lithology for self-building and detection models.

However, the above GERS datasets are generally confined to limited lithology categories, lack diversity, and most do not contain high-resolution remote sensing images and are not made public. Therefore, we propose and publish for the first time a dataset called Multi-GL9, which combines multi-source data from Landsat 8 (15-m spatial resolution) and Gaofen-2 satellite (1-meter spatial resolution), covering nine categories of GERS elements.

Currently, the trend in GERS interpretation focuses on multi-source and multi-modal data fusion, small training samples, and expert knowledge integrated with DL models. Multi-modal, high-resolution datasets are becoming a research hotspot, which can better capture the complexity and diversity of GERS elements. However, publicly available high-quality datasets are still scarce, especially in covering more lithology categories and subtle features. The research we proposed aims to fill this existing gap by constructing and publishing the Multi-GL9 dataset to provide a high-resolution, multi-category GERS element dataset to support precise research in GERS interpretation. It not only helps to improve the accuracy of current geological element interpretation but also provides a solid data foundation for future research.

2.2. Image Classification Based on Deep Learning (CNNs and Transformer)

DL methods such as CNNs and Transformers have demonstrated powerful capabilities in rapidly processing remote sensing big data and have been widely used in tasks [32,33] such as object detection [34] and image classification [35].

With the rapid developments in DL technologies such as ResNet [36] and self-attention mechanisms [37], DL and remote sensing image processing can be combined to automatically perform GERS elements classification tasks. For example, Latifovic et al. [38] used CNN and combined aerial and satellite remote sensing data to predict geological categories to complete geological mapping. Sang et al. [39] used CNN to identify image content to confirm lithology distribution and complete the production of high-resolution geological maps. Shirmard et al. [40] proposed a framework that combines CNN and traditional ML methods to achieve high-precision lithology mapping using a variety of remote sensing data. Subsequently, researchers optimized CNN architecture, such as in VGG [41], GoogLeNet [42], DenseNet [43], etc., to effectively extract image features through deeper convolution layers and also improved the ability to identify GERS element features [44]. This technology has been widely used in geological mapping [45]. For example, Fu et al. [46] proposed ResNeSt-50 based on ResNet, combined with a channel-level attention mechanism and multi-path network, which significantly improved the accuracy of lithology classification. Huang et al. [47] proposed a rock image classification model based on EfficientNet and a triple attention mechanism. Through transfer learning and optimized network structure, it achieved a test set accuracy of up to 93.2%, showing strong robustness and generalization capabilities. Although CNNs have performed well in the field of image classification, they still have the disadvantage of being unable to capture the characteristics and global information of GERS images in GERS interpretation. It is necessary to combine methods such as attention mechanism and optimization of network structure to make up for its shortcomings.

Because the Transformer is not limited by the local receptive field, it allows flexible attention to various areas of the image, surpassing traditional CNNs, and achieving some progress in the field of GERS. For example, Dong et al. [48] proposed the ViT-DGCN model, which enhances the feature extraction performance of remote sensing images by combining ViT [49]. The FaciesViT model proposed by Koeshidayatullah et al. [50] has shown significant performance improvement and repeatability in image-based lithology classification tasks. Jacinto et al. [51] proposed solving the rock stratigraphy prediction problem using a Transformer-based algorithm. The CNN-transformer model proposed by Cheng et al. [52] can effectively capture local and global mineral characteristics and performs better than the traditional CNN’s model. However, the above models have problems in GERS interpretation such as complex training, weak generalization ability, and high data requirements.

ViT often has difficulty in capturing local features when processing GERS images. In contrast, we consider Swin [53]’s hierarchical feature extraction, which has high computational efficiency and better local and global feature fusion. However, Swin has some cross-scale information that is not effectively integrated, which may lead to the loss of specific details or features when processing complex GERS images. Based on this, we propose RSWFormer, which can effectively capture key features in GERS images by fusing multi-scale features and sharing high-dimensional and low-dimensional semantic information.

3. Methodology

This section introduces the RSWFormer model designed specifically for GERS element interpretation in detail. RSWFormer consists of two main modules: the Multi-stage Geosemantic Hierarchical Sampling (MGHS) module and the Multi-scale Geological Context Enhancement (MGCE) module. The cascade of these two modules enables the RSWFormer to provide richer and more detailed geological information interpretation when handling GERS element interpretation tasks.

First let us define this task. In GERS image classification, we give a dataset consisting of N images Data = ${\left\{I_{\mathrm{k}}\right\}}_{k=1}^N$, where each image of the input model is $I_{\mathrm{k}}\in {\mathbf{R}}^{224\times 224\times 3}$. The model is trained by means of a training set and the effectiveness of the model is evaluated using a validation set, with the aim of classifying each $I_{\mathrm{k}}$ into one of t categories ${\mathbf{M}}_{\mathbf{t}}$ = {$M_1$, $M_2$, $M_3$, …, $M_t$} and make predictions in unknown GERS regions. We use the RSWFormer model to accomplish this task.

3.1. Overall Architecture and Processes

The overall architecture and data processing flow of RSWFormer is shown in Figure 2. It mainly includes MGHS and MGCE modules, whose functions are as follows:

• The MGHS module aims to extract high-dimensional geological features, in which the LTG block effectively captures and analyzes geological data at each stage to achieve the extraction of feature information from local to global. More details are presented in Section 3.2.
• The MGCE module enhances the understanding of contextual semantics by fusing geological semantic information at different scales. Detailed information is introduced in Section 3.3.

The specific processing flow is as follows:

• The image size is $H\times W\times C(C=3)$ and is first input into a Patchify stem (Inchannel = 3, Outchannel = embed_dim, Kernel_size = Stride = Patch_size, called a Patchify stem). It uses a $4\times 4$ convolution kernel (stride 4) to extract $H/4\times W/4$ high-resolution features and outputs features with a channel count $C^{\ast }$ of 96 followed by a LayerNorm [54] (LN).
• In four consecutive stages ($Stag{e}_{1-4}$), the depth of the feature map is gradually deepened and its spatial dimension is reduced at the same time, aiming to capture more abstract and advanced geological remote sensing image features. In order to maintain a high spatial resolution and effectively fuse features and retain key feature information, downsampling is not performed in the $Stag{e}_4$. The output feature maps are labeled $F_{1-4}$.
• $F_{1-4}$ performs dimension transformation through a $1\times 1$ size convolution kernel to unify them to the dimension $C^{\wedge }$. Subsequently, the feature maps are upsampled by 2 times one by one and added together to obtain $P_{1-4}$. Parameter information at each stage is shown in

  Table 1. Parameter table at different stages.

  /	Outchannel	Iterations
  Stage1	192	2
  Stage2	384	2
  Stage3	768	6
  Stage4	768	2

  .
• By upsampling $P_{2-4}$ to the same resolution as $P_1$ through bilinear interpolation and splicing in the channel dimension, finally, a $4\times$$C^{\wedge }$ feature with rich GERS details is formed to be used in the classification task.

3.2. Multi-Stage Geosemantic Hierarchical Sampling Module (MGHS)

GERS elements usually show complex interactions, with diverse characteristics and uneven distributions. The traditional self-attention mechanism cannot fully consider the local characteristics of GERS images and has high computational complexity.

We adopted the hierarchical construction method in ST [53] work, W-MSA reduces computational complexity and SW-MSA’s cross-window communication capabilities. The alternate use of the two MSAs can capture local to global geological semantic information, which we call the LTG (Local To Global) block. Specifically, this is as follows:

• As shown in Figure 3a, in W-MSA, the feature map is divided into different windows according to $M\times M$ (M = 7), and multi-head self-attention is used within each window. Using this method, the computational complexity is $7\times 7$, which is smaller than the computational complexity of $56\times 56$ when using the MSA method.
• As shown in Figure 3b, in SW-MSA, the window position in W-MSA is offset by $Window\_size//2$ in the x and y directions (for example, Window_size = $4\times 4$, then offset by 2 pixels). This is combined with the masked MSA method, and each offset window is calculated independently to improve the ability to capture global information.

In addition, during the downsampling process, the spatial size of the feature map is halved while the count of channels is doubled to enhance the feature representation of the model through dimensional reorganization.

The original image is processed by Patchify Stem to obtain X, and we set $F_0=X$. $F_{1-4}$ are calculated as follows:

$$F_i=\left\{\begin{array}{c}{W}_d\left(LTG\left(F_{i-1}\right)\right),i=1,2,3\\ LTG\left(F_{i-1}\right),i=4\end{array}\right.$$

(1)

where index i is used to identify the processing order of each stage. The details are as follows:

• When i takes the value of 1, the initial feature $F_0$ ($F_{i-1}$) first passes through $LTG$ in $Stag{e}_1$ (one characteristic of a Transformer is that it does not change the resolution and dimension of the input data).
• Secondly, when passing through the $W_d$ layer, it halves the H and W of the feature map and uses the linear transformation weight to reduce the merged patch channel dimension from the original four times to two times.
• Then, it generates feature $F_1$, completing the conversion of doubling the quantity of channels and halving the spatial size.
• Finally, $F_1$ is used as new input data to further generate $F_2$ and then $F_2$ is used to generate $F_3$.

It is worth noting that in the process of generating $F_4$, it is not downsampled, so the $W_d$ layer is removed, thereby retaining the original data dimension. The entire processing flow is shown in module A in Figure 2.

Through this hierarchical and progressive processing method, we can more comprehensively and carefully mine the complex semantic information from local to global GERS images.

3.3. Multi-Scale Geological Context Enhancement Module (MGCE)

Traditional classification networks extract high-level features in images through multi-stage processing. Although high-level features possess richer semantic information, they have lower resolution and are therefore less capable of perceiving details. On the contrary, low-level features contain rich location and detail information and have higher resolution; at the same time, due to fewer convolutional layers, their semantic information is weaker and more susceptible to noise. In FPN [55], a top-down approach is used to gradually fuse high-level and low-level features. PANet [56] proposes a top-down and bottom-up two-way fusion path.

The above design optimizes the feature fusion and can significantly enhance the model’s understanding of GERS images. The specific process of MGCE is as follows:

• Firstly, considering the idea of feature fusion of different scales, we use $Con{v}_1(·)$ (kernel size is $1\times 1$) to adjust the channel to $C^{\wedge }=192$ for $F_{1-4}$.
• Then, we obtain the $P_{1-4}$ by twice up-sampling $U{p}_2(·)$ operation, and the computational procedure is summarized as follows:

  $$P_j=\left\{\begin{array}{ll}U{p}_2\left(P_{j+1}\right)+{Conv}_1\left(F_{j+1}\right),& j=1,2\\ P_{j+1}+{Conv}_1\left(F_{j+1}\right),& j=3\\ {Conv}_1\left(F_{j+1}\right),& j=4\end{array}\right.$$

  (2)
• Next, $P_{1-4}$ are dimensioned to the same width and height as $P_1$ using the bilinear interpolation method $BI(·)$.
• Finally, the feature maps are connected in the channel dimension using the concatenation function $Concat(·)$.
• Afterwards, the feature maps are normalized by the LN layer. The calculations are as follows:

  $$Y_M=W_L\left(AAP\left(Concat\left[P_1;BI\left(P_{j=2,3,4}\right)\right]\right)\right),M=1,2,3,\dots ,t$$

  (3)

  where $AAP$ represents Adaptive Average Pooling, $W_L$ is the weight of changing the channel dimension, and $Y_M$ is the most probable among the t categories.

In GERS images, this structure can not only capture the diversity of geological structures and features at different scales, but it can also retain the underlying detailed information and incorporate high-level semantic information.

4. Experiments and Analysis

4.1. Study Area and Dataset

The study area is located in the high mountains and hilly area in western China with variable topography and extreme climatic conditions. The topography of the area includes steep slopes, areas covered with snow and ice, mudslides, and rain and snow erosion phenomena with sparse natural vegetation and a high degree of rock exposure. The study area contains rock types such as granite, gabbro, diorite, etc. The variability of lithology increases the complexity of GERS interpretation, which meets the needs of this study for the interpretation of geologic elements. The multi-source dataset (Multi-GL9) mainly consists of two remote sensing satellite images: Landsat-8 and Gaofen-2.

This study used the multispectral satellite Landsat-8 carrying OLI and TIRS sensors, which can provide visible information for exploration lithology mapping, including Band 2 (Blue 0.452–0.512; 30 m), Band 3 (Green 0.533–0.590; 30 m), Band 4 (Red 0.636–0.673; 30 m), and Band 8 (Panchromatic, 15 m). We chose Bands 4, 3, 2, and 8 to synthesize 15 m resolution images and removed Band 8 information after synthesis. Gaofen-2 is a high-spatial resolution imagery remote sensing satellite with Band 4 (NIR 0.77–0.89; 3.2 m), Band 3 (Red 0.63–0.69; 3.2 m), Band 2 (Green 0.52–0.59; 3.2 m) and Panchromatic (0.45–0.90; 0.8 m). We used Bands 4, 3, 2, and Panchromatic to synthesize the new image, and then removed the panchromatic information for a new image with a spatial resolution of up to 1 m. As shown in Figure 4 for the study area of the Gaofen-2 image, Figure 4a is used for manual labeling to construct the dataset and for model training and validation, while Figure 4b is used for model testing or prediction.

In view of the complexity of GERS images, the interpretation of specific GERS elements usually relies on human intuitive judgment and understanding. The specific labeling process is as follows:

• Preprocessing of Figure 4a (stretching, sampling, and denoising).
• The uniqueness, diversity, and serious homogenization of GERS elements have led to a complex classification system and a lack of unified standards. Based on this, we first built on the theoretical foundation of constructing 9 and 13 types of GERS elements in [6,7] and then conducted in-depth research on a large number of geological survey reports (covering a wide area) based on field investigations and on-site data collection. Next, with the help of texture features captured by remote sensing satellites and the interaction between GERS elements, we analyzed the differences in tonal and texture features of the data in Figure 4a. Therefore, we finally divided the GERS elements into nine categories: slate, granite, gabbro, diorite, silt, surface water, soil, arenaceous rock, and carbonate rock.
• As shown in Figure 5a, crop GERS data in a top-to-bottom, left-to-right, non-overlapping window sliding manner, and the cropping size was set to $512\times 512$ in consideration of the high-spatial resolution of Gaofen-2, while the cropping size of Landsat-8 data was set to $64\times 64$.
• When performing manual annotation, we specify the label category according to the principle of the largest area in the image block. For mixed transition zones, we divide them into a certain type of elements based on prior knowledge identification and the proportion of each geological element component; for the finely distributed and smaller areas, they are divided into existing categories or the category names are redefined based on geological theoretical knowledge.

During the entire annotation process, geographical coordinates were used to ensure the consistency of the cropping position and the label data, thereby constructing the Gaofen-2 scene dataset containing geographical coordinates, with a total of 1668 images. Meanwhile, the same annotation process was followed for the GERS image in Figure 1 to produce the Landsat-8 scene dataset, totaling 899 images. Figure 6 presents a partial sample of the two datasets, while

Table 2. The number of images per category from Gaofen-2 and Landsat-8 datasets.

Serial No.	Name	Gaofen-2	Landsat-8
1	Slate	207	103
2	Granite	207	101
3	Diorite	175	82
4	Gabbro	148	102
5	Arenaceous Rock	267	102
6	Soil	247	144
7	Surface Water	167	65
8	Silt	105	100
9	Carbonate Rock	145	100
Total number of datasets		1668	899

details the number of images in each GERS element class. Figure 7 shows the distribution of the quantity of images in each category in the two datasets in the form of histograms.

4.2. Experimental Environment and Evaluation Metrics

Our research utilizes PyTorch to build the model and calculate the loss using a cross-entropy-based loss function with the following relevant Equation (4), where $p\left(x\right)$ denotes the observed sample distribution, $q\left(x\right)$ is the predictive distribution of the model, and t is the total number of categories. A decrease in the loss value indicates that the label predicted by the model is getting closer to the true label.

$$\mathrm{Loss}=-\sum _{i=1}^{t}P\left(x_i\right)log\left(q\left(x_i\right)\right)$$

(4)

We use the following training configuration: the learning rate is set to $1\times {10}^{-4}$, the batch size is 16, the total number of iterations is 100, and the AdamW algorithm is used. The dataset is randomly divided into training and test sets according to an 8:2 ratio, specifically, 1337 out of 1668 Gaofen-2 images are used for training and 331 are used as validation, cropped and resized to $224\times 224$, whereas 899 Landsat-8 images are similarly divided and up-sampled and resized to $224\times 224$ to facilitate training. The hardware and software setup for the experimental environment is detailed in the provided

Table 3. Experimental configuration.

Experimental Platform	Parameter Name
GPU	RTX 4090
CPU	Intel (R) Xeon (R) Gold 6430
Memory	32 GB
DL Framework	Pytorch 1.10.0
Operation System	Windows 10
Programming language	Python 3.8

, which provides the necessary computational power for the experiments.

We explored multiple models based on CNNs and Transformer (shown in Section 4.4) and present the classification results using overall accuracy(OA) as in Equation (5), which indicates the proportion of the model that correctly classifies the entire dataset. To provide insights into RSWFormer’s classification capabilities in the field of GERS, class-specific accuracies are also presented. True positive (TP) is a positive example predicted to be a positive example, true negative (TN) is a negative example predicted to be a negative example; false positive (FP) is a negative example predicted to be a positive example; and false negative (FN) is a positive example predicted to be a negative example.

$$OA=\frac{TP+TN}{TP+FN+FP+TN}$$

(5)

4.3. Analysis of Results

On the Multi-GL9 dataset with 9 GERS elements, we used the proposed RSWFormer model to conduct experiments according to the configuration in Section 4.2 and evaluated the classification performance of different GERS elements through OA and specific category accuracy (see

Table 4. RSWFormer’s OA under the Gaofen-2 and Landsat-8 datasets and the interpretation accuracy of each category (%).

Dataset	SL	GR	DI	GA	AR	SO	SW	SI	CR	OA
Gaofen-2	90.24	90.24	94.29	100.0	77.36	97.96	100.0	95.24	93.10	92.15
Landsat-8	95.00	85.00	56.25	75.00	40.00	82.14	100.0	100.0	90.00	80.23

Slate (SL), Granite (GR), Diorite (DI), Gabbro (GA), Arenaceous Rock (AR), Soil (SO), Surface Water (SW), Silt (SI), Carbonate Rock (CR).

). The OA of RSWFormer on the Gaofen-2 dataset is as high as 92.15%. The interpretation accuracies of gabbro and surface water are both 100%, and the other categories’ accuracies are all above 90%. In the Landsat-8 dataset, the OA is 80.23%, and the classification accuracy values of surface water and silt also reach 100%. This shows that RSWFormer has high classification accuracy on high-resolution datasets and is efficient and accurate in analyzing and identifying geological features.

Then, we used ArcGIS10.2 software to crop the image in Figure 4b to obtain a small area containing a variety of GERS features for small area interpretation testing. The overall effect of small-area interpretation is good, and the water body contour curve and soil body category in the small area are successfully predicted, as shown in the yellow and green boxes in Figure 5b. The soil body is widely distributed, which is consistent with the actual spatial distribution. The spatial distribution of other categories including silt and carbonate rock is basically consistent. However, a small number of misclassifications are still obvious, such as the red box in Figure 5b, which misclassifies carbonate rock into silt and arenaceous rock. This may be caused by the imbalance of a small number of samples and the overlap or similarity of geological characteristics.

Next, to further validate the effectiveness and generalization of RSWFormer for regional lithology mapping, we adopted the sliding window method to classify the prediction from left to right and from top to bottom using overlapping windows. Specifically, Figure 4b was scaled to 0.5, 0.75, 1.0, 1.25, 1.5, and 1.75, setting the window size to $64\times 64$ and the stride to 32. Accumulating the prediction results of the six scaling values to obtain the final classification values, an interpretation plot was generated using the same color palette as the test. As shown in Figure 8, the distribution of soil and silt in the blue box is very accurate, the distribution of various types of rocks (e.g., slate, granite, gabbro, carbonate rock, etc.) is highly consistent with the actual terrain, and the surface water boundaries in the green box are basically correctly interpreted. This shows that RSWFormer has the ability to generalize over a wide range of unknown regions.

Finally, we use the model weight with OA = 80.23% and scale it at the same ratio to predict the wider Landsat-8 entire image. As shown in Figure 9, the outlines of each GERS element were accurately identified and the boundaries of water bodies, soil distribution patterns, and the spatial distribution of rocks such as diorite, gabbro, and granite were successfully depicted. However, as in Figure 8, there is still a large amount of “salt and pepper” misclassification in the surface water interpretation.

To sum up, RSWFormer shows excellent overall interpretation results in the GERS element identification task, fully verifying its effectiveness and generalization in this field. Its prediction results are highly consistent with the terrain characteristics and lithology distribution of the area, effectively meet the interpretation needs of GERS elements in a wide area and can effectively assist manual experts in visual interpretation.

4.4. Comparative Experiment of Multi-GL9 Dataset in the Same Area

For a comprehensive assessment of RSWFormer’s performance, we compared multiple models, including EfficientNet [57], RegNet [58], CeiT [59], DeiT [60], ViT [49], and PVT [61].

4.4.1. Comparative Experiments and Analysis of Gaofen-2 Dataset

According to

Table 5. OA and class-specific accuracy on Gaofen-2 dataset (%).

Model	SL	GR	DI	GA	AR	SO	SW	SI	CR	OA
EfficientNet	70.73	87.80	40.00	58.62	39.62	83.67	96.97	95.24	82.67	70.69
RegNet	73.17	87.80	74.29	58.62	45.28	85.71	100.0	90.48	82.76	75.83
CeiT	82.93	82.93	80.00	86.21	64.15	93.88	96.97	90.48	72.41	82.48
DeiT	85.37	90.24	68.57	96.55	73.58	93.88	100.0	100.0	72.41	85.80
ViT	92.68	97.65	88.57	89.66	86.79	95.92	96.97	100.0	72.41	91.24
PVT	87.80	95.12	85.71	93.10	88.68	95.92	100.0	100.0	79.31	91.54
RSWFormer	90.24	90.24	94.29	100.0	77.36	97.96	100.0	95.24	93.10	92.15

Slate (SL), Granite (GR), Diorite (DI), Gabbro (GA), Arenaceous Rock (AR), Soil (SO), Surface Water (SW), Silt (SI), Carbonate Rock (CR).

, on the high-resolution dataset, the recognition accuracy of surface water reaches 100%, followed closely by silt and soil, both of which have recognition accuracies of more than 95%, which may be related to the larger coverage area and relatively single geological features. However, for lithologies such as slate, granite, gabbro, diorite, and carbonate rock, the interpretation accuracy is relatively low due to the high similarity between them and their unique complexity and diversity. Despite this, the interpretation accuracies of these lithologies in RSWFormer still remain above 90%. In particular, interpretation accuracy of gabbro reaches 100%.

The data show that compared with CNNs, the average interpretation accuracy of the Transformer model in the GERS interpretation task is improved by about 10%. This improvement is mainly due to the attention mechanism that enhances the model’s understanding of the interrelationships between GERS elements, allowing it to more accurately identify transition areas between different lithologies and their geological characteristics. In addition, it also shows that CNNs still show great potential for development in the GERS interpretation task. Overall, we found that RSWFormer achieved 92.15% OA in the GERS interpretation task, which is 0.61% higher than the current optimal model PVT and 0.91% higher than the ViT model, proving its efficient recognition ability. Figure 10 shows the confusion matrix of RegNet, Ceit, PVT, and RSWFormer on the Gaofen-2 dataset.

4.4.2. Comparative Experiments and Analysis of Landsat-8 Dataset

We performed the same comparison experiment on the Landsat-8 dataset, and the data are shown in

Table 6. OA and class-specific accuracy on Landsat-8 dataset (%).

Model	SL	GR	DI	GA	AR	SO	SW	SI	CR	OA
EfficientNet	70.00	45.00	31.25	30.00	50.00	39.29	76.92	80.00	65.00	53.11
RegNet	80.00	75.00	50.00	50.00	60.00	75.00	92.31	60.00	65.00	67.23
CeiT	70.00	100.0	50.00	45.00	15.00	71.43	92.31	70.00	55.00	62.71
DeiT	75.00	95.00	65.50	70.00	95.00	75.00	100.0	85.00	95.00	76.27
ViT	95.00	95.00	68.75	60.00	30.00	75.00	100.0	95.00	95.00	78.53
PVT	90.00	95.00	50.00	70.00	45.00	75.00	100.0	90.00	95.00	78.53
RSWFormer	95.00	85.00	56.25	75.00	40.00	82.14	100.0	100.0	90.00	80.23

Slate (SL), Granite (GR), Diorite (DI), Gabbro (GA), Arenaceous Rock (AR), Soil (SO), Surface Water (SW), Silt (SI), Carbonate Rock (CR).

. RSWFormer achieves a 100% recognition accuracy on the surface water and silt categories. In the slate and carbonate rock categories, the accuracy also exceeded 90%. Overall, compared with Transformer, low-resolution GERS images increase the ambiguity between categories, and there is less information in local areas, which reduces the feature extraction effect, resulting in relatively poor performance of the CNN. In contrast, RSWFormer improves performance by 1.7% when comparing the optimal PVT and ViT models.

In addition, the average interpretation accuracy of diorite and arenaceous rock is relatively low, which may be due to the fact that these two rock types are more variable in their natural environment and the effect of noise is more significant in low-resolution images, making it difficult to accurately capture their complex and variable features. Figure 11 shows the confusion matrix of RSWFormer and other models.

Finally, in two datasets of comparative experiments, we found that ViT’s interpretation performance exceeded CeiT and DeiT. This phenomenon may be due to the relatively complex GERS features in the Multi-GL9 dataset, resulting in CeiT and DeiT being insufficiently sensitive to these features and performing poorly. On the contrary, ViT has strong global information processing capabilities and can more effectively capture and analyze the widely distributed geological structures and features in GERS images. Therefore, based on the above data, RSWFormer shows excellent performance in the field of GERS interpretation. It shows relatively balanced and stable performance in identifying images with complex features and in diverse GERS element classification tasks.

5. Conclusions

This paper constructs a multi-source dataset Multi-GL9 that has the characteristics of diverse categories, complex lithology, and variable distribution. A Transformer-based fusion network RSWFormer is proposed, which uses the MGHS module to extract multi-scale geological features and the MGCE module to fuse geological semantic information of different scales, solving the problems of high inter-class similarity and intra-class differences, gradational boundaries, and complex distribution of elements in the GERS interpretation task. Experiments show that this method has wide applicability and advantages in mapping tasks over spatially large areas.

In the future, we will consider coupling geological remote sensing knowledge graphs to correct misclassified GERS elements. We will introduce remote sensing data from radar, optical, and thermal infrared data to integrate and combine advanced DL technology and geological expert knowledge to further improve the accuracy and robustness of GERS element interpretation. Our research not only provides a solid technical foundation for GERS interpretation but also provides important inspiration for further research and innovation in areas such as building identification and crop classification in agricultural monitoring.