TENet: A Texture-Enhanced Network for Intertidal Sediment and Habitat Classification in Multiband PolSAR Images

Abstract

This paper proposes a texture-enhanced network (TENet) for intertidal sediment and habitat classification using multiband multipolarization synthetic aperture radar (SAR) images. The architecture introduces the texture enhancement module (TEM) into the UNet framework to explicitly learn global texture information from SAR images. The study sites are chosen from the northern part of the intertidal zones in the German Wadden Sea. Results show that the presented TENet model is able to detail the intertidal surface types, including land, seagrass, bivalves, bright sands/beach, water, sediments, and thin coverage of vegetation or bivalves. To further assess its performance, we quantitatively compared our results from the TENet model with different instance segmentation models for the same areas of interest. The TENet model gives finer classification accuracies and shows great potential in providing more precise locations.

1. Introduction

Intertidal zones are the transitional areas between terrestrial and marine environments [1,2]. The surfaces of these areas are usually of different kinds, including vegetation, mussel beds, and different types of sediments. Under the influence of tidal processes, the intertidal zones are endowed with dynamic conditions and high species diversity and productivity [3,4].

The intertidal ecosystems play a crucial role in ecological systems, including carbon storage [5], nutrient cycling [6], storm protection [7], and serving as nursery grounds for marine organisms [8]. Additionally, the intertidal regions can be reclaimed for various commercial and recreational purposes [9]. However, intertidal zones are also recognized as environmentally delicate and sensitive areas, which can be influenced by warming waters, coastal erosion, species invasion, coastal development, and many other factors [10]. Recently, there have been growing anthropogenic threats exposed to the intertidal zones, including tourism, excessive fishing, oil and gas production, and high nutrient loads [11], which makes continuous monitoring and frequent classification of surface types mandatory.

The understanding of valuable coastal environments is constrained by a lack of data. In situ observations of intertidal zones are scarce due to the inaccessibility of most areas [12]. Satellite remote sensing offers a valuable resource for monitoring coastal zones [13,14]. High-resolution multispectral remote sensing data obtained from satellite-borne optical sensors have been successfully employed for sediment and habitat classification on intertidal flats [15,16]. However, the use of these sensors is constrained to daylight conditions and can be hindered by cloud cover and atmospheric haze [11]. The synthetic aperture radar (SAR), operating as an active microwave sensor, stands out among existing remote sensors due to its ability to acquire high-resolution imagery with multiple bands and polarizations, independent of weather and light conditions. These advantages make SAR an ideal tool for studying intertidal zones [17].

Most of the current approaches employ traditional machine learning algorithms to construct SAR classification schemes [12,18,19], which often requires a significant amount of time and effort to define and manually extract features. Moreover, existing models tend to categorize the surfaces of intertidal zones into a limited number of types, which often fail to meet the practical demands and requirements.

In recent years, deep learning frameworks have shown great potential in the field of remote sensing applications, exhibiting impressive capabilities and performance [20,21,22,23,24,25,26]. There exist already some data-driven models based on optical images for sediment and habitat classification [27,28]. However, there is still very little research reported on data-driven approaches for classifying intertidal sediments and habitats using SAR images.

So far, several methods have been specifically designed for semantic segmentation tasks in SAR images. Wu et al. [29] developed a fully convolutional network (FCN) [30]—based model with a transfer learning strategy, which proved to be effective in achieving accurate polarimetric SAR (PolSAR) scene segmentation with limited training data. We refer to this model as TL-FCN in this paper. Wang et al. [31] proposed HR-SARNet, a deep neural network specifically designed for scene segmentation using high-resolution SAR data. The aforementioned methods have demonstrated effectiveness in accomplishing SAR semantic segmentation tasks. For the intertidal sediment and habitat classification, we can further consider the specific characteristics of intertidal areas to enhance the classification results.

The utilization of multiband and multipolarization SAR data has been employed for the classification of intertidal zones in some studies [32,33]. Gade et al. [34] proposed an approach applied to multifrequency, copolarized spaceborne imaging radar-C/X band (SIR-C/X) SAR images for the classification of sediments. Geng et al. [35] examined the tidal impacts on the polarimetric characteristics of mudflats and aquaculture farms by analyzing C-band Radarsat-2 (RS2) and X-band TerraSAR-X (TSX) SAR data. Wang et al. [12] introduced an FCDK-RF classification scheme for intertidal sediments and habitats, which was validated using SAR data from different bands (L-band, C-band, and X-band), separately. These studies have demonstrated the usefulness and effectiveness of incorporating multiband and multipolarization SAR data for accurate intertidal classification using traditional machine learning algorithms. Based on these findings, we have considered and incorporated multiband and multipolarization SAR data in our data-driven approach. Specifically, in our work, we integrated full-polarization RS2 and ALOS PALSAR-2 (ALOS2) data to classify sediments and habitats in intertidal flats.

In the intertidal zones, texture features have proven to be highly valuable in the classification of sediments [36] and habitats [37]. Texture features contain both local structural and global statistical features [38]. Deep learning models are highly effective in extracting local structural features such as boundaries, smoothness, and coarseness [39]. However, there is a lack of well-defined systems to extract and utilize global statistical texture information for convolutional neural network (CNN)–based semantic segmentation. Zhu et al. [40] demonstrated that easily computable textural features have the potential for general applicability in various computer vision tasks related to image classification. Therefore, taking inspiration from their research, this paper introduces a texture-enhanced network, referred to as TENet, taking statistical texture information into consideration for the classification of intertidal sediments and habitats in SAR imagery.

The subsequent sections of this paper are structured as follows: In Section 2, we present the whole processing chain diagram and the structural details of the TENet model. Section 3 provides an overview of the dataset, including the area of interest and the SAR data used herein. Section 4 shows the verification methods employed and comprehensive analyses of the model results. Finally, in Section 5, we conclude this paper with insightful discussions.

2. System Architecture

Our classification framework is split into two parts. The first part involves the preprocessing of SAR data and optical sensor data. The second part involves utilizing the processed outcomes to feed the TENet model and obtain test results. In this section, we first give a detailed flow diagram of the proposed method. Next, we explain two involved classic polarimetric decomposition algorithms. Lastly, we introduce the proposed TENet model.

2.1. Processing Procedure

Figure 1 depicts the data processing workflow in this paper. During the preprocessing stage of the PolSAR images, we initially execute radiometric calibration on RS2 and ALOS2 SAR data to convert the image pixel values from a digital number (DN) to a standard geophysical measurement unit of radar backscatter. Next, the polarization scattering matrices $\left[\mathbf{S}\right]$ are converted into coherence matrices $\left[\mathbf{T}\right]$. Following that, a multilooking operation is performed to mitigate speckle noise. Afterwards, slant-to-ground-range conversion and terrain correction are carried out to eliminate geometry-dependent radiometric distortions. To further reduce speckle, a refined Lee polarization filter with a window size of 7 × 7 (range direction × azimuth direction) is employed. Finally, polarimetric decomposition algorithms are applied to extract information on backscattering mechanisms. All the preprocessing operations are performed on each channel separately.

Rational polynomial coefficient (RPC) orthorectification and resizing procedures are performed on the classification data obtained from SPOT-4 and Landsat 8. Finally, both the SAR data and classification data are reprojected to the Universal Transverse Mercator (UTM) zone 32N system, and a georegistration is applied under the same geographical coordinates. Lastly, the processed SAR data and classification map are divided into training sets and testing sets for the TENet model.

2.2. SAR Polarimetric Decomposition

During the preprocessing procedure, two classic polarimetric decomposition algorithms are involved: Cloude-Pottier (CP) and Freeman-Durden (FD). The fundamental principle driving polarimetric decomposition techniques is to analyze and interpret the polarimetric characteristics of radar signals [41]. These decomposition techniques aim to decompose the radar return signal into different scattering mechanisms, enabling the extraction of important information about the observed targets or scenes.

Eigenvalues and eigenvectors are employed in CP polarimetric decomposition to describe the predominant scattering mechanisms exhibited by individual targets [42]. For the analysis of CP polarimetric decomposition, three main parameters with physical interpretations can be utilized: entropy H, alpha angle $\alpha$, and anisotropy (AN).

The FD polarimetric decomposition is a physically derived model that can be employed to describe the polarimetric backscatter from naturally existing scatterers [43]. For the FD decomposition method, we can extract three orthogonal scattering components, including volume scatter (indicated as vol), double-bounce scatter (indicated as dbl), and first-order Bragg scatter (indicated as odd).

2.3. Texture Enhancement UNet

2.3.1. The Overall Architecture

Figure 2 displays the whole network structure of the TENet model, which is built on the UNet model. This network draws inspiration from the Statistical Texture Learning Network (STL-Net), which is used for semantic segmentation in computer vision [40].

Recent deep learning models for semantic segmentation primarily emphasize the acquisition of high-level features, potentially overlooking abundant details and structural information in PolSAR images. To address this limitation, researchers have introduced skip connections to retain low-level features. However, the extracted low-level features tend to exhibit low quality, particularly when dealing with SAR images affected by speckle noise. Consequently, the model tends to gradually disregard the ambiguous texture details during the training process. Traditional methods for SAR image classification in the intertidal zone often incorporate global textural information, such as intensity column diagrams. Inspired by this idea, TENet is introduced, utilizing a texture enhancement module (TEM) to effectively preserve and transfer improved low-level features.

As shown in Figure 2, for the TENet, the encoding phase consists of four convolutional layers, while the decoding phase consists of four corresponding deconvolutional layers. The input images have dimensions of 512 pixels for both width and height, while the number of channels can vary depending on the experimental settings. The decoder directly receives the feature maps from the corresponding encoder layer, except for the first layer. We designate the value of F as 64. After the decoding phase, a 1 × 1 convolution is used to generate the prediction mask. Further details regarding the TEM will be presented in the following part of this section.

2.3.2. Texture Enhancement Module

The TEM is adopted to improve the texture details by capturing the global distribution of low-level features. The inputs to the TEM are the feature maps obtained from the initial encoding layer, and its outputs are directly connected to the corresponding decoding layer.

We attempted to concatenate the output of the TEM with the encoding feature maps and then sent them to the decoding layer; however, the results turned out to be worse. A possible explanation is that applying the TEM operation to the encoding feature maps, which already contain local textural information, may result in redundancy. The output of the TEM is already capable of capturing sufficient low-level details without adding noise, making the concatenation unnecessary and potentially detrimental to the overall performance. Therefore, we do not use the plain skip connection in the TEM.

The input feature maps of the TEM are represented as $\mathbf{A}\in {\mathbb{R}}^{H\times W\times C}$, where H, W, and C denote the height, width, and number of channels of the feature maps, respectively. To obtain the global average feature map $g\in {\mathbb{R}}^{1\times 1\times C}$, global average pooling is applied to each feature map. The cosine similarity between g and A is then computed for every pixel in the feature map and is denoted as follows:

$${\mathbf{S}}_{i,j}=\frac{g·{\mathbf{A}}_{i,j}}{{\parallel g\parallel }_2·{∥{\mathbf{A}}_{i,j}∥}_2}$$

(1)

where ${\mathbf{A}}_{i,j}$ represents each pixel in the feature maps, with $i\in [1,W],j\in [1,H])$. The cosine similarity of each pixel is denoted as ${\mathbf{S}}_{i,j}(i\in [1,W],j\in [1,H])$.

After reshaping into ${\mathbb{R}}^{H\times W}$, $\mathbf{S}$ is divided into $\mathbf{N}$ parts, denoted as $L=\left[L_1,L_2,\dots ,L_N\right]$, by equally partitioning N points between the minimum and maximum values of $\mathbf{S}$. The process involves creating $\mathbf{N}$ partitions that span the range of values in $\mathbf{S}$.

To obtain the representation of the quantization encoding matrix $\mathbf{E}\in {\mathbb{R}}^{H\times W\times N}$, the values of ${\mathbf{S}}_{i,j}(i\in [1,W],j\in [1,H])$ are quantized with $\mathbf{N}$ functions:

$${\mathbf{E}}_{i,n}=\left\{\begin{array}{cc}\hfill 1-\left|{\mathbf{L}}_n-{\mathbf{S}}_i\right|\hfill & \mathrm{if}-\frac{0.5}{N}\le {\mathbf{L}}_n-{\mathbf{S}}_i<\frac{0.5}{N}\\ 0\hfill & \mathrm{else}\end{array}\right.$$

(2)

where n represents the nth level of ${\mathbf{L}}_n$. In a departure from the conventional argmax operation or the binary-based one-hot encoding approach, the quantization encoding strategy aims to circumvent the issue of gradient disappearance that may arise during backpropagation. With the quantization encoding matrix $\mathbf{E}$ at hand, we can proceed to produce the quantization counting map $\mathbf{C}$.

Specifically, we concatenate $\mathbf{L}$ and the average $\mathbf{E}$ for each feature map to obtain the quantization counting map $\mathbf{C}\in {\mathbb{R}}^{N\times 2}$:

$$\mathbf{C}=Concat\left(\mathbf{L},\frac{{\sum }_{i=1}^{HW}{\mathbf{E}}_{i,n}}{{\sum }_{n=1}^N{\sum }_{i=1}^{HW}{\mathbf{E}}_{i,n}}\right)$$

(3)

where $Concat$ refers to the concatenate operation in the channel dimension.

Since $\mathbf{C}$ encodes the relative statistics of $\mathbf{A}$, we can obtain the absolute relative statistics of $\mathbf{D}$ by concatenating the global average feature map g. Then g is upsampled to ${\mathbb{R}}^{N\times C}$, and $\mathbf{C}$ is sent to a multilayer perceptron ($MLP$) to increase the dimension. The final $\mathbf{D}$ can be expressed as follows:

$$\mathbf{D}=Concat\left(MLP\right(\mathbf{C}),g)$$

(4)

Similar to traditional histogram equalization methods, the statistics map $\mathbf{D}$ requires the reconstruction of quantization levels. To achieve this, we employ a graph reasoning module that performs relation reasoning using a graph convolutional network (GCN) in the interaction space:

$$\mathbf{X}=Softmax\left({\varphi }_1{\left(\mathbf{D}\right)}^T·{\varphi }_2\left(\mathbf{D}\right)\right)$$

(5)

$${\mathbf{L}}^{\prime }={\varphi }_3\left(\mathbf{D}\right)·\mathbf{X}$$

(6)

where ${\mathbf{L}}^{\prime }$ is the reconstructed levels, and operations ${\varphi }_1$, ${\varphi }_2$, and ${\varphi }_3$ are performed using 1 × 1 convolutions. The GCN structure here is designed to work as a self-attention mechanism. The final output is the quantization encoding map $\mathbf{E}$ on ${\mathbf{L}}^{\prime }$ obtained through matrix multiplication:

$$\mathbf{R}={\mathbf{L}}^{\prime }·\mathbf{E}$$

(7)

The final output of the TEM is the reshaped ${\mathbf{R}}^{C_2\times H\times W}$.

3. Dataset

3.1. Study Area

As shown in Figure 3, the study area is situated in the northern region of the German Wadden Sea, specifically between the islands of Amrum and Föhr. This region is part of the largest coherent intertidal area globally, known as the Wadden Sea, spanning over 500 km along the North Sea coast of the Netherlands, Denmark, and Germany [44].

The dominant surface types in this area are sediments, which can be either pure or mixed with mud. The distribution of these sediments is strongly influenced by local hydrodynamic forces. Additionally, the region also includes vegetated areas and bivalve beds, primarily consisting of Pacific oysters, cockles, and blue mussels. To achieve a fine-grained classification, the surfaces in the study area are categorized into six types: land, water, sediments, bright sands/beach, bivalves, seagrass, and thin coverage of vegetation or bivalves. The thin coverage of vegetation or bivalves denotes areas where the vegetation or bivalves are sparsely distributed or occupy a smaller portion of the overall area.

Figure 4 illustrates a classification map of the study area. The classification for the bright sands/beach, sediments, and bivalves is based on Landsat 8 OLI data collected between 2014 and 2016. The classification of seagrass and bivalves is based on SPOT-4 data obtained in August 2015 and April 2016. The information derived from Landsat 8 OLI is refined through a manual postprocessing step that involves adjusting the data using the identified locations of the bivalves in the SPOT-4 data. This postprocessing step ensures that the derived information aligns more accurately with the actual locations of the bivalves.

The entire classification map, with dimensions of 1187 × 1699 pixels, was divided into three sections for our study. The areas shown in Figure 4a,b were allocated for training purposes, while Figure 4c was reserved for testing. This division ensured that approximately 70% of the data were used for training, while the remaining 30% were used for testing, thereby preventing any data leakage in the testing process.

It is worth noting that the bivalves, seagrass, and thin coverage classes are very limited in both the training and testing sites. Additionally, the beach class includes only a small number of samples in the testing sites, despite accounting for a relatively higher proportion in the training site. These imbalances in data distribution and the differences between the training and testing datasets may potentially have an adverse impact on the final classification results.

3.2. PolSAR Data

Two Single-Look Complex (SLC) SAR images were used for the study area, with pixel sizes ranging from 1 m × 1 m to 5 m × 5 m. Figure 5 displays VV-polarization images obtained from RS2 and ALOS2 on 24 December 2015 and 29 February 2016, respectively. Further details regarding the SAR data can be found in

Table 1. SAR acquisition information. Water levels are measured at Wittdün.

Sensor/Band	Date/Time	Low Tide Time/Water Level	Water Level
RS2/C	24 December 2015/05:43 UTC	05:25 UTC/−103 cm	−94 cm
ALOS2/L	29 February 2016/23:10 UTC	23:46 UTC/−176 cm	−171 cm

. The sediments and habitats on the surface of intertidal zones can only be detected when they are exposed during low tide. Consequently, it is necessary to use only SAR data acquired within a time window of 60 min before and 60 min after low tide [45].

In these images, the beach and seagrass classes are observed as dark patches, which could be attributed to the residual water present, resulting in a smooth surface. Conversely, the bivalves contribute to a rougher intertidal surface, leading to their appearance as bright patches.

4. Experiments

4.1. Experimental Setup

All experiments were conducted in PyTorch on a GeForce RTX 2080 Ti GPU. The models were trained using the Adam optimizer with a momentum of 0.9 and a weight decay of 0.0005. The initial learning rate was set to 0.0001. For our multiclass segmentation task, the common cross-entropy loss was adopted.

The training site shown in Figure 4c was randomly cropped into patches of size $512\times 512$. Random flipping augmentation was applied to the training dataset. The models were trained with a batch size of 4 for approximately 10,000 steps. Under this setup, the training of the TENet model typically requires up to 3 h.

4.2. Evaluation Metrics

We assess the semantic segmentation results of different methods using five metrics: per-class F1 score ($F1$), mean F1 score ($mF1$), mean intersection over union ($mIoU$), average accuracy ($AA$), and overall accuracy ($OA$).

By comparing the reference classifications and prediction results, we can generate the confusion matrix:

$$P=\left\{p_{ij}\right\}\in {\mathbb{N}}^{k\times k}$$

(8)

where $p_{ij}$ represents the number of pixels that belong to class i and are classified as class j. The total number of classes is denoted as k, which in our case is seven. Notably, $p_{ii}$ represents the number of correctly classified pixels. The average precision $\left(P\right)$ and recall $\left(R\right)$ can be calculated as follows:

$$P=\frac{1}{k}\sum _{i=1}^k\frac{p_{ii}}{{\sum }_{j=1}^k{p}_{ji}},R=\frac{1}{k}\sum _{i=1}^k\frac{p_{ii}}{{\sum }_{j=1}^k{p}_{ij}}$$

(9)

The $F1$ score is a harmonic mean of P and R, which is particularly useful for imbalanced classes. $F1$ can be computed as follows:

$$F1=2\times \frac{P\times R}{P+R}$$

(10)

The $mF1$ is computed by averaging the individual F1 scores for each class.

The $mIoU$ is a stringent metric used for evaluating image segmentation. It considers every pixel and is calculated as follows:

$$\mathrm{mIoU}=\frac{1}{k}\sum _{i=1}^k\frac{p_{ii}}{{\sum }_{j=1}^k{p}_{ij}+{\sum }_{j=1}^k{p}_{ji}-p_{ii}}$$

(11)

The $OA$ is the ratio of correctly classified pixels to the total number of pixels in the testing set. It is given by the following:

$$\mathrm{OA}=\frac{{\sum }_{i=1}^k{p}_{ii}}{{\sum }_{i=1}^k{\sum }_{j=1}^k{p}_{ij}}$$

(12)

The $AA$ is the average accuracy across all classes:

$$\mathrm{AA}=\frac{1}{k}\sum _{i=1}^k\frac{p_{ii}}{{\sum }_{j=1,j\ne i}^k{p}_{ij}+p_{ii}}$$

(13)

4.3. Comparison of Results

We compare our TENet model with two classic semantic segmentation models, Deeplab V3 Plus [46] and UNet [47]. Two frameworks specifically designed for SAR image semantic segmentation, HR-SARNet [31] and TL-FCN [29], are also concluded as control groups. To ensure a fair comparison, we maintain consistent hyperparameters across all models, including learning rate, batch size, optimizer, etc.

Note that we used an 11-channel input for all models, consisting of 3 CP component channels (H, AN, $\alpha$) (when the reciprocity theorem holds, the HV and VH channels can be substituted with their average value, reducing the need for four polarimetric channels to three. This test can be performed, for instance, by applying the GLRT criterion devised in [48]), 4 intensity channels (HH, HV, VH, HH) for RS2 data, and 4 intensity channels (HH, HV, VH, HH) for ALOS2 SAR data. This configuration was chosen based on the results of our ablation study.

Table 2. Comparing quantitative results of different semantic segmentation models for intertidal sediment and habitat classification.

Model	F1 (%)							mF1 (%)	mIoU (%)	AA (%)	OA (%)
	Landmask	Seagrass	Bivalves	Beach	Water	Sediments	Thin Coverage
DeeplabV3+	97.49	18.09	0.28	3.37	79.73	78.74	0.00	39.67	34.02	40.21	84.25
UNet	96.39	13.83	3.18	15.09	79.65	77.23	3.09	41.21	34.41	42.87	83.04
HR-SARNet	96.31	18.39	10.05	3.99	78.91	78.32	0.00	40.85	34.27	41.80	83.14
TL-FCN	95.82	9.05	9.30	16.08	80.17	77.25	0.00	41.09	34.31	42.52	83.01
TENet	97.11	18.87	2.30	18.49	79.63	77.75	1.49	42.23	35.43	43.69	83.95

presents the quantitative results obtained by different models. In general, we observe that our TENet model achieves the highest performance in terms of $mF1,mIoU$, and $AA$, and is only marginally (0.30%) below Deeplab V3 Plus in terms of $OA$.

Due to the limited number of training and testing pixel samples for classes of seagrass, bivalves, beach, and thin coverage, their metrics are not as good as those of land, water, and sediments. The features of the low-accuracy classes are also more difficult to learn.

Considering the extreme sample imbalance of our sediment and habitat classification task, we find that the $AA$ metric is more suitable for assessing the model’s capabilities, compared with the overall accuracy $OA$. In fact, the visualization results presented in Figure 6 indicate that the Deeplab V3 Plus model tends to overfit on the land, water, and sediments classes.

Compared with the basic framework UNet, our proposed TENet model demonstrates improvements in mean metrics $mF1,mIoU,AA$, and $OA$ by 1.02%, 1.02%, 0.82%, and 0.91%, respectively. When contrasted with the basic skip connections in UNet, integrating the TEM enhanced the model’s performance, providing clear evidence of the TEM’s effectiveness over plain skip connections.

It is worth mentioning that the basic UNet model outperforms the more complex HR-SARNet and TL-FCN models in terms of $mF1,mIoU$, and $AA$. This observation suggests that our application may not be content with the very high-resolution conditions in HR-SARNet, and the additional branches for FCN in TL-FCN do not contribute significantly to our task.

To intuitively demonstrate the superiority of TENet, we present the prediction masks of the testing sites in Figure 6. A comparison between the UNet baseline and our proposed model reveals that TENet exhibits the ability to predict relatively smoother but more precise results. This improved performance can be attributed to the effect of the TEM.

We note that RS2 and ALOS2 images are acquired at different times during various tide cycles, leading to variations in environmental backgrounds (such as wind speed and weather conditions) and water levels within the same area. These factors, in addition to the frequency difference, can also impact the radar backscatter recorded by the RS2 and ALOS2 sensors.

4.4. Ablation Study

We comprehensively evaluated our TENet model on different input sources. The same set of hyperparameters is adopted across all experiments for a fair comparison.

4.4.1. Multiband Input

We first investigated the multiband design choices of the RS2 (C band) and ALOS2 (L band) data. For each band, we concatenated the three CP components and four intensity channels as input. There are five design choices in

Table 3. Comparing quantitative results of different train and test dataset inputs for intertidal sediment and habitat classification.

Train Dataset	Test Dataset	F1 (%)							mF1 (%)	mIoU (%)	AA (%)	OA (%)
		Landmask	Seagrass	Bivalves	Beach	Water	Sediments	Thin Coverage
RS2	RS2	93.01	15.28	0.11	11.22	76.82	73.65	0.72	38.69	31.75	40.53	79.93
ALOS2	RS2	24.32	1.13	0.00	0.09	65.93	35.33	0.00	18.11	12.16	22.81	39.92
RS2	ALOS2	87.21	0.94	0.00	2.53	0.42	8.18	0.00	14.18	11.94	23.03	47.30
ALOS2	ALOS2	95.61	17.90	5.70	18.15	77.77	77.87	1.91	42.13	34.48	42.29	82.73
RS2+ALOS2	RS2+ALOS2	97.11	18.87	2.30	18.49	79.63	77.75	1.49	42.23	35.43	43.69	83.95

and Figure 7.

RS2 and ALOS2 are different 7-channel SAR data, the “Training” keyword means that we use this kind of data as TENet input, and the “Testing” keyword means that we use this kind of data for performance testing. The “+” operation between RS2 and ALOS2 means that we concatenate them in the channel dimension.

The quantitative results are shown in Table 3. The “Training: RS2 + ALOS2, Testing: RS2 + ALOS2” setting obtains the best results in all average metrics, which proves that a combination of SAR data from different bands indeed helps the model in learning. The ALOS2 data behaves much better compared with the RS2 data. A possible explanation may be that the ALOS2 data (L band) emits longer waves whose penetration ability is stronger. This ability is very important to classify cover types in the intertidal zone like thin coverage.

The results of the “Training: ALOS2, Testing: ALOS2” setting are very close to our best results. Adding RS2 data makes the performance of seagrass, bivalves, and thin coverage classes drop slightly. More complex multiband fusion networks can be designed for better feature extraction. When the training and testing data are from different bands, a dramatic drop happens in all the metrics. This result shows that SAR image features of different bands reflect different backscatter mechanisms.

The visualization results of the five design choices are shown in Figure 7. The seagrass, bivalves, and thin coverage classes nearly disappear under training and testing for the RS2 setting. However, the classification of training and testing on the ALOS2 setting gets the prediction maps closer to our proposed method. The models trained and tested from different bands data confuse the dominating pixels and lose the power of detecting the classes bivalves and thin coverage.

4.4.2. Multipolarization Input

We investigate the design choices of a combination of multipolarization SAR inputs. For every possible combination, we use both RS2 and ALOS2 data as input. There are seven design choices in

Table 4. Comparing quantitative results of different input combinations of intensity channels (I), FD components (FD), and CP components (CP) for intertidal sediment and habitat classification.

Input	F1 (%)							mF1 (%)	mIoU (%)	AA (%)	OA (%)
	Landmask	Seagrass	Bivalves	Beach	Water	Sediments	Thin Coverage
I	95.65	9.39	7.36	13.31	78.31	76.93	2.94	40.56	33.69	41.02	82.48
FD	96.25	10.73	5.40	8.09	77.18	77.36	2.63	39.66	33.24	39.64	82.93
CP	96.34	12.35	1.56	14.94	77.82	77.89	0.00	40.13	33.70	40.76	83.51
FDI	96.14	11.25	6.69	13.58	79.41	77.43	1.71	40.89	34.17	41.97	83.06
FDCP	94.47	18.16	11.86	14.18	66.03	72.94	0.26	39.70	31.47	42.74	78.07
FDCPI	96.34	14.44	3.09	16.13	78.97	78.04	1.03	41.15	34.40	41.99	83.55
CPI	97.11	18.87	2.30	18.49	79.63	77.75	1.49	42.23	35.43	43.69	83.95

and Figure 8.

The keyword “I or FD or CP” signifies that we only use either four intensity channels or three FD components or three CP components as the TENet input. The FD and CP components can be combined using band concatenation as “FDCP” or unified with the four intensity channels separately as “FDI” or “CPI”. In the final control group, we utilize intensity channels, FD components, and CP components together as “FDCPI”.

Table 4 presents the quantitative results of the seven comparison groups. Among them, the “CPI” setting demonstrates the best overall performance. In theory, the “FDCPI” setting contains the most comprehensive information compared with other settings. However, it only shows a slight improvement (0.29%) in the sediments class compared with “CPI”. One possible explanation for this is that the concatenation method employed for fusing intensity and polarimetric decomposition information is too simplistic. This information may interfere with each other during feature extraction, leading to limited performance gains. In general, the different combinations of FD components, CP components, and intensity channels enhance the final average metrics compared with using them individually.

Figure 8 illustrates the visualization results of the seven design choices. The “CP” setting demonstrates proficiency in identifying the sediments class but struggles to detect the thin coverage class, aligning with the quantitative results. The “FDCP” setting significantly aids in identifying bivalve areas. One possible explanation is that polarization characteristics can better reflect the features of specific classes, such as bivalves, compared with intensity information. These results further validate the importance of incorporating different polarimetric decomposition components as the model inputs.

5. Discussion

The TENet model can obtain excellent classification results for the classes landmask, water, and sediments, but it fails to perform effectively in certain specific categories. One potential reason is that it is extremely challenging to learn the intricate characteristics of seagrass and bivalves with a limited amount of training data.

In the future, we can enlarge our dataset and adopt few-shot learning strategies to help achieve better classification results. In addition, more appropriate fusion mechanisms for multiband and multipolarization SAR data can be designed. Additional polarimetric decomposition components can be incorporated to differentiate sediments and habitats in intertidal flats. Furthermore, the combination of multiple sensors, such as SAR and multispectral data, holds great promise for significantly improving the performance of the model and realizing full-time all-weather operation in future intertidal monitoring.

6. Conclusions

In this paper, we propose a TENet model for the classification of sediments and habitats in intertidal zones. We evaluate the proposed model using fully polarimetric SAR data from ALOS2 (L-band) and RS2 (C-band), and our classification results are generated with higher accuracies (where sediments are further classified into bright sands/beach and other sediments, and habitats are further subdivided into bivalves, seagrass, and thin coverage of vegetation or bivalves).

The experimental results show that the incorporation of the TEM improves the performances by explicitly learning global texture information. The TENet model exhibits improved classification accuracies across all average metrics in comparison with the UNet baseline. The visualization results further validate that TENet is capable of offering more precise locations.

The comparative experiments verify the effectiveness and potential of the multiband and multipolarization systems for classification tasks in intertidal zones. By considering the information from multiple bands, the TENet model can leverage the complementary nature of SAR data to improve classification results. The best classification results are obtained when we combine ALOS2 and RS2 data. For the multifrequency input, the choice of the “CPI” setting achieved the highest average performances compared with the other six comparison groups. We note that the CP components show greater effectiveness compared with the FD components in our framework in terms of all the average metrics. Overall, our research demonstrates the effectiveness of the TENet model and emphasizes the importance of considering global texture, multiband, and multipolarization SAR information for the accurate classification of sediments and habitats in intertidal zones.