Super Resolution for Mangrove UAV Remote Sensing Images

Abstract

Mangroves play a crucial role in ecosystems, and the accurate classification and real-time monitoring of mangrove species are essential for their protection and restoration. To improve the segmentation performance of mangrove UAV remote sensing images, this study performs species segmentation after the super-resolution (SR) reconstruction of images. Therefore, we propose SwinNET, an SR reconstruction network. We design a convolutional enhanced channel attention (CEA) module within a network to enhance feature reconstruction through channel attention. Additionally, the Neighborhood Attention Transformer (NAT) is introduced to help the model better focus on domain features, aiming to improve the reconstruction of leaf details. These two attention mechanisms are symmetrically integrated within the network to jointly capture complementary information from spatial and channel dimensions. The experimental results demonstrate that SwinNET not only achieves superior performance in SR tasks but also significantly enhances the segmentation accuracy of mangrove species.

1. Introduction

Mangroves are communities of salt-tolerant evergreen trees and other plant species found in tropical and subtropical intertidal zones. They provide important ecosystem services such as nutrient cycling, carbon sequestration, and mitigation of coastal disasters [1]. The ecological functions of mangroves have been degrading over the past few decades due to the impacts of climate change, natural disasters, and human disturbances [2,3]. Therefore, accurate classification and real-time monitoring of mangrove species are crucial for their protection and restoration. Traditionally, obtaining information on mangrove species requires expensive, laborious, and time-consuming field surveys, and surveyors often find it difficult to access mangrove areas [4]. In recent years, remote sensing technologies have been widely used for mangrove monitoring and assessment due to their excellent spatial and textural features and high-resolution multispectral imagery [5]. These technologies include spectrometer measurements, high-resolution aerial imagery, medium- to high-spatial-resolution satellite remote sensing imagery, and hyperspectral imagery [6]. Most remote sensing technologies are used for large-scale forest resource surveys and cannot capture detailed distributions of mangrove species [7]. In recent years, the rapid development of UAV technology has provided a new data source for the classification of mangrove communities [8,9]. Its flexibility, ability to operate below clouds, low cost, and centimeter-level high spatial resolution make it a complement to satellite remote sensing with broad development prospects. However, accurate classification of mangrove species remains challenging due to the diversity of species and canopy structures. Even with high-spectral-resolution and high-spatial-resolution remote sensing data, it is difficult to distinguish between different tree species [10].

With the development of deep learning [11,12,13], SR reconstruction technology can enhance image details in the network, improving the effectiveness of image segmentation [14].

Based on this technology, this study conducted research on mangrove species recognition using SR reconstruction of UAV remote sensing images. This study proposed SwinNET, an SR reconstruction network of mangrove UAV remote sensing images based on the improved SwinIR [15]. In recent years, in the field of image SR, the effectiveness of Single-Image SR (SISR) based on Transformer [16] models has gradually surpassed traditional Convolutional Neural Network (CNN) methods [17]. The Swin Transformer [18] is a visual model based on the Transformer structure, which has good characteristics such as global perception and dynamic weighting, making it perform well in various visual tasks. SwinIR, as an SR model based on the Swin Transformer, has also demonstrated excellent performance.

Based on this, this study proposed SwinNET, an improved mangrove UAV remote sensing image reconstruction network based on SwinIR. Specifically, similarly to SwinIR, this network consists of shallow feature extraction, deep feature extraction, and high-quality image reconstruction modules. The shallow feature extraction module uses convolutional layers to extract shallow features, which are directly transmitted to the reconstruction module to preserve low-frequency information. The deep feature extraction module is mainly composed of residual attention modules. We design the CEA in the residual attention module to enhance the network’s feature reconstruction on channel attention. Additionally, we introduce the NAT [19] into the network to help the model better focus on leaf detail features. CEA and NAT are symmetrically integrated within the attention mechanism to capture complementary information from both the channel and spatial dimensions, reflecting a balanced and structured design. Finally, a convolution layer is added to the end of the block for feature enhancement. In the image reconstruction layer, the Pixel-Shuffle method is used to merge shallow and deep features. The experimental results show that our network performs well in the SR reconstruction network, and the images reconstructed after SR significantly improve the mangrove image segmentation performance.The contributions of this study are as follows:

(1)

Designed a CEA channel attention module, which combines ECA attention to enhance the network’s feature extraction on channel attention.

(2)

Introduced the NAT module into the SR network to enhance the network’s ability to extract leaf detailed features.

(3)

The images reconstructed using an SR reconstruction network enhance the segmentation performance of mangrove species in UAV remote sensing images.

2. Methods

2.1. Network Architecture

To improve the species recognition performance of mangrove UAV remote sensing images, we propose a novel super-resolution reconstruction network based on an improved SwinIR architecture. As illustrated in Figure 1, the overall framework consists of three main components: shallow feature extraction, deep feature extraction, and image reconstruction. In the shallow feature extraction stage, a convolutional layer is applied to extract low-level features from the input image. The deep feature extraction stage leverages a series of Residual Mixed Attention Modules (RMAMs) to enhance texture and structural information. Each RMAM contains six Mixed Attention Modules (MAMs), which symmetrically integrate both CEA and NAT to capture channel dependencies and the detailed textural features of mangrove leaf structures from complementary perspectives. Finally, the image reconstruction stage fuses the extracted shallow and deep features and upsamples them using pixel shuffle to generate the high-resolution output.

Specifically, in the shallow feature extraction stage, similarly to SwinIR, a convolutional layer is used to extract shallow features to address edge and texture feature extraction in remote sensing images. This provides more effective input for subsequent feature extraction processing.

In the deep feature extraction stage, this study designed RMAMs to extract deep features $F_D\in R^{H\times W\times C}$ from shallow features. The formula is as follows:

$$F_D=H_D\left(F_S\right)$$

(1)

where $H_D(·)$ denotes the deep feature extraction module.

Specifically, the final deep feature $F_D$ is extracted by each residual attention module and the final convolutional layer. The formula is as follows:

$$F_i=H_{RMAM}\left(F_i\right),i=1,2,3,···,6.$$

(2)

where $H_{RMAM}\left(·\right)$ represents the i-th RMAM module.

The RMAM is mainly composed of the MAM, and the MAM is mainly composed of the CEA module and the NAT module. Given an input feature $F_k$, the formula is as follows:

$$F_n=LN\left(F_k\right)$$

(3)

$$F_m=CEA\left(F_n\right)+NAT\left(F_n\right)+F_k$$

(4)

$$F_i=MLP\left(LN\left(F_m\right)\right)+F_m$$

(5)

where $F_n$ and $F_m$ are intermediate features.

In the image reconstruction layer, the Pixel-Shuffle method is used to merge shallow and deep features. The upsampling method helps effectively retain information for image detail recovery, thereby improving the quality of image reconstruction.

2.2. Residual Mixed Attention Module

The RMAM mainly consists of MAM, which integrate the CEA and NAT modules in a symmetric structure to extract complementary features from the channel and spatial dimensions.

The CEA module, as shown in Figure 1, is designed to enhance the model’s capability in capturing rich channel-wise features, thereby improving the extraction of textural features of mangrove leaves during super-resolution reconstruction. The CEA module strengthens the representation of the important textural and structural features of mangrove canopies, which contributes to generating clearer and more informative high-resolution images.

In particular, the initial convolutional layers extract richer and more abstract features from the input feature maps while simultaneously reducing their channel dimensions. Nonlinear activation functions are applied after each convolution to introduce nonlinearity, enabling the model to capture more complex patterns and relationships. The subsequent convolutional layers further refine the features and apply Batch Normalization to stabilize training and accelerate convergence. Finally, the ECA module introduces a lightweight yet effective channel attention mechanism that adaptively emphasizes informative channels, enhancing the accuracy and efficiency of feature representation. Through the integration of these architectural components, the module significantly improves the extraction of fine-grained venation patterns and structural details, thereby enhancing the super-resolution reconstruction quality of UAV-based mangrove remote sensing imagery.

In addition, NAT is integrated into the network to enhance feature extraction and detail capture capabilities, thereby improving the modeling of fine spatial details. Given the complex and diverse texture patterns present in mangrove images, a detail-sensitive mechanism like NAT is particularly effective in enhancing the reconstruction quality of mangrove leaf textures.

Specifically, NAT employs an innovative neighborhood attention mechanism that enables the more flexible and localized processing of each pixel, while maintaining high computational efficiency. This allows the model to more accurately capture and reconstruct fine image details. Moreover, NAT preserves translational invariance, which is essential for maintaining spatial consistency in mangrove images. At the core of NAT is its ability to restrict self-attention to the local neighborhood of each pixel, rather than computing attention globally across the entire image. This approach significantly reduces computational complexity while retaining critical spatial information.Additionally, NAT inherits the local perceptual capabilities of traditional convolutional networks, enabling the model to be more sensitive and accurate in capturing subtle structural variations. This design makes NAT particularly well-suited for UAV-based mangrove super-resolution reconstruction tasks, where a balance between efficiency and detail fidelity is essential. By incorporating NAT, the network gains a more powerful and flexible means of enhancing complex textures and fine-grained features in high-resolution mangrove imagery.

3. Dataset

The study area selected for this research is the Shankou Mangrove Nature Reserve located in Hepu County, Beihai City, Guangxi, China. It is primarily distributed along the eastern and western coastal zones of the Shatian Peninsula, covering an area of approximately 8000 hectares. The reserve is situated in a humid subtropical climate zone of southern Asia and consists of marine areas, land areas, and extensive intertidal zones on both the eastern and western sides of the peninsula. The location of the study area is shown in Figure 2.

Throughout the aerial imaging process, this study carefully considered natural environmental factors that could interfere with image acquisition to ensure image quality and data consistency. In particular, attention was paid to weather conditions, wind speed, and tidal variations. Aerial missions were preferentially scheduled during periods of overcast weather with soft lighting and low wind conditions to reduce issues such as overexposure, shadow interference, and abnormal image contrast caused by direct sunlight. Additionally, low wind speeds contributed to the stable flight of the UAV platform and minimized the shaking of mangrove leaves, thereby improving image clarity and spatial coverage accuracy and avoiding blurring or displacement caused by motion.

To obtain the data, this study used DJI Mavic series drones equipped with high-resolution RGB camera sensors to capture images at a flying height of 10 m. A flight altitude of 10 m was chosen to reduce the impact of UAV airflow on mangrove leaves. During the data processing stage, with the assistance of mangrove species experts, the research team carefully selected 500 high-quality mangrove images from the original UAV imagery. These images have a resolution of 5280 × 3956 pixels, resulting in a Ground Sampling Distance (GSD) of approximately 2.78 mm per pixel, as shown in Figure 3. To construct a high-quality dataset suitable for super-resolution image training and to ensure the preservation of edge regions during cropping, a sliding window strategy was employed. Specifically, each image was divided into multiple sub-images, each with a size of 480 × 480 pixels. These images cover a variety of typical mangrove species, including Bruguiera gymnorrhiza, Rhizophora stylosa, and Aegiceras corniculatum, featuring rich species diversity and structural characteristics. They provide a comprehensive representation of the ecological composition of the mangrove forests within the study area.

4. Experiments

4.1. Experimental Setup

In our experiments, this study employed the AdamW optimization algorithm with ${\beta }_1$ set to 0.9, ${\beta }_2$ set to 0.999, and a batch size of 32. In this study, the size N of the network is set to 6, and the initial learning rate is set to $2.5\times {10}^{-4}$; we introduced a weight decay rate of 0.01 to effectively prevent overfitting. During the training phase, this study utilized the MultiStepLR strategy to efficiently adjust the learning rate at different training stages. For example, in the case of a scaling factor of 2, the model underwent a total of 1,500,000 iterations to ensure sufficient learning and optimization. Specifically, we multiplied the learning rate by 0.5 at training steps 300,000, 500,000, 700,000, 1,000,000, 1,300,000, and 1,450,000 to achieve timely decay, helping the model maintain optimal performance at different training stages.

4.2. Results on Image SR

Table 1. Comparison between SwinNET and other SR reconstruction algorithms in the UAV remote sensing mangrove SR reconstruction dataset.

Method	Training Dataset	×2		×3		×4
		PSNR (dB)	SSIM (%)	PSNR (dB)	SSIM (%)	PSNR (dB)	SSIM (%)
FSRCNN	mangrove	31.31	90.19	28.16	85.92	26.07	79.30
VDSR	mangrove	31.87	90.64	28.57	86.02	26.56	79.97
EDSR	mangrove	32.25	91.11	29.03	86.38	26.91	80.11
RDN	mangrove	32.44	90.70	28.62	86.81	26.66	79.61
RCAN	mangrove	32.57	91.08	29.12	87.30	26.90	80.02
DRLN	mangrove	32.38	91.23	29.75	88.39	26.81	80.16
SAN	mangrove	32.06	90.11	30.13	89.45	26.86	80.21
IGNN	mangrove	32.13	90.65	30.04	89.55	26.73	81.33
ELAN	mangrove	32.17	90.80	30.32	90.07	26.64	82.09
SwinIR	mangrove	32.65	91.24	30.55	90.32	27.02	82.09
SwinNET (ours)	mangrove	34.04	94.43	30.59	90.21	28.52	83.35

The best results have been marked in bold.

shows the comparison of SwinNET with FSRCNN [20], VDSR [21], EDSR [22], RDN [23], RCAN [24], DRLN [25], SAN [26], IGNN [27], ELAN [28], and SwinIR on the dataset of UAV mangrove remote sensing images. The experimental results demonstrate that at scaling factors of ×2, ×3, and ×4, our method achieves 34.04/94.43, 30.59/88.55, and 28.52/83.35, respectively. Compared to SwinIR, our method achieves performance improvements of +1.39/3.19, +0.18/0.10, and +0.04/1.66. To better illustrate the comparison results, the visual comparisons corresponding to Table 1 are presented in Figure 4.

Table 2. Comparison of SwinNET and other SR reconstruction algorithms in DF2K dataset.

Method	Scale	Set5		Set14		BSDS100		Urban100		Manga109
		PSNR	SSIM	PSNR	SSIM	PSNR	SSIM	PSNR	SSIM	PSNR	SSIM
SRCNN	×2	36.59	95.31	32.44	90.61	31.31	88.72	29.49	89.41	35.51	96.60
FSRCNN		36.95	95.55	32.59	90.90	31.44	89.10	29.78	90.12	36.61	97.02
VDSR		37.47	95.79	33.02	91.21	31.88	89.53	30.68	91.32	37.16	97.40
EDSR		37.03	95.96	33.89	91.95	32.27	90.12	32.82	93.40	39.00	97.68
RDN		37.62	96.03	33.51	92.06	32.33	90.14	32.82	93.48	39.06	97.62
RCAN		37.26	96.14	33.08	92.07	32.37	90.25	33.28	93.76	39.39	97.58
DRLN		37.24	96.10	33.22	92.05	32.34	90.25	33.30	93.82	39.12	97.79
SAN		37.88	96.12	33.67	92.02	32.40	90.20	33.08	93.63	39.22	97.85
IGNN		38.03	96.10	33.07	91.87	32.39	90.15	33.23	93.73	39.26	97.78
ELAN		38.09	96.11	33.17	92.03	32.38	90.30	33.26	93.86	39.41	97.84
SwinIR		38.15	96.12	33.33	92.05	32.36	90.27	33.31	93.23	39.12	97.89
SwinNET		38.10	96.15	33.91	92.06	32.97	91.33	33.35	94.19	39.44	98.02
SRCNN	×3	32.72	90.84	29.25	82.06	28.36	78.61	26.22	79.87	30.44	91.14
FSRCNN		33.09	91.33	29.31	82.08	28.50	79.05	26.43	80.76	31.09	92.04
VDSR		33.59	92.10	29.68	83.15	28.81	79.84	27.14	82.90	32.01	93.34
EDSR		34.14	92.13	30.13	84.12	29.22	80.88	27.73	84.47	33.16	93.69
RDN		34.16	92.18	30.23	84.13	29.22	80.91	27.75	84.49	33.07	93.74
RCAN		34.23	92.39	30.18	84.13	29.31	81.08	27.07	84.95	33.44	93.99
DRLN		34.18	93.01	30.14	84.10	29.33	81.17	27.15	84.12	33.65	93.09
SAN		34.13	92.19	30.21	84.13	29.28	81.02	27.84	84.61	33.23	93.90
IGNN		34.15	92.32	30.24	84.18	29.24	80.95	27.94	84.86	33.38	94.92
ELAN		34.15	93.07	30.25	84.19	29.35	81.23	28.30	84.38	33.69	93.09
SwinIR		34.20	92.35	30.24	84.13	29.20	80.38	28.26	84.14	33.38	93.89
SwinNET		34.35	92.62	30.30	84.20	29.37	82.31	28.51	85.88	33.76	94.58
SRCNN	×4	30.38	86.27	27.43	75.09	26.80	70.95	24.43	72.15	27.54	85.49
FSRCNN		30.72	86.51	27.55	75.45	26.93	71.47	24.58	72.77	27.84	86.08
VDSR		31.31	88.25	27.93	76.75	27.23	72.26	25.09	75.30	28.72	88.60
EDSR		32.43	83.59	28.70	78.73	27.64	74.10	26.56	80.30	30.95	91.42
RDN		32.39	89.84	28.80	78.64	27.67	74.16	26.60	80.19	30.99	91.43
RCAN		32.56	89.95	28.85	78.89	27.77	74.34	26.76	80.81	31.20	91.65
DRLN		32.55	90.01	28.86	78.91	27.80	74.40	26.94	81.14	31.44	91.89
SAN		32.57	90.00	28.89	78.84	27.77	74.28	26.70	80.61	31.12	91.64
IGNN		32.47	89.98	28.75	78.85	27.75	74.34	26.83	80.82	31.27	91.81
ELAN		32.64	90.18	28.88	79.06	27.77	74.52	27.13	81.58	31.61	92.22
SwinIR		32.65	90.24	29.05	79.44	27.89	74.88	27.46	82.52	32.05	92.48
SwinNET		32.79	90.32	29.23	79.54	28.13	75.43	27.84	83.22	32.16	92.61

The best results are marked in bold.

presents a performance comparison between the proposed method, SwinNET, and a series of other methods on the DF2K (DIV2K and Flicker2K [29]) dataset, including traditional interpolation methods, early classic SR algorithms, and current mainstream methods such as SwinIR. The experimental results show that SwinNET achieves excellent PSNR (dB) and SSIM (%) across all test datasets. At a scaling factor of ×4, SwinNET’s PSNR/SSIM values on the Set5, Set14, BSD100, Urban100, Manga109, and mangrove datasets are 32.79/90.32, 29.23/79.54, 28.13/75.43, 27.84/83.22, 32.16/92.61, and 28.52/83.35, respectively. Compared to SwinIR, our method achieves performance improvements of +0.14/0.08, +0.18/0.10, +0.24/0.55, +0.38/0.70, and +0.11/0.13 on these datasets. Our method also performs well at other scaling factors.

4.3. Visual Comparison

A visual comparison of the super-resolution results of six mangrove UAV images is presented in Figure 5. The SwinNET-based approach demonstrates superior performance in restoring fine textures and structural details by effectively leveraging both channel and neighborhood features. In particular, the reconstructed results show more accurate texture recovery and more complete structural information, resulting in clearer and more natural visual quality compared to other models. As shown in Figure 5a, the results from other methods appear noticeably blurred and fail to reconstruct the leaf edges, whereas the proposed method successfully preserves fine boundary details and the intricate structure of the mangrove canopy.

4.4. Results of Mangrove Species Classification

The mangrove images captured by the drone were processed using the labeling tool LabelMe. Under the guidance of mangrove species experts, this study meticulously annotated different tree species in the images, such as Rhizophora stylosa, Avicennia marina, and Bruguiera gymnorrhiza. To meet the input requirements of the deep learning model, this study cropped the selected regions of the images to 512 × 512 pixels and used a 128-pixel sliding window method to extract 1320 images. Among these, 924 images were used for training, 132 for testing, and 264 for validation.

To evaluate the enhancement effect of super-resolution reconstruction on the original images, this study employed the classical and relatively simple segmentation network FPN for comparative experiments. The purpose was not to pursue state-of-the-art segmentation accuracy but rather to isolate and demonstrate the improvement brought by the SR reconstruction itself.

The experimental results are shown in

Table 3. Segmentation results of drone-based HR original images and SR reconstructed mangrove remote sensing images.

	HR Images		SwinNET
Class	IOU	ACC	IOU	ACC
Rhizophora stylosa	91.85	97.79	93.42	98.34
Bruguiera gymnorhiza	66.35	69.09	74.13	80.26
Aegiceras corniculata	74.36	74.64	95.56	96.86

The three classified mangrove species are Rhizophora stylosa, Avicennia marina, and Bruguiera gymnorrhiza.

, the performance of the FPN network in the segmentation task of different mangrove tree species is significantly different. Under the conditions of original HR images and SR images that are enlarged four times, the segmentation performance of Rhizophora stylosa, Avicennia marina, and Bruguiera gymnorrhiza all improved. For Rhizophora stylosa, the Intersection over Union(IoU) was 91.85% and accuracy (ACC) was 97.79% in the original HR image, which increased to 93.42% IoU and 98.34% accuracy in the ×4 image. The segmentation performance of Avicennia marina improved significantly, with an IoU of 66.35% and ACC of 69.09% in the original HR image, which increased to 74.13% IoU and 80.26% ACC in the ×4 image. Bruguiera gymnorrhiza had an IoU of 74.36% and ACC of 74.64% in the original HR image, which increased to 95.56% IoU and 96.86% ACC in the ×4 image.

The experimental results indicate that using ×4 enlarged images significantly improves the performance of the FPN network in mangrove species segmentation tasks, especially in the segmentation of Avicennia marina and Bruguiera gymnorrhiza, which show particularly outstanding performance. Especially for Bruguiera gymnorrhiza, SR processing significantly improved the segmentation effect, indicating that reasonable SR processing can significantly improve the segmentation performance of mangrove images.

5. Conclusions

This study proposes a method for mangrove species recognition based on drone remote sensing images. By combining SR reconstruction technology with deep learning, the resolution of mangrove images is improved, enabling the accurate identification of mangrove species. Our model, based on the SwinIR-improved SwinNET for mangrove drone remote sensing image reconstruction, thoroughly exploits the channel and neighborhood features of images, accurately restoring the details and textures of the original high-resolution images. The experimental results of the drone-based mangrove remote sensing image dataset show that SwinNET achieves high performance at scale factors of 2×, 3×, and 4×, with improvements over SwinIR, demonstrating better SR reconstruction effects. Additionally, this study conducted mangrove species segmentation experiments using the classic segmentation network FPN, which shows significantly improved performance in the segmentation tasks of Rhizophora stylosa, Avicennia marina, and Bruguiera gymnorrhiza when using 4× enlarged images. The proposed method achieved certain results in the processing and species recognition of mangrove drone remote sensing images, providing reference for the protection and restoration of mangrove ecosystems.

Although this study has achieved promising results, it is limited by the scope of the study area and the inability to distinguish all mangrove species, particularly under complex environmental conditions such as adverse weather. Additionally, the generalizability of the model to other mangrove sites with different ecological characteristics remains to be validated. Moreover, this study only employed the classical FPN segmentation network for species classification, which may limit the comprehensiveness of the evaluation. In future work, we plan to integrate multispectral imagery and other complementary data sources to enhance species classification and improve model robustness. We also acknowledge the need to evaluate our approach using additional segmentation models, such as U-Net and DeepLabV3+, to further validate the effectiveness of super-resolution reconstruction across a broader range of architectures. Furthermore, the potential for real-time, onboard UAV processing will be explored to enable more efficient and responsive monitoring of mangrove ecosystems.