TriEncoderNet: Multi-Stage Fusion of CNN, Transformer, and HOG Features for Forward-Looking Sonar Image Segmentation

Abstract

Forward-looking sonar (FLS) image segmentation is essential for underwater exploration with remaining challenges including low contrast, ambient noise, and complex backgrounds, which both existing traditional and deep learning-based methods fail to address effectively. This paper presents TriEncoderNet, a novel model that simultaneously extracts local, global, and edge-related features through three parallel encoders. Specifically, the model integrates a convolutional neural network (CNN) for local feature extraction, a transformer for global context modeling, and a histogram of oriented gradients (HOG) encoder for edge and shape detection. The key innovations of TriEncoderNet include the CrossFusionTransformer (CFT) module, which effectively integrates local and global features to capture both fine details and comprehensive context, and the HOG attention gate (HAG) module, which enhances edge detection and preserves semantic consistency across diverse feature types. Additionally, TriEncoderNet introduces the hierarchical efficient transformer (HETransformer) with a lightweight multi-head self-attention mechanism to reduce computational overhead while maintaining global context modeling capability. Experimental results on the marine debris dataset and UATD dataset demonstrate the superior performance of TriEncoderNet. Specifically, it achieves an mIoU of 0.793 and mAP of 0.916 on the marine debris dataset, and an mIoU of 0.582 and mAP of 0.687 on the UATD Dataset, outperforming state-of-the-art methods in both segmentation accuracy and robustness in challenging underwater environments.

1. Introduction

With the continued development of ocean engineering, seabed exploration, and marine mapping have become critical components of marine research. Underwater sonar detection technology plays an important role in marine research. Among them, the forward-looking sonar (FLS) sensor has become an indispensable imaging device in underwater exploration due to its mature technology and low cost. FLS images effectively capture the underwater environment without being constrained by lighting conditions, offering rich visual information. Consequently, analysis and processing methods for FLS images have garnered significant attention in recent years [1,2]. One of the key tasks in this domain is FLS image segmentation, which serves as a foundational step for underwater detection and advanced semantic analysis [3]. While natural image segmentation has achieved remarkable progress in recent years, these algorithms cannot be directly applied to FLS images due to their unique characteristics. FLS image segmentation presents two major challenges: (1) Due to the influence of the water’s physical properties on acoustic wave propagation, phenomena such as attenuation and scattering occur, leading to a reduction in the contrast between the background and targets in FLS images, which weakens the semantic information. (2) Underwater environmental noise and ocean reverberation can cause significant interference, and the complex background often makes target feature extraction challenging [4,5,6].

Sonar image segmentation methods are mainly divided into two categories, traditional methods and deep learning-based methods. Traditional sonar image segmentation methods mainly include thresholding-based, Clustering-based, Markov random field (MRF)-based and level set-based methods [7,8,9,10,11,12]. For example, Li et al. [9] proposed an MRF-based method that formalizes the image segmentation problem as an energy minimization problem, leveraging MRF to capture statistical correlations within local regions and optimizing the energy function using a level-set approach to refine segmentation boundaries. Zhang et al. [10] proposed a level-set algorithm combining non-local means and heterogeneity filtering to improve FLS image saliency and segmentation accuracy for small underwater targets. However, reliance on handcrafted features limits robustness and generalizability.

Among the deep learning-based methods, convolutional neural networks (CNNs) are widely used in sonar image segmentation tasks with remarkable results due to their superior feature extraction capabilities. Typical CNN-based algorithms include U-Net [13], DeepLabv3+ [14], FCNN [15] and SegNet [16]. U-Net [13] employed a U-shaped structure that extracts high-level features during encoding and restores spatial resolution during decoding. Skip connections directly transfer encoder features to the decoder, helping recover fine details. DeepLabv3+ [14] utilized dilated convolutions to expand the receptive field, effectively capturing multi-scale context and addressing challenges posed by varying object sizes and blurred edges. MAANU-Net [17] replaced traditional skip connections with an atrous pyramid structure in the encoder–decoder pipeline, enabling more effective feature fusion. It also employed a nested U-shaped structure to integrate features from multiple encoders, further enhancing segmentation accuracy. LMA-Net [18] enhanced inference speed with a lightweight multi-scale attention mechanism and introduced a multi-scale feature attention gate to fuse large spatial features with decoding-stage features, effectively preserving high-precision details from upper-layer features. Despite the excellent performance of these methods in dealing with local features and multi-scale information, they are still insufficiently considered in global feature extraction. How to effectively extract global contextual information and effectively fuse it with local features is the key to further improving the performance of sonar image segmentation.

Vision Transformer (ViT) [19] migrated from the field of Natural Language Processing (NLP) to Computer Vision (CV). With its powerful global context modeling capabilities, it has achieved remarkable results in image segmentation tasks [19,20,21,22,23]. Swin Transformer [20] introduced a hierarchical structure with sliding windows, restricting self-attention to local regions and gradually expanding the window size with increasing network depth, effectively capturing multi-scale features while reducing computational complexity. RegionViT [21] integrated the strengths of Vision Transformers and regional information. By employing Regional Self-Attention, it captured local feature relationships and incorporated global information, effectively addressing the dependencies between local regions and global context. Although Transformer-based image segmentation methods demonstrate remarkable capabilities in capturing global contextual relationships, they still face challenges in representing local features effectively.

To address this limitation, some approaches incorporate CNNs to enhance the expression of local features. By integrating local and global features, these methods aim to achieve a better balance between modeling local details and global context. iFormer [24] combined Inception’s convolution and pooling operations with Transformer self-attention, optimizing image feature extraction and representation. Using a channel-splitting mechanism and frequency ramp structure, it improved efficiency while effectively modeling fine details and global information. TransUNet [25] integrated Transformer modules into the U-Net framework, combining CNNs for local feature extraction with Transformers to encode contextual sequences. By leveraging self-attention, it captured global dependencies, enabling precise detail extraction and long-range interactions for better understanding of complex image structures. Currently, only a few sonar image segmentation methods employ Transformers for global context extraction, such as SonarNet [26]. This approach combined Transformers for global context modeling with CNNs for local feature extraction. In the final stage of the decoder, it fused these two types of features with different semantic meanings, significantly improving segmentation accuracy and highlighting the importance of global context features in sonar image segmentation. While these methods effectively integrate the respective strengths of Transformers and CNNs, they are constrained by a sequential paradigm where local features are extracted first, followed by global feature extraction. This design limits the comprehensive utilization of the interdependencies between local and global features across the network layers, thereby impeding deeper interaction and integration of these features. The Sonar image segmentation methods are summarized in

Table 1. Summary of Sonar image segmentation methods.

Categories	Methods	Advantages	Disadvantages
Traditional methods	Thresholding-based methods [7]	Minimal training requirement, Low computational complexity.	Limited robustness and generalizability.
	Clustering-based methods [8]
	Markov random field (MRF)-based methods [9]
	Level set-based methods [10,11]
Deep learning methods	Typical CNN-based algorithms include U-Net [13], DeepLabv3+ [14], FCNN [15], SegNet [16], MAANU-Net [17] and LMA-Net [18].	Dealing with local features and multi-scale information well.	Considering global feature extraction insufficiently.
	Transformer-based methods including Swin Transformer [20], RegionViT [21]	Capturing global contextual relationships remarkably	Insufficient performance in representing local features.
	Hybrid approaches: iFormer [24], TransUNet [25], SonarNet [26]	Balanced performance between modeling local details and global context.	Limited comprehensive utilization of the interdependencies between local and global features.

.

Beyond single-stream architectures, ensemble-based strategies have also emerged to boost segmentation performance. For instance, Carisi et al. [27] and Dhiyanesh et al. [28] utilize bagging or stacking to aggregate predictions from independently trained models, while Das et al. [29] and Dang et al. [30] employ weighted fusion or multi-layer stacking to refine final outputs. However, these decision-level ensemble paradigms often suffer from high computational redundancy due to running multiple heavy backbones. More importantly, they typically lack deep feature interaction, as independent branches cannot correct or guide each other during the encoding process. Consequently, developing a tightly coupled, multi-branch framework that enables stage-wise deep feature fusion—allowing global and local features to dynamically modulate each other at multiple resolutions—presents a more efficient and effective alternative to simple prediction averaging.

To address the aforementioned challenges, we propose a novel model with three-encoder architecture, TriEncoderNet, which utilizes three parallel encoders to separately extract local high-resolution features, global contextual features, and histogram of oriented gradients (HOG) features. Feature fusion is performed at each downsampling stage, where local features capture fine-grained texture information of the target, global features provide a comprehensive view of the target through expanded receptive fields, and HOG features represent the target’s local edges and shape information. Considering the significant semantic differences between these features, we specifically design the CrossFusionTransformer (CFT) module and the HOG Attention Gate (HAG). The core objective of the CFT module is to deeply integrate local perceptual features with global contextual features. Through the CFT module, we can fully exploit the complementarity between local and global information. Local features enable precise modeling of minute details, while global features help mitigate the insufficiency of local information or reduce noise interference, thus enhancing the model’s ability to adapt to complex scenes while ensuring segmentation accuracy. The HAG module fuses HOG features with image features, maintaining semantic consistency across different feature types. This fusion improves the model’s perception of edge information, optimizing segmentation accuracy in edge regions and preventing the loss of edge details. Moreover, considering that pure Transformers often introduce excessive parameters and computational overhead when extracting global features, we design a hierarchical efficient Transformer (HETransformer). Our contributions are as follows:

• A novel model TriEncoderNet is proposed in this work, which utilizes three parallel branches to extract different types of features simultaneously. CNNs are used to extract local high-resolution features. Transformers are employed to capture global contextual information. A HOG encoder is used to extract gradient features. The segmentation performance is significantly improved by effectively fusing the advantages of these three features.
• TriEncoderNet introduces the CFT module and the HAG module. These two modules enable the deep fusion of features with different semantic meanings at multiple scales. The CFT module enhances information representation by effectively integrating the complementarity of local and global features, while the HAG improves the model’s ability to perceive edge information, significantly boosting segmentation accuracy.
• A HETransformer is desired in TriEncoderNet, incorporating a lightweight multi-head self-attention (LMSA) mechanism. The LMSA mechanism facilitates cross-window interactions within local regions, captures global context, and effectively reduces computational complexity.
• Through comparative experiments on two publicly available datasets, TriEncoderNet outperforms state-of-the-art sonar segmentation algorithms, achieving the best results in terms of performance.

This paper is organized as follows: The related works are reviewed in Section 2. The proposed method, including the overall architecture and the design of each modules, is detailed in Section 3. Section 4 introduces the datasets and the evaluation metrics used in the experiments and the comparative results and visualization analysis are also demonstrated. The ablation study is followed in Section 5. In Section 6, we draw a conclusion of this work.

2. Related Work

2.1. CNN-Based Sonar Image Segmentation Methods

CNN-based sonar image segmentation algorithms, including modular architecture [31], CNN-based encoder–decoder structure [32], Seg2Sonar network [33], CNN with dynamic multi-scale dilated convolutions [34] and dual-channel segmentation network [35], perform well in local feature extraction but often struggle with capturing global features. Without a global perspective, the model cannot fully capture the deep semantic relationships between the target and background, making it more vulnerable to noise and reducing segmentation accuracy and robustness. Therefore, integrating global features is crucial for enhancing segmentation precision and model stability.

2.2. Transformer-Based Image Segmentation Methods

Vision Transformer (ViT) [19] used self-attention mechanisms on image patches to capture global dependencies, surpassing the limitations of traditional CNNs and showcasing the potential of Transformers in visual tasks. CoaT [36] achieved cross-scale information interaction by integrating serial and parallel blocks, effectively enabling the modeling of image features from fine to coarse and vice versa. Focal Transformer [37] introduced a strategy that combined fine-grained local attention with coarse-grained global attention, facilitating the capture of both short-range and long-range dependencies. Rajani et al. [38] introduced a multi-scale encoder for sonar image segmentation, combining patch merging with efficient convolutions to enhance feature extraction while reducing computational complexity.

2.3. Hybrid Transformer–CNN Models

To extract both global and local features, various hybrid Transformer–CNN models have been developed. H2Former [39] introduced the Multi-Scale Channel Attention (MSCA) module, which dynamically selected relevant features by fusing multi-scale global and local features while modeling inter-channel dependencies. UNETR [40] used a Transformer encoder to capture global features and convolutional operations at multiple resolutions for local feature extraction. ST-UNet [41] employed a dual-encoder structure with the Swin Transformer for global modeling and CNNs for local extraction, using a Spatial Interaction Module (SIM) to model pixel-level correlations between different feature semantics. TopFormer [42] employed Vision Transformers to extract global features from a Token Pyramid. During upsampling, it fused multi-scale local tokens with global features, achieving scale-aware semantic extraction and producing high-quality segmentation results. UNetFormer [43] combined CNNs for local detail extraction and a Transformer-based decoder with a global–local attention mechanism to integrate both global and local context.

However, most of the existing models only use high-level features that have been processed by CNNs, ignoring the importance of using low-level features in establishing remote dependencies. In addition, the semantic difference between local and global features is large, which makes the fusion of these two features not ideal. Moreover, the traditional Transformer has too much computation in extracting global features, which leads to inefficiency, especially when processing high-resolution images, as the computational overhead increases significantly.

3. Method

3.1. Overall Architecture

The overall architecture of the proposed TriEncoderNet is shown in Figure 1. It aims to achieve more accurate image segmentation by extracting in parallel local detail features, global features, and Histogram of Oriented Gradients (HOG) features through multiple encoders. To address the high computational overhead associated with Transformer models when processing global features, TriEncoderNet introduces a HETransformer to effectively reduce computational complexity. Additionally, TriEncoderNet proposes the CFT module, which fuses CNN features and Transformer features at each downsampling stage of the encoder. This allows low-level features from different encoders to complement each other, preserving more detailed information. Furthermore, the CFT module utilizes a Cross-Attention Block to effectively integrate features with different semantic information, thereby enhancing the model’s ability to capture diverse feature types. To better leverage HOG features, TriEncoderNet incorporates an HAG module to achieve deep integration of HOG and image features. To further explore the correlation between HOG features and image features, an Attention Map Dropout mechanism is introduced, enabling the network to focus more effectively on the contribution of HOG features to edge information extraction during training.

TriEncoderNet takes Forward-Looking Sonar (FLS) images $\mathbf{X}\in {\mathbb{R}}^{512\times 512}$ as input. Initially, the input images are fed into both the HOG feature encoder and the CNN encoder. The image features are processed through a CNN block to generate CNN feature maps ${\mathbf{X}}_{\mathrm{cnn}}^i\in {\mathbb{R}}^{{\displaystyle \frac{H}{2^i}}\times {\displaystyle \frac{W}{2^i}}\times C}$ where H, W, and C represent the height, width, and number of channels of the image, respectively. While i represents the encoder layer index, $i=(0,1,2,3,4)$. Each CNN block consists of two convolutional layers, two batch normalization layers, and ReLU activation functions, designed to extract rich image features. The extracted CNN features are then simultaneously processed by multiple modules. Firstly, the image features are fed into the Transformer encoder to capture global contextual relationships, while also being passed to the next stage of the CNN encoder for further extraction of local detail features. Within the Transformer encoder, the input image features undergo a patch embedding operation, converting them into feature tokens $Z_{trans}^i\in {\mathbb{R}}^{n\times d}$, where $n={\displaystyle \frac{HW}{2^{2i}{P}^2}}$, d is the dimension of tokens. These tokens are then processed by the HETransformer for global context modeling, followed by downsampling via a patch merge layer. The tokens are subsequently converted back into feature maps through a feature mapping operation. The feature maps from both the Transformer and CNN encoders are then fed into the CFT module for feature fusion. The CFT module leverages a cross-attention mechanism to deeply integrate features from different encoders. The fused features are further processed by the HAG module, where they are combined with HOG features to achieve deep integration and enhance the representation of semantic information from three distinct sources. TriEncoderNet fuses semantic information at each scale and enhances feature representation through the mutual fusion of features across each scale. The fused features are finally passed through a decoder to restore the image resolution.

To improve the readability of the mathematical formulations, the key parameters and notations used in this paper are summarized in

Table 2. Description of key symbols and parameters used in TriEncoderNet.

Symbol	Description
$H,W,C$	Height, width, and channel number of the input image
P	Patch size used in Patch Embedding
d	Dimension of the feature vectors (tokens)
n	Number of tokens
M	Window size in Window Partition
k	Downsampling factor in LMSA
$Z_{trans}$	Feature vectors from the Transformer encoder
$Z_{cnn}$	Feature vectors from the CNN encoder
$Z_{hog}$	Feature vectors from the HOG encoder
$Map$	Attention maps generated in HAG

.

In the following sections, we will provide a detailed description of key modules, including the HETransformer, CFT, HOG Extract Block, and HAG, along with their respective functionalities and contributions to the proposed architecture.

3.2. Hierarchical Efficient Transformer (HETransformer)

Before image features are processed by the HETransformer, they undergo a Patch Embedding operation to generate feature vectors. This process consists of two steps. First, the image is divided into multiple patches using Patch Partition, where each patch has a size of $P\times P$. Next, these patches are mapped into a high-dimensional space through a Linear Embedding layer, producing feature vectors $Z_{\mathrm{trans}}^i$ of dimension d. These feature vectors $Z_{\mathrm{trans}}^i$ are then fed into the HETransformer for self-attention computation. The architecture of the HETransformer is shown in Figure 2.

In the standard Transformer architecture [19], global self-attention is computed across all feature tokens, resulting in a computational complexity that scales quadratically with both the number of tokens (n) and their dimensionality (d). This high computational cost poses significant challenges during both training and inference. To mitigate this, the HETransformer reduces the number of tokens (n), substantially decreasing the computational overhead. As shown in the figure, the HETransformer adopts a dual strategy. First, feature tokens are transformed back into feature maps via a Feature Mapping operation and processed by the LMSA module. Second, a Windows Partition strategy [41] is employed, where $M\times M$ patches are divided into multiple non-overlapping windows. Within each window, local self-attention is computed to model remote dependencies among patches, replacing the computationally expensive global self-attention. This step produces feature tokens $Z_w^i\in {\mathbb{R}}^{n^{\prime }\times d}$, where $n^{\prime }=M^2$, significantly reducing the overall computational complexity. To further enhance global context modeling, the HETransformer incorporates a Shifted Windows Partition strategy [41] in subsequent stages. By shifting the positions of windows, this approach ensures the effective integration of global information. Together, these strategies reduce the computational complexity of self-attention while preserving the ability to capture meaningful global dependencies. The computational formulation of the HETransformer is as follows:

$${\widehat{Z}}_{trans}^i=LMSA(FM\left(Z_{trans}^i\right),WP\left(Z_{trans}^i\right))+Z_{trans}^i$$

(1)

$${\dot{Z}}_{trans}^i=MLP\left(LN\left({\widehat{Z}}_{trans}^i\right)\right)+{\widehat{Z}}_{trans}^i$$

(2)

$${\tilde{Z}}_{trans}^i=SLMSA(FM\left({\dot{Z}}_{trans}^i\right),SWP\left({\dot{Z}}_{trans}^i\right))+{\dot{Z}}_{trans}^i$$

(3)

$${\ddot{Z}}_{trans}^i=MLP\left(LN\left({\tilde{Z}}_{trans}^i\right)\right)+{\tilde{Z}}_{trans}^i$$

(4)

where $LMSA(·)$ and $SLMSA(·)$ represent the LMSA mechanism and shifted LMSA mechanism, $FM(·)$ denotes Feature Map, $WP(·)$ and $SWP(·)$ refer to Windows Partition and Shifted Windows Partition, $MLP(·)$ stands for Multi-Layer Perceptron, and $LN(·)$ represents Layer Normalization.

The LMSA mechanism is depicted in Figure 2. LMSA takes two inputs, which are used to generate the query, key, and value. Specifically, LMSA performs a depth-wise convolution along the channel dimension of the input Feature Map, reducing its resolution by a factor of k, resulting in feature maps denoted as $x^i\in {\mathbb{R}}^{{\displaystyle \frac{H}{2^{i}k}}\times {\displaystyle \frac{W}{2^{i}k}}\times d}$. Here, k is a hyperparameter, and the depth-wise convolution uses a kernel size of $k+1$, a stride of k, and a padding of $k/2$. The downsampled feature map is then subjected to a patch embedding operation, converting it into feature vectors. These vectors are further divided into windows using the Windows Partition method, where ${\displaystyle \frac{M}{k}}\times {\displaystyle \frac{M}{k}}$ patches are grouped into windows to create feature vectors $Z_{dw}^i\in {\mathbb{R}}^{n^{″}\times d}$, where $n^{″}={\displaystyle \frac{k^2}{M^2}}$. Subsequently, a Layer Normalization (LN) operation is applied to generate $V_{dw}^i$ and $K_{dw}^i$. The second input to LMSA, $Z_w^i$, undergoes a Layer Normalization operation directly to produce $Q_w^i$. The LMSA calculation is formally expressed as follows:

$$V_{dw}^i=Z_{dw}^i{W}_V^{dw},K_{dw}^i=Z_{dw}^i{W}_K^{dw},Q_w^i=Z_w^i{W}_Q^w$$

(5)

The corresponding weight matrices for $V_{dw}^i$, $K_{dw}^i$ and $Q_w^i$ are $W_V^{dw}$, $W_K^{dw}$ and $W_Q^w$, respectively.

Finally, the attention mechanism is calculated using the following formula:

$$Attention(Q,K,V)=Softmax({\displaystyle \frac{{Q_w^i{K}_{dw}^i}^{\mathrm{T}}}{\sqrt{d}}})V_{dw}^i$$

(6)

The computational complexity of LMSA is $O(({\displaystyle \frac{2}{k^2}}+1)n{d}^2+({\displaystyle \frac{2M^2+2k+1}{k^2}}+2)nd)$. In contrast, the computational complexities of the MSA modules in ViT and Swin Transformer are $O(4n{d}^2+2n^{2}d)$ and $O(4n{d}^2+2M^{2}nd)$, respectively. The computational complexity of LMSA is significantly lower than these classical approaches.

3.3. CrossFusionTransformer (CFT)

Features from different encoders often exhibit significant semantic differences. However, these features also possess unique strengths that are complementary to one another. CNN encoders excel at capturing local features, effectively extracting high-precision local details and progressively deriving higher-level semantic features through multiple convolutional layers. In contrast, Transformer encoders are adept at capturing global relationships, capable of handling long-range dependencies and complex contextual information. They demonstrate strong capabilities in modeling interactions between distant regions within an image. To integrate and align the features from both types of encoders, we propose the CFT module, as illustrated in Figure 3.

First, the features from different encoders are transformed through Patch Embedding to obtain the transformer feature vectors ${\mathrm{Z}}_{trans}^i\in {\mathbb{R}}^{n\times d}$ and the CNN feature vectors $Z_{cnn}^i\in {\mathbb{R}}^{n\times d}$, respectively. Subsequently, the two feature vectors undergo an initial fusion process, which can be expressed by the following equation:

$${\mathrm{Z}}_f^i=PE\left(FM\left(Concat({\mathrm{Z}}_{trans}^i,Z_{cnn}^i)\right)\right)$$

(7)

Here, $PE(·)$ represents Patch Embedding, and ${\mathrm{Z}}_f^i\in {\mathbb{R}}^{n\times d}$ denotes the feature vector after the initial fusion.

However, simple feature fusion cannot eliminate the semantic gap between features. To address this, the transformer feature vector ${\mathrm{Z}}_{trans}^i$, the image feature vector ${\mathrm{Z}}_{cnn}^i$, and the fused feature vector ${\mathrm{Z}}_f^i$ are introduced. ${\mathrm{Z}}_f^i$ serves as supplementary information. These multi-source features are used to extract the query vector, key vector, and value vector in parallel. The specific calculations for this process are as follows:

$$V_f^i={\mathrm{Z}}_f^i{W}_V^f,K_{cnn}^i=Z_{cnn}^i{W}_K^{cnn},Q_{cnn}^i=Z_{cnn}^i{W}_Q^{cnn}$$

(8)

$$K_{trans}^i=Z_{trans}^i{W}_K^{trans},Q_{trans}^i=Z_{trans}^i{W}_Q^{trans}$$

(9)

The matrices $W_V^f$, $W_K^{trans}$,$W_K^{cnn}$, $W_Q^{trans}$, and $W_Q^{cnn}$ represent the weight matrices corresponding to $V_f^i$, $K_{trans}^i$, $K_{cnn}^i$, $Q_{trans}^i$, and $Q_{cnn}^i$, respectively. After obtaining the query, key, and value vectors from the multi-source features, they are fed into Cross Attention (CA) Blocks for further processing. Specifically, one set of features ($V_f^i$, $K_{trans}^i$, $Q_{trans}^i$) is fed into one CA Block, while the other set ($V_f^i$, $K_{cnn}^i$, $Q_{cnn}^i$) is fed into another CA Block. Each CA Block performs identical computational operations. The detailed calculation process within a CA Block is as follows:

$$CAttention(Q_{cnn}^i,K_{cnn}^i,V_f^i)=Softmax({\displaystyle \frac{{Q_{cnn}^i{K}_{cnn}^i}^{\mathrm{T}}}{\sqrt{d}}})V_f^i$$

(10)

The output results of the dual-branch CA Block are normalized in parallel and then further processed by a Multi-Layer Perceptron (MLP). Finally, the feature vectors are fused through a concatenation operation. The fused features are subsequently mapped back into image features through Feature Mapping. A series of convolution operations is then applied to reduce the number of channels in the feature maps, enabling deeper feature integration and producing the final output.

3.4. Hog Extract Block, HogConv Block and HOG Attention Gate

Inspired by research on ship classification in radar images [44] and facial recognition [45] using HOG features, we observed that HOG features excel in capturing edge information of objects in images. The specific structure of the HOG Extract Block is shown in Figure 4, and its details are described below.

First, the input FLS image undergoes preprocessing, with the specific steps outlined as follows:

$$X^{\prime }=max\{X-mean\left(X\right),0\}$$

(11)

where X represents the input FLS image, $mean(·)$ denotes the computation of the grayscale average. Next, the gradient for each pixel in the preprocessed image is calculated as follows:

$$G(x,y)=\sqrt{G_x{(x,y)}^2+G_{\mathrm{y}}{(x,y)}^2}$$

(12)

$$\theta (x,y)=arctan{\displaystyle \frac{G_{\mathrm{y}}(x,y)}{G_x(x,y)}}$$

(13)

where $(x,y)$ represents each pixel in the image. The pixel gradient is described using the gradient magnitude $G_x(x,y)$ and gradient direction $\theta (x,y)$. The specific calculation processes for gradient magnitude and gradient direction are as follows:

$$G_x(x,y)=V(x+1,y)-V(x-1,y)$$

(14)

$$G_y(x,y)=V(x,y+1)-V(x,y-1)$$

(15)

where $V(·)$ represents the pixel value at each point in the image. The image is first divided into smaller cells, and the gradient magnitude and direction are calculated for each cell. The gradient magnitude and direction of each cell are then mapped into a histogram based on predefined orientation bins. Subsequently, neighboring cells are grouped into blocks, and block-level normalization is applied to the histograms of these cells. Finally, the combined histograms from all cells are concatenated to form the HOG feature vector. The resulting HOG feature vector is then resized to serve as the input HOG feature for the HOG encoder.

To effectively extract HOG features, TriEncoderNet designs the HogConv Block, as illustrated in Figure 4. This module consists of three sets of convolutional operations, each followed by Batch Normalization and the ReLU activation function to introduce nonlinearity. Specifically, the first convolutional operation employs a 1 × 1 kernel and maintains the same number of channels, aiming to extract low-level features from the image. The second convolutional operation utilizes a 3 × 3 kernel with a stride of 1, doubling the number of channels to enhance the network’s representation capacity and depth. Next, a pooling operation is performed to achieve down-sampling. The final convolutional operation also uses a 3 × 3 kernel with a stride of 1, maintaining the channel count, and focuses on extracting higher-level semantic features.

For the HOG features and the image features from the CFT module, TriEncoderNet designs a HAG to achieve feature fusion, as illustrated in Figure 5. First, the two features are concatenated and then undergo initial feature fusion through a 1 × 1 convolutional operation, followed by Batch Normalization and a ReLU activation function. Next, we employ P Attention Extract branches to generate attention maps. By incorporating HOG features, these branches can automatically identify critical regions associated with the HOG features, particularly the object edges and contour regions that are crucial for FLS image segmentation tasks. Each Attention Extract branch consists of two convolutional layers designed to reduce the feature dimensionality, with an activation function introduced between the layers to add nonlinearity. Finally, the attention maps ${\mathrm{Map}}_m\in {\mathbb{R}}^{{\displaystyle \frac{H}{2^i}}\times {\displaystyle \frac{W}{2^i}}}$ are obtained through a Sigmoid activation function, where $Map=[Ma{p}_1,Ma{p}_1,\dots ,Ma{p}_P]$.

To encourage the model to explore more edge feature information brought by HOG features, TriEncoderNet proposes an Attention Drop mechanism. This mechanism randomly eliminates s attention maps by setting all values in these s maps to zero. The remaining $N=P-s$ attention maps are then merged into a final attention map $Ma{p}_{final}\in {\mathbb{R}}^{{\displaystyle \frac{H}{2^{\mathrm{i}}}}\times {\displaystyle \frac{W}{2^{\mathrm{i}}}}}$. The detailed calculation is as follows:

$$Max(Ma{p}_1(x,y),Ma{p}_2(x,y),\dots ,Ma{p}_N(x,y))$$

(16)

where $x\in [1,{\displaystyle \frac{H}{2^{\mathrm{i}}}}]$,$y\in [1,{\displaystyle \frac{W}{2^{\mathrm{i}}}}]$. Finally, $Ma{p}_{final}$ is multiplied element-wise with the previously fused image features. Through multiple Attention Extract branches and the Attention Drop mechanism, the HAG assigns different weights to each feature region, further highlighting the shape and edge information brought by the HOG features. This not only effectively integrates HOG features with image features but also significantly enhances the utilization efficiency of the edge information provided by HOG features.

The intuition behind this design stems from the specific degradation mechanisms of underwater acoustics. While CNNs capture local texture details often obscured by speckle noise, and Transformers model global context to distinguish valid targets from reverberation shadows, the HOG encoder introduces explicit geometric priors to sharpen boundaries that are typically blurred in FLS imagery. By fusing these complementary views, the model achieves robust segmentation in complex underwater environments.

4. Experiment

This section provides a detailed introduction to the datasets used in the experiments, evaluation metrics, comparisons with other models and visualized comparison results.

4.1. FLS Image Dataset

Marine Debris Dataset [46]: This dataset consists of 1868 FLS images collected using an ARIS Explorer 3000 Sonar. It includes 11 target categories (excluding the background). Notably, Standing bottle and Shampoo bottle account for only 2% and 3%, respectively, while Wall accounts for 25%. Categories like Standing bottle, Hook, Drink carton, and Shampoo bottle only account for 1%, 2%, 2%, and 2% of pixels, respectively, whereas Wall comprises 44%. This dataset suffers from significant class imbalance, especially for smaller objects. Sample images of Marine Debris Dateset are shown in Figure 6.

UATD Dataset [47]: This dataset was collected using a Tritech Gemini 1200ik sonar in lakes and oceans. It contains 9200 FLS images across 10 target categories: sphere, cube, cylinder, human, plane, circular cage, square cage, metal barrel, tire, and ROV. While the original dataset was annotated with bounding boxes, we further refined the annotations using LabelMe to provide semantic segmentation for objects in each category across the entire dataset. Sample images of UATD Dateset are shown in Figure 7.

4.2. Evaluation Metrics

We comprehensively evaluate TriEncoderNet from two aspects: segmentation accuracy and segmentation efficiency. Segmentation accuracy is assessed using mean Pixel Accuracy (mAP), Intersection over Union (IoU), and mean Intersection over Union (mIoU) as evaluation metrics. Segmentation efficiency is measured in terms of model size and inference time.

mAP measures the proportion of correctly classified pixels within each class, averaged across all classes. The calculation formula is as follows:

$$mAP={\displaystyle \frac{1}{k+1}}\sum _{i=0}^k{\displaystyle \frac{p_{ii}}{{\sum }_{j=0}^k{p}_{ij}}}$$

(17)

$p_{ij}$ represents the number of pixels that belong to class i but are misclassified as class j. Similarly, $p_{ii}$ represents the number of pixels correctly classified as class i. K represents the total number of classes.

IoU calculates the ratio between the intersection and the union of the ground truth A and the segmentation result B. The specific formula is shown as follows:

$$\mathrm{IoU}={\displaystyle \frac{|A\cap B|}{|A\cup B|}}$$

(18)

mIoU calculates the IoU for each class and then computes the average across all classes. The specific formula is shown as follows:

$$\mathrm{mIoU}={\displaystyle \frac{1}{k}}\sum _{i=1}^k{\mathrm{IoU}}_i={\displaystyle \frac{1}{k}}\sum _{i=1}^k{\displaystyle \frac{|A_i\cap B_i|}{|A_i\cup B_i|}}$$

(19)

where k is the total number of classes, $A_i$ is the ground truth for class i, and $B_i$ is the segmentation result for class i.

4.3. Implementation Details

Due to the absence of an official division for the Marine Debris Dataset and the UATD dataset, we randomly divided the training, validation, and testing sets in a 7:2:1 ratio. The input FLS images were resized to $512\times 512$, and data augmentation techniques such as random flipping, random cropping, random scaling, and color jittering were employed. During training, the Adam optimizer [48] was used for 160 epochs with a batch size of 8. A cosine annealing learning rate schedule was applied to accelerate model convergence, starting with an initial learning rate of $1\times {10}^{-4}$. To mitigate the class imbalance in the datasets, we utilized a joint loss function combining Dice Loss and Cross-Entropy (CE) Loss [49]. All hyperparameters were tuned on the validation set, and the final model performance was evaluated on the test set. We conducted training and inference for TriEncoderNet and baseline models using an NVIDIA Geforce RTX 3090 Ti GPU. The hyperparameters and training details are listed in

Table 3. Implementation details and hyperparameters.

Category	Parameter	Value
Training Settings	Optimizer	Adam
	Initial Learning Rate	$1\times {10}^{-4}$
	Batch Size	8
	Epochs	160
	Weight Decay	$1\times {10}^{-4}$
	Learning Rate Scheduler	Cosine Annealing
Model Architecture	Input Size	$512\times 512$
	Patch Size (P)	4
	Window Size (M)	7
Data Augmentation	Random Flip	Probability 0.5
	Random Rotation	[−10°, 10°]
	Random Scaling	[0.5, 2.0]
	Color Jittering	Brightness, Contrast

. For comparison experiments, we retrained state-of-the-art (SOTA) methods with their recommended parameter settings. The same dataset partitioning strategy, training duration, batch size, and image augmentation methods as our approach were applied to ensure fairness.

4.4. Comparison with the SOTA Methods

We conducted comparative experiments between TriEncoderNet and state-of-the-art (SOTA) methods on the Marine Debris Dataset and UATD dataset. The SOTA methods include CNN-based models such as U-Net [13], DeepLabv3+ [14], FCNN [15], UNet3+ [50], SegNet [16], as well as Transformer-based models such as PVT [22], TransUNet [25], Swin Transformer [20], MT-UNet [51], COAT [36], and SAM [49].

Marine Debris Dataset: The quantitative analysis results of TriEncoderNet and SOTA methods on the Marine Debris Dataset are shown in

Table 4. Quantitative results of TriEncoderNet and SOTA methods on the Marine Debris Dataset.

Model	IoU ↑												Evaluation Index ↑
	Background	Bottle	Can	Chain	Drink Carton	Hook	Propeller	Shampoo Bottle	Standing Bottle	Tire	Valve	Wall	M_IoU	MPA
U-Net	0.99	0.81	0.54	0.69	0.69	0.65	0.54	0.73	0.49	0.86	0.59	0.80	0.699	0.816
Deeplabv3+	0.99	0.79	0.61	0.62	0.76	0.68	0.70	0.86	0.53	0.89	0.57	0.87	0.739	0.846
FCNN	0.98	0.76	0.59	0.57	0.69	0.64	0.63	0.76	0.41	0.86	0.50	0.83	0.685	0.801
Unet3+	0.99	0.81	0.63	0.64	0.79	0.76	0.70	0.83	0.46	0.90	0.64	0.85	0.750	0.872
SegNet	0.99	0.75	0.57	0.63	0.75	0.73	0.73	0.81	0.49	0.90	0.65	0.89	0.741	0.851
PVT	0.99	0.83	0.56	0.70	0.71	0.68	0.64	0.76	0.46	0.87	0.60	0.82	0.718	0.848
TransUNet	0.99	0.71	0.67	0.69	0.72	0.59	0.73	0.76	0.31	0.90	0.42	0.86	0.696	0.814
Swintransformer	0.99	0.82	0.67	0.63	0.74	0.67	0.79	0.80	0.50	0.87	0.63	0.88	0.749	0.871
MT-Unet	0.99	0.84	0.54	0.69	0.77	0.70	0.71	0.84	0.44	0.90	0.65	0.88	0.746	0.870
COAT	0.99	0.80	0.56	0.71	0.70	0.67	0.53	0.72	0.51	0.88	0.61	0.82	0.708	0.823
SAM	0.99	0.78	0.59	0.66	0.78	0.74	0.73	0.84	0.46	0.90	0.64	0.88	0.749	0.886
Our	0.99	0.85	0.70	0.73	0.81	0.77	0.79	0.86	0.54	0.91	0.66	0.90	0.793	0.916

, where the bold font indicates the highest metric values. We evaluated the IoU metric for each category and used mIoU and mAP to assess overall performance. TriEncoderNet demonstrated superior performance across all three evaluation metrics.

Specifically, compared to the baseline model U-Net, TriEncoderNet improved mAP and mIoU by 9.4% and 10%, respectively. Among CNN-based methods, TriEncoderNet outperformed the best-performing UNet3+ by 4.3% and 4.4% in mAP and mIoU, respectively. Compared to the best-performing Transformer-based method, SAM-Sonar, TriEncoderNet still achieved improvements of 4.4% in mAP and 3% in mIoU. Furthermore, TriEncoderNet consistently led in IoU for each individual category compared to all other models.

UATD Dataset: The quantitative analysis results of TriEncoderNet and SOTA methods on the UATD dataset are presented in

Table 5. Quantitative results of TriEncoderNet and SOTA methods on the UATD Dataset.

Model	IoU ↑										Evaluation Index ↑
	Background	Cube	Ball	Cylinder	Human Body	Tire	Cage	Metal Bucket	Plane	Rov	M_IoU	MPA
U-Net	0.99	0.43	0.41	0.25	0.39	0.46	0.47	0.40	0.40	0.41	0.461	0.523
Deeplabv3+	0.99	0.38	0.41	0.26	0.34	0.40	0.45	0.43	0.35	0.41	0.442	0.491
FCNN	0.99	0.32	0.31	0.16	0.31	0.30	0.43	0.47	0.22	0.20	0.371	0.421
Unet3+	0.99	0.41	0.45	0.26	0.40	0.42	0.49	0.52	0.45	0.30	0.469	0.558
SegNet	0.99	0.39	0.47	0.28	0.42	0.44	0.50	0.41	0.42	0.38	0.470	0.550
PVT	0.99	0.38	0.45	0.30	0.41	0.45	0.51	0.42	0.39	0.37	0.467	0.516
TransUNet	0.99	0.36	0.60	0.26	0.43	0.49	0.45	0.58	0.43	0.36	0.495	0.588
Swintransformer	0.99	0.40	0.62	0.29	0.46	0.49	0.61	0.60	0.54	0.50	0.550	0.632
MT-Unet	0.99	0.39	0.43	0.28	0.36	0.43	0.48	0.46	0.26	0.42	0.450	0.515
COAT	0.99	0.45	0.40	0.26	0.41	0.41	0.50	0.38	0.41	0.42	0.463	0.521
SAM	0.99	0.51	0.59	0.27	0.45	0.51	0.59	0.61	0.56	0.46	0.554	0.641
Our	0.99	0.53	0.62	0.30	0.48	0.52	0.64	0.62	0.62	0.50	0.582	0.687

, where the bold font indicates the highest metric values. We evaluated the IoU metric for each category and used mIoU and mAP for overall assessment. TriEncoderNet achieved the best results across all segmentation tasks for the UATD dataset.

Compared to the baseline model U-Net, TriEncoderNet improved mIoU and mAP by 12.1% and 16.4%, respectively. Among CNN-based methods, TriEncoderNet outperformed the best-performing SegNet by 11.2% and 13.7% in mIoU and mAP, respectively. Compared to the best-performing Transformer-based method, Swin Transformer, TriEncoderNet achieved improvements of 3.2% in mIoU and 5.5% in mAP. Additionally, TriEncoderNet maintained a consistent lead in IoU for every individual category compared to all other models.

Robustness Analysis: Furthermore, to empirically quantify the model’s robustness against acoustic interference, we conducted stress tests using synthetic Gaussian and Speckle noise on the Marine Debris Dataset in

Table 6. Quantitative robustness analysis of TriEncoderNet and baseline methods under synthetic noise on Marine Debris Dataset.

Model	Noise Type	Intensity (${\mathit{\sigma }}^2$)	Metrics
			mIoU	Performance Drop
U-Net	None	0	0.699	-
	Gaussian	0.01	0.642	−8.2%
	Gaussian	0.05	0.515	−26.3%
	Speckle	0.05	0.588	−15.9%
	Speckle	0.10	0.473	−32.3%
Swin Transformer	None	0	0.749	-
	Gaussian	0.01	0.695	−7.2%
	Gaussian	0.05	0.580	−22.6%
	Speckle	0.05	0.645	−13.9%
	Speckle	0.10	0.535	−28.6%
TriEncoderNet (Ours)	None	0	0.793	-
	Gaussian	0.01	0.771	−2.8%
	Gaussian	0.05	0.705	−11.1%
	Speckle	0.05	0.748	−5.7%
	Speckle	0.10	0.682	−14.0%

. Under severe noise conditions (e.g., Speckle noise with ${\sigma }^2=0.10$), the baseline U-Net and Swin Transformer suffered catastrophic performance drops of 32.3% and 28.6%, respectively. In contrast, TriEncoderNet exhibited superior resilience, limiting the degradation to only 14.0%. This confirms that the HOG-based multi-stage fusion effectively preserves structural integrity even when pixel-level coherence is compromised by noise.

4.5. Computational Efficiency Analysis

As shown in Figure 8, it is evident that CNN-based models are significantly smaller in size and inference speed compared to transformer-based models, primarily due to the inevitable computational overhead of global attention mechanisms. TriEncoderNet incorporates both CNN and transformer-based parallel encoders. Despite this, its model size is still smaller than most transformer-based models, and its inference speed outperforms many of them. This improvement is attributed to the design of our HETransformer, which significantly reduces computational overhead compared to classical transformers. Although TriEncoderNet is not the smallest in terms of model size or the fastest in inference speed, it achieves the highest segmentation accuracy at 91.6%. With a model size of 49.07 MB and an inference speed of 37 fps, TriEncoderNet reduces the model size by 274.96 MB and improves inference speed by 185% compared to SAM, the second most accurate model. TriEncoderNet achieves the best segmentation performance with minimal computational overhead.

To provide a standardized comparison of computational cost independent of hardware platforms, we evaluated the Giga Floating-point Operations (GFLOPs) with an input size of $512\times 512$. As presented in

Table 7. Comparison of computational cost (GFLOPs) with SOTA methods. Input size is standardized to $512\times 512$.

Method	GFLOPs (G)
U-Net	126.4
DeepLabv3+	30
FCNN	152.3
UNet3+	195.4
SegNet	160.2
PVT	46.5
TransUNet	128
SwinTransformer	80.5
MT-UNet	75.1
CoaT	94.8
SAM	325.7
TriEncoderNet (Ours)	54

, our TriEncoderNet achieves a competitive computational cost compared to other SOTA methods.

4.6. Visualization Analysis

To visually and comprehensively validate the performance of TriEncoderNet, we conducted a comparative analysis of segmentation results on the Marine Debris Dataset and UATD dataset. This analysis included comparisons with SOTA methods such as U-Net [13], Unet3+ [14], SegNet [16], TransUNet [25], and SAM [49].

Marine Debris Dataset: We performed visualization analysis on the test set of the Marine Debris Dataset, which included scenarios with complex-shaped objects (e.g., chains) and small-scale targets (e.g., hooks, drink cartons). As shown in Figure 9, TriEncoderNet demonstrated precise segmentation of underwater objects even in complex backgrounds. While other methods were also capable of segmenting various underwater targets, they were prone to errors due to prominent noise interference and complex backgrounds, leading to imprecise segmentation.

In Scene 1, where the chain is located in a complex background, CNN-based methods produce discontinuous segmentation results for the chain. In contrast, TriEncoderNet achieves precise and continuous segmentation. This is attributed to its dual ability to extract both local features and global contextual information, enabling it to effectively capture relevant target features. In Scene 2, TriEncoderNet is the only model capable of accurately segmenting the edges of the wall and cans. This advantage is due to the integration of HOG features, which enhance the capture of contour details. Other methods are significantly affected by noise, leading to incomplete or imprecise segmentation results. In Scene 3, for valve segmentation, only TriEncoderNet manages to clearly capture the branch details along the valve’s edges. This performance is made possible by its deep-level extraction of local detail features and the seamless fusion of HOG features with image characteristics. Other methods generally fail to address these crucial edge details effectively. In Scene 4, hooks are small-scale targets. CNN-based methods, while strong in local feature extraction, fail to segment the upper portion of the hook due to their lack of global contextual modeling. Transformer-based methods can roughly capture the upper portion of the hook’s shape but fall short in edge precision due to insufficient attention to local details. Only TriEncoderNet successfully achieves the most accurate segmentation, combining the strengths of both local detail extraction and global context awareness. In Scene 5, bottle segmentation demonstrates TriEncoderNet’s robustness. SegNet fails to fully segment the bottle’s shape, and other models inaccurately segment the bottle neck due to noise interference and low contrast with the background. TriEncoderNet, however, accurately captures the bottle’s complete shape, showcasing its superior robustness to noise and ability to capture fine details.

UATD Dataset: For the UATD dataset, we analyzed test set scenarios including similar objects (e.g., cube and cage), small-scale targets (e.g., ROV and ball), and sparse objects (e.g., human body), as shown in Figure 10.

On the UATD dataset, Scene 1 and Scene 2 involve similar targets (e.g., square cage and cube) and small-scale objects (e.g., tires). CNN-based methods are heavily affected by underwater noise and fail to segment the edges of the square cage or the full tire region. TriEncoderNet, however, achieves complete segmentation of these targets, demonstrating its adaptability and accuracy in challenging environments. In Scene 3, both TransUNet [25] and SAM failed to segment the plane’s wings, lacking the ability to extract high-precision local details. Furthermore, all other methods overlooked the extraction of potential HOG features during the overall segmentation process, resulting in inferior performance compared to TriEncoderNet. In Scene 4, where the target ROV is a small-scale object with weak semantic information and significant background noise, only TriEncoderNet achieved precise segmentation, demonstrating its exceptional capability in handling small objects. Only TriEncoderNet achieved precise segmentation, demonstrating its exceptional capability in handling small objects. In Scene 6, when segmenting the human body, the low contrast between the target and the background led all other methods, except TriEncoderNet, to fail in segmenting the limbs of the human body. TriEncoderNet’s superior performance is attributed to its robust ability to extract detailed features and effectively model global contextual information.

The visualization analysis on the Marine Debris Dataset and UATD dataset reveals the primary challenges in underwater target segmentation: low contrast, subtle target features, and interference from complex backgrounds and noise. Traditional segmentation methods (CNN-based and Transformer-based models) show notable limitations when dealing with complex-shaped objects, small-scale targets, and sparse objects. These limitations include insufficient local detail extraction, inadequate global context modeling, and reduced segmentation accuracy under noisy conditions. In contrast, TriEncoderNet, with its multi-stage feature fusion, deep modeling of local details and global context, and the extraction and integration of HOG features, achieves precise segmentation across various challenging scenarios. It particularly excels in segmenting small-scale targets, edge details, and low-contrast objects, highlighting its robustness and adaptability for underwater FLS image segmentation.

5. Ablation Study

5.1. Ablation Study of Hand-Crafted Feature Fusions

We conducted comprehensive ablation experiments on the hand-crafted features introduced in TriEncoderNet, comparing several different types, including Canny edge features [52], grayscale histogram features [53], and Local Binary Pattern (LBP) features [54]. As shown in

Table 8. Performance comparison of different hand-crafted features on Marine Debris Dataset and UATD datasets.

Hand-Crafted Feature	Marine Debris Dataset		UATD Datasets
	mIoU	mAP	mIoU	mAP
None	0.780	0.902	0.570	0.671
Canny	0.785	0.908	0.575	0.677
Grayscale histogram	0.776	0.890	0.557	0.657
LBP	0.778	0.895	0.564	0.668
Sobel	0.782	0.909	0.572	0.673
Unnormalized HOG	0.785	0.906	0.562	0.667
Learned Gradients	0.786	0.901	0.576	0.670
HOG (Ours)	0.793	0.916	0.582	0.687

, the performance of different hand-crafted feature fusions on the two test datasets is summarized, with bold font indicating the highest metric values.

From the experimental results, it is evident that HOG features outperform the other three feature types, leading to more accurate segmentation results and proving the effectiveness of HOG features. This superiority can be attributed to the fact that HOG features excel at capturing the distribution of local gradient orientations, effectively describing edge shapes and object structures in sonar images. By modeling the directional information of these edges, HOG features significantly enhance segmentation performance. Additionally, the results show that grayscale histogram features and LBP features negatively impact the segmentation performance. Compared to not incorporating any hand-crafted features, their inclusion results in a decrease in accuracy. We hypothesize that grayscale histogram features are overly sensitive to local brightness variations, potentially introducing noisy textures that hinder sonar image segmentation. On the other hand, LBP features, being global descriptors, fail to preserve local structural information, adversely affecting segmentation precision. While Canny edge features provide some benefits for sonar image segmentation, their susceptibility to noise interference limits their effectiveness. Consequently, the choice of appropriate hand-crafted features remains a research-worthy direction for further exploration.

Although gradient-based methods like Sobel operators and learned convolution layers offer marginal improvements over the baseline, they still fall short of the proposed HOG module. This performance gap is attributed to HOG’s histogram binning and block-wise normalization, which provide robust statistical representations against the speckle noise and signal attenuation inherent in FLS images—challenges that raw gradients and unnormalized features fail to address effectively. In contrast, grayscale histograms and LBP features degrade performance due to their sensitivity to local brightness variations and lack of structural preservation. Consequently, the normalized HOG descriptor proves to be the most effective geometric prior for enhancing segmentation in complex underwater environments.

Ablation Study of Proposed Blocks

To comprehensively validate the effectiveness of each module in TriEncoderNet and its contribution to overall model performance, we conducted extensive ablation experiments. The results, as presented in

Table 9. Ablation Experiment of the Proposed Modules on Marine Debris Dataset and UATD Dataset.

Model	Ours Modules				Marine Debris Dataset		UATD Dataset
	HET	CFT	HAG	HOG	mIoU	mAP	mIoU	mAP
CNN-baseline					0.699	0.816	0.461	0.523
Transformer-baseline					0.752	0.880	0.523	0.636
CT					0.764	0.886	0.555	0.650
CT + HET	✓				0.769	0.889	0.561	0.661
CT + CFT		✓			0.773	0.895	0.563	0.663
CT + CFT + HET	✓	✓			0.780	0.902	0.570	0.671
CTH				✓	0.775	0.891	0.559	0.657
CTH + HAG			✓	✓	0.783	0.896	0.566	0.669
CTH + CFT + HET	✓	✓		✓	0.787	0.910	0.575	0.676
CTH + CFT + HET + SE	✓	✓		✓	0.785	0.908	0.576	0.673
CTH + CFT + HET + CBAM	✓	✓		✓	0.788	0.904	0.576	0.669
CTH + CFT + HET + HAG (Ours)	✓	✓	✓	✓	0.793	0.916	0.582	0.687

, demonstrate the impact of each module on two test datasets, with bold font indicating the highest metric values. U-Net was employed as the CNN-baseline, while Vision Transformer was used as the Transformer-baseline. The CNN–Transformer (CT) model features a dual-encoder architecture that combines CNN and Vision Transformer encoders, using simple concatenation as the fusion mechanism. The CNN–Transformer–Hog (CTH) model extends this structure by adopting a tri-encoder architecture, incorporating CNN, Vision Transformer, and HogExtract as the Hog feature encoder. While its structure aligns with TriEncoderNet, all fusion modules in CTH are replaced with concatenation.

Effectiveness of the Tri-Encoder Architecture: The results for the CT model demonstrate the advantages of using dual encoders to extract both global and local information. Compared to the CNN-baseline and Trans-baseline, which extract only one type of feature, CT achieves substantial improvements in segmentation accuracy. Specifically, on the Marine Debris Dataset, CT achieves a 6.5% increase in mIoU and a 7% increase in mAP compared to the CNN-baseline. This demonstrates that the combination of global and local feature extraction significantly enhances model performance.

Building on the CT model, the introduction of Hog features in the CTH model further improves segmentation accuracy. By incorporating Hog features through a third encoder, CTH supplements the global and local features with texture and contour information. This addition leads to notable performance gains across various evaluation metrics. For example, on the Marine Debris Dataset, CTH achieves higher mIoU and mAP scores than CT, showcasing the effectiveness of the tri-encoder structure.

Effectiveness of HETransformer: Replacing the Vision Transformer in CT with the proposed HETransformer (CT + HET) results in further improvements in segmentation accuracy on both datasets. Specifically, CT + HET achieves a 0.5% and 0.3% increase in mIoU and mAP, respectively, on the Marine Debris Dataset, and a 0.6% and 1.1% increase in mIoU and mAP, respectively, on the UATD dataset. Furthermore, integrating HETransformer into CT + CFT results in even greater performance gains. For instance, on the Marine Debris Dataset, mIoU and mAP improve by 0.7% each, while on the UATD dataset, the improvements are 0.7% and 0.8%, respectively. These results validate the efficiency of HETransformer, which not only reduces computational complexity but also enhances the ability to extract global features, leading to improved segmentation accuracy.

Effectiveness of CFT: Replacing the simple concatenation operation in CT with the proposed CFT fusion module (CT + CFT) significantly improves performance, particularly on the UATD dataset, where mIoU and mAP both increase by over 1%. When applied to CT-HET, the CFT module further boosts segmentation accuracy. The improvement stems from the CFT module’s ability to model the correlations between global and local features across multiple scales, enabling deeper integration of semantically distinct feature information. This contrasts with simple concatenation, which cannot fully capture the complementary relationships between such features.

Effectiveness of HAG: Adding the HAG module to CTH and CTH + CFT + HET further enhances segmentation performance. Unlike the straightforward concatenation of Hog features in CTH, the HAG module effectively captures detailed textures and contour features brought by Hog, resulting in higher segmentation accuracy. For example, on the UATD dataset and Marine Debris Dataset, adding HAG to CTH + CFT + HET leads to measurable improvements in both mIoU and mAP, showcasing the module’s ability to exploit the benefits of Hog features for refining segmentation precision. Additionally, replacing HAG with standard SE or CBAM blocks leads to performance degradation. This confirms that HAG’s specific cross-modal design, utilizing HOG priors and Attention Dropout, is more effective than standard self-attention at suppressing underwater noise.

The ablation experiments highlight the efficacy of each module in TriEncoderNet. The dual-encoder architecture of CT effectively combines global and local features, while the tri-encoder structure of CTH leverages Hog features to further enhance segmentation accuracy. Additionally, the integration of HETransformer, CFT, and HAG modules contributes to substantial improvements in segmentation precision, validating the overall design of TriEncoderNet and its computational efficiency.

6. Conclusions

In this study, we proposed a novel deep learning model to address the challenges of FLS image segmentation. The integration of the CFT module facilitates a seamless fusion of local and global features at multiple scales, enhancing the model’s ability to adapt to complex underwater environments. The HAG further improves edge detection, ensuring the preservation of critical boundary information. The HETransformer optimizes computational efficiency without compromising global context modeling capabilities. Comparative experiments on publicly available datasets demonstrate the effectiveness of TriEncoderNet. Despite the superior performance, we acknowledge certain limitations in our evaluation. Currently, the experiments are conducted on two public datasets. While these datasets cover various underwater scenarios, they may not fully represent extreme conditions such as cross-domain adaptation or synthetic-to-real transfer. Future work will focus on extending the evaluation to a wider range of benchmarks and exploring the model’s generalization capabilities on unannotated real-world sonar data.