Underwater Image Feature Extraction and Classification Using a Multi-Scale Vision Transformer with Cross-Scale Biased Attention Fusion

Abstract

Underwater image analysis is affected by light scattering, wavelength-dependent attenuation, low contrast, and suspended particles, which reduce the discriminative visual features. Current multi-scale Vision Transformers are not well-suited to these degradations because they cannot effectively fuse features across scales to achieve accurate classification. Although Vision Transformers (ViTs) can model long-range interactions, single-scale patch tokenization remains suboptimal for underwater images, where both fine-grained textures and global structures are important. This study proposes a Multi-Scale Vision Transformer (MS-ViT) with Cross-Scale Biased Attention Fusion (CSBAF) for underwater image classification. Before transformer encoding, the CSBAF introduces a learnable source–target scale-pair bias and an input-dependent scale-reliability gate. This differs from standard multi-scale fusion and cross-attention methods, which mainly concatenate features or exchange information between scale branches. The proposed design enables the model to emphasize reliable scales while suppressing degraded-scale responses. A hybrid dataset containing 14,000 images from the Roboflow Aquarium and RUIE datasets across five classes was used for evaluation. MS-ViT with CSBAF achieved 88.9% accuracy and an 88.8% F1-score, outperforming the CNN baseline by 7.6% and state-of-the-art transformer models, including UWFormer, DP-ViT, and CvT, by 2.3–4.2%. Ablation studies showed a 1.7% accuracy improvement over simple multi-scale concatenation, whereas cross-dataset testing achieved 84.4% accuracy, indicating reasonable cross-dataset robustness. These results demonstrate that explicit scale–aware fusion can improve transformer-based underwater visual understanding.

1. Introduction

Underwater computer vision is a key enabling technology in various scientific and practical applications, including marine biodiversity monitoring, autonomous underwater vehicle navigation, ecological surveys, deep-sea reconnaissance, and coral reef health monitoring [1]. Underwater imaging conditions are less stable than terrestrial imaging conditions because water has different physical properties as an imaging medium [2]. The absorption and scattering of light are wavelength-dependent within water, resulting in haze, loss of contrast, color shift, and uneven illumination [3]. These physical effects directly impact computer vision systems and can degrade object detection, classification, and even scene understanding, particularly in underwater imaging environments [4].

Feature extraction in computer vision systems is degraded by absorption, scattering, and low underwater visibility [5]. All of these result in low signal-to-noise ratios, a lack of sharpness in the edges, and an uneven color distribution [6]. Underwater image analysis is further complicated by variations in water turbidity, natural light, and artificial illumination, which create underwater visual heterogeneity that even high-level deep learning models struggle to handle [7].

Recently, Convolutional Neural Networks (CNNs) have attracted much attention in image analysis and have demonstrated reliable performance in image classification, object detection, and segmentation of terrestrial images [8]. CNNs are trained to learn hierarchical local features using convolution filters that capture edges, textures, and shapes at increasing levels of abstraction [9]. However, their architecture suffers from the inability to model long-range spatial dependencies and global contextual relations [10]. This is significant in underwater applications where object boundaries are frequently obscured by haze, lighting conditions are often uneven, and color is distorted [11]. For instance, the scene context of coral formations and stones covered with algae may be similar at the local level but distinct at the higher level, which CNNs may be unable to capture [12].

Recently, however, Vision Transformers (ViTs), which learn long-range image information using self-attention, have been proposed as an alternative [13]. In contrast to CNNs, ViTs break down images into non-overlapping patches, transform the patches into a sequence of tokens, and encode them with transformer models that were initially created for natural language processing [14]. This enables each patch token to focus on all other tokens, which allows the model to learn local visual information and the overall image context [15]. Several studies have shown the good potential of ViTs in underwater vision systems [16]. Fathy et al. created ViT-ClarityNet, a hybrid CNN-transformer model that significantly enhances the clarity of underwater images [17]. Chen et al. proposed UWFormer, a multi-scale transformer designed for semi-supervised underwater image enhancement [18]. Li et al. presented a transformer-based architecture with attention-guided fusion for detecting underwater objects [19]. Sun et al. introduced a dual-path transformer (DP-ViT) designed to detect sonar targets [20]. Together, these studies show that transformer-based architectures hold substantial promise for advancing underwater computer vision compared to the performance of traditional CNNs [21].

Recent progress shows that underwater visual recognition is shifting away from traditional CNNs into transformer-based, benchmark-driven, and data-efficient paradigms [22]. Zhang et al. introduced CATNet, a cascaded attention transformer network to classify marine species images and presented the MSID dataset comprising 8605 underwater images across seven categories [23]. Yang et al. introduced FishAI, which is based on the Vision Transformer method to hierarchically classify marine fish at multiple taxonomic ranks, and SwinFishNet demonstrated strong transfer learning performance to classify fish species on three evaluation datasets [24]. More recently, AQUA20 established a challenging benchmark for underwater species classification under turbidity, low illumination, and occlusion, and indicated that even robust modern backbones had a large margin for improvement [25]. Recent developments have also focused on data-efficient learning; for example, FishAI 2.0 has also been used for multimodal few-shot classification. Rost demonstrated that semi-supervised learning on frozen DINOv3 embeddings can be competitive with only a small percentage of labeled data [26].

Despite recent advances, underwater image classification still faces major challenges owing to severe visual degradation [5]. Current underwater transformer models mainly focus on enhancement, detection, and general feature extraction [27]. Common multi-scale designs, such as Swin Transformer, hierarchical ViTs, and CrossViT-like fusion, usually rely on fixed window partitioning, hierarchical down sampling, feature concatenation, or branch-level cross-attention [28]. These methods improve multi-scale representation, but they do not explicitly estimate the reliability of each patch scale under changing underwater conditions, such as turbidity, low contrast, blur, and color attenuation [29]. Fine-scale patches can capture coral textures, algae boundaries, and small fish features; however, they may become unreliable when local details are degraded [30]. Coarse-scale patches preserve object-level structure and scene context, but they may miss small discriminative details [31]. Therefore, this study addresses adaptive scale fusion for underwater image classification using a learnable scale-pair bias and input-dependent scale-reliability gating [32].

Although CNNs and ViTs have improved visual recognition, their limitations remain important in underwater classification [33]. CNNs mainly capture local features and may miss long-range contextual cues, while standard ViTs use fixed patch tokenization that cannot flexibly balance fine textures and global structure [34]. Related underwater transformer models also have task-specific limitations: UWFormer mainly focuses on enhancement; DP-ViT is designed for sonar target detection; and CvT still relies on fixed tokenization [18]. Therefore, these methods do not explicitly perform reliability-aware scale selection under changing underwater degradation [35]. This gap motivates the proposed MS-ViT with CSBAF.

To address these limitations, this study proposes a Multi-Scale Vision Transformer (MS-ViT) with Cross-Scale Biased Attention Fusion (CSBAF) to classify underwater images. The proposed framework simultaneously processes patch embeddings of 8 × 8, 16 × 16, and 32 × 32 to capture both local texture and global structural context. In contrast to current multi-scale transformer architectures, which use either simple concatenation or independent scale processing, CSBAF introduces learnable scale-pair bias parameters and a scale-reliability gate to enable adaptive inter-scale routing according to the current input scene. A hybrid dataset was created from 14,000 images from the Roboflow and RUIE datasets, which included five classes: fish, coral, sand, rocks, and algae.

2. Methodology

The proposed Multi-Scale Vision Transformer (MS-ViT) and CSBAF for underwater image feature extraction and classification are presented in this section. The idea of multi-scale patch embeddings (8 × 8, 16 × 16, and 32 × 32 patches) was proposed to integrate fine-grain textures with global structural details, unlike traditional Vision Transformers that rely on a single fixed patch size. This multi-scale approach tackles the main problems in underwater imaging, including color distortion, noise, and low contrast. The methodology consists of a Cross-Scale Biased Attention Fusion (CSBAF) module, which combines multi-scale patch features with learnable scale-pair biases, and a reliability gate to boost the model’s robustness. It uses a hybrid dataset, which combines the Roboflow Aquarium and RUIE datasets for training and evaluation. The framework is supported by a specialized preprocessing pipeline that addresses the specifics of underwater images. A large-scale analysis is conducted that involves ablation studies, cross-dataset generalization, and understanding of the complexity of the model to prove the effectiveness and efficiency of the MS-ViT with CSBAF. All experiments were implemented using Python 3.10, PyTorch 2.0.1, CUDA 11.8, and an NVIDIA RTX 3090 GPU with 24 GB VRAM. The same software and hardware environment was used for model training and evaluation unless otherwise stated.

2.1. Overview of the Proposed Framework

The overall design included three steps: data preprocessing, feature extraction using an enhanced vision transformer, and classification. The proposed pipeline combines architectural improvements and data-driven approaches to enhance robustness against variations in underwater conditions. As shown in Figure 1, the architecture of the Multi-Scale Vision Transformer (MS-ViT) with CSBAF is designed for underwater image feature extraction and classification.

Figure 1 presents the proposed MS-ViT with CSBAF workflow, showing how preprocessing, multi-scale patch embedding, adaptive cross-scale fusion, transformer encoding, and classification are connected in one pipeline. It also highlights that the proposed model differs from standard ViT by using multiple patch scales instead of a single fixed patch size.

2.2. Multi-Scale Vision Transformer Architecture with Cross-Scale Biased Attention Fusion

2.2.1. Multi-Scale Patch Embedding

In standard Vision Transformers (ViTs), an input image I ∈ RH × W × CI is divided into fixed-size patches, which are then flattened and linearly projected into a shared embedding space. However, using a single fixed patch size may not sufficiently capture local textures and global structures in underwater images. This is especially difficult because of the degradation of the boundaries of objects by haze, blur, and attenuation due to the wavelength dependence of the color of objects.

To address this limitation, the proposed model uses multi-scale patch embedding, which splits the same input image into three patch sizes: 8 × 8, 16 × 16, and 32 × 32. These patch sizes were selected to represent fine, intermediate, and coarse spatial details while maintaining a manageable computational cost. Because all images were resized to 512 × 512 pixels, the 8 × 8 branch captured local texture details, such as coral patterns, algae boundaries, and small fish structures. The 16 × 16 branch captures both texture and object-level shape, whereas the 32 × 32 branch captures broader structural and scene-level contexts under turbidity, blur, low contrast, or color attenuation. Therefore, these patch sizes were not chosen as arbitrary hyperparameters; they were selected to cover important spatial cues for underwater classification. The ablation results also support their usefulness, as multi-scale patch embedding with simple concatenation improves accuracy compared with the single-scale ViT baseline. In the proposed framework, the minimal feature size refers to the smallest image region explicitly processed by the model. Because all input images were resized to 512 × 512 pixels, the finest branch used 8 × 8 pixel patches, producing 64 × 64 fine-scale tokens per image. Therefore, the smallest feature region modeled by the network was 8 × 8 pixels. This value is reported in pixels rather than physical units, such as millimeters or centimeters, because the datasets do not provide camera calibration, imaging distance, or physical-scale information.

Surface roughness is not measured directly, but is addressed at the image feature level. In underwater images, rough surfaces such as coral, rocks, sand and algae appear as local texture variations, irregular boundaries and non-uniform intensity patterns. The proposed MS-ViT with CSBAF captures these roughness-related cues through the 8 × 8 fine-scale patch branch, while the 16 × 16 and 32 × 32 branches provide broader contextual information to distinguish actual surface texture from noise, turbidity, and illumination variation. The CSBAF module then adaptively fuses these multi-scale features by emphasizing the most reliable scale under degraded underwater conditions. Since the datasets do not include depth maps, surface profiles, or physical roughness annotations, surface roughness is represented implicitly through learned visual texture features rather than quantified in metric units.

This allows the model to capture both fine-grained textures (using smaller patches) and larger structural contexts (using larger patches). Patches from each scale were flattened and projected into a unified embedding space using learnable linear projections.

The tokens produced from the three different patch scales are then represented in a unified token space before adaptive fusion. The multi-scale token embeddings are computed as:

$$\mathrm{Z}=[Z_8,Z_{16},Z_{32}]+E_{pos}$$

(1)

where $Z_8,Z_{16},Z_{32}$ represent the embeddings of the 8 × 8, 16 × 16, and 32 × 32 patch tokens, respectively, and $E_{pos}$ denotes the positional encoding, which helps the model capture spatial relationships between patches. This multi-scale representation enables the model to capture fine textures and object boundaries with small patches (8 × 8). The scene-level structure and long-range context are preserved using larger patches (32 × 32). Discrimination between visually similar underwater objects, such as fish and coral, can be improved by leveraging features from multiple scales. The adaptive fusion of these multi-scale embeddings is then carried out by the Cross-Scale Biased Attention Fusion (CSBAF) module, which allows the model to dynamically select the most relevant features from each scale, based on the task at hand.

2.2.2. Cross-Scale Biased Attention Fusion

The proposed CSBAF module differs from existing multi-scale ViT and CrossViT-style fusion methods by combining learnable scale-pair bias with an input-dependent scale-reliability gate. Conventional multi-scale ViTs usually concatenate tokens from different patch sizes or process scale branches independently, while CrossViT-style methods exchange information through cross-attention between branches. In contrast, CSBAF explicitly models the relative importance between source and target scales and dynamically adjusts each scale’s contribution according to the visual condition of the underwater image. This allows less reliable scales to be suppressed under turbidity, low contrast, or texture degradation. Therefore, the main novelty of CSBAF is adaptive reliability-aware scale routing, rather than multi-scale tokenization alone. The novelty of CSBAF lies not in using an attention bias alone, but in how the bias is combined with input-dependent scale reliability and applied inside cross-scale attention before transformer encoding. Standard cross-attention mainly transfers information between token branches through query–key interactions, but it does not explicitly determine whether the source scale is reliable under degraded underwater conditions. Conventional gated fusion usually assigns weights after feature extraction or after feature aggregation. In contrast, CSBAF introduces a source–target scale-pair bias and modulates this bias using an input-dependent reliability score before attention-based fusion. This allows the model to regulate scale contribution during cross-scale attention rather than only after fusion.

More specifically, CSBAF differs from standard cross-attention and generic gated fusion in three aspects. First, the learnable scale-pair bias models directional source–target scale relationships, such as fine-to-coarse and coarse-to-fine interactions. Second, the reliability gate makes the scale interaction input-dependent, allowing unstable scales to be suppressed when turbidity, blur, low contrast, or color distortion reduces their usefulness. Third, fusion is performed before the transformer encoder, so the encoder receives adaptive scale–aware representations instead of independently processed scale branches.

Existing multi-scale approaches often concatenate patch embeddings or exchange information between separate scale branches without explicitly modeling input-dependent scale reliability. To address this, we propose CSBAF, which adaptively fuses tokens from multiple scales using bias-aware attention and reliability-based scale routing.

Let $S=\{8,16,32\}$ denote the set of patch scales used in the proposed model. For each scale $s\in S$, the corresponding token embedding is denoted as $X_s\in {\mathbb{R}}^{N_s\times D}$, where $N_s$ is the number of tokens at scale $s$, and $D$ is the embedding dimension. In the cross-scale attention formulation, $s$ denotes the source scale, from which information is transferred, and $t$ denotes the target scale, into which information is fused. Therefore, the attention operation from source scale $s$ to target scale $t$ is denoted as $A_{t\leftarrow s}$.

For each source–target scale pair $\left(t,s\right)$, the query, key, and value projections are computed as:

$$Q_t=X_t{W}_t^Q,K_s=X_s{W}_s^K,V_s=X_s{W}_s^V$$

(2)

where $W_t^Q$, $W_s^K$, and $W_s^V$ are learnable projection matrices. This formulation explicitly separates the target scale used to generate queries from the source scale used to generate keys and values. Thus, information is transferred from the source scale $s$ to the target scale $t$, while the attention query is generated from the target branch.

To model the relative importance between different scale pairs, CSBAF introduces a learnable scale-pair bias $b_{t,s}$. In addition, an input-dependent scale-reliability gate $g_s$ is computed for each source scale:

$$g_s=\sigma \left(\mathrm{MLP}\left(\mathrm{GAP}\left(X_s\right)\right)\right)$$

(3)

where $\mathrm{GAP}(\cdot )$ denotes global average pooling over tokens, $\mathrm{MLP}(\cdot )$ is a lightweight multilayer perceptron, and $\sigma (\cdot )$is the sigmoid activation function. The value $g_s\in [0,1]$ represents the reliability of the source scale $s$. A higher value indicates that the scale contains more reliable information, whereas a lower value suppresses unstable scale responses caused by turbidity, blur, low contrast, or color distortion. The reliability gate is designed as a lightweight global average pooling and MLP module to estimate scale-level reliability rather than token-level saliency. Global pooling summarizes the overall response of each scale branch, while the MLP maps this descriptor to a reliability score. This design is suitable for underwater images because degradation such as blur, turbidity, low contrast, and color distortion often affects the usefulness of an entire scale. For example, fine-scale tokens may become unreliable when local textures are degraded, while coarse-scale tokens may still preserve global structure. Thus, scale-level reliability provides an efficient way to guide adaptive scale routing without adding excessive complexity. The ablation results further support this design, as full CSBAF outperforms static scale-pair bias alone, confirming the benefit of input-dependent reliability estimation. The scale-pair bias and source-scale reliability gate are combined as:

$${\tilde{B}}_{t,s}=\lambda b_{t,s}+g_s$$

(4)

where $b_{t,s}$is a learnable scalar bias for the source–target scale pair $\left(t,s\right)$, $g_s$is the input-dependent reliability score of the source scale, and $\lambda$is a fixed scaling coefficient used to control the magnitude of the learnable bias term. In this study, $\lambda$is kept fixed rather than learned, so that the adaptive behavior is controlled by the learnable scale-pair bias $b_{t,s}$and the input-dependent reliability gate $g_s$. In all experiments, $\lambda$ was fixed to 1.0 to keep the scale-pair bias on the same numerical range as the reliability gate and to avoid introducing an additional tunable parameter. The resulting ${\tilde{B}}_{t,s}$is broadcast to the attention-logit matrix for the corresponding source–target scale pair.

The cross-scale attention from source scale $s$to target scale $t$is then computed as:

$$A_{t\leftarrow s}=\mathrm{Softmax}\left({\displaystyle \frac{Q_t{K}_s^T}{\sqrt{d_k}}}+{\tilde{B}}_{t,s}\right)$$

(5)

where $d_k$is the key dimension. The fused representation for target scale $t$is obtained by aggregating information from all source scales:

$${\widehat{X}}_t={\sum }_{s\in S}{A}_{t\leftarrow s}{V}_s$$

(6)

The fused representations from all target scales are then concatenated and projected into a unified feature space:

$$X_{\mathrm{fuse}}=\left[{\widehat{X}}_8;{\widehat{X}}_{16};{\widehat{X}}_{32}\right]W^O$$

(7)

where $W^O$is a learnable output projection matrix. Finally, the class token and positional encoding are added before the fused tokens are passed into the transformer encoder:

$$X_{\mathrm{enc}}=\left[x_{\mathrm{cls}};X_{\mathrm{fuse}}\right]+E_{\mathrm{pos}}$$

(8)

The fusion is performed before the transformer encoder because early scale interaction allows the encoder to refine already fused scale-aware representations. If scale fusion is delayed until after independent encoder processing, each branch may suppress useful fine- or coarse-scale evidence before cross-scale interaction occurs. In contrast, pre-encoder fusion preserves scale-specific information before global contextual modeling. This design choice is further validated by the fusion-placement comparison reported in Section 3.10, where pre-encoder fusion achieves higher accuracy than within-encoder and post-encoder fusion.

The design rationale of CSBAF can be summarized in three points:

• Pairwise scale-bias parameters learn preferred interactions between fine and coarse token branches.
• The reliability gate suppresses unstable scales affected by turbidity, blur, low contrast, or color distortion.
• Pre-encoder fusion preserves scale-specific evidence before global transformer encoding.

2.2.3. Transformer Encoder with Multi-Head Self-Attention

The fused tokens produced by the CSBAF module are passed through a stack of standard transformer encoder layers with Multi-Head Self-Attention (MHSA). Each encoder layer follows the conventional Vision Transformer design, including MHSA, feed-forward layers, residual connections, and layer normalization. Since these operations are well established in transformer-based vision models, their full mathematical derivation is not repeated here.

In the proposed framework, MHSA is used to refine the CSBAF-enhanced multi-scale token representation by modeling contextual relationships across the entire underwater image. Unlike a standard ViT that receives tokens from a single patch scale, the proposed encoder operates on adaptively fused tokens generated from the 8 × 8, 16 × 16, and 32 × 32 patch branches. These tokens combine fine-grained texture information with broader structural context.

This is particularly useful for underwater image classification because visually similar categories, such as coral, rocks, sand, and algae, may share overlapping local textures under turbidity, low contrast, and color distortion. By applying MHSA after CSBAF, the model can capture long-range dependencies while preserving reliability-weighted multi-scale information.

Therefore, the main contribution of this study is not the redesign of the standard transformer encoder, but the use of CSBAF to provide adaptive scale-aware token representations before global contextual modeling.

2.2.4. Feature Aggregation and Classification

After CSBAF fusion, a learnable class token is appended to the fused token sequence and passed through the transformer encoder. The final output corresponding to the class token is used as the global image representation. This representation is then fed into a fully connected classification layer followed by softmax activation to generate class probabilities for the five underwater categories: fish, coral, sand, rocks, and algae. Since the classification head follows the standard Vision Transformer classification design, detailed softmax derivations are omitted for brevity. The classifier is intentionally kept simple so that the performance improvement can be mainly attributed to the proposed multi-scale representation and CSBAF module rather than to a complex classification head.

2.3. Hybrid Dataset Construction and Distribution

A hybrid dataset with two datasets, the Roboflow Aquarium dataset and the Real-world Underwater Image Enhancement (RUIE) dataset, was generated to enhance the generalization of the underwater environment across a broad spectrum. Roboflow has controlled and well-annotated images, and RUIE adds degradation to images in real-world scenarios, such as turbidity, noise, and change in illumination.

The integration process involved:

• the selection of common object classes (fish, coral, sand, rocks, algae)
• annotation standardization
• the removal of duplicate and low-quality samples.

To ensure consistency between the Roboflow Aquarium and RUIE datasets, all images were classified into five image-level classes: fish, coral, sand, rocks, and algae. Class mapping was performed by selecting only the images that clearly met one of these five classes, even though the dominant visual content was present. Images that were labeled with unclear terms, had multiple dominant object categories, were very blurry, or did not have clear visual content were removed during the dataset, cleaning phase. RUIE is primarily an image enhancement dataset and classification labels were applied to the images only when the dominant class was clearly visible. Duplicate and near-duplicate images were removed before dataset splitting to avoid train/test leakage. The dataset was further split into training, validation, and test sets using a stratified split to ensure class distribution across the training, validation, and test data. Original data sources and licensing were also examined, and only images suitable for academic research were used.

The final dataset contains approximately 14,000 images across five classes. The five target classes were selected because they were common, visually identifiable, and consistently represented across the combined dataset sources after label harmonization. These classes also cover both object-level underwater categories, such as fish and coral, and habitat-level categories, such as sand, rocks, and algae, making the classification task more representative of real underwater scenes. The sample distribution of the hybrid dataset is summarized in

Table 1. Distribution of samples in the hybrid dataset.

Class	Description	Total Images	Training	Validation	Testing
Fish	Various fish species	5000	3500	750	750
Coral	Coral formations	3000	2100	450	450
Sand	Seabed regions	2500	1750	375	375
Rocks	Rocky structures	2000	1400	300	300
Algae	Algae formations	1500	1050	225	225
Total		14,000	9800	2100	2100

.

The class distribution remains moderately imbalanced, with fish being the largest class and algae being the smallest. This imbalance reflects the available image distribution after label cleaning and removal of ambiguous samples. To reduce bias toward the majority class, the training stage used class-weighted cross-entropy loss and class-preserving data augmentation. Model performance was also reported using accuracy, precision, recall, F1-score, and class-wise analysis rather than accuracy alone.

Figure 2 is a pie chart showing the class distribution in the dataset. The biggest fraction is fish (35.7%), after which come coral (21.4%), sand (17.9%), rocks (14.3%), and algae (10.7%). The imbalance regarding classes is shown in the chart, where fish will be dominant, and the algae will be underrepresented, which could affect the model performance unless addressed.

Figure 3 shows that the samples of the dataset include fish, coral, sand, rocks, and algae. The images indicate the visual heterogeneity, different levels of illumination, and noise of the underwater conditions, and the difficulties of extracting features.

The hybrid dataset used in this study consists of images from multiple sources and is classified into five categories of underwater objects. The dataset is divided into training and testing subsets, and the sample distribution of images across five classes is summarized to indicate class balance and sample availability. Preprocessing was applied to standardize the images for model training; details of the preprocessing pipeline are described in Section 2.4.

Figure 4 shows the class-wise distribution of images after dataset splitting. The distribution follows the same overall class trend as the full dataset, with fish having the largest number of samples and algae having the smallest. The data set is still moderately imbalanced, but the proportions of the classes in both the training and testing sets are maintained via stratified split. This allows for some representation within each class at training and evaluation time.

2.4. Preprocessing Pipeline

Underwater images often suffer from low contrast, color attenuation, uneven illumination, blur, noise, and turbidity. Therefore, a preprocessing pipeline was applied before model training and evaluation to standardize the input images and reduce low-level visual variability. The goal of this pipeline was to improve input consistency while preserving the original semantic content of each image.

First, all images were resized to 512 × 512 for the multi-scale patch embedding module. This resolution enables producing tokens of 8 × 8, 16 × 16 and 32 × 32 in a resolution-independent manner, without discarding local texture and global context. Second, the pixel value was normalized in the range [0, 1] in order to achieve stable training and to decrease the numerical difference between images.

Data augmentation was applied to the training set to improve robustness in various underwater imaging conditions. These operations comprised random rotations, brightness and color jittering. Random rotation resulted in a greater diversity of viewpoints, while brightness adjustment, reduced sensitivity to lighting changes, and color jittering made the model less sensitive to underwater color attenuation. These operations were used only during training. Validation and test images were processed with deterministic resizing and normalization only, and random augmentation therefore did not affect the evaluation results.

The proposed procedure does not involve signal averaging or repeated acquisition of multiple image sets. During both training and inference, each underwater image is processed as a single input sample. Noise and degradation are handled through resizing, pixel normalization, brightness adjustment, color jittering, and the learned multi-scale feature representation of MS-ViT with CSBAF. Therefore, this single-image classification framework achieves robustness without multi-frame averaging or external enhancement techniques.

The preprocessing pipeline can be summarized as follows: resize all images to 512 × 512 pixels, normalize pixel values to the [0, 1] range, apply random rotation, brightness adjustment, and color jittering only during training, and use deterministic resizing and normalization for validation and testing.

Figure 5 presents representative raw and preprocessed underwater images. The examples show variations in lighting, visibility, contrast, and color distribution, illustrating how preprocessing standardizes the inputs while preserving the main underwater content needed for classification.

2.5. Model Training Strategy

The proposed model was trained using supervised learning with five categorical labels. The Adam optimizer was used because of its adaptive learning ability.

The training process used a step-decay learning rate scheduler to decrease the learning rate with time. Overfitting was avoided due to the early stopping.

The model hyper parameters that are used for the model training are presented in

Table 2. Training parameters used in the experiments.

Parameter	Value
Optimizer	Adam
Learning Rate	0.0001
Batch Size	16
Epochs	50
Loss Function	Weighted Cross-Entropy
Learning Schedule	Step Decay

, such as the optimizer type, learning rate, batch size, epoch, loss function, and learning rate schedule. These parameters were chosen to facilitate the convergence and enhance the model results. For efficient weight updates, the Adam optimizer was employed and the weight-decay learning rate schedule was used to control the learning process while training the network. An early stopping technique was also used to reduce overfitting.

A class-weighted cross-entropy loss was used during training to address the moderate class imbalance in the hybrid dataset. Class weights were calculated inversely from the number of training samples in each class, assigning a higher penalty to minority classes such as algae and rocks, and a lower penalty to the majority class, fish. This reduced the model’s bias toward the dominant class during optimization. In addition, data augmentation was applied only to the training set using random rotation, brightness adjustment, and color jittering, while the validation and test distributions were kept unchanged. The training, validation, and test sets were also stratified to preserve class distribution across all subsets.

Loss function: Weighted cross-entropy loss was used to reduce the effect of class imbalance during training. Unlike standard cross-entropy, the weighted form assigns a larger penalty to minority classes and a smaller penalty to majority classes. This encourages the model to learn discriminative features for underrepresented categories such as algae and rocks, rather than optimizing mainly for the dominant fish class.

$$L=-{\displaystyle {\sum }_{c=1}^c}{w_{c}y}_c\log \left({\widehat{y}}_c\right)$$

(9)

where $C$ is the number of classes, $y_c$ is the ground-truth label, ${\widehat{y}}_c$ is the predicted probability for class$c$, and $w_c$is the class weight computed from the training-set class frequency.

Optimizer Adam: The Adam optimizer was used for weight updates, combining momentum and adaptive learning rates for efficient training shown in equation

$$\left\{\begin{array}{c}{m}_t={\beta }_1{m}_{t-1}+\left(1-{\beta }_1\right)g_t\\ {v_t=\beta }_2{v}_{t-1}+\left(1-{\beta }_2\right)g^{2}t\\ {\theta }_{t+1}={\theta }_t-{\displaystyle \frac{\alpha }{\sqrt{v_t}+ϵ}}{m}_t\end{array}\right.$$

(10)

where $m_t$ exponentially decaying average of past gradients, $v_t$ exponentially decaying average of squared gradients, α learning rate (0.0001), ${\beta }_1,{\beta }_2$ momentum parameters (typically 0.9 and 0.999), ϵ small constant to prevent division by 0.

Learning rate scheduler step decay: The learning rate scheduler learns to decrease the learning rate as training advances, which assists the model with achieving good convergence as shown in equation

$${\alpha }_t={\alpha }_0\cdot {\gamma }^{\left({\displaystyle \frac{t}{s}}\right)}$$

(11)

where $\alpha$ is the learning rate at epoch$t$, ${\alpha }_0$ is the initial learning rate (0.0001), γ is the decay rate (typically 0.1), and $s$ is number of epochs before each decay step. To reduce overfitting risks, the training applied early stopping and model check Pointing, ending the training only when validation loss did not decrease within a fixed number of epochs.

3. Results and Discussion

This section presents a detailed discussion of the proposed multi-scale Vision Transformer (MS-ViT) with CSBAF system for underwater feature extraction and classification. Nevertheless, in contrast to the original manual, the evaluation has been extended to further include comparisons to the state of the art (SOTA) transformer-based models, ablation studies, evaluation of cross-dataset generalization, attention map visualization, and complexity analysis. These additions contribute to further vetting of the proposed approach and counteract the limitations noted in the review.

3.1. Evaluation Metrics

The proposed model was evaluated according to classical metrics of performance or classification, accuracy, precision, recall, and F1-score. These measures provide a reasonable evaluation of classification accuracy, particularly when the dataset is unbalanced in terms of classes. Furthermore, the silhouette score was employed to evaluate the quality of feature clustering in the learned embedding space, which consisted of data about the separability of the various classes of underwater objects.

Accuracy: Accuracy is the ratio of the number of correct classifications to the instances that were analyzed. It gives a rough understanding of the effectiveness of the model in the classification of the underwater features of fish, coral, sand, rocks, and algae below, in the equation.

$$\mathrm{A}\mathrm{c}\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{y}={\displaystyle \frac{\mathrm{T}\mathrm{P}+\mathrm{T}\mathrm{N}}{TP+TN+FP+FN}}$$

(12)

• TP (true Positives): Correctly predicted positive instances
• TN (true Negatives): Correctly predicted negative instances
• FP (false Positives): Incorrectly predicted positive instances
• FN (false Negatives): Incorrectly predicted negative instances.

Precision: The true positives divided by the predicted positive instances are equal to precision. It demonstrates how effective the model is in not only recognizing the targeted class but also reducing the misclassifications of other classes in equation

$$Precision={\displaystyle \frac{TP}{TP+FP}}$$

(13)

The high accuracy index suggests that the model will be able to regulate the number of false positives, which is a significant factor in underwater work since mistakes in the background recognition could cause serious errors.

Recall: Recall or true positive rate tells the percentage of actual positive instances that are correctly identified out of all positive instances. It illustrates the model’s ability to identify all important objects in the aquatic environment, as in the equation.

$$Recall={\displaystyle \frac{TP}{TP+FN}}$$

(14)

A high recall score indicates that the model successfully recognizes the majority of actual underwater features, thus reducing the risk of missing important ones.

The F1 score integrates precision and recall with the harmonic mean, so that both types of errors are given equal weight; since the underwater object categories have different precision and recall characteristics, the F1 score is instrumental below the equation

$$F1Score=2\times {\displaystyle \frac{Precision\times Recall}{Precision+Recall}}$$

(15)

F1 Score: The F1 score is critical for the overall reliability of the model and the verification that it obtains a well-balanced balance of precision and recall for a variety of object classes.

Silhouette Score: The silhouette score measures the quality of the feature-based clustering, quantifying the degree to which the classes are separated based on the representation of the model on the features. The score is between −1 and 1, and the higher the better, which means that clusters are more distinct, and classes are less mixed, as shown in the equation

$$SilhouetteScore={\displaystyle \frac{b-a}{\mathrm{m}\mathrm{a}\mathrm{x}(\mathrm{a},\mathrm{b})}}$$

(16)

where $a$ represents the average intra-cluster distance (cohesion), while $b$ represents the average nearest-cluster distance (separation).

The importance of the silhouette score is that it is used in underwater feature extraction because it quantifies how well the model can distinguish visually similar objects, such as corals and rocks, which have overlapping coloration and patterns in water.

In this study, the silhouette score was used only as an auxiliary feature-separability metric rather than a primary classification metric. The embeddings used for silhouette-score computation were extracted from the final class-token representation of each model, immediately before the fully connected classification layer. These embeddings represent the learned feature space after feature extraction and contextual modeling. The silhouette score was computed using the ground-truth class labels to evaluate whether samples from the same underwater class formed compact clusters and whether different classes were well separated. As a result, the silhouette score provides a clue to the discriminative power of the learned representations, particularly among visually similar classes, like coral, rocks, sand, and algae. The accuracy, precision, recall, and F1-score were, however, used as the main metrics for assessing classification performance.

3.2. Model Performance Across Epochs

This section presents the accuracy curves during training, validation, and testing of all models (CNN, CvT, DP-ViT, UWFormer, and MS-ViT + CSBAF) over 50 epochs. The proposed MS-ViT with CSBAF achieved the highest accuracy during training, indicating its learning ability. Figure 6 shows the accuracy trend for all models at the three phases.

The accuracy curves of the CNN, CvT, DP-ViT, UWFormer and the proposed MS-ViT over 50 epochs for training, validation, and testing are illustrated in Figure 6. The proposed model has the best overall accuracy among the baseline models. The curves were employed to compare learning behavior and generalization between training, validation, and testing subsets in identical experimental setup.

3.3. Evaluation Metric Summary

Five metrics, namely, accuracy, precision, recall, F1-score, and silhouette score, were used to evaluate the proposed model. Accuracy is the overall correct level of classification while precision is the model’s ability to not make false positive predictions. Recall is used to measure the performance of the model in predicting real class samples and F1-score reflects a balance between precision and recall. An auxiliary feature separability metric, called the silhouette score, was employed to examine the separability of the learned embeddings for the underwater classes that are visually similar. All the results reported here were obtained under a mixed underwater data set, with about 14,000 images belonging to five classes and with the same training/validation/test split, preprocessing pipeline, and evaluation protocol. These metrics together give an overall evaluation of the classification performance and representation quality, as seen in Figure 7.

The hybrid underwater test set includes 2100 images across five classes: fish, coral, sand, rocks, and algae. Accuracy, precision, recall, and F1-score are reported as percentages, while the silhouette score represents feature separability and is scaled for radar-chart visualization.

3.4. Quantitative Comparison with State-of-the-Art Models

To evaluate the effectiveness of the proposed framework, we compared the proposed MS-ViT with CSBAF against a traditional CNN baseline and transformer-based models, including UWFormer, DP-ViT, and Convolutional Vision Transformer (CvT). To ensure a fair and reproducible comparison, all baseline models were implemented or adapted and evaluated under the same experimental protocol as the proposed MS-ViT with CSBAF. Specifically, CNN, CvT, DP-ViT, UWFormer, and the proposed model were trained and tested using the same hybrid dataset, identical train/validation/test splits, the same preprocessing pipeline, the same training strategy, and the same evaluation metrics. For transformer-based models originally designed for related underwater vision tasks, the backbone was adapted to the five-class underwater image classification task by replacing the original task-specific output layer with the same classification head used for all models. No literature-reported results were used in the final quantitative comparison. This protocol ensures that the reported performance differences are mainly attributable to architectural differences rather than differences in datasets, preprocessing, training settings, or evaluation conditions. In addition, the same input resolution of 512 × 512 pixels, batch size of 16, Adam optimizer, weighted cross-entropy loss, step-decay learning rate schedule, and 50 training epochs with early stopping were used for all compared models. UWFormer was included as a related underwater transformer baseline because it uses multi-scale transformer-based feature modeling, although it was originally designed for enhancement. For fair adaptation to classification, its task-specific enhancement head was removed and replaced with the same five-class classification head used for the other models.

The reliability of reported performance differences was assessed by training and testing each model over five independent runs with different random seeds, same dataset split, preprocessing pipeline, and training protocol. The data are presented as mean ± standard deviation. The proposed MS-ViT with CSBAF was then compared with each baseline model, using paired t-tests to determine if the accuracy differences were statistically significant. This statistical validation is significant as the gains over strong transformer baselines are moderate. The comparative performance results are summarized in

Table 3. Performance comparison with CNN and transformer-based SOTA models over five independent runs.

Model	Accuracy (%)	Precision (%)	Recall (%)	F1-Score (%)	Silhouette Score
CNN	81.3% ± 0.8	81.2% ± 0.7	80.8% ± 0.8	80.8% ± 0.7	0.65 ± 0.02
CvT	84.7% ± 0.6	84.2% ± 0.6	84.9% ± 0.6	84.4% ± 0.6	0.72 ± 0.02
DP-ViT	85.4% ± 0.7	85.1% ± 0.6	85.7% ± 0.7	85.3% ± 0.6	0.74 ± 0.01
UWFormer	86.6% ± 0.5	86.2% ± 0.5	86.9% ± 0.5	86.5% ± 0.5	0.76 ± 0.01
Proposed MS-ViT with CSBAF	88.9% ± 0.4	88.4% ± 0.4	89.6% ± 0.4	88.8% ± 0.4	0.80 ± 0.01

.

The statistical significance results are summarized in

Table 4. Statistical significance test between the proposed MS-ViT with CSBAF and baseline models.

Comparison	Mean Accuracy Difference (%)	p-Value	Interpretation
Proposed vs. CNN	+7.6%	<0.001	Statistically significant
Proposed vs. CvT	+4.2%	<0.01	Statistically significant
Proposed vs. DP-ViT	+3.5%	<0.01	Statistically significant
Proposed vs. UWFormer	+2.3%	<0.05	Statistically significant

.

The statistical analysis shows that the proposed MS-ViT with CSBAF consistently improves performance over all compared baselines. The largest improvement is observed over the CNN baseline, while the gains over transformer-based models are smaller but still statistically supported. Therefore, the results should be interpreted as consistent and reliable improvements rather than large absolute gains. The manuscript has been revised to avoid overstating the performance margins.

The comparison of the performance of various models for underwater image classification is illustrated in Figure 8a, which presents the accuracy, precision, and recall of various models such as CNN, CvT, DP-ViT, UWFormer, and MS-ViT equipped with CSBAF, with MS-ViT outperforming the others. Figure 8b compares the feature separability (silhouette score), where MS-ViT with CSBAF achieves the highest score, indicating better class distinction.

3.5. Ablation Study

An ablation study was conducted to determine the contribution of each major component of the proposed framework. In addition to preprocessing and multi-scale tokenization, the study explicitly isolates the effect of the proposed Cross-Scale Biased Attention Fusion (CSBAF) module in a staged manner. Four experimental configurations were considered: baseline ViT without preprocessing, ViT with preprocessing only, ViT with multi-scale patch embedding and simple token concatenation, and the proposed MS-ViT with full CSBAF. This design makes it possible to separate the gain obtained from simple multi-scale tokenization from the additional gain achieved by adaptive cross-scale biased fusion. For a finer methodological diagnosis, additional fusion-strategy and fusion-placement experiments are reported in Section 3.10. These experiments compare simple concatenation, CrossViT-style fusion, static scale-pair bias only, the full CSBAF module, and three fusion-placement variants: pre-encoder, within-encoder, and post-encoder fusion. This extended decomposition is important because it distinguishes ordinary multi-scale interaction from the proposed bias-aware routing mechanism, making the architectural novelty of the full model easier to verify experimentally.

Table 5. Ablation study results.

Configuration	Accuracy (%)	F1-Score (%)	Silhouette Score
Baseline ViT single patch 16 × 16, no preprocessing	83.4%	83.1%	0.72
ViT + Preprocessing	85.6%	85.3%	0.75
ViT + Multi-scale Patch Embedding simple concatenation	87.2%	87.0%	0.78
Proposed MS-ViT with CSBAF	88.9%	88.8%	0.80

summarizes the contribution of the different components of the proposed framework. The comparison among the baseline Vision Transformer, the model with preprocessing, the model with multi-scale patch embedding and simple concatenation, and the full proposed model shows that each component contributes positively to the final result.

The baseline ViT achieves 83.4% accuracy under underwater distortion. Preprocessing improves this to 85.6% by reducing low-level variation. Adding multi-scale patch embedding with simple concatenation yields 87.2% accuracy, confirming the value of multi-scale tokenization. Critically, the MS-ViT with CSBAF achieves 88.9% accuracy, which is a 1.7% absolute gain over simple concatenation. This demonstrates that the proposed adaptive fusion mechanism, rather than merely multi-scale tokenization, constitutes the key architectural innovation.

3.6. Class-Wise Performance Analysis

In order to understand more about the behavior of the model, performance by classes was checked. These results indicate that the highest accuracy in the fish category is obtained by the model using the properties of shape and texture, which leads to higher accuracy. Classes such as sand and algae show lower performance because they have high intra-class variability and strong inter-class visual similarity with rocks, coral, and seabed regions.

Figure 9 reveals the performance of the proposed MS-ViT with CSBAF model on each of the classes based on accuracy, precision, recall, F1-score, and silhouette score. The figure demonstrates a few observations to simulate the performance of the models with various types of underwater objects. It may be observed that the model performs better at the classes of fish and coral that possess distinguishing structural characteristics, whereas the model performs relatively poorly at the visually close classes, such as sand and algae. The effectiveness of feature clustering is also proved by the inclusion of the silhouette score, indicating that the model offered in this paper is learning well separated and discriminative representations of the various classes.

Figure 9 shows that fish and coral achieve higher performance due to stronger structural cues and larger representation in the dataset, whereas algae, sand, and rocks remain more challenging because of visual similarity and class imbalance under underwater degradation.

The class-wise performance comparison is reported in

Table 6. Class-wise performance comparison of CNN, CvT, DP-ViT, UWFormer, and the proposed MS-ViT for underwater image classification.

Model	Class	Accuracy (%)	Precision (%)	Recall (%)	F1-Score (%)
CNN	Fish	86.8%	85.9%	86.5%	86.2%
	Coral	82.3%	81.8%	81.2%	81.5%
	Sand	79.0%	78.6%	78.0%	78.3%
	Rocks	80.1%	79.6%	79.2%	79.4%
	Algae	77.0%	76.5%	75.8%	76.2%
CvT	Fish	90.2%	88.8%	90.0%	89.5%
	Coral	86.0%	84.9%	85.5%	85.2%
	Sand	82.7%	81.5%	82.8%	82.1%
	Rocks	84.0%	82.8%	84.0%	83.4%
	Algae	80.6%	79.4%	80.5%	80.0%
DP-ViT	Fish	90.8%	89.4%	90.8%	90.1%
	Coral	86.9%	85.7%	86.3%	86.0%
	Sand	83.9%	82.6%	83.7%	83.2%
	Rocks	85.2%	83.9%	85.0%	84.5%
	Algae	81.8%	80.5%	81.6%	81.2%
UWFormer	Fish	92.1%	90.5%	92.2%	91.3%
	Coral	88.2%	86.8%	87.6%	87.4%
	Sand	85.1%	83.7%	84.9%	84.5%
	Rocks	86.5%	85.1%	86.2%	85.8%
	Algae	83.0%	81.6%	82.8%	82.5%
MS-ViT (Proposed) with CSBAF	Fish	93.0%	91.8%	92.5%	92.1%
	Coral	90.2%	89.2%	89.9%	89.5%
	Sand	87.3%	86.3%	87.0%	86.6%
	Rocks	88.5%	87.4%	88.2%	87.8%
	Algae	84.7%	83.8%	84.5%	84.1%

.

The class-wise silhouette score comparison is reported in

Table 7. Class-wise silhouette score comparison of CNN, CvT, DP-ViT, UWFormer, and the proposed MS-ViT with CSBAF for underwater image classification.

Model	Fish	Coral	Sand	Rocks	Algae
CNN	0.72	0.65	0.60	0.62	0.56
CvT	0.79	0.73	0.68	0.70	0.64
DP-ViT	0.81	0.75	0.70	0.72	0.66
UWFormer	0.83	0.77	0.72	0.74	0.68
MS-ViT(Proposed) with CSBAF	0.86	0.82	0.78	0.79	0.75

.

The class-wise results indicate that performance differences are affected by both dataset imbalance and visual similarity among underwater categories. Fish achieved the highest performance because it has the largest number of samples in the dataset and usually contains clearer structural cues, such as body shape, fins, and boundaries. Coral also performed well because its texture and structural patterns are more distinguishable than those of background-like classes.

In contrast, algae showed the lowest performance because it has the smallest number of samples and strong visual overlap with rocks, coral, and sand under turbidity, low contrast, and color attenuation. Sand and rocks also showed relatively lower scores because their textures and boundaries are often similar, especially under uneven illumination. Although class-weighted cross-entropy loss and class-preserving augmentation were used to reduce imbalance effects, the remaining performance gap suggests that minority and visually overlapping classes still require more balanced and diverse training samples.

3.7. Cross-Dataset Generalization

Cross-dataset evaluation was conducted to test the strength of the proposed model. The model was trained using the Roboflow data set and tested using the RUIE data set, and vice versa. Such an assessment configuration makes it possible to assess the generalization of the model to other undersea conditions and domain switches, which is essential for possible real-world applications. Cross-dataset experiments were also used to measure robustness, where the model is trained on a specific dataset and tested on a separate dataset. Such an arrangement is similar to those encountered in the field, where changes in domains are the norm.

Table 8. Cross-dataset generalization performance.

Training Dataset	Testing Dataset	Accuracy (%)	F1-Score (%)	Silhouette Score
Roboflow	RUIE	84.4%	84.1%	0.74
RUIE	Roboflow	85.2%	84.9%	0.75

shows the generalizing power of the suggested model with the assistance of various datasets. The model is trained on one dataset (Roboflow/RUIE) and tested on the other, which is an approximation to real-world domain shifts. The findings reveal that the model performs well on various datasets, which demonstrates that the system is strong to changes related to the underwater environment. This discussion highlights that the composite dataset approach and cross-domain preprocessing pathway are efficient in improving the cross-domain flexibility.

The cross-dataset results show reasonable robustness under domain shift, but they should be interpreted cautiously. Accuracy drops from the in-domain result of 88.9% to 84.4% and 85.2%, showing that dataset shift still affects performance. This drop may be due to differences in image quality, illumination, turbidity, class distribution, and labeling consistency between Roboflow Aquarium and RUIE. Although the five target classes were harmonized, minor annotation and visual differences may still influence the results. Therefore, the experiment supports reasonable cross-dataset robustness, but not complete domain invariance.

3.8. Attention Map Visualization

Attention-map visualization was used to analyze which image regions influenced model predictions. This analysis helps examine whether each model focuses on relevant underwater regions or responds to background degradation. Figure 10 presents the comparative attention-map visualization used to examine the image regions influencing model predictions.

Attention responses from CNN, CvT, DP-ViT, UWFormer, and the proposed MS-ViT with CSBAF were compared using the same test images, 512 × 512 input resolution, preprocessing pipeline, and inference protocol. The proposed model shows more focused attention on semantically meaningful regions, including object boundaries, fish bodies, and coral structures, while reducing attention to background noise, haze, and turbidity. Compared with the baseline models, the proposed method produces more concentrated attention on relevant underwater regions and less response to background degradation. This supports the role of CSBAF in improving feature focus and interpretability beyond the general attention behavior of Transformer models.

3.9. Model Complexity Analysis

Model complexity was evaluated using parameter count, floating-point operations (FLOPs), and inference time. This analysis was included to clarify the trade-off between classification performance and computational cost for possible underwater deployment. The proposed MS-ViT with CSBAF has higher parameters and FLOPs than the baseline models. Therefore, it should not be considered a lightweight model in terms of theoretical computational complexity. Its lower measured inference time only indicates runtime efficiency under the same experimental environment, not lower overall complexity. All complexity and inference-time measurements were obtained using the same input resolution, batch size, preprocessing pipeline, and runtime environment for each model. Hardware and software configuration: Inference-time measurements were performed on an NVIDIA RTX 3090 GPU with 24 GB VRAM, CUDA 11.8, and PyTorch 2.0.1, using the same input resolution, batch size, preprocessing pipeline, and inference mode for all models.

The computational time budget can be divided into offline training and online inference. Offline training includes preprocessing, multi-scale patch generation, and supervised model training for a maximum of 50 epochs with early stopping. Since this stage is completed before deployment, it does not affect real-time inference. During deployment, only the forward pass is required. As shown in

Table 9. Model complexity comparison.

Model	Parameters (M)	FLOPs (G)	Inference Time (ms)
CNN	12	4.1	36
CvT	20.1	6.5	32
DP-ViT	22.8	7.2	35
UWFormer	25.4	8.1	38
Proposed MS-ViT Multi-scale with CSBAF	46.8	10.2	30.1

, the proposed MS-ViT with CSBAF requires approximately 30.1 ms per image under the same experimental environment. However, these results should be interpreted as an accuracy–inference trade-off rather than evidence of a lightweight model. The proposed model has 46.8 M parameters and 10.2 GFLOPs, which are higher than all compared baselines. Its lower measured inference time indicates competitive runtime performance only under the tested hardware/software environment, but it does not remove the higher parameter and FLOP cost. Further optimization, such as pruning, quantization, or knowledge distillation, may be required for highly resource-constrained underwater deployment.

CNN, CvT, DP-ViT, UWFormer, and the proposed MS-ViT with CSBAF were compared on the same 2100-image test set using an identical dataset split, preprocessing, training, and classification protocol. Rows show true labels, while columns show predicted labels for the five underwater classes. The confusion matrix comparison shows that the proposed MS-ViT with CSBAF reduces inter-class confusion across the five categories and remains competitive with the reference baselines. The result supports the value of multi-scale representation and adaptive fusion for improving feature discrimination in underwater scenes. Figure 11 presents the confusion matrix comparison for all evaluated models on the same test set.

3.10. Comparison with Alternative Multi-Scale Fusion Strategies

To further strengthen the ablation analysis, the CSBAF module was decomposed into its main components and compared with alternative multi-scale fusion strategies under the same experimental protocol. The goal of this analysis was to isolate the effect of multi-scale token concatenation, cross-scale interaction, static scale-pair bias, and input-dependent reliability gating. Since the accuracy gain of the full CSBAF module over simple multi-scale concatenation is moderate, each configuration was evaluated over repeated runs and reported using mean ± standard deviation. This allows the contribution of the bias term and the reliability gate to be examined more clearly.

Table 10. Component-wise ablation of multi-scale fusion strategies.

Fusion Strategy	Description	Accuracy (%)	F1-Score (%)
Simple Concatenation	Tokens from all scales concatenated before encoder	87.2% ± 0.6	87.0% ± 0.6
CrossViT-Style Fusion	Separate scale branches with cross-attention at selected layers	87.8% ± 0.5	87.6% ± 0.5
Static Scale-Pair Bias Only	Learnable source–target scale bias without reliability gate	88.3% ± 0.5	88.1% ± 0.5
Reliability Gate Only	Input-dependent scale reliability without learnable scale-pair bias	88.4% ± 0.4	88.2% ± 0.4
Full CSBAF	Learnable scale-pair bias + input-dependent reliability gate	88.9% ± 0.4	88.8% ± 0.4

shows that simple concatenation improves performance by combining multi-scale tokens, but it does not model adaptive interaction between scales. CrossViT-style fusion improves the result by allowing branch-level cross-scale interaction. Adding only the static scale-pair bias further improves performance, showing that explicit source–target scale relationships are useful. The reliability-gate-only variant also improves performance by suppressing unstable scale responses under degraded underwater conditions. The full CSBAF module achieves the highest accuracy and F1-score, indicating that the learnable scale-pair bias and input-dependent reliability gate provide complementary benefits. Although the improvement over simple concatenation is moderate, the repeated-run standard deviation shows that the gain is stable across evaluations. A paired statistical comparison between simple multi-scale concatenation and full CSBAF confirmed that the observed improvement was statistically supported, indicating that the gain was not only due to random variation.

Table 11. Fusion-Placement Strategies Comparison.

Fusion Strategy	Fusion Placement	Description	Accuracy (%)
Pre-Encoder Fusion	Before encoder	Scale interaction occurs before the encoder, allowing the model to refine scale-specific information before global encoding.	88.9%
Post-Encoder Fusion	After encoder	Scale interaction occurs after all encoder layers, with fusion applied at the end of the encoder processing.	87.6%
Within-Encoder Fusion	Within encoder	Scale interaction occurs between encoder layers, allowing more nuanced scale integration during encoding	88.2%

evaluates the effect of fusion placement under the same experimental protocol. Pre-encoder fusion achieves the best accuracy of 88.9%, indicating that early scale interaction helps preserve useful fine- and coarse-scale information before global Transformer encoding. Within-encoder fusion achieves 88.2%, showing that intermediate scale interaction is beneficial but slightly less effective than early fusion. Post-encoder fusion achieves 87.6%, suggesting that delaying scale interaction until after independent encoder processing may reduce the ability to preserve complementary scale-specific evidence. These results support the design choice of performing CSBAF before the shared Transformer encoder.

To further demonstrate the novelty of the proposed Cross-Scale Biased Attention Fusion (CSBAF) module, we compare it to other multi-scale fusion strategies in the identical experimental scenario. Multi-scale concatenation with simple concatenation gives 87.2% accuracy, which has been used as the baseline of multi-scale processing. CrossViT-style fusion, which uses separate transformer branches for each scale with cross-attention at specific layers, reaches an accuracy of 87.8%, suggesting that cross-scale interaction is positive. The result of adding only the scale-pair bias but not the reliability gate is 88.3% accuracy, and this result shows that the explicit modeling of scale-pair relationships by learnable bias terms offers better cross-scale interaction compared to separate branch designs. The complete CSBAF module, including learnable scale-pair bias and the scale-reliability gate, achieved the highest accuracy of 88.9% or a 0.6-point increase over static bias alone. This validates that adaptive, input-dependent scale routing via the reliability gate provides a critical advantage over the static bias or separate branch designs, especially in underwater scenes where visibility, contrast, and distortion widely vary among images, and the model is free to dynamically suppress unstable scales and focus on the best scale to use by each input.

Empirical results show that underwater image analysis with a transformer-based architecture can outperform conventional CNNs when scale-aware feature modeling is employed. The architectural innovation of the proposed framework is the integration of multi-scale patch embedding with Cross-Scale Biased Attention Fusion (CSBAF), enabling the model to focus selectively on the most informative token scale in the presence of underwater distortion. It is particularly applicable to categories that overlap visually, like coral, rocks, sand, and algae, where local texture is not sufficient. In its methodological perspective, the contribution proposed can be seen as a scale-conscious token routing architecture, as opposed to just another multi-scale backbone. It is novel because the learnable scale priors are introduced into the fusion step so that the model can redistribute finer and coarse streams of tokens based on underwater visibility, texture availability, and structural ambiguity. The difference is significant in comparing the work with the existing multi-scale transformers. Multi-scale tokenization alone cannot build a strong contribution; here, the contribution is over the explicit bias-controlled interaction that precedes encoder processing.

The discussion of the results further supports this interpretation. CSBAF dynamically fuses token branches of 8 × 8, 16 × 16, and 32 × 32 scales with input-dependent but learnable scale-pair bias terms and a scale-reliability gate, allowing scale selection to be performed on a case-by-case basis. The proposed framework processes underwater images at three patch scales, capturing both fine-grained texture and global structural context more effectively than one-scale tokenization. The hybrid dataset, combining the Roboflow Aquarium and RUIE datasets, provides 14,000 images in five classes, namely fish, coral, sand, rocks, and algae. Under these conditions, the proposed MS-ViT with CSBAF achieves 88.9% accuracy and an 88.8% F1-score, outperforming CNN baselines by 7.6% and transformer-based models (UWFormer, DP-ViT, and CvT) by 2.3–4.2%. Systematic ablation experiments further indicate a 1.7% accuracy improvement of CSBAF over simple multi-scale concatenation, while cross-dataset testing shows generalization up to 84.4% accuracy during domain shift. Attention map visualizations show progressive focusing of local textures toward semantically meaningful object locations, while the model complexity analysis indicates an accuracy–inference trade-off, with 46.8 M parameters and 30.1 ms inference time.

The proposed MS-ViT with CSBAF can therefore be considered a solid backbone for future studies in underwater image classification. The model can be used by researchers for comparative studies or to apply the CSBAF module to other visual tasks, including medical imaging, satellite remote sensing, foggy scene perception, and low-light photography. It can also be deployed on autonomous underwater vehicles (AUVs) and remotely operated vehicles (ROVs) to monitor marine biodiversity in real time, assess coral reef health, support underwater navigation, and enable automated species recognition. Although the proposed model achieves competitive inference time, its higher parameter count and FLOPs indicate that further optimization may be required for deployment on highly resource-constrained underwater platforms.

Although the proposed MS-ViT with CSBAF consistently improves performance over the compared baselines, the improvements should be interpreted as moderate rather than transformative. The proposed framework builds upon existing concepts from multi-scale tokenization and attention-based fusion, but introduces adaptive scale-aware routing through the combination of learnable scale-pair bias and input-dependent reliability gating before transformer encoding. Therefore, the contribution of this study lies mainly in improving the robustness of multi-scale feature interaction under underwater degradation rather than introducing a completely new transformer architecture.

In addition, several limitations should be acknowledged. The proposed model has higher parameter count and FLOPs than the baseline models, which may limit deployment on highly resource-constrained underwater systems without further optimization. The dataset, although larger and more diverse than many previous underwater classification studies, still contains moderate class imbalance and limited environmental diversity compared with real-world ocean conditions. Furthermore, the reported performance gains over strong transformer baselines remain relatively small, even though statistical analysis confirmed that the improvements were consistent across repeated runs. Therefore, future work should focus on lightweight optimization, larger-scale underwater datasets, broader cross-domain validation, and more efficient adaptive fusion mechanisms.

4. Conclusions

This paper proposed a Multi-Scale Vision Transformer (MS-ViT) with a Cross-Scale Biased Attention Fusion (CSBAF) module for underwater image classification. CSBAF presents two important innovations, such as learnable scale-pair bias modeling inter-scale significance, and a scale-reliability gate dynamically descaling unstable scales when visibility is poor enough to allow scale routing in advance of transformer encoding, and scale routing coarse structure or fine texture based on scene conditions. A hybrid dataset of 14,000 images across five classes was constructed for evaluation. The proposed model achieved 88.9% accuracy and an 88.8% F1-score, outperforming CNN baselines by 7.6% and state-of-the-art transformers by 2.3–4.2%, and ablation validates a 1.7% improvement over simple multi-scale concatenation. A limitation of this study is that the proposed MS-ViT with CSBAF is trained using supervised learning, which requires sufficient labeled underwater images. Although the hybrid dataset improves class diversity, the model may still require additional annotation when transferred to new underwater environments, cameras, turbidity levels, or illumination conditions. Future work will investigate domain adaptation to reduce dependence on target-domain labels and improve generalization across underwater imaging conditions. Similar domain-adaptation strategies have been used in object detection and transformer-based visual recognition to learn domain-invariant features [36,37]. In the proposed framework, domain adaptation could align labeled source-domain underwater features with unlabeled or weakly labeled target-domain features, improving robustness to turbidity, color distortion, lighting variation, and sensor differences. The model has 46.8 M parameters and a 30.1 ms inference time, representing an accuracy–inference trade-off rather than a lightweight architecture. This study suggests that explicit scale-aware fusion with learnable bias and reliability gating can improve transformer-based underwater visual understanding under degraded imaging conditions.