State of the Art: Analysis of Deep Learning Techniques in Images Acquired in an Aquatic Environment

Abstract

The oceans and other marine ecosystems are indispensable to life, so the understanding and knowledge of their biodiversity is crucial to the use of their resources and exploration. These environments are complex and difficult to access, so different types of remote sensing technologies are used to study them. These intelligent sensors can collect a massive amount of data, which, once reviewed and analyzed, can help to draw conclusions and increase knowledge of these underwater environments. Manually reviewing and organizing through this large amount of information is both time-consuming and costly. Therefore, it is advisable to employ automated techniques from machine learning and deep learning fields. In recent years, these methods have proven to be efficient and have obtained very good results in solving different problems applied to the marine world: image enhancement, image classification, segmentation and object detection. This paper presents a systematic review, conducted in accordance with the PRISMA 2020 guidelines, aimed at summarizing the methods used to address underwater problems and their reported results.

1. Introduction

Most of our planet is covered by oceans, and many of them are unexplored. In recent decades, advances in sensorial technology have improved the imaging of ocean biodiversity. All of this, as well as the increasing development of robotic vehicles, made progress in the monitoring of the continental waters and deep seas [1].

Comprehensive knowledge of marine ecosystems and their biodiversity is essential for the sustainable utilization of their resources. To effectively explore and preserve the extensive biodiversity present within underwater ecosystems, systematic monitoring and rigorous analysis of collected data are required.

The ocean floor is inaccessible to humans due to its constituent features such as extreme pressure, cold temperatures and darkness. In this way, imaging or video devices prove to be valuable tools for oceanographic surveillance. In addition to underwater cameras or ROVs (Remotely Operated Vehicles), there are wired observatories; platforms connected to the coast that have multiple types of sensors to collect a wide variety of data [2]. These platforms can acquire image and/or video material during consecutive years, from which animals of different species can be identified and counted [3,4,5,6,7,8].

The acquisition of images of the biodiversity of the oceans has increased dramatically over the last two decades [9], supporting a revolution in the monitoring of marine communities at all depths of the continental margins and on the seabed [10]. However, the large amounts of image and video data generated cannot be processed manually. That can be solved by automatic processes of classification, recognition and labelling. These automatic processes are carried out thanks to artificial intelligence techniques. These techniques are grouped into different branches, such as Computer Vision (CV), Machine Learning (ML) and Deep Learning (DL).

CV techniques have been commonly used for image processing, feature detection and pattern recognition [11]. This branch of artificial intelligence is widely used to process and enhance images, improving their quality and brightness [12,13]. These types of techniques have worked quite well, but sometimes they are not powerful enough to achieve good results. In addition, they are techniques that require multiple manual adjustments.

ML algorithms are quite popular and well-known in the world of image classification and data analysis since these algorithms are able to learn from the data. These methods have been applied to underwater image classification, marine animal identification, and a variety of other underwater tasks [14,15].

Image analysis, pattern recognition, and object detection in DL frequently employ deep neural networks. However, it was not until a few years ago that these techniques were introduced in the marine world, enabling the detection and classification of plants and animals, the improvement and restoration of underwater images and consequently, to gain a greater knowledge about the sea. Applying DL methods to these types of problems presents significant challenges, largely due to the intricate nature of the images, as they are very likely to be noisy or of poor quality. However, if applied correctly, they could help to track and count marine species without the need for constant supervision.

A number of reviews have addressed the topics of fish classification and detection, with some, such as [16], encompassing over 90 cited works. Other papers, despite containing deep state-of-the-art examinations, are not up-to-date because they were published some years ago [17]. The review contained in [18] compares more than 80 articles covering fish classification methods, feature extraction techniques, and classification algorithms. However, despite the inclusion of DL techniques, the article focuses considerably on more traditional methods such as SVM (Support Vector Machine), PCA (Principal Components Analysis), and classification algorithms based on manual features. Other reviews can also be found that focus on aerial images [19], or on more specific species (such as corals [20]) that, despite being very complete reviews, do not provide such a holistic and adaptable view of the field, due to the lack of variety in some aspects. In contrast, the present review extends the scope by providing a detailed and systematic analysis of 132 studies, offering a broader and more comprehensive synthesis of the existing literature.

The structure of this paper is as follows: Section 2 introduces the PRISMA methodology. Section 3 gives a background, setting the stage with fundamental concepts and relevant literature critical to understanding the subsequent sections. Section 4 delineates the various problem types and applications related to the study, explaining how these problems manifest in practical scenarios. Section 5 provides an overview of deep learning approaches applied specifically to underwater and water-related fish images, which is subdivided into detailed discussions, ranging from image processing and enhancement techniques to aquatic species detection, classification, and segmentation techniques, including subsections detailing the applications of different deep learning models to different types of images. In Section 6, we discuss the implications of our findings. Finally, the paper concludes with Section 7, where we draw conclusions from the research conducted.

2. Methodology

This systematic review aims to examine the application of DL techniques applied to solve underwater image problems, encompassing convolutional neural networks (CNNs), transformer-based architectures, and hybrid multimodal approaches. The review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) protocol (accessed on 11 August 2025: www.prisma-statement.org) [21,22], ensuring a transparent, rigorous, and reproducible process.

The PRISMA methodology offers a rigorous and transparent framework for conducting systematic reviews, ensuring that the search, screening, and selection of studies are replicable and free from unnecessary bias. By requiring explicit documentation of inclusion and exclusion criteria, PRISMA enhances the reproducibility and credibility of the review process. This structure also facilitates the synthesis of large and heterogeneous bodies of literature, enabling researchers to identify trends, gaps, and methodological patterns across studies. For technical domains, such as underwater engineering or deep learning applications, PRISMA provides a clear protocol to handle diverse sources (e.g., conference papers, journal articles, preprints) while maintaining methodological consistency.

Despite its strengths, PRISMA also has limitations when applied to rapidly evolving or highly interdisciplinary technical fields. Its structured nature can reduce flexibility in exploring emerging topics that may not yet have well-established terminology or indexing in academic databases. Additionally, the meticulous documentation and multi-phase screening process can be time-consuming and resource-intensive, particularly when large volumes of literature are retrieved. In technical research, PRISMA’s emphasis on predefined criteria may lead to the exclusion of innovative or unconventional studies that could offer valuable insights but fall outside the initial scope. Researchers must, therefore, balance methodological rigour with adaptability to ensure the review remains comprehensive and relevant.

2.1. Introduction

A comprehensive literature search was conducted in IEEE Xplore, Scopus, Web of Science, and arXiv, covering publications from January 2015 to August 2025. This time frame was selected to capture developments from the introduction of AlexNet to the emergence of state-of-the-art architectures such as Vision Transformers (ViT) and Segment Anything Models (SAMs). Search strings combined DL-related terms with underwater detection keywords: (“deep learning” OR “neural network” OR “machine learning”) AND (“underwater” OR “fish” OR “coral” OR “plankton” OR “underwater plants”) AND (“image classification” OR “object detection” OR “image restoration” OR “image enhancement”).

The initial search retrieved 19,275 articles. This search was refined to a final 132 papers. Figure 1 shows an adapted PRISMA flow diagram showing the number of articles in each phase.

This systematic review was guided by the following research questions, designed to structure the search strategy, selection criteria, and synthesis of results:

• RQ1: What advanced deep learning architectures have been applied to the analysis of marine species imagery captured both underwater and above water?
• RQ2: What datasets and image acquisition methods are most frequently used in these studies?
• RQ3: What evaluation metrics are reported, and how do performance levels vary across methods and application contexts?
• RQ4: What limitations, challenges, and future research directions are identified in the literature regarding the use of deep learning for marine species image analysis?

These findings will be presented in Section 6 of this paper.

2.2. Screening

Studies were selected according to predefined inclusion and exclusion criteria designed to ensure relevance, methodological rigour, and reproducibility. The scope of this review was limited to the application of Artificial Intelligence (AI) methods to the analysis of images containing marine species, captured either underwater or above water.

Table 1. Selection criteria.

Criterion Category	Inclusion Criteria	Exclusion Criteria
Publication type	Peer-reviewed journal articles, conference papers, and peer-reviewed preprints from recognized repositories (e.g., arXiv).	Abstract-only publications, posters, editorials, commentaries, theses, or non-indexed reports.
Methodological scope	Application of AI techniques (deep learning, CNNs, transformers, hybrid approaches) to the analysis of marine species images.	AI methods without application to marine species images.
Domain relevance	Imagery of marine species captured underwater (e.g., coral reefs, open ocean) or above water (e.g., shoreline, aerial surveys).	Studies focusing exclusively on non-marine species or terrestrial ecosystems.
Methodological transparency	Detailed description of model architecture, datasets, and training/testing procedures.	Insufficient methodological detail to enable replication.
Temporal coverage	Published between January 2015 and August 2025.	Publications outside the specified date range.
Language and availability	Written in English (or accessible language for review team) with full-text available.	Languages not accessible to the review team or full text unavailable.
Data modality	Image-based studies (still images or video frames) of marine species.	Studies using only non-visual modalities (e.g., acoustic, sonar-only, textual).
Complexity of methods	Use of advanced deep learning methods (e.g., CNNs, Vision Transformers, hybrid architectures).	Use of only traditional machine learning methods (e.g., SVM, random forest, k-NN) without deep learning models.
Access status	Articles available as open access or through freely accessible repositories.	Articles requiring paid subscription or inaccessible through institutional or open channels.

below presents a summary of the selection criteria.

2.3. Inclusion

After applying the inclusion and exclusion criteria, a comprehensive full-text review was conducted on the 132 qualifying studies. This process aimed to capture the breadth of research on advanced deep learning methods applied to marine species imagery, encompassing both underwater and above-water contexts. The review considered variations in neural network architectures, target species, imaging conditions, and dataset composition, as well as reported performance metrics and validation approaches.

Through this systematic assessment, the review seeks to establish a consolidated view of the methodologies currently employed, identify patterns in their application, and highlight the most effective strategies for accurate and efficient species detection. The synthesis of these findings will inform best practices and guide the development of more robust AI-based tools for marine biodiversity research and conservation.

The distribution of articles published over the years is shown in Figure 2. A general upward trend is observed until 2018, followed by fluctuations in subsequent years, mainly attributable to selection criteria and the availability of studies during the analyzed period. The increase recorded in recent years highlights the growing attention to current DL approaches, although it should be noted that the data for the last few years are influenced by the time frame of the literature search.

2.4. Data Extraction and Synthesis

Each included study was reviewed individually, and its key characteristics were extracted into comparative tables. The extracted variables included the article reference, the name of the dataset used, the type of task or problem it solved, the model architecture, the chosen evaluation metrics, and the values associated with each metric.

The extracted information was organized into tables to facilitate analysis and comparison between studies. These tables summarize the performance of the different models across various datasets and tasks, providing a consistent basis for comparison.

For each study, quantitative performance results were extracted from the models when available. These included accuracy, precision, recall, F1 score, mean average precision (mAP), Intersection Over Union (IoU), and, for image enhancement or restoration tasks, PSNR (Peak Signal-to-Noise Ratio), SSIM (Structural Similarity Index Measure), PCQI (Perception-based Colour Quality Index), UCIQE (Underwater Colour Image Quality Evaluation), UIQM (Underwater Image Quality Measure), UICM (Underwater Image Colourfulness Measure), and other task-specific metrics reported in the original publications, among others, depending on the study.

3. Background

Artificial intelligence (AI) is the branch of computer science that attempts to solve tasks using computers for which human intelligence is required. In turn, this branch is composed of sub-branches that encompass different techniques that can solve problems automatically. Some of the best-known branches are the following: Computer Vision, Machine Learning and Deep Learning.

Computer Vision (CV) is considered a part of the artificial intelligence family, as they share topics such as pattern recognition and learning techniques. Its aim is to imitate human vision perception and reasoning by means of computers. In other words, computer vision is the analysis and processing of videos or images. CV tasks range from methods for acquiring, processing, analyzing, and understanding digital images to extracting high-dimensional data from the real world to produce numerical or symbolic information.

Machine Learning (ML), which is considered as a subset of AI, is the study of algorithms that computer systems use to perform a specific task based on pattern recognition and inference, without the need for explicit instructions. In ML, the programme infers its own rules through a process called training. In the case of supervised learning, this process consists of analyzing a sample of data (known as the training set or training data) in order to predict some properties or labels. Under some specific conditions, the model trained in this way can generalize to previously unseen data and thus produce the predictions for these.

Deep Learning (DL) is a sub-branch of machine learning, where the most basic computational model is an Artificial Neural Network (ANN), which is inspired from the human brain functionality. ANNs were first introduced in 1943 in the paper [23] by W. McCulloch and W. Pitts. They presented a computational model based on how brain neurons work with each other to perform more complex tasks using logic of propositions [24]. Early neural networks played a fundamental role in the development of ML, but they also had significant limitations. The concept of a perceptron was introduced by Frank Rosenblatt in 1958 [25]. A perceptron is a supervised linear classification algorithm designed to identify and select subsets within larger datasets based on learned criteria. However, despite its historical importance, the perceptron could not solve nonlinearly separable problems, such as the XOR problem.

But it was not until the 1980s that the interest in the ANNs was regained due to various advances made in this field, like the contribution of Hopfield in [26], who introduced recurrent neural network models. Hopfield’s networks demonstrated that neural systems could reach stable states and function as associative memory. Another fundamental advance came with the development and popularization of the backpropagation algorithm, particularly thanks to the work of Rumelhart, Hinton, and Williams [27]. Backpropagation provided an efficient mechanism for training multilayer perceptron (MLP) networks. These networks consist of an input layer, one or more hidden layers, and an output layer, and are capable of learning complex and nonlinear mappings. By iteratively propagating error gradients from the output layer backwards through the network, backpropagation enables the network parameters to be updated using gradient-based optimization. The breakthrough of backpropagation, together with increased computational power and the availability of data, laid the foundations for modern deep learning. Contemporary deep neural networks extend the MLP paradigm through deeper and more specialized architectures, enabling the learning of more complex feature representations.

These networks with a greater number of layers are called deep neural networks (DNN). These networks are considered hard to train, besides being a costly process, but they can also have better results and reduce error [28]. These models are capable of learning hierarchical feature representations directly from raw data, leading to significant performance improvements over classical machine learning approaches in many tasks. In CV, this evolution has resulted in a diversity of DL architectures, such as Convolutional Neural Networks (CNNs), autoencoders, and, more recently, Transformer-based models; each addressing different aspects of visual perception and representation.

4. Types of Problems in Images and DL Applications

During the last few years, artificial intelligence has been used to solve different problems related to images in multiple fields, like agriculture [29,30,31,32], medicine [33,34,35,36] or meteorology [37,38,39], to name a few. However, in a general way, we can differentiate the following applications that are generally used in all areas: image processing, image classification, object detection and multimodal understanding and reporting. We can see the breakdown of these applications in Figure 3.

Digital Image Processing (DIP) is the technical analysis of a digital image by applying computer vision or machine/deep learning techniques. DIP involves low-level operations such as noise reduction, contrast enhancement and sharpening or blurring. These kinds of operations are made in order to improve further classification results, object detection results or simply achieve increased image enhancement [40].

Image classification is the process of taking an input (in this case an image) and outputting a class or a probability that the input belongs to a class or category. In other words, image classification refers to the labelling of images into one of the already predefined categories.

In order to carry out this labelling, it is necessary to extract information or characteristics from the images. This is performed by pre-processing the images, usually using computer vision techniques.

The next steps would be training and classification, both of which can be performed using machine learning or deep learning techniques [41].

Object detection is an area of computer vision and is related to image processing that deals with detecting instances of semantic objects in digital images and videos. Object Detection is modelled as a classification problem where windows of fixed sizes slide through an input image to find the possible locations of the objects and then feed these patches to an image classifier. The first object detection framework was proposed by Paul Viola and Michael Jones in 2001 [42]. It was motivated by the problem of face detection, even though it can be trained to detect a variety of elements. The framework was composed of a Haar-like feature Selector and an Adaboost algorithm for classification.

Even though object detection can be carried out using HOG Features, methods for object detection can generally be categorized into two main groups, based on deep learning techniques: Region-based Convolutional Neural Networks (R-CNNs) and regression-based object detectors. The first group contains detectors like R-CNN [43]; the other group contains detectors that model the object detection problem as a regression problem, like YOLO (You Only Look Once) [44] or SSD (Single Shot Multibox Detector) [45].

Recent advances in Vision-Language Models (VLMs) and Large Language Models (LLMs) have made it possible to understand and generate multimodal reports as a higher-level task in the analysis of aquatic imagery. This paradigm allows images to be modelled in conjunction with natural language, sensor metadata or other relevant information. In underwater and aquatic scenarios, approaches based on these multimodal language models enable applications such as scene-level understanding and description, automated report generation and semantic summarisation of detected visual observations.

5. Results

CV techniques and ML algorithms fit well when performing image processing, item classification or object detection tasks, and therefore they have been used together to solve different problems in different areas [46,47], among them, the analysis of aquatic or underwater species field [48,49,50,51,52,53]. DL techniques are more advanced than CV and ML, and although computationally they tend to be more expensive, the results are often better. Recently, these techniques have been used in the underwater world to classify or detect species and even monitor them and better understand their behaviour [54,55].

5.1. Image Processing, Enhancement and Restoration

The processing of an underwater image is quite challenging, due to the underwater conditions, which include lack of light, water currents and other phenomena that cause distortion and noise. Even though computer vision techniques are more widespread (probably because they are older), DL has proven to be a highly effective approach for the enhancement and processing of underwater images, with neural networks playing a central role in this field. A wide range of neural network architectures have been designed and implemented to tackle the specific challenges associated with the underwater environment. Similar image quality degradation issues have also been examined in other application domains, such as medical imaging, further reinforcing the general importance of enhancement and denoising as preprocessing steps under challenging acquisition conditions [56].

Some reference datasets exist in the literature for underwater image enhancement and restoration research. The dataset called Underwater Image Enhancement Benchmark (UIEB) [57] provides a comprehensive set of degraded underwater images paired with reference-enhanced counterparts, enabling the evaluation of colour correction and dehazing methods. The Underwater Video Enhancement Benchmark (UVEB) [58] extends this concept to video sequences, supporting the assessment of temporal consistency in enhancement algorithms. The U45 dataset [59] offers 45 particularly challenging underwater scenes for testing robustness under extreme visibility, turbidity, and lighting conditions. Finally, the Enhancement of Underwater Visual Perception (EUVP) dataset [60] contains paired and unpaired underwater images collected across varying depths and environments, facilitating both supervised and unsupervised training. Together, these datasets form a comprehensive foundation for developing and benchmarking underwater image enhancement techniques.

Although CNNs are primarily used for image classification and detection, they can also be used to improve and restore image quality, like in [61], where they train a CNN to dehaze and perform image enhancement. Compared to other state-of-the-art methods, their approach produces results that closely resemble the ground truth images. They carried out two experiments in which they picked up the error obtained when classifying the treated images, in turn comparing them with Automatic Colour Enhancement (ACE) [62] and histogram equalization techniques. The error obtained with their technique in the validation set was 14.1%, while with ACE it was 15.7%, and with the histogram 20.5%.

While considered distinct networks, there are other types of networks that contain a CNN-type architecture, like Generative Adversarial Networks (GANs). Generative modelling uses unsupervised learning to find patterns in data, allowing the model to generate new, unseen examples [63]. In [64], they perform underwater image enhancement by a Conditional Generative Adversarial Network (cGAN), achieving a clear image by a multi-scale generator. Although they achieved the results of the current state of the art, there still exist several limitations, as they achieved an SSIM (Structural Similarity Index Measure) value of 0.6552.

Another experiment based on GANs is performed in [65]. They built two networks called UGAN (Underwater Generative Adversarial Network) and UGAN-P (with Gradient Difference Loss), and they compared their results to CycleGAN (Cycle Generative Adversarial Network) [66]. Both UGAN and UGAN-P achieved better results. In addition, they performed the detection of swimmers both in original images and in images enhanced by the Mixed-Domain Periodic Motion (MDPM) tracker [67]. Out of 500 frames of each dataset, in the original images, there were only 42 correct detections, while in the generated images, there were 147 detections.

In Li et al. [68], the Water Generative Adversarial Network (WaterGAN) is used to produce realistic underwater images and the Underwater Image Restoration Network for correcting the colour. WaterGAN learns a realistic representation from real unlabelled underwater images; it leverages terrestrial images along with associated depth information to generate artificial underwater images. They compared their colour restoration method to other kinds of methods like histogram equalization or the Shin et al. approach [69], among others. Although the proposed method obtained very good results in terms of colour accuracy and colour consistency, the histogram equalizer also obtained high and sometimes better results. They tested the presented model on multiple datasets, including B3DO (The Berkeley 3D Object Dataset), UW RGB-D Object (Underwater RGB-Depth Object dataset) and NYU Depth (New York University Depth dataset), among others. An autoencoder is a type of neural network that learns without supervision and uses backpropagation for training. The aim of an autoencoder is that the output data is the same as the input data [70]. For that purpose, they learn the input data features so they can represent it later in the output layer. Autoencoders comprise three principal components: the encoder, the latent-space representation (also referred to as the code), and the decoder. In [71], authors developed an Underwater Denoising Autoencoder (UDAE) model using a denoising autoencoder with a U-Net [33] architecture as a CNN architecture for underwater image restoration. They compared their results with the UGAN [65], and concluded that their method performed better than the network based on a GAN, obtaining a lower value of MSE (Mean Squared Error). They also obtained an SSIM value of 0.9653, while the UGAN achieved a value of 0.9186.

Li et al. [72] introduced a method that eliminates scatter while maintaining colour in underwater images. They found that their method and colour correction of images influence the classification results. In addition, they propose a more comprehensive image quality assessment [73] called index Qu as a rule to compare results of the algorithm, which achieved a value of 0.7929.

Variational autoencoders are also a resource used for image quality enhancement, such as in [74], where a novel probabilistic network named PUIE-Net (Probabilistic Network for Underwater Image Enhancement) is introduced to model the enhancement distribution of degraded underwater images. Built upon the U-Net framework, the PUIE-Net architecture improves underwater images by modifying aspects like colour and contrast, while keeping the original content intact. It uses a technique called probabilistic adaptive instance normalization (PAdaIN), which combines a conditional variational autoencoder (CVAE) [75] with adaptive instance normalization (AdaIN) [76], enabling effective extraction of image features for enhancement. The network has two branches: the upper estimates the prior for a raw underwater image, and the lower constructs posterior distributions using the original image and a reference. They tested the proposed method on two datasets: images from UIEB (Underwater Image Enhancement Benchmark) and RUIE (Real-world Underwater Image Enhancement dataset) [77], which only contain raw underwater images. The network was trained with the modified set from the UIEB dataset. Evaluating on the RUIE dataset, they reached a value of 4.512 and 3.7 in Mean Opinion Score (MOS) and Natural Image Quality Evaluator (NIQE) [78], respectively. MOS was used to quantify the subjective evaluation.

Although the use of reinforcement learning is more common in solving other types of problems, it has also been used for image enhancement. In [79], they made use of a Markov decision process (MDP), where states are represented by feature maps, actions are represented by enhancement methods (basic adjustment, colour tuning, correction, and deblurring), and rewards are represented as improvements in image quality (determined by a pixelwise loss and a perceptual loss). A deep Q network is responsible for selecting an image enhancement action, which causes a state to change from one to another at each step of the MDP. To train and test the network, data obtained from the Underwater Image Enhancement Benchmark (UIEB) dataset was used, a dataset containing 950 authentic underwater images. They evaluated the images qualitatively (visual inspection) and quantitatively. For the quantitative evaluation, they chose 3 metrics: NIQE, Underwater Colour Image Quality Evaluation (UCIQE) [80], and Underwater Image Quality Measure (UIQM) [81], with which they achieved the values 36.5574, 0.5962, and 4.34436, respectively, on the challenging image set they preselected.

The study in [82] proposes a Multiscale Dense Generative Adversarial Network (MDGAN) to enhance underwater images. The researchers worked with a dataset that included both synthetic and real underwater images. Their proposed method achieved better performance on both metrics than other methods such as FE (Fusion Enhance), RB (retinex-based), UDCP (Underwater Dark Channel Prior), CycleGAN, WSCT (Weakly Supervised Colour Transfer), and UGAN. In qualitative tests, the method showed significant improvements in colour correction and detail recovery in underwater images compared to other existing methods.

The work in [83] proposes DNnet (Dynamic range and Normalization framework), a compact neural network for enhancing underwater images, especially in 4K. It introduces novel modules like FAN (Fast Average Normalization) and CDR (Channel Dynamic Range) to efficiently restore colour and detail. DNnet achieves real-time performance with high accuracy, outperforming larger models on datasets like UIEB (Large Scale Underwater Image Dataset) and LSUI (Large Scale Underwater Image Dataset).

Another underwater image enhancement method called UIEVUS (Underwater Image Enhancement method designed for Various Underwater Scenes) is presented in [84]. This method addresses colour distortion and uneven illumination. It decomposes images into illumination and reflection maps via Retinex theory, enhances them using GAN-based modules, and fuses results for high-quality outputs. The proposed method demonstrates state-of-the-art results on metrics like PSNR (Peak Signal-to-Noise Ratio) (23.55 dB) and UIQM (3.26), improving visual quality and downstream tasks (e.g., object detection).

This paper [85] proposes CCL-Net, a two-stage network based on cascaded contrastive learning (CCL) for underwater image enhancement. Stage 1 (CC-Net) corrects colour distortion in Lab space, while Stage 2 (HR-Ne, a Haze Removal Network) removes haze using multi-scale features. Cascaded contrastive learning progressively refines results by using raw images (Stage 1) and intermediate outputs (Stage 2) as negative samples. The CCL-Net achieved the best value of UIQM (3.021) and the second-best value of UCIQE (0.464) on the UIEB-T90 dataset.

This work [86] proposes a lightweight U-Net variant for underwater image enhancement, using simplified channel attention and SK fusion to reduce model complexity. Using the EUVP paired dataset for training, it delivers state-of-the-art performance with 9.27 million parameters and 2.74 billion FLOPS (Floating-point Operations)—outperforming nine baselines in PSNR (21.565 dB), SSIM (0.879), and efficiency. Ablation studies validate design choices like LayerNorm and GELU (Gaussian Error Linear Unit) activation. The method excels in restoring colour/contrast in challenging underwater scenes (tested on UIEB), demonstrating practical deployment potential.

UDNet (Uncertainty Distribution Network) proposes an unsupervised framework for underwater image enhancement, eliminating the need for paired training data [87]. It integrates uncertainty modelling via SGMCSS (Statistically Guided Multicolour Space Stretch module for reference generation), CVAE (feature extraction), and PAdaIN (adaptive normalization). Evaluated on 8 datasets, UDNet outperformed 10 methods in quantitative metrics (PSNR, SSIM, UIQM) and qualitative results. It excels in generalizing to unseen/unpaired data, addressing challenges like colour distortion and low contrast.

The Transformer model, proposed by Google in 2017 [88] and has achieved successful results in Natural Language Processing (NLP) tasks, due to its multi-attention mechanism. Later, the Vision Transformer (ViT) [89] was introduced, which is capable of extracting image features, achieving outstanding results in various computer vision tasks. ViTs divide images into fixed-size patches, use them as tokens, and apply self-attention to model global context efficiently. The Swin Transformer, introduced in 2021 [90], was designed as an enhanced version of ViT, offering better performance and greater computational efficiency. In [91] they propose a hybrid UNet-MaxViT (Unet with Multi-axis Vision Transformer) architecture for underwater image enhancement, integrating UNet’s local feature extraction with MaxViT’s global multi-axis attention to address colour distortion, low contrast, and detail loss. Evaluated on UIEB, EUVP, and UFO-120 (Dataset for Simultaneous Enhancement and Super-Resolution (SESR) of underwater imagery) datasets, it achieves state-of-the-art results: PSNR 22.91 (UIEB) and 26.12 (EUVP), outperforming 12+ methods including PhISH-Net (Physics-Inspired Network) and URSCT-SESR (U-Net-based reinforced Swin-Convs Transformer for simultaneous enhancement and super-resolution). The model’s efficiency (10.82 GFLOPs (Giga FLOPS), 0.015 s/image) enables real-time deployment. Visual results demonstrate enhanced colour fidelity and structural preservation in complex underwater conditions.

Multi-scale transformers have proven to be an effective approach for image enhancement and restoration tasks by effectively capturing information at various spatial resolutions. By processing images through hierarchical representations, these models combine fine-grained local details with broader global context, which is crucial for addressing complex degradations such as noise, blur, and colour distortion. This multi-scale architecture allows for more accurate and coherent enhancement compared to single-scale methods, improving the fidelity and visual quality of restored images. This paper introduces UWFormer, a new multi-scale Transformer network for enhancing underwater images [92]. UWFormer uniquely combines a Nonlinear Frequency-aware Attention (NFA) module and a Multi-Scale Fusion Feed-forward Network (MSFN) within a semi-supervised learning framework. A key innovation is the Subaqueous Perceptual Loss (SPL), used to generate reliable pseudo-labels from unlabelled data. UWFormer surpasses leading methods in several benchmarks (such as EUVP, UIEB, U45 and RUIE (Retrieval-based Unified Information Extraction), restoring colour balance, contrast, and detail without typical artefacts such as colour casts or over-enhancement.

The Segment Anything Model (SAM) is a prompt-driven framework designed for versatile, large-scale image segmentation. SAM is able to precisely separate different objects and areas without needing to retrain for specific tasks. Its ability to separate foreground and background with high precision has made it a powerful tool across diverse fields, from medical imaging to remote sensing. In underwater vision, SAM offers a unique advantage by isolating scene elements that are often degraded differently due to scattering, turbidity, and colour distortion. The following work proposes a discriminative underwater image enhancement method using SAM to segment foreground/background regions [93]. It applies adaptive colour compensation and pyramid-based fusion separately to each region, eliminating foreground-background crosstalk. High-frequency edge fusion restores blurred details.

Table 2. Deep Learning techniques for underwater image enhancement and processing.

Ref	Dataset	Problem Type	Algorithms	Metric	Value (Highest)
[61]	Images obtained by an autonomous underwater vehicle	Underwater image enhancement	CNN	Visual inspection and error rate in posterior classification	14.1%
[64]	UIEB dataset and synthetic underwater image dataset	Underwater image enhancement	Conditional generative adversarial network (cGAN)	PSNR/SSIM (Structural Similarity Index)	17.715 dB/0.6552
[65]	Subsets from Imagenet, YouTube videos and images from Flicr™	Underwater image enhancement	GAN (U-Net) called UGAN and UGAN-P (with Gradient Difference Loss)	Gradient difference loss metrics in red, blue, green and orange patch/distances in image space	9.39, 5.50, 3.25, 5.79/94.91 (mean)
[68]	B3DO [94], UW RGB-D Object [95], NYU Depth [96] and Microsoft 7- scenes [97] datasets and 3 underwater datasets collected in a laboratory, Jamaica and Australia	Underwater image generation and restoration	WaterGAN	Euclidean distance/variance of intensity-normalized colour in RGB-space (red, green, blue)	(0.0484, 0.2132, 0.1431)/(0.0005, 0.0007, 0.0006)
[71]	Images from the Internet (aquariums, underwater images with artificial light and processed images)	Underwater image enhancement and restoration	Underwater Denoising Autoencoder (UDAE)	MSE/SSIM	0.0028/0.9653
[72]	JAMSTEC database	Underwater image enhancement	Contrast enhancement method	Index Qu/SSIM	0.7929/0.6027
[74]	Modifed UIEB dataset and RUIE dataset	Underwater image enhancement	U-Net, CVAE, AdaIN	PSNR/SSIM/DeltaE/NIQE/MOS	UIEB dataset: 21.86 dB/0.870/9.556/3.626/4.2
[79]	UIEB dataset	Underwater image enhancement	MDP and deep Q network	NIQE/UCIQE/UIQM	41.334, 0.6412, 4.5935
[82]	Images from ImageNet and SUN (Scene Understanding) database [98]	Underwater image enhancement	Multiscale dense generative adversarial network (GAN)	UCIQE/UIQM	0.6028 ± 0.0282/5.0973 ± 0.4163
[83]	UIEB dataset, LSUI [99] and UVEB	Underwater image enhancement	DNnet	PSNR/SSIM/MSE/UIQM	UIEB dataset 26.335 dB/0.910/0.368 × ${10}^3$/2.994
[84]	UIEB and LSUI datasets for training, UIEB190, LSUI850, OceanEx (full-reference); C60, RUIE dataset Color90, UPoor200, U45 (non-reference).	Underwater image enhancement	UIEVUS Framework: Integrates Retinex decomposition with GAN-based enhancement	Full-reference: PSNR/SSIM. Non-reference: UIQM	Full-reference: UIEB190 = 23.55 dB/0.90 Non-reference: U45 = 3.26
[85]	UIEB-T90 (90 images from UIEB), UIEB-C60 (60 challenging images from UIEB), EUVP-T515 (515 images from EUVP), SQUID-T16 (16 images from SQUID), UIE-T78 (78 images from RUIE dataset).	Underwater image enhancement	CCL-Net	PSNR, SSIM, UIQM, UCIQE	RUIE-T78: UIQM = 3.168, UCIQE = 0.447 UIEB-T90: PSNR = 20.181 dB, SSIM = 0.866, UIQM = 3.021, UCIQE = 0.464
[86]	EUVP and UIEB datasets	Underwater image enhancement	Improved U-Net	NIQE, UCIQE, PSNR, SSIM	4.393/0.430/21.565 dB/0.879
[87]	UIEBD (Unsupervised, no ground truth). For testing: EUVP, UFO, UIEBD (paired); DeepFish [100], FISHTRAC, FishID, RUIE dataset, SUIM (Segmentation of Underwater Imagery dataset) (unpaired).	Underwater image enhancement	UDNet framework integrates SGMCSS Module, CVAE Module and PAdaIN Block.	Full-Reference: PSNR, SSIM, MAD (Most Apparent Distortion), GMSD (Gradient Magnitude Similarity Deviation). No-Reference: UIQM, MUSIQ (Multi-scale Image Quality Transformer), NIQE, UCIQE. Comparison: PSNR, SSIM, UIQM, UCIQE.	Full-Reference: EUVP: PSNR = 22.96 dB, SSIM = 0.771, UIQM = 3.265, UCIQE = 0.749 No-Reference: UCCS: UIQM = 3.974, UCIQE = 0.713
[91]	UIEB, EUVP and UFO-120	Underwater image enhancement and restoration	Hybrid UNet-MaxViT	PSNR, SSIM, PCQI (Perception-based Colour Quality Index), UCIQE, UIQM, UICM, UIConM (Underwater Image Contrast Measure), CCF (Colourfulness Contrast Fog density index).	UIEB: 22.91 dB/23.5286/0.9341/0.6460/1.602/9.6405/1.1865/37.9325.
[92]	UIEB, EUVP, EUVPUN, RUIE dataset, U45	Underwater Image Enhancement	UWFormer	PSNR, SSIM, LPIPS, UIQM, UCIQE	EUVP: PSNR = 24.40, SSIM = 0645, LPIPS = 0.129, UCIQE = 0.431 U45: UIQM = 3.227/UCIQE = 0.440
[93]	U45, SIUM (Segmentation of Underwater Imagery) [101], UIEB, SQUID [102]	Underwater image enhancement	SAM	UIQM, CCF, AG, BC(e) (Blind Contrast Restoration Assessment in edge visibility), BC(r) (Blind Contrast Restoration Assessment in edge pixel gradient values), URanker (Ranking-based underwater image quality assessment)	UIEB: 3.544/48.508/27.165/1.168/3.889/1.964

summarizes the most relevant deep learning-based approaches for underwater image enhancement, including the corresponding references discussed above.

5.2. Underwater Species Detection and Classification

The classification of fish is the most recurrent problem, and it has also been solved by deep learning techniques. The images used, again, can vary greatly, as the background can be static, where the photos have been obtained in an aquarium or out of water (in a controlled environment), or dynamic background in an uncontrolled environment, where the images have been obtained on the sea or river bottom and are much more complex. As shown in Figure 4, classification and detection are the final stages in a typical underwater image processing workflow, which begins with image acquisition and involves several intermediate steps such as enhancement and feature extraction.

Deep learning techniques usually obtain better results than machine learning, both in controlled and uncontrolled environments. Again, items using datasets obtained in controlled environments such as tanks or aquariums are many [103,104,105,106,107,108], as are works using images of fish out of water [109,110,111,112], in markets or on fishing boats, for example. However, datasets generated in uncontrolled environments, such as the open sea, are more popular in deep learning articles [113,114,115,116,117,118,119,120]. In such environments, factors like water currents, particle clouds, and low light make accurate detection and classification challenging.

Many papers use datasets provided by fixed long-term underwater observatories (FUOs), which use multiple sensors to collect data (videos, images, etc.) from the marine world around them. In these datasets, the problem of counting and tracking species is usually addressed in order to analyze the marine environment to which the images belong.

The LoVe (Lofoten–Vesterålen) Observatory is a FUO situated in Norway that provides a large dataset of images and multiple data from different sensors (like temperature or turbidity). This observatory’s data has been used in some papers [121,122].

At present, there are few public datasets with underwater fish images, and most are relatively simple in terms of complexity. SeaCLEF (Sea Content-based Conference and Labs of the Evaluation Forum, https://www.imageclef.org/lifeclef/2017/sea, 5 March 2026) is a benchmark dataset developed as part of the CLEF (Conference and Labs of the Evaluation Forum) initiative, focusing on marine biodiversity monitoring. It typically includes images and videos of underwater species, habitats, and environmental conditions, aiming to support research in species recognition, habitat classification, and environmental assessment in marine contexts. Fish4Knowledge (https://homepages.inf.ed.ac.uk/rbf/fish4knowledge/, 3 May 2026) is a large-scale dataset containing millions of underwater video frames collected from coral reefs in Taiwan. It provides annotations for fish detection, tracking, and species classification, enabling research in underwater computer vision, ecological monitoring, and automated marine life surveys. The NOAA Fisheries datasets (https://www.st.nmfs.noaa.gov/aiasi/DataSets.html, 3 May 2026) contain images and videos collected by the U.S. National Oceanic and Atmospheric Administration for fishery monitoring and management. It includes various fish species in different environments, both in situ and on deck, supporting research in species identification, size estimation, and population assessment. LifeCLEF (Life-based Conference and Labs of the Evaluation Forum) is an annual challenge and dataset series dedicated to biodiversity identification, covering multiple domains such as plants, birds, mammals, and marine species. The marine branch includes underwater imagery for species recognition and habitat mapping, serving as a benchmark for AI-driven biodiversity assessment.

5.2.1. Aerial, Plankton, Aquatic Plants and Coral Detection and Classification

Although the use of aerial image datasets is not so widespread, some researchers have used them, obtaining high detection and classification results. In [123], two different Faster R-CNN models, one with a ZF (Zeiler and Fergus model) and the other VGG (Visual Geometry Group model)-based, are used for stingray detection on aerial images. The experiments showed that the VGG model achieved higher average precision, reaching the 99.7% in one of the three test datasets used.

The study in [124] presents results on two different datasets, the first containing underwater images and the second composed of aerial images. They chose a pretrained RetinaNet as the object detector. For tracking, they applied the Simple Online Realtime Tracker (SORT) algorithm over the detections performed by the detection method. For the aerial dataset, the average precision for each of the three classes was 0.4 for Ray, 0.25 for Diver and 0.75 for Shark. The lower results were due to varying annotation counts for each class. In the initial dataset, there were more false positives because certain fish were detected but not annotated.

In [125], the authors developed a solution for whale detection and counting in aerial images. The system was composed of two steps: first, GoogleNet-Inception v3 CNN architecture was applied for whale detection; second, a Faster R-CNN model built on the Inception-Resnet v2 CNN architecture performed the counting. They obtained high results in both detection and counting.

Although they used a low-complexity CNN, in [126] they used aerial dataset to perform species detection and classification over aquatic areas. For detection, they initially halved the size of each frame to reduce the number of pixels to be processed. Subsequently, they identified a homogeneous background by calculating a fine-grained saliency map, where low-value areas represent the background and high-value areas represent possible detections. Finally, the coordinates of the local maxima in high-saliency areas were identified, extracting a window around each coordinate to classify it and determine whether it contained an animal or was part of the background. The performance of the CNN differed depending on the species and factors like morphology, spacing, behaviour, and habitat uniformity. The overall precision of the model for detecting seal and sea turtles was below 0.27 for both classes, while for detecting gannets it was 0.74.

In [127], the authors proposed a human-in-the-loop method for cetacean detection in images. First, generic deep learning models produce a binary land cover map to filter irrelevant images and identify key samples for annotation. Next, an active learning strategy refines a segmentation model with selected images undergoing manual and AI-assisted annotation, iteratively improving performance. Finally, a human reviewer verifies all model detections, correcting them, if necessary, to improve the quality of the final analysis. The selected model was a U-Net architecture with EfficientNet-b3 as the encoder. The trained model managed to detect 84% of the detections on the whole dataset observed by one of the experts who analyzed the dataset.

Most studies using aerial datasets over the sea aim to detect whales, probably because they are large mammals that can be seen from afar. The dataset used in [128] was collected during an aerial survey over Cumberland Sound Bay, Nunavut, Canada, in August 2014. A pre-trained and fine-tuned Faster-RCNN model was used for all experiments. To evaluate the effect of the sliding window approach on object detection, patches of four different sizes (256 × 256, 512 × 512, 768 × 768, and 1024 × 1024) were cropped with a 20% overlap of the full images. Experimental results indicate that increasing the input image size improves the model performance.

We can find a study that presents a solution for detection in aerial images based on YOLOv8 in [129]. This paper proposes PGG-YOLO (P2P5hGhostConv-G_Ghostbottleneck-YOLO), a lightweight fish detection model for identifying turned white belly fish in pond environments using UAV (Unmanned Aerial Vehicle) imagery. It enhances YOLOv8 with GhostConv, G-Ghost bottleneck, and a small object detection layer. The model achieves a high mean average precision (mAP50 = 99.4%) and runs efficiently (10.3 ms/frame) with a compact model size (8.62 MB). A custom dataset was built from 3000 annotated UAV images. The method shows excellent detection and counting accuracy, aiding practical aquaculture management and insurance evaluation.

The methods and studies cited in this section are organized and detailed in

Table 3. Deep Learning techniques for underwater object detection in aerial images.

Ref	Dataset	Problem Type	Algorithms	Metric	Value (Highest)
[123]	Aerial images	Stingray detection and classification	Two different Faster R-CNN models (one with a ZF and other VGG based)	Average precision	99.7%
[124]	Underwater images and Aerial images	Marine species classification	RetinaNet as the object detector/Simple Online Realtime Tracker (SORT) for tracking	Average precision	Aerial: 0.4 for Ray, 0.25 for Diver and 0.75 for Shark
[125]	Three datasets combining images from Google Earth38, free Arkive83, NOAA Photo Library84, and NWPU-RESISC45 dataset	Whale detection and counting	GoogleNet Inception v3 CNN architecture for detection and Faster R-CNN based on Inception-Resnet v2 CNN architecture for counting	F1-measure	Detection: 81%, Counting: 94%
[126]	Aerial images	Gannets, seals sea turtles	CNN	Average Recall/average precision	0.826/0.403
[127]	Aerial images	Cetacean detection and classification	U-Net with EfficientNet-b3	Accuracy/f1-score/recall/precision	91.37%/95.49%/98.96%/92.26%
[128]	Aerial images	Beluga whale detection	Faster-RCNN	Intersection over Union	0.79
[129]	Custom dataset built from UAV imagery	Detection and counting of turned white belly fish	PGG-YOLO based on YOLOv8	Precision/Recall/F1-score/mAP50	99.52%/97.66%/98.58%/99.4%

.

As plankton is fundamental in the marine ecosystem, it is very important to carry out monitoring tasks to understand its behaviour and to know its habits, and several papers focus on studying them.

In 2016, Lee et al. [130] used the WHOI-Plankton database [131] containing about 3 million images of approximately 100 plankton species. However, 90% of the images were for only 5 types of plankton. To solve this, they proposed a method using CNN and transfer learning (training with CIFAR10, the Canadian Institute For Advanced Research dataset). For all classes, they obtained an accuracy value of 92.80%, while for the 5 main classes it was 94.77%.

In [132], they used another databased called Planktonset [133]. They developed different versions of a deep convolutional neural network, with different input image sizes. Even though the loss value was over 0.6, they showed effectiveness in plankton classification.

In the paper [134], they proposed ZooplanktonNet, which is a deep convolutional network for Zooplankton classification. For training the net, they used a zooplankton dataset with 9460 images that involves 13 classes. The final accuracy value obtained by this network was 93.7%.

The use of transfer learning is a currently recurrent and successful technique. In [135] it is also used to improve the classification of plankton. They carried out experiments in three datasets (WHOI, ZooScan and one by Kaggle), comparing different structures of the state of the art, such as AlexNet [136], VGG16 or ResNet50 (Residual Network with 50 layers) among others, with their ensemble method. They obtained a maximum F-measure value of 0.953, 0.897 and 0.926 on the WHOI, ZooScan and Kaggle datasets, respectively.

The authors in [137] propose a novel detection strategy by combining a cycle adversarial network and a densely connected YOLOV3 model (YOLOV3-dense), which improves rare taxa detection by increasing the data volume and reducing feature loss. Starting from the WHOI-Plankton dataset, they generated two augmented datasets using a CycleGAN to produce fake data from the original unpaired data to increase the data volume of rare taxa. The results show that this strategy achieves a mean precision (mAP) of 97.21% and 97.14% on the two datasets, outperforming models such as YOLOV3-tiny, YOLOV3, and Faster-RCNN, especially in rare taxa detection, where the precision improves by 4.02% on average. Moreover, the detection time is competitive, being significantly lower than that of the Faster-RCNN model, indicating that the proposed model is efficient and effective for real-time monitoring of the plankton ecosystem.

Ref. [138] analyses the performance of Transformer architectures on a plankton dataset, comparing their classification accuracy and computational speed with various CNN and Transformer neural networks. They propose a novel inter-class similarity distillation algorithm based on feature prototypes, allowing a smaller network to improve on plankton recognition guided by a larger network. The feature extraction capability of Swin-B was chosen after comparing a selection of Transformer neural networks and CNNs, which was afterwards transferred to five lighter networks and evaluated on a taxonomic dataset of in situ plankton images. Finally, the four models that performed better and the Swin-B were tested on images of the coastal waters of Guangdong, China.

In [139], five experiments were carried out to assess the impact of data augmentation, preprocessing, the use of different models, the combination of models, and class clustering. Several deep learning algorithms were compared, and different methods of data augmentation, fine-tuning, and training of all layers were evaluated. Finally, the results of the networks were combined using ensemble learning, showing significant improvements in average classification rates. Models were trained using two Kaggle plankton datasets from Oregon State University’s Hatfield Marine Science Centre. DenseNet achieved success rates of 78% for the larger ensemble (118 classes) and 92% for the smaller one (38 classes).

To facilitate comparison, the main features of the previously mentioned studies are listed in

Table 4. Deep Learning techniques for plankton classification.

Ref	Dataset	Problem Type	Algorithms	Metric	Value (Highest)
[130]	WHOI-Plankton database	Plankton classification	CNN and transfer learning (training with CIFAR10)	Accuracy	92.80%
[132]	Planktonset	Plankton classification	Deep convolutional neural network	Loss	60%
[134]	Zooplankton dataset	Zooplankton classification	ZooplanktoNet	Accuracy	93.7%
[135]	WHOI, ZooScan and one by Kaggle	Plankton classification	Their ensemble method compared to AlexNet, VGG16 or ResNet50 and more	F-measure	0.953, 0.897 and 0.926
[137]	Datasets generated from the WHOI-Plankton database	Plankton classification	YOLOV3-dense	mAP	97.21%
[138]	Dataset collected by PlanktonScope in the coastal area of Guangdong	Plankton classification	Transformers (Swin-T, ViT-B, and Swin-B) and CNNs (ResNet50, ResNet101, ResNet152, MobileNet V2, ShuffleNet)	Precision/recall	92.38%, 91.73%
[139]	Dataset captured by Oregon State University’s Hatfield Marine Science Centre	Plankton classification	Inceptionv3, InceptionResNetv2, DenseNet, ResNet, and VGG-16	Precision/recall/f1-score/accuracy/loss	92%, 92%, 92%, 0.93, 0.31

.

Other papers, like [140], focused on detecting and identifying underwater plants situated on the ocean bottom, like Posidonia. They compare different ML and DL techniques’ performance. The selected methods include, on the one hand, combinations of techniques like SVM algorithm and artificial neural network (ANN), with Gabor filters, texture descriptors and co-occurrence matrix, and on the other hand, a CNN. Even though the training time was higher with the CNN, the detection hit ratio was better than with the rest of the methods.

The paper [141] goes further, as it aims to identify and segment Posidonia Oceanica semantically. To do this, they use a network architecture that can be divided into an encoder and decoder, called VGG16-FCN8 (for which good results have been seen above [142]). They used six datasets to train and test the network and, in turn, different case studies. All of them obtained an AUC (Area Under the Curve) value of over 95%.

The study in [143] presents a deep learning-based segmentation system that analyses surveillance camera images to detect Posidonia, as well as providing information on coastal features such as shoreline changes, erosion and sediment accumulation. The system was tested on images of Torre Canne beach, in Puglia, Italy, provided by the “former Puglia Interregional Basin Authority”. The convolutional neural networks used in the analysis include Detectron2, Mask R-CNN and a proprietary CNN model, which obtained the highest results.

This paper [144] proposes an unsupervised learning framework to classify underwater lake vegetation without labelled data. It applies ConvNeXt for features, reduces dimensions with UMAP (Uniform Manifold Approximation and Projection), and clusters using multi-algorithm voting. The approach enables high-precision classification (up to 97.32% accuracy) and supports the construction of unbiased, large-scale ecological datasets. It was validated on public and private datasets from different lake environments. The method reduces labeling efforts, offering a cost-effective solution for lake ecosystem monitoring.

Within the study of aquatic vegetation, algae represent a key component in both freshwater and marine ecosystems. Their detection and monitoring have gained special relevance in the context of water quality control. In this regard, the authors of [145] propose AlgaeNet, a scalable deep learning model designed to detect floating algae Ulva prolifera in MODIS (Moderate Resolution Spectroradiometer) and SAR (Synthetic Aperture Radar) images. AlgaeNet builds upon the established U-Net architecture with two specific modifications: multi-channel physical information input and a new loss function designed to address the problem of imbalanced samples. The model achieved better results for the SAR image dataset, and as for the MODIS dataset, it barely achieved an Intersection over Union (IoU) value of 48.57%. Adding high-resolution SAR images increases the algae detection by 63.66% compared to MODIS images.

The study in [146] introduces SFD-YOLO (Seafloor-Debris-YOLO), an optimized deep learning framework combining super-resolution reconstruction (SRR) and object detection to automate seafloor debris monitoring in turbid waters. Using a custom dataset from Thailand’s Koh Tao Island, the RDN SRR (Residual Dense Network with Super-Resolution Reconstruction) model achieved top image enhancement (PSNR: 41.02 dB, SSIM: 95.08%). SFD-YOLO, enhanced with attention mechanisms and dynamic convolution, detected debris at 91.2% mAP on high-resolution images and 89.7% on RDN-reconstructed images. The approach significantly outperformed traditional methods, offering a cost-effective solution for marine conservation.

As part of this research [147], deep learning is used to automate eelgrass monitoring by classifying its presence or absence in underwater videos. A custom dataset (8324 images) from Danish transects was annotated via the SeagrassFinder platform. Vision Transformers (ViT) achieved top performance (96% Receiver Operating Characteristic Area Under the Curve, commonly called AUROC) with UW-enhanced images. A novel temporal coverage estimation method provided ecological insights beyond pixel-based approaches. The pipeline reduces manual annotation costs and supports scalable marine conservation efforts, demonstrating ViT’s efficacy in challenging underwater conditions.

Table 5. Deep Learning techniques for aquatic plants identification.

Ref	Dataset	Problem Type	Algorithms	Metric	Value (Highest)
[140]	Dataset obtained in the coastal areas of Mallorca	Detecting and identifying underwater plants (Posidonia)	SVM and ANN, with Gabor filters, texture descriptors and co-occurrence matrix, and on the other hand, a CNN	Mean of the detection hit ratio	Dataset1: 96.37%, Dataset2: 93.35%
[141]	Self-created with an AUV in Palma Bay, Cala Blava and Valdemossa port	Identify and segment Posidonia Oceanica semantically	Encoder and decoder, called VGG16-FCN8	AUC	95%
[143]	Images of Torre Canne beach in Puglia	Posidonia detection	Mask R-CNN, Detectron2, custom CNN	Accuracy/loss/IoU	90.85%/0.18/0.68
[144]	DeepSeagrass datasets and two private datasets: Erhai Lake and Wuhan East Lake datasets (China)	Underwater vegetation classification	ConvNeXt. UMAP and clustering (K-Means, Agglomerative Clustering and Birch)	Accuracy/precision	DeepSeagrass: 97.32 ± 0.36%/90%; Erhai Lake 92.43 ± 0.97%/93%; Wuhan East Lake: 96.15 ± 0.60%/95%
[145]	SAR and MODIS images	Ulva prolifera detection	AlgaeNet (U-Net based model)	Accuracy/precision/recall/f1-score/IoU	99.83%/95.46%/92.32%/93.86%/88.43%
[146]	Custom seafloor debris dataset from Koh Tao (Thailand), COCO and TrashCan dataset	Automated seafloor debris detection and classification	SFD-YOLO (enhanced YOLOv8)	mAP@0.5	91.2% (TrashCan pretraining)
[147]	Custom dataset from Lynetteholm project (Denmark)	Presence/absence classification of eelgrass and temporal coverage estimation in underwater videos	Different versions of ResNet, InceptionV3, DenseNet and ViT	Accuracy, AUROC, Calibration Error (CE)	ViT: 0.902/0.959/0.087

presents a comparative summary of the studies discussed above.

Since corals are vitally important to marine life, several jobs involve the task of monitoring and detecting corals.

Mahmood et al. [148] proposed a method that combined hand-crafted features and CNN features to perform coral classification. The MLC dataset was used to train and validate the method, which contains 2055 images collected over three years and annotated with nine different labels, five of the labels being corals. This automatic method is based on a pre-trained VGGnet for feature extraction and a two-layer Multilayer Perceptron (MLP) for classification. Three experiments were performed with different combinations of data and different feature representations. The highest achieved results were an accuracy value of 84.5% and an Average Class Precision (ACP) of 0.69.

The same year, they used the system proposed in their previous work [148] to detect coral reefs in images [106] with a CNN based on VGGnet. A subset of the Australian benthic Benthoz15 dataset [149] was used to fine-tune the weights of the network. They performed three different experiments considering different compositions of data from the years 2011, 2012 and 2013. For all experiments, the achieved accuracy was higher than 92%. They also analyzed unlabelled coral data from three sites of the coral reef of the Abrolhos Islands from two years, 2010 and 2013. The results concluded that the area covered by coral decreased during this period.

The presented workflow on [150] makes possible the analysis of the activity of cold-water coral polyps over a certain period of time. The CNN they use achieved an accuracy value of 0.96 on the test and validation set. The network was applied to one of the datasets obtained by the LoVe Observatory.

In [151], they demonstrated the effectiveness of using drone imagery combined with deep neural networks to monitor coral reefs, providing accurate, high-resolution information in a cost-effective manner. The study used RGB drone images captured over the North Bay coral reef on Lord Howe Island, Australia. Five sets of orthomosaic images were collected over one year using a DJI Phantom 4 Pro drone. The images were segmented into 4096 × 4096 pixel blocks for model training. A deep neural network architecture, mRES-uNet (Multi Resolution U-net architecture network), was used, which is an adaptation of the classic U-Net model for image segmentation. This improved version incorporates “MultiRes blocks” to learn features at different scales and “Res paths” to improve the connection between the encoder and decoder layers. For unbleached corals they achieved a precision of 0.96, recall of 0.92, and a Jaccard index of 0.89, whereas for the bleached corals they gained a precision of 0.28, recall of 0.58, and a Jaccard index of 0.23.

This study introduces CoralClassify, a deep learning framework using modified ResNet50 to automate coral health monitoring [152]. A dataset of 1700 images (balanced healthy/bleached classes) was compiled from Flickr, StructureRSMAS (Structure Rosenstiel School of Marine and Atmospheric Science dataset), and ReefBase. Custom augmentation resolved class imbalance, while hyperparameter optimization ensured efficiency. The model achieved 87.6% accuracy in just 15 epochs, outperforming existing methods in speed and performance. This approach enables scalable reef monitoring for conservation, with potential for mobile deployment in future work.

The HKCoral benchmark introduces a densely annotated dataset for coral growth form segmentation in challenging underwater environments [153]. It benchmarks 17 semantic segmentation algorithms, revealing significant performance gaps (best baseline: SegFormer (Segmentation framework which unifies Transformers with lightweight MLP decoders), 72.88 mean Intersection over Union; mIoU). The proposed complementary architecture, fusing original and enhanced images, achieved a mIoU value of 73.54. This approach enables accurate coral cover estimation and 3D distribution mapping, reducing sampling bias in ecological surveys. The dataset and methods advance automated coral reef monitoring for conservation.

For clarity, the works cited above are categorized and detailed in

Table 6. Deep Learning techniques for coral analysis.

Ref	Dataset	Problem Type	Algorithms	Metric	Value (Highest)
[106]	A subset of the Australian benthic Benthoz15 dataset	Detect coral reefs in images	A CNN based on VGGnet	Accuracy	92%
[148]	MLC (Moorea Labelled Coral) dataset	Coral classification	For feature extraction: Pre-trained VGGnet; for classification: a two-layer Multilayer Perceptron (MLP)	Accuracy	84.5%
[150]	LoVe Observatory dataset	Analysis of the activity of cold-water coral polyps in a certain period of time	CNN	Accuracy	96%
[151]	RGB drone images captured over the North Bay coral reef on Lord Howe Island, Australia	Coral classification	mRES-uNet	Overall accuracy/average Jaccard index	85.74%/0.56
[152]	Dataset created with images taken from Flickr, StructureRSMAS and ReefBase	Coral health classification	CoralClassify framework (Modified ResNet50)	Accuracy/Precision/ Recall/F1-Score	87.6%/87.99%/87.06%/87.37%
[153]	HKCoral Dataset (collected mostly in Hong Kong)	Coral growth form segmentation	Complementary Architecture (based on a CNN): Fuses original and enhanced underwater images to improve segmentation.	mIoU	73.54

.

5.2.2. Classification of Underwater Animal Species

Deep learning methods have demonstrated strong performance in classifying underwater images. Most of these approaches rely on CNN-based neural networks, as convolutional neural networks remain the leading model for image recognition and classification tasks. These networks contain many layers that transform their input with convolution filters of a small extent [154]. In 1998, LeCun et al. [155] presented the first CNNs called LeNet-5, even though they were already working with them in 1989 [156]. However, as with papers using classical machine learning algorithms, many other papers choose not to focus on a single technique and to compare various deep learning techniques in terms of ranking and performance results.

CNNs are extensively utilized in deep learning for the analysis of visual data, especially in tasks such as image classification and object detection. Their capacity to autonomously learn hierarchical features has made CNNs the predominant choice for classifying various species of marine animals.

Rimavicius et al. [120] performed a comparison between a DNN, DBN (Deep Belief Network) and CNN classifying Norwegian seabed species. The study utilized data gathered in the Norwegian Sea by a remotely operated vehicle (ROV). Each of the images was segmented and labelled for further classification. They generate four datasets. The first one (DS1) was composed of the patches extracted for every region of each of the images and was augmented from 4589 to 18356 samples. As the first dataset was composed of five imbalanced classes, the second dataset (DS2) was composed of the dataset DS1 and some additional samples of the less numerous classes. The dataset DS3 contained information on 111 features of each of the images. The fourth dataset (DS4) was created by reshaping image patches into data arrays. They use the CNN on the datasets DS1 and DS2, while the DNN and DBN were used on the datasets DS3 and DS4. The highest overall accuracy value they obtained was 88.74% by region and 92.78% by pixel, with the CNN on the dataset DS1.

In the paper [105], the DeCAF (Deep Convolutional Activation Feature) is applied on two datasets: Taiwan sea fish and the Monterey Bay Aquarium Research Institute (MBARI) benthic animal. Hand-designed features and CNN features classification result’s errors are compared. They concluded that even though CNN features provided satisfying results, hand-designed features worked better with low-quality or noisy images.

Liang et al. [104] present a classification system for ornamental fish, which also helps the user by controlling the temperature of the aquarium depending on the species, as well as calculating if the fish is moving correctly, in order to know if it is sick. They chose a CNN for the classification, which obtained an accuracy value of 98.5%.

In [111] they created a dataset by taking pictures of fishes they took out of the water. They trained a CNN network to be able to differentiate four classes of carp, obtaining a 100% accuracy value.

Qiu et al. [157] explore other transfer learning strategies, which consisted of pre-training the network, first on the ImageNet dataset and then pre-training it again on the Fish4-Knowledge (F4K) dataset. They implement three types of Bilinear CNNs [158] and evaluate them with different combinations of techniques for enhanced data augmentation. The implementation of B-CNN plus refined squeeze-and-excitation blocks obtained the highest results on both datasets, Croatian fish dataset [159] and QUT (Queensland University of Technology) fish dataset [160], with values of 83.92% and 71.80% of accuracy, respectively.

Another approach that employs CNN for fish recognition can be seen in [161]. They designed three different structures of CNNs and evaluated their accuracy results over several iterations. They achieved similar values of accuracy (over 96%) with two of the models, but the other model had a problem of overfitting.

Many papers like [162] examine how deep learning approaches can be an efficient solution for underwater imagery analysis. In the fish recognition task, the researchers used a CNN optimized with SGD (Stochastic Gradient Descent) on the Fish4Knowledge dataset and achieved a test accuracy of 98.57%.

Qin et al. [163] created a deep learning approach to identify live fish within the Fish4Knowledge dataset, primarily utilizing a ConvNet and then classifying with a linear SVM. The foreground of the images was extracted for input into the network using a technique grounded in sparse and low-rank matrix decomposition [164]. They attained state-of-the-art performance, achieving an accuracy of 98.57% on the test dataset.

Rathi et al. [165] employed the same dataset and developed a method integrating CNNs with advanced preprocessing methods, such as Gaussian blurring, morphological operations, and Otsu’s thresholding. They evaluate their method on the Fish4Knowledge dataset, using Adam optimizer and three different activation function: ReLU (Rectified Linear Unit), Softmax and tanh. They achieved an accuracy value of 96.29% using the ReLu activation function, while with the tanh and Softmax did not reach the 75% of accuracy.

Understanding the behaviour of animal species can help discover if there are problems or abnormalities in the environment that lead to changes in their routines. Related to this, in article [166] they prepared a water tank to monitor a small school of fish: to watch their movement, their speed when eating or resting, etc. With the help of a CNN, they performed the classification of six types of fish behaviour. They achieved a test accuracy value of 0.825.

Although not directly related to diseases, paper [167] does seek to create a solution to improve the quality of life and health of certain fish. In fishing industry one of the main challenges is production loss. This problem is usually caused by bad handling of the fish; thus, it can be very stressful, it may end in fish death. In order to detect that kind of stressful situations, Jovanović and his fellows presented a novel algorithm based on using of CNNs for splash.

A paper dealing with this topic is [168], in which they present an improved version of convolutional network based tracker (CNT), called Fast-CNT2. One of the improvements of this network is that can perform multi-target tracking. For background modelling they use the improved GMM as the processing technique that does not require training.

Although it is not a very common task, probably due to the complexity it can entail, some articles investigate the genus classification of specimens using DL techniques. In [169], they introduce a convolutional neural network architecture that is tailored for classifying the gender of Chinese mitten crabs using images of their shell and abdomen. Their network obtained higher accuracy values than other models such as BPNN (Back Propagation Neural Network) and CNN-SVM.

The studies cited and analyzed in the previous paragraphs and whose methods are based on CNNs are organized and detailed in

Table 7. Application of CNNs in underwater images.

Ref	Dataset	Problem Type	Algorithms	Metric	Value (Highest)
[104]	Images from the internet	Ornamental fish classification	CNN model	Accuracy	98.5%
[105]	Taiwan sea fish and Monterey Bay Aquarium Research Institute (MBARI) benthic animal	Underwater species classification	The DeCAF (Deep Convolutional Activation Feature) framework (CNN)	Overall error	2.09 ± 0.51%/0.83 ± 1.78%
[111]	Self-constructed	Carp species classification	CNN-based method	Accuracy	100%
[120]	Data collected in the Norwegian sea by a remotely operated vehicle (ROV)	Classify Norwegian seabed species	DNN, DBN and CNN	Accuracy	88.74% by region and 92.78% by pixel
[157]	Train on ImageNet and F4K, Test on the Croatian fish dataset and QUT fish	Fish classification	Three types of Bilinear CNNs	Accuracy	83.92% and 71.80%
[161]	SeaCLEF	Fish species classification	CNN	Accuracy	96%
[162]	Fish4Knowledge	Fish recognition	CNN with SGD as the optimizer	accuracy	98.57%
[163]	Fish4Knowledge	Live fish recognition	Deep architecture for live fish recognition composed principally of a ConvNet and a linear SVM	Accuracy	98.57%
[165]	Fish4Knowledge	Fish classification	CNNs and several preprocessing techniques like Gaussian blurring, morphological operations and Otsu’s thresholding	Accuracy	96.29%
[166]	Own dataset (water tank)	Fish behaviour classification	CNN	Accuracy	82.5%
[167]	Videos collected in a water tank	Anomaly detection in carp and koi ponds by analyzing behaviours	Method based on CNN	Accuracy	99.9%
[168]	Videos obtained at the culture pond of the Department of Aquaculture of National Taiwan Ocean University	Detect stressful situations when fishing fish in industry	An improved version of convolutional network-based tracker CNT (Fast-CNT2)	-	-
[169]	Self-created dataset	Chinese mitten crab’s gender classification	Custom CNN	Accuracy	98.90%

.

VGG16 is a convolutional neural network model proposed by K. Simonyan and A. Zisserman [170] widely used in image classification tasks. In [171], authors use a CNN based on a VGG-16 for the classification of freshness in dead fish, which can be an interesting issue in the fishing industry. They achieved high accuracy value: 98.21%. This method could be applied to other fish diseases that can be seen with the naked eye. Figure 5 shows the architecture of a VGG-16.

Becken et al. performed a research with two objectives related to the aesthetic value of the Great Barrier Reef (GBR) [172]. The first objective has the aim of determine the aesthetic value using eye tracking over photos from the GBR, while the second objective consist in the recognition of objects in underwater images of the GBR and automatically assessing their aesthetic value. The images they used were taken from different sources, and many of them were photoshopped to add species or remove them. For the first objective, 21 images were used, and 705 usable surveys were performed, but for the second one they created a dataset composed of 4909 images with 50 different species to detect and classify. Three different CNNs were compared in the experiments carried out to detect and classify species: ZF, CNN-M and VGG-16. The VGG-16 achieved the highest value of mAP (0.824) in the whole sample.

The work in [103] is part of the development of the FishCam monitoring system [173] that is designed for semi-automatic monitoring of fish migration, and it describes a second classification step of the study that classifies the different fish species. As training data, they used the images extracted from the videos their collected in a controlled environment created in a river for three years, as well as the measures of the length of the fish and the date of the migration. A pretrained VGG-16 in the ImageNet dataset [174,175] was used for classification. The highest accuracy was obtained by the network configuration where the classification is based on the images, in addition to the length and the date; they achieved an accuracy of 89.4%.

In [176], the authors developed a system for fish detection and classification, composed of two branches, one for image-level classification and the other for instance-level classification. The image-level classification estimates a probability value for each of the classes the image may belong to, considering all the elements found in the image. The instance-level classification performs fish detection, object pose estimation and horizontal alignment before class prediction is made. The final prediction is the average value of the classification made in the two branches. Object detection was performed by SSD and YOLOv2. They evaluated three different CNNs in the instance classification problem: ResNet50, VGG-16 and InceptionV3. In the image-level classification, just a fully convolutional network (FCN), which was a modified VGG-16, is used. The adaptive prediction average achieved the lowest loss value, which was 0.604.

In [177], authors use a dataset provided by Kaggle, which contains 3777 images extracted from video footage of a fishing boat. They generated other datasets, one with annotated images and the other two applying transformations to the original and annotated images, creating two datasets with noisy images, containing 12,275 images each one. For classification, two VGG-16 networks were used, one of them with transfer learning from a pre-trained network on the ImageNet dataset. Better results were obtained by the simple VGG-16 model for the dataset composed of noisy original images, achieving 99.38% of accuracy, and values of precision, recall and F-score above 0.91. One reason why transfer learning did not work so well could be because of the difference between the images contained in ImageNet and those used for testing.

In [178] they propose a combination of VGG-16 with Darknet for fish classification. The training session lasted 72 min for 500 epochs covering the 951 photographs. The annotated dataset was downloaded from roboflow.ai. These annotations included bounding boxes to locate fish within the images and labels to classify them into one of three classes: “Adipose,” “Non-Adipose,” or “Unknown” (representing fish that were out of view). The precision they achieved was 0.7.

The study in [179] developed and compared two CNN architectures for fish species classification: VGG-16 and a reduced version of VGG-16, VGG-8. The latter is a simplified version of VGG-16, specifically designed to reduce the time and memory resources required for training, without significantly compromising accuracy. VGG-8 has a total of 8 layers, with 6 convolutional layers and 2 fully connected layers, making it more efficient in terms of training time and memory usage. The VGG-16 architecture achieved high classification accuracy with micro-average, recall, and F1-score accuracy. However, it required more training time and memory resources due to its higher depth and number of parameters. The scaled-down version, VGG-8, also showed competitive performance with micro-average, recall, and F1-score accuracy. Although the accuracy was slightly lower than that of VGG-16, VGG-8 was significantly more efficient in terms of time and resource usage.

The researchers in [180] developed a modified version of the VGGNet model, named MLR-VGGNet. It incorporates multi-level residual (MLR) blocks to improve feature extraction capabilities. Several models were compared, including VGG16, VGG19, ResNet50, Inception V3, Xception, and the newly proposed MLR-VGG16 and MLR-VGG19 (Multi-Level Residual VGG) models. Both models achieved high accuracy values on both datasets, outperforming existing models such as ResNet50 and Inception V3. The proposed MLR-VGG16 model achieved the best performance with a test accuracy of 98.46% on the Fish-gres dataset [181], outperforming other state-of-the-art models.

In [182] we can find another of the few studies related to fish disease detection. The authors compared the performance of multiple algorithms, both ML and DL. Neural networks obtained better precision, recall, and f-score values for each class, in addition to achieving higher accuracy values. The ResNet-50 model obtained an accuracy of 99.28%, making it one of the best pre-trained models for this task, although the VGG16+VGG19 combination achieved the highest precision.

Table 8. Application of VGGs in underwater images.

Ref	Dataset	Problem Type	Algorithms	Metric	Value (Highest)
[103]	Self-constructed dataset from Austrian river	River fish classification	Pretrained VGG-16	Accuracy	89.4%
[171]	Dataset obtained from a fish farm (Khorramabad, Lorestan province, Iran)	Freshness diagnosis of common carp	Method based on CNN (VGG-16)	Accuracy	98.21%
[172]	Images from Flickr	Aesthetic value of underwater images and species recognition	ZF, CNN-M and VGG-16 Conditional generative adversarial network (cGAN)	mAP	82.4%
[176]	Kaggle fishing boat dataset	Fish detection, object pose estimation and horizontal alignment before class prediction	SSD and YOLOv2 as detectors, VGG-16 as pose estimation, VGG-16 and Inception V3 as classification	Loss	60.4%
[177]	Kaggle fishing boat dataset	Fish classification	Two VGG-16 networks (one with transfer learning)	Accuracy	99.38%
[178]	Fish Image dataset from roboflow.ai	Fish classification	VGG-16 and Darknet	Precision/recall	70%/80%
[179]	Fish4Knowledge	Fish classification	VGG-8 and VGG-16	Micro average: Precision/recall/f1-score	99%/99%/99%
[180]	Fish4Knowledge and Fish-gres dataset (out of water fish images)	Fish classification	MLR-VGG16 and MLR-VGG19	Accuracy	Fish-gres: 98.46%/Fish4Knowledge: 97.09%
[182]	Dataset collected from various public sources	Fish disease classification	ML algorithms, VGG16, VGG19, ResNet-50, VGG16+VGG19 and VGG16+Inception V3	Accuracy	99.64%

summarizes the most relevant VGG-based approaches for underwater image classification, including the corresponding references discussed above.

AlexNet, GoogLeNet (Inception), and ResNet represent landmark convolutional neural network architectures that have significantly advanced image analysis tasks.

Sungbin Choi describes the work done by his team in the LifeCLEF fish task 2015 [183]. They performed foreground detection and fish species classification with the GoogleNet (pretrained on ImageNet). As the images used are temporarily connected (since they have been obtained from videos), the classification result has been refined by comparing the contiguous frames of each k-frame. The best precision result obtained was 0.81.

Jäger et al. described in their paper the results obtained in the recognition task in SeaCLEF 2016 [184]. For the generation of bounding boxes, they use adaptive background mixture models, followed by erosion and a blob detection method to filter the smallest blobs. The generated proposals are utilized to extract CNN features, specifically from AlexNet (pretrained on Large Scale Visual Recognition Challenge 2012 dataset, better known as ILSVRC 2012). Finally, they used an SVM for discriminating between fish and background. They obtained a precision of 0.66, a lower value than in 2015, which was of 0.81. In 2017, they introduced a multi-object tracking approach that built upon the improved method developed by Mothes and Denzler [185], as well as their own earlier research. The method combines CNN activations with a two-stage graph-based tracker, which deals with the problem of object’s occlusion [186]. To compare their method results, they used a subset of the SeaCLEF 2016 dataset. Even their approach achieved a Multi-Object Tracking Accuracy (MOTA) value of 87.6, they did not exceed the result obtained by the method of Mothes and Denzler.

The filtering deep convolutional network (FDCNet) is a proposed model for classifying underwater species [119]. They also build a dataset called Kyutech10K, which contains seven classes, 10,728 images, and 1489 videos derived from the database of the Japan Agency for Marine–Earth Science and Technology (JAMSTEC). The proposed structure overcomes the underwater degraded images descattering problem due to the use of the Underwater Dark Channel Prior (UDCP) estimator and deep convolutional neural fields (DCNF), which calculate the disparity and remove the noise from the images. Then, a modified GoogleNet performs classification. With that combination of techniques, they achieved an accuracy rate exceeding 92%.

Even though aerial drones are useful for recording images of the sea from the top, the development of underwater drones enables recording images of organisms under the sea, directly from their environment, which could be more representative. The authors in [107] developed an underwater drone that has a 360° panoramic camera embedded. However, they performed classification on a dataset collected from a Google search, composed of 100 images of each of the four kinds of fish, which they augmented up to 86.400 images. The recognition accuracy rate they achieved was 87%, 85% and 67% for the three networks they used, AlexNet, GoogleNet and LeNet, respectively. Although data augmentation is a common strategy for addressing the problem of limited data availability, extreme data augmentation can have adverse consequences. Operations such as rotation and blurring can improve training stability, but they do not generate entirely new samples, increasing the risk of overfitting. Furthermore, when the original images are obtained from controlled sources (for example, in aquariums or in properly lit areas) and high accuracy scores are achieved, this does not necessarily mean that the model will generalise correctly in real-world aquatic conditions.

Many papers have shown that transfer learning can obtain better results when classifying, like [187], where they compare two networks, AlexNet and GoogleNet, following this strategy. The dataset they used was provided by The Nature conservancy through the Kaggle competition “The Nature Conservancy Fisheries Monitoring” (https://www.kaggle.com/c/the-nature-conservancy-fisheries-monitoring, 3 May 2026.). Without transfer learning, they barely reached a success rate higher than 80%, whereas with it, they achieved an accuracy of at least 96% with both networks.

This paper [112] introduces SuperFish, a mobile app for recognizing 38 Mauritian fish species using dual approaches: traditional computer vision (kNN, 96% accuracy) and deep learning (Inception-v3, 98% accuracy). The custom dataset (1520 images) was processed via contour extraction and geometric/colour features for traditional methods, while deep learning used transfer learning.

The most recent studies on underwater image classification and detection tend to employ more advanced deep learning architectures than those used in earlier work. Whilst more classical networks such as AlexNet provided a fundamental starting point for the adoption of convolutional networks in this domain, subsequent research has exploited deeper and more specialised architectures, incorporating attention mechanisms, transfer learning and optimised design for complex underwater scenes. Consequently, the performance differences observed between older and more recent studies should be interpreted as the result of the natural evolution of architectures and available resources, rather than as a direct comparison between methods evaluated under equivalent conditions.

Among these more recent studies is the one presenting DAMNet, wich introduces a dual-attention mechanism, CBAM (Convolutional Block Attention Module) with multi-stage stacking to classify underwater biological images [188]. It achieves 96.93% accuracy on a 7-class dataset, outperforming benchmarks like ResNet50 (91.11%) and GoogLeNet (95.24%). The Gravity Optimizer reduces loss (0.1860) and accelerates convergence. Though effective overall, fish classification lags (91.84%) due to intra-class diversity. The model addresses challenges like turbidity and low contrast in marine imagery.

The authors of [189] address the task of distinguishing raw underwater images from enhanced ones, which is relevant for underwater image processing pipelines. The authors propose a modified ResNet-18 that augments a pre-trained backbone with additional fully connected layers to better model underwater degradations. Transfer learning is applied by freezing early layers, while newly added layers focus on noise, colour distortion, and illumination variability. Class imbalance is addressed through data augmentation and downsampling. Experiments on SAUD (Subjectively Annotated UIE benchmark Dataset) and MSRB (Marine Snow Removal Benchmarking) datasets show that the approach outperforms standard CNN and Transformer-based baselines while remaining computationally lightweight.

Table 9. Application of deeper CNNs in underwater images for classification and detection.

Ref	Dataset	Problem Type	Algorithms	Metric	Value (Highest)
[107]	Images from Google Search (86.400 with data augmentation)	Marine species classification	AlexNet, GoogleNet and LeNet	Accuracy	87%
[112]	Dataset collected mostly from open fish markets	Identification of fish species which are found in Mauritian waters	InceptionV3 model	Accuracy	98%
[119]	Kyutech10K dataset (10,728 images and 1489 videos)	Underwater species classification	A modified GoogleNet	Accuracy	92%
[183]	Pretrained on ImageNet, SeaCLEF	Fish species classification	GoogleNet	Precision	81%
[184]	SeaCLEF 2015	Fish species classification	AlexNet, SVM	Precision	66%
[186]	SeaCLEF 2016	Fish detection and classification	A method based on the refined method of Mothes and Denzler [185]	MOTA value	87.6%
[187]	Kaggle fishing boat dataset—The Nature conservancy	Fish classification	AlexNet and GoogleNet	Accuracy	+96% (with transfer learning)
[188]	Dataset created from multiple sources: OceanDark, RUIE, UIEB, UFO-120, EUVP	Multiclass classification: Fish, Turtles, Sea Urchins, Sea Cucumbers, Corals, Humans, Wreckage	DAMNet	Overall Accuracy/Precision/Recall/F1-Score/Loss	96.93%/96.70%/96.78%/96.74%/ 0.1860
[189]	SAUD [190], PHISMID [191], MSRB [192]	Binary underwater image classification (raw vs. enhanced)	Modified ResNet-18	Accuracy, Precision, F1-score, AUC-ROC	SAUD: 96%/99%/95%/96%

presents a comparative summary of the studies that apply complex and deep neural networks (such as ResNet or Inception) analysed previously.

5.2.3. Object Detection and Segmentation Applied on Aquatic Images

Object detection presents greater complexity and challenge compared to classification, as it requires not only identifying the presence of an item within an image but also accurately classifying it. In marine environments it is even more complicated because it is a hostile environment, and the images used usually contain a lot of noise and can be blurry, and they are often not well lit.

Although object detection can be accomplished with HOG features, contemporary methods are typically divided into two main categories, both leveraging deep learning approaches: region-based convolutional neural networks and regression-based object detectors. Region-based methods include models such as R-CNN [43] and its faster versions, Fast R-CNN [193] and Faster R-CNN [194]. Spatial Pyramid Pooling (SPP-net) and Mask R-CNN [195], among other object detectors, can also be included in this family.

In [196] they propose PVANet, a lightweight deep neural network designed specifically for detecting fish in underwater images. It integrates C.ReLU (to reduce early-stage computation), Inception modules (for multi-scale features), and HyperNet (to fuse hierarchical features). Evaluated on the ImageCLEF dataset (24 k images, 12 species), it achieves 89.95% mAP, surpassing Faster R-CNN by 7.25% with faster inference (0.089 s/image). The design addresses missed small-fish detection in prior work through optimized multi-scale feature extraction.

Region-Based Convolutional Neural Networks (R-CNN) generate a collection of bounding boxes from an input image, with each bounding box enclosing an object and indicating its classification. This family of nets is slower compared to others, but obtains high classification and detection results, so it has been widely used in the underwater field. There are other faster variants, such as Fast R-CNN, where the convolution operation is performed only once per image and a feature map is generated. Faster R-CNN is considerably faster than its predecessors, since instead of using a selective search algorithm to identify region proposals as R-CNN and Fast R-CNN do, a separate network called RPN (Region Proposal Network) is used.

Controlling the population of many species of fish in some areas is a necessity, because many of them have adapted to other habitats in which they do not have predators, thus their population would increase heavily. In [114], they designed a real-time detection system using an underwater robot (called OpenROV) and implementing a red lionfish detection system, for dealing with the problem of this specie’s invasion. The detection is made by a Matlab implementation of a R-CNN, which was trained with 1000 frames. They obtained a true positive detection rate of 93%.

Li et al. [197] employed the Fast R-CNN framework for fish detection and identification on a subset of the SeaCLEF 2014 dataset. The model architecture was based on AlexNet, which had been pretrained on the ImageNet (ILSVRC2012) dataset. Performance analyses were conducted by comparing the Deformable Part Model (DPM), Region-based CNN (R-CNN), and two implementations of Fast R-CNN; one incorporating truncated Singular Value Decomposition (SVD) to compress fully connected layers. Fast R-CNN yielded the highest mean Average Precision (mAP) at 81.4%, followed by R-CNN at 81.2%, and Fast R-CNN with SVD at 78.9%. Although the Fast R-CNN with SVD had slightly lower precision than R-CNN, it demonstrated substantially greater training and testing efficiency.

The study in [114,198] presents an enhanced Faster R-CNN model for identifying marine organisms, including sea cucumbers, sea urchins, scallops, starfish, and algae. They improve the model replacing the VGG16 network by Res2Net101 in the feature extraction module; the Online Hard Example Mining (OHEM) algorithm is used to address positive/negative bounding box imbalance; and the use of IoU was replaced by Generalized IOU (GIOU) and implementing Soft-NMS (non-maximum suppression) over standard NMS, which is simpler and easily adapted to different object detection algorithms. Initially, 2372 image samples of underwater environments (such as sea cucumbers, sea urchins, scallops, starfish and algae) were collected. Data augmentation was performed using the Mosaic technique, which consists of randomly combining four images into one by random arrangement, zooming and cropping. Through ablation experiments (removal tests), the impact of each improvement was evaluated independently. The results showed that the improved model achieves a mAP@0.5 of 71.7%, 3.3% higher than the original Faster RCNN model.

The research in [199] proposed two multi-stream fusion approaches based on Faster R-CNN: Shared RPN Fusion, which uses a single region proposal network (RPN) shared between two CNNs to improve the detection of moving objects; and Shared Classifier Fusion which uses two RPNs, one for appearance and one for motion, but shares a single classifier. Using the LifeClef 2015 Fish (LCF-15) dataset, the Shared RPN Fusion architecture achieved an F-score of 83.16%, surpassing other leading fusion methods for underwater fish detection.

In [115] they present the Video and Image Analytics for Marine Environments (VIAME) open-source computer vision library. This platform contains several modules to detect and classify fishes or other organisms; while some of them are specific to individual applications, others are more general. Faster R-CNN was also added to the system as an object detector due to its generality. They performed the network training on data collected at the Monterey Bay Aquarium Research Institute. In [116] they also used a Faster R-CNN for object detection and fish abundance estimation. They used three classification models from [170,200,201] (which are CNNs) in combination with the RPN (Region Proposal Network). The second experiment performed, in which only species with acceptable training samples were considered, obtained better results. The model in [170], a modified VGG-16, achieved a mAP value of 0.824.

The aim of the study in [202] is to detect and analyse the behavioral trajectory of fish in a simulated aquatic environment with different levels of ammonium chloride to observe how these conditions affect fish behavior and vitality. To do so, they compare the performance of Faster R-CNN and YOLOv3. The experimental results indicate that fish movement decreases significantly in the presence of ammonia, with fish remaining motionless for longer as the ammonia concentration increases. Faster R-CNN achieved a higher average recognition success rate, but YOLOv3 showed superior efficiency in detection speed.

Detecting small animals, such as shrimp, is more complex, as they can sometimes blend into the background if they do not stand out too much and models are not able to generalize well. The dataset used in [203] includes 20,000 images of shrimp captured in various aquatic conditions. The images were obtained in RGB and grayscale, with different formats and lighting conditions, both during the day and at night, to increase the robustness of the model. They built a CNN using a transfer learning technique based on the Faster R-CNN and InceptionV2 architecture. The model showed good real-time performance, although with a slight delay in frames per second (FPS), resulting in a slow-motion video effect.

This study [204] uses Faster R-CNN to automatically detect, identify, and count 11 deep-water snapper species in BRUVS footage from New Caledonia. Using 12,100 annotated fish images, the model achieved strong performance for well-represented species (F-measure up to 0.87). A semi-automatic protocol (expert verification of AI detections) significantly improved accuracy (F-measure up to 1.0). Automatic abundance estimates correlated highly with manual counts (*r* = 0.85), rising to 0.96 in semi-automatic mode, demonstrating feasibility for fisheries monitoring. The approach addresses data scarcity in deep-sea fisheries management while reducing manual processing time.

Residual Neural Network (ResNet) was presented in [205] with an innovative architecture designed to address the issue of vanishing or exploding gradients. This network uses the technique called skip connections, which skips training from a few layers. Several papers have used this network to solve problems in underwater images. Some papers have focused on the detection and counting of jellyfish, such as in [118]. This paper introduces a system called Jellytoring, which uses a deep object detection neural network to automatically identify and measure various jellyfish species. The researchers chose to implement the Faster R-CNN model with Inception ResNet v2. They reached an F1 score of 93.8% in the jellyfish detection task. In 2017, Labao et al. [117] proposed a 152-layer Fully Convolutional Residual Network (ResNet-FCN) [206] for fish segmentation and identification. Six videos were chosen from various locations within the Verde Island Passage (Philippines). Each video presents unique challenges, including abrupt changes in illumination, reduced visibility, or the presence of marine snow; a phenomenon resulting from suspended particles in underwater environments. Due to the quantity of frames, they used weakly labeled ground truth derived from the adaptative Gaussian mixture-based background subtraction [207]. The average precision obtained was 65.91%, since although the accuracy obtained for most of the sites was greater than 70%, for some of the sites it was quite low. However, the achieved average recall was higher, about 84%.

The articles in which different types of R-CNNs are applied in underwater images are listed in

Table 10. Application of region-based networks in underwater images.

Ref	Dataset	Problem Type	Algorithms	Metric	Value (Highest)
[114]	OpenROV images	Marine species detection	R-CNN	True positive detection rate	93%
[115]	Data from Monterey Bay Aquarium Research Institute	Detect and classify fishes or other organisms	Video and Image Analytics for Marine Environments (VIAME) open-source computer vision library	ROC (AUC)	70%
[116]	Videos obtained in marine waters of beaches and estuaries across southeast Queensland	Object detection and fish abundance estimation	Faster R-CNN	mAP	82.4%
[117]	Six videos recorded at various locations within the Verde Island Passage, Philippines	Fish segmentation and identification	Fully Convolutional Residual Network (ResNet-FCN)	Precision/accuracy/recall	65.91%/70%/84%
[118]	Self-created with a GoPro at the seafloor	Jellyfish detection	Faster R-CNN-based implementation of the Inception ResNet v2	F1-score	93.8%
[196]	ImageCLEF	Fish detection and classification	PVANet to DPM, R-CNN, Fast R-CNN and Faster R-CNN	mAP	90%
[197]	SeaCLEF 2014	Fish detection and classification	Fast R-CNN based on an AlexNet	mAP	81.4%
[198]	Self-created underwater dataset	Underwater marine species detection and classificarion	Faster RCNN with Res2Net101	Average precision/mAP/f1-score	43%/71.7%/55.3%
[199]	LifeClef 2015	Fish detection and segmentation	Two multi-stream fusion approaches based on Faster R-CNN	F1-score/mAP	83.16%/73.69%
[202]	Self-created in a tank at laboratory	Fish detection and trajectory analysis	Faster R-CNN and YOLO-V3	Accuracy/proportion of lost points	98.13%/1.87%
[203]	Images downloaded randomly from various sources	Shrimp detection	Faster R-CNN InceptionV2	Precision/recall/f1-score/accuracy	97%/97%/96%/96%
[204]	Underwater baited video footage from New Caledonia (South Pacific)	Automated detection, species identification, and counting of deep-water snapper species	Faster R-CNN with Inception-ResNet V2 backbone	Recall/Precision/F-measure	Best specie (Etelis coruscans): 0.91/0.84/0.87

.

The second category of detectors approaches object detection as a direct regression problem, simultaneously predicting object classes and bounding box coordinates. Representative models in this group such as YOLO (You Only Look Once) [44] (as well as the improved versions, like YOLO9000 [208], YOLOv4 [209] and the latest YOLOv9 [210]) and SSD (Single Shot Multibox Detector) [45] detect objects by processing information through the network just once. These detectors are faster than the R-CNN family, but they are not as accurate detecting small objects.

SSDs perform object detection in a single forward pass through a neural network by simultaneously predicting bounding boxes and class probabilities at multiple scales. Unlike two-stage detectors (like R-CNN family), SSDs eliminate the need for a separate region proposal step, resulting in faster inference while maintaining competitive accuracy, especially for real-time applications. Figure 6 shows the operation of the YOLO network detection operation.

The Kyutech10K dataset was partially used in [211], for training a system for underwater real-time recognition and tracking of four different organisms using their previous underwater image enhancement method [72] and YOLO. They compared the performance of YOLO with other two methods: Tracking Learning Detection (TLD) [212] and MedianFlow [213]. YOLO and Medianflow obtained good results, but YOLO showed many advantages, such as the adaptative bounding boxes in different frames for the same object. While YOLO achieved relatively high precision and recall scores for each species, it failed to detect some objects and could not continue with tracking in those cases. There are other jobs that use tracking to monitor fish health, like work done in [214]. This paper introduces a framework for detecting anomalies in carp and koi pond health monitoring by analyzing the behaviours of these species, quantifying changes in their movement patterns, and identifying unusual behaviours through tracking. The framework is based on YOLOv3.

The authors in [176] developed a fish detection and classification system that relies primarily on object detection techniques, specifically SSD and YOLOv2. The system features two branches: the image-level branch provides overall class probabilities based on all elements in the image, while the instance-level branch focuses on detecting individual fish, estimating their pose, and aligning them horizontally before classification. Three CNN architectures were tested for instance-level classification, whereas the image-level classification used a fully convolutional network based on a modified VGG-16. By combining predictions from both branches through adaptive averaging, the system achieved a minimum loss of 0.604, demonstrating the effectiveness of SSD and YOLOv2 for accurate fish localization and subsequent classification.

In [215] they present two improved models (YOLO-Fish-1 and YOLO-Fish-2) that are based on YOLOv3, which is much lighter than its next version, YOLOv4. YOLO-Fish1 and YOLO-Fish2 achieved an average accuracy of 76.56% and 75.70%, respectively, for fish detection in real marine environments, which is significantly better than YOLOv3. However, they did not manage to surpass the results of YOLOv4, since it reached 81.02% average precision.

In [216] they adapted YOLOv3 by training it with both real and synthetic images, using lobster parts placed on varied backgrounds from the Benthoz15 underwater set. The synthetic data approach allowed to address the variability and complexity of the underwater environment and improve the model’s capacity to detect lobsters even when they are partially hidden in images.

In [217], a streamlined model was introduced using MobileNetv2 in combination with YOLOv4. They tested their network on three datasets: PASCAL VOC (Pattern Analysis, Statistical Modelling, and Computational Learning Visual Object Classes dataset), Brackish [218] and Underwater Robot Picking Contest (URPC 2020) datasets. They reached a mAP of 81.67% and 92.65% on the PASCAL VOC dataset and brackish dataset, respectively. On the URPC dataset the mAP achieved was lower: 79.54%. In [219] they have chosen the latest version of YOLO optimized with Res2Net to perform fish detection on a self-built fish dataset, reaching 95.4% mAP.

The authors of [220] investigate the detection and classification of seven species of jellyfish and fish: Cyanea purpurea, Rhizostoma pulmo, Phacellophora camtschatica, Agalma okeni, Aurelia aurita, Phyllorhiza punctata, Rhopilema esculentum, and fish. The dataset used in this study contains 11,926 images that were obtained through tracking technology and laboratory captures, and were divided into training, verification, and test sets. The developed model is an improved version of the YOLOv4-tiny algorithm, which is adapted for real-time jellyfish detection. Improvements include the addition of a CBAM [221] attention mechanism module to improve the feature extraction capability, especially for small and hidden targets. The improved YOLOv4-tiny algorithm achieved a detection accuracy of 95.01% and a detection speed of 223 FPS (frames per second), outperforming other compared algorithms such as YOLOv4, YOLOv5, YOLOv6, YOLOv7 and YOLOv8.

Other research on a version of a YOLO algorithm applied to jellyfish detection and classification can be found in [222]. The model developed in this study is JF-YOLO, based on an improved version of the YOLOv4 model for jellyfish detection. The dataset used presents challenges such as turbid water, high jellyfish density, pixel blur, transparency, scale variations, and illumination. Improvements to the developed model include, among others, residual dilated convolution (RDC) and super feature creation for feature fusion, and modification of PANet by removing SPP (Spatial Pyramid Pooling) to better handle scale variation and high target density. The JF-YOLO model achieved a recall of 85.74%, outperforming various versions of YOLOv4 and YOLOv3. In addition, JF-YOLO showed real-time performance with a speed of 41 FPS.

The study in [223] used a dataset composed of 1462 images of green turtles (Chelonia mydas) taken on Little Liuqiu Island, Taiwan. The images were collected using drones (UAVs) and underwater cameras between 2016 and 2021, along with additional images obtained from a local Facebook group. The study evaluated and compared three object detection models based on deep learning algorithms: YOLOv3, YOLOv5s, and YOLOv5l. The study concluded that, despite its more complex architecture and higher number of layers, YOLOv5l underperformed compared to YOLOv5s and YOLOv3, which stood out for the high accuracy achieved. YOLOv3 performed best on the smaller dataset of 781 images, reaching 98.25% precision, 97.68% recall, and a 97.96% F1-score.

The authors of [224] seek to determine how many shrimps are present in culture tanks by analyzing images, and they assess the precision and efficiency of various deep learning methods and models. The main dataset includes 1379 images of shrimp in RAS culture tanks, captured with an iPhone 11 mini. Two variants of the Faster R-CNN model and two variants of YOLOv5m6 were used, with the latter achieving the best mean absolute percentage error (MAPE) value.

In [225] they developed a model called NAM-YOLOv7, which is an improvement of the YOLOv7 algorithm that incorporates a normalization-based attention module (NAM) [226] that improves detection accuracy by focusing on relevant features of fish images. The aim of the study is the rapid detection of fish with Spring Viremia of Carp (SVC) symptoms in aquaculture. To do so, they created a dataset of 1814 images obtained from videos of fish, specifically zebrafish, infected with the SVC virus in a controlled environment.

The following study also focused on the detection of different types of turtles, although this time the images were obtained using a binocular camera in a controlled environment with a blue PVC background [227]. The dataset utilized in this research comprises more than 11,000 images of Chinese soft-shelled turtles. An enhanced object detection model, referred to as YOLOv7-SS, was developed based on the YOLOv7 architecture. This model integrates both the SE (Squeeze-and-Excitation) and SimAM (Simultaneous Attention Mechanism) attention mechanisms. These attention mechanisms improve the model’s ability to focus on relevant features in turtle images, thus improving the accuracy of plastron and shell detection.

Diffusion models are a class of generative models that create data by progressively transforming noise into structured outputs through a learned denoising process. Inspired by thermodynamic diffusion, these models iteratively reverse a gradual noising procedure applied to training data, enabling high-quality image, audio, and video synthesis. Diffusion models have recently gained popularity due to their ability to generate diverse and realistic samples, often surpassing traditional GANs in stability and image fidelity. Their flexible framework supports conditional generation, inpainting, and super-resolution, making them powerful tools in modern generative modelling. This research introduces AIT-YOLOv7 (Advanced imaging technique-YOLOv7), a novel framework combining diffusion models, CBAM, and MSTB (Modified Swin Transformer Block) with YOLOv7 to enhance underwater object detection [228]. It addresses challenges like turbidity and low visibility using the TrashCan dataset (22 classes, 7212 images). The system achieves 81.4% mAP@0.5, outperforming other methods by up to 24.21%. This advancement supports marine conservation by enabling the precise identification of marine debris and biological organisms. The integration of denoising, resolution enhancement, and real-time tracking significantly improves reliability in underwater environmental monitoring.

A fairly updated version of YOLO is used in [229]. The model developed in the study is YOLOv8m, a medium-sized variant of the YOLOv8 model, which is based on the Darknet architecture. In order to improve underwater object detection, image enhancement techniques such as contrast stretching and gamma correction were applied to the images. The dataset employed in this study comprises 1434 images representing diverse fish species, including manta rays, jellyfish, and penguins, among others, organized into seven distinct classes for detection purposes. The YOLOv8m model reached an F1 score of 64.31%, outperforming Faster-RCNN (52.8%) and SSD (54.5%) in precision and recall balance.

The paper [230] introduces YOLOv8-TF (Transformer-enhanced YOLOv8), an advanced object detection model designed for identifying fish species in underwater environments. By incorporating transformer blocks into YOLOv8, the model effectively captures global context, and its class-aware loss mechanism addresses significant class imbalance found in the SEAMAPD21 dataset (Southeast Area Monitoring and Assessment Programme Dataset 2021). Achieving a state-of-the-art performance of 87.9% mAP@0.5 on SEAMAPD21, YOLOv8-TF surpasses previous YOLO variants. While computationally heavier (30.56 M parameters), it maintains real-time speed (116 FPS). Validated on Pascal VOC and MS COCO, it demonstrates broad applicability for underwater ecological monitoring.

Accurate detection of abnormal fish behaviour in aquaculture is essential. The investigation in [231] proposes an improved detection algorithm based on YOLOv9, named DDEYOLOv9 (an enhanced high-precision detection algorithm based on YOLOv9), to detect abnormal fish behaviours early in industrial aquaculture settings, allowing farmers to take swift action to safeguard fish health and prevent economic losses. They also created the Takifugu rubripes Abnormal Behaviour Dataset, which includes five categories of fish behaviours. Experimental results achieved precision, recall and mAP values of 91.7%, 90.4%, and 94.1%, respectively.

Due to the increasing interest in the fishery industry and activities, many approaches focus on monitoring fishing vessels and analyzing images/videos obtained during those activities. In [232], fish tracking and segmentation are performed on videos recorded in a vessel. They introduced a tracking approach that integrates a deep convolutional neural network with an object detector and applies a Kalman filter in 3D. The object segmentation is updated and refined with the result of the RGB-depth image derived from stereoscopic images. The RGB-D images derived from the stereo videos are useful for obtaining the length of the detected objects. The proposed rescoring and segmentation method, in combination with the SSD model, obtained a Multiple Object Tracking Accuracy (MOTA) of 96.3%, which was quite better than the result obtained by the combination of FCNT (Faster Convolutional Network based Tracker) +SSD and the CFNet (Correlation Filter Network) +SSD, which did not reach the 90%. In [113], the authors present a real-time and robust underwater live crab detector called Faster MSSDLite (An SSD with MobileNetV2 as its backbone), which performed better than traditional SSD in the test results. As the backbone of this single-shot multi-box detector, they selected MobilNetV2. They also applied a denoising and enhancement method for underwater images effectively. An average precision of 99.01% was achieved by the network.

The paper [233] presents Foc_YOLOXn_ASFF, an enhanced version of YOLOX-nano with Adaptively Spatial Feature Fusion for detecting and classifying underwater fish. The system incorporates focal loss to address class imbalance and utilizes Adaptively Spatial Feature Fusion (ASFF) to enhance multi-scale feature learning. The model was trained on a custom dataset featuring four fish species from marine farms. It achieved an AP50 score of 98.9%.

In the line of research on fish diseases, we can find the work in [234]. This investigation introduces DCW-YOLO, an enhanced YOLOv10-based model for detecting skin diseases in Asian croaker fish. It replaces CIoU loss with NWD (Normalized Gaussian Wasserstein Distance) loss and adds C2f-D-LKA (Deformable Large Kernel Attention integrated into a C2f module (Cross-Stage Partial with full concatenation)) and DySample modules to improve accuracy and efficiency. A custom underwater dataset was created and augmented to train the model. DCW-YOLO achieves 96.87% mAP50 and runs at 90.91 FPS, outperforming other models. It enables fast, accurate, and lightweight fish health monitoring in aquaculture environments.

RetinaNet is a one-stage object detector that uses focal loss to address class imbalance. It combines high accuracy with efficient inference, making it suitable for detecting small and densely packed objects in images. Levy et al. [124] performed detection and tracking with RetinaNet [235] on two different datasets, the first containing underwater images [236] and the second is composed of aerial images. They concluded that the use of transfer learning could improve classification results. The average precision (AP) of fish detection was 0.74 for the underwater dataset.

An overview of research that applied different versions of YOLO and other types of single-shot networks is presented in

Table 11. Application of single-shot detectors in underwater images.

Ref	Dataset	Problem Type	Algorithms	Metric	Value (Highest)
[113]	Self-created by the cruise muddy underwater video monitoring system at Changzhou	Live crabs’ detector	Faster MSSDLite	Average precision/f-score	99.01%/98.94%
[211]	Kyutech10K	Real-time recognition and tracking of four different organisms	YOLO, Tracking Learning Detection (TLD) and MedianFlow	AUC	53.1%
[124]	Underwater images and Aerial images	Marine species classification	RetinaNet as the object detector/Simple Online Realtime Tracker (SORT) for tracking	Average precision	Aerial: 40% for Ray, 25% for Diver and 75% for Shark
[176]	Kaggle fishing boat dataset	Fish detection, object pose estimation and horizontal alignment before class prediction	SSD and YOLOv2 as detectors, VGG-16 as pose estimation, VGG-16 and Inception V3 as classification	Loss	60.4%
[214]	Self-created dataset in a water tank	Analysis of motion patterns for koi fish health monitoring	YOLOv3	Heatmap visualization of fish locations and different plots comparison	-
[215]	DeepFish and OzFish [237] datasets	Fish detection	YOLO-Fish-1 and YOLO-Fish-2 (based on YOLOv3)	Precision/recall/f1-score/AP	95%/96%/94%/96.15%
[216]	Images of lobsters captured in Western Australia using AUV and synthetic images	Western rock lobster detection	YOLOv3	mAP	46.9%
[217]	PASCAL VOC, Brackish and URPC Dataset	Underwater animal detection	A model based on MobileNet v2 and YOLOv4	mAP	92.65%
[219]	Self-created underwater dataset in a lake, in a laboratory tank and gathered from the internet	Fish detection	A model based on Res2Net and YOLOv5	Precision/recall/mAP	95.7%/88%/95.4%
[220]	Self-created with crawler technology and on laboratory	Jellyfish species and fish detection	An improved version of the YOLOv4-tiny	Precision/recall/mAP/f1-score	92.62%/89.69%/95.01%/0.91
[222]	Images from videos about jellyfish through the web crawler	Jellyfish detection	JF-YOLO (based on YOLOv4)	mAP/recall	92.67%/85.74%
[223]	Self-created dataset collected by UAVs and images gathered from Facebook	Turtle detection	YOLOv3, YOLOv5s, and YOLOv5l	Precision/recall/f1-score	97.21%/97.82%/97.51%
[224]	Images of shrimp in Recirculating Aquaculture System (RAS) culture tanks	Shrimp detection	Faster RCNN and YOLOv5m6	MAPE	5.48
[225]	Self-created dataset in a tank	Healthy and sick fish with SVC detection	NAM-YOLOv7	Precision/recall	97.3%/93.8%
[227]	Self-created; images obtained in a controlled environment (out of water)	Chinese soft-shelled turtle detection	YOLOv7-SS	Precision/recall/mAP	95.38%/94.68%/89.82%
[228]	TrashCan dataset	Underwater object detection and classification (marine debris, biological organisms and submerged artefacts)	AIT-YOLOv7	mAP@0.5	81.40%
[229]	Not specified	Fish (sharks, jellyfish and other species) detection	YOLOv8m	Precision/recall/mAP/f1-score	68.6%/61.24%/66.7%/64.31%
[230]	Pascal VOC, SEAMAPD21 and MS COCO datasets	Identification of underwater fish Species	YOLOv8-TF	mAP50/mAP50:95	Pascal VOC: 94.60/SEAMAPD21: 61.2%
[231]	Self-created underwater dataset in a tank in a laboratory	Abnormal fish behaviour detection and classification	A model based on YOLOv9	Precision/recall/mAP	91.7%/90.4%/94.1%
[232]	Self-created dataset on a fishing vessel	Fish tracking and segmentation	A deep convolutional neural network and an object detector with a Kalman filter in 3D + SSD/YOLOv2	Multiple Object Tracking Accuracy (MOTA)	96.3%
[233]	Custom underwater fish dataset	Fish detection and classification	Foc_YOLOXn_ASFF (base model YOLOX-nano)	AP50/AP75/AP50-95/AR	98.2%/93.3%/84.6%/87.0%
[234]	Self-created dataset: images captured in a semi-submerged underwater cage	Fish disease detection	DCW-YOLO (base model YOLOv10)	Precision/recall/mAP50/mAP50:95	95.46%/90.12%/96.87%/75.04%

.

Transformers, originally developed for sequence modelling, have been adapted for computer vision tasks through architectures like Vision Transformers (ViT). Their ability to capture long-range dependencies makes them a powerful alternative to traditional CNNs.

A paper featuring a model based on ViTs is [238]. This paper proposes the ADANSE ViT, featuring Amended Locale Self Attention (ALSA) and External Attention (EA) mechanisms, for classifying underwater fish and shrimp species, particularly on small datasets. It achieved high accuracy on a small proprietary dataset (701 images, 4 species) and the large WildFish benchmark. The ADANSE ViT significantly outperforms existing ViT models, achieving accuracies up to 92.3% (proprietary) and 93.9% (WildFish) across various image resolutions. Key innovations include learnable temperature parameters in ALSA, dual memory units in EA, and Layer Normalization for stability on small data. This demonstrates the potential for efficient and accurate underwater species classification even with limited data resources.

Some of the same authors of the previous work introduced ADANSE-TL [239], an integrated model combining a novel ADANSE ViT (for semantic features) with DenseNet-169 Transfer Learning (for local features) to classify marine species in challenging underwater images. On a self-collected dataset (701 images, 4 species) and benchmark sets (Croatian, Blue Bot), it delivered state-of-the-art accuracy: 96.21% on self-collected data and 95.09% on Blue Bot, outperforming other CNN variants and classifiers. Key innovations include a hybrid image enhancement pre-processing step and a feature fusion mechanism unifying local and semantic representations. The model demonstrates robustness across diverse underwater conditions and datasets.

In [240], IMViT, an improved Vision Transformer, is introduced for detecting and classifying fish species in challenging underwater images. To overcome ViT’s limitations on smaller datasets, IMViT integrates convolutional layers and residual units from ResNet before the transformer, enhancing its ability to extract low-level features and inductive biases. Evaluated on the “Fish-Dataset” (~9k images, 9 species), IMViT achieved state-of-the-art results: 95.73% accuracy, 95.31% precision, 95.14% recall, and 94.92% F1-score, significantly outperforming CNNs (ResNet, VGG) and the standard ViT. The hybrid architecture demonstrates strong potential for fish monitoring tasks in complex underwater environments.

Table 12. Application of Transformers for image classification.

Ref	Dataset	Problem Type	Algorithms	Metric	Value (Highest)
[238]	Proprietary Dataset (4 categories) and WildFish Dataset	Underwater Species Classification (specifically fish and shrimp species)	ADANSE ViT	Accuracy	Propietary: 92.3%/WildFish: 93.9%
[239]	Self-collected dataset, Croatian Fish dataset and Blue Bot dataset	Self-collected dataset	ADANSE-TL (ADANSE ViT and DenseNet-169)	Accuracy/Loss/ Precision/ Recall/ F1-Score	96.21%/ 0.174/96.32%/96.17%/96.20%
[240]	A Large-Scale Dataset for Segmentation and Classification [241]	Fish detection and fish species classification	Improved Vision Transformer (IMViT)	Accuracy/ Precision/ Recall/ F1-Score	95.73%/ 95.31%/95.14%/94.92%

lists previous articles that use transformers for object detection.

Autoencoders constitute a category of unsupervised neural networks that are intended to extract concise representations of input data by means of an encoding–decoding process. Beyond their traditional applications in image reconstruction and denoising, autoencoders have also proven effective in supporting object detection and segmentation tasks. By learning salient feature representations in an unsupervised manner, they can enhance downstream models or serve as building blocks within more complex architectures, especially when labelled data is scarce.

In the paper [242], the authors propose a novel classification convolution autoencoder (CCAE) to improve accuracy classification. They trained the net on ImageNet and Fish4Knowledge and obtained quite high results: 0.7375 of accuracy on ImageNet, 0.9928 on Fish4Knowledge.

O’Byrne et al. [243] trained a deep encoder–decoder network called SegNet (which they combined with an SVM for better results), using 2500 images. Even though the images were rendered in a virtual underwater environment, they obtained good results that could be extrapolated to a real environment, as they achieved an MIoU of 87% in a real environment.

Another approach based on the SegNet encoder–decoder can be seen in [244]. They performed fish segmentation on images of fish out of water. They achieved high values of IoU with the Nile tilapia fish, but when they applied the same method to other species, the mean IoU went down to 31.8%.

Object segmentation is a further step than object detection, since, in addition to detecting the location of the object and classifying it, the exact shape of the object must be segmented. Although this type of task can be performed with VC and ML techniques, DL approaches usually have better and more accurate results. In [245], DeepLabv3+ was adapted for underwater image segmentation by adding an unsupervised colour correction method (UCM) unit [246] to the encoder, enhancing both image quality and segmentation performance. The Mean Intersection over Union (MIoU) score they reached was 64.65%.

Mask R-CNN extends the Faster RCNN network by adding a branch that includes a binary mask that aims to predict whether the image pixel contributes to the given part of the object or not. This framework is made up of essential elements such as a backbone network, a feature extraction network called FPN (Feature Pyramid Network), a region proposal network called RPN and another RoI (Region of Interest) proposal network. Figure 7 shows the different layers that make up the architecture of the Mask R-CNN.

In [247] they replace the baseline ResNet50/101 network in Mask R-CNN with multiple residual blocks, constructing a 32-layer feature extraction network to decrease the network’s training parameters without compromising detection performance. Using transfer learning, they pre-trained the network with the MS-COCO (Microsoft Common Objects in Context) dataset. To test the network, they used a dataset acquired using a Tritech Gemini 720i sonar inside a water tank.

U-Net is a convolutional neural network from the University of Freiburg, designed to work with limited training data and deliver precise image segmentation by extending fully convolutional architectures [33]. Although initially developed for biomedical image segmentation, it has been used in multiple fields, as well as in the classification of aquatic species. The research in [248] employs an enhanced deep neural network (DNN) design for fish segmentation in underwater videos, utilizing distributed learning and edge computing to boost both energy efficiency and real-time processing capabilities. Their primary model is a customized U-Net, which integrates Pix2Pix blocks—commonly found in generative adversarial networks—to handle upsampling. To decrease the number of trainable parameters and accelerate training, they apply transfer learning by incorporating pre-trained weights from MobileNetV2, originally trained on the ImageNet dataset. The model was trained using a Distributed Computing System (DCS) on AWS (Amazon Web Services) SageMaker, leveraging 20 identical computers to distribute the workload. In distributed training with 20 nodes, the Sparse Categorical Cross Entropy (SCCE) loss was 0.053, improving by 18% compared to training on a single computer (loss of 0.065).

The classification model in [249] is based on the U-Net architecture. For training and testing the network, they created two datasets from internet videos. The first dataset featured clownfish and the second featured cavefish. They achieved an mIoU value of 88.19%.

The behaviour of the fish is not the only thing to be monitored. Möller et al. [250] tracked the size and behaviour of a sponge. They used images from a fixed underwater observatory (FUO) called Lofoten-Vesterålen (LoVe). They performed the sponge segmentation by U-Net successfully.

Some studies only focus on studying a single species, as can be seen in [251]. This study uses a dataset of images featuring roman seabream (Chrysoblephus laticeps), a fish native to southern Africa. The developed model is an implementation of Mask R-CNN designed for localization, classification, counting and tracking of Roman bream in uncontrolled underwater environments. The Mask R-CNN model achieved an mAP70 of 80.28% on the previously unseen test dataset.

The dataset used in the experiment in [252] is composed of 1824 images obtained from different underwater environments. The images come from several existing datasets, such as Semantic Segmentation of Underwater Imagery (SUIM) [101], DeepFish, and RockFish, as well as videos captured by underwater drones in Peru. The study compared two deep neural network architectures for semantic segmentation: DeepLabV3+ and U-Net. Optimal performance was achieved with a U-Net-scSE (U-net with Spatial Channel Squeeze-and-Excitation module) model that incorporated a ResNeSt-269e encoder. The training process used a combined loss function featuring Chan-Vese-based learning active contour loss, cross-entropy loss, and Dice loss.

This paper [253] introduces a neural network for underwater fish segmentation, addressing challenges like colour distortion and blur using ResNet50 with an enhanced feature pyramid (PAFE) and attention mechanisms. The model attains 95.1% MIoU and 0.901 F1-Score, surpassing U-Net and PSPNet (Pyramid Scene Parsing Network). Data augmentation (Mixup, CutMix) and a new deep-sea dataset validate robustness in real-world conditions. The PAFE module improves multi-scale feature extraction, balancing accuracy and computational efficiency.

AASNet (Agricultural Aqua Segmentation Network) is an advanced deep learning framework developed to provide precise and efficient underwater fish instance segmentation for use in smart fisheries [254]. It tackles key challenges like lighting/colour variations and extreme class imbalance through two core innovations: the Linear Correlation Attention (LCA) module for robust feature correlation and the Dynamic Adaptive Focal Loss (DAFL) for handling data imbalance. Evaluated on UIIS (Underwater Image Instance Segmentation dataset) [255] and USIS10K (Underwater Salient Instance Segmentation 10K dataset) [256] datasets, AASNet achieves state-of-the-art performance (31.7 mAP on UIIS, 47.4 mAP on USIS10K) while maintaining high inference speed (28.9 ms/image). Its accuracy and efficiency make it ideal for real-time agricultural uses, such as monitoring fish farms and assessing health.

This research [257] introduces a self-supervised approach to fish segmentation, eliminating the requirement for manual annotation by employing feature alignment and spatiotemporal consistency. It employs a CoaT Transformer (Co-Scale Conv-Attentional Image Transformer) to capture multi-scale features—combining local details and global context—for improved robustness in low-visibility underwater scenes. The method achieves 50.0 ${\mathcal{J}\&\mathcal{F}}_{mean}$ (Jaccard Index & Dice Coefficient mean) on the Seagrass dataset [258] and 63.3 on YouTube-VOS [259], outperforming previous self-supervised approaches. The approach is computationally efficient thanks to parallel processing and anchor sampling, although it still faces challenges in scenarios with heavy occlusion and high background similarity.

Swin Transformers are hierarchical vision transformers designed to efficiently model images at multiple scales using shifted windows for self-attention. This architecture enables better capture of both local and global context, making it well-suited for dense prediction tasks like segmentation. Swin Transformers have demonstrated state-of-the-art performance in semantic and instance segmentation benchmarks, often outperforming traditional CNN-based methods. Thanks to their flexibility and strong feature representation, Swin Transformers have become a popular backbone choice in modern segmentation models.

Following this line, we can find works like [260], where they present SwinConvMixerUNet, a novel hybrid model combining Swin Transformer’s attention mechanisms and ConvMixer’s efficient feature mixing within a U-Net structure for underwater image semantic segmentation. It addresses challenges like low visibility, light scattering, and particulate matter in underwater environments, crucial for tasks like fish habitat monitoring and marine ecosystem preservation. On the SUIM dataset, the model achieves an mIoU of 84.83%, surpassing DeepLabV3, PSPNet, UNet, and other models. The architecture effectively handles complex underwater conditions through patch partitioning, hierarchical Swin Transformer blocks, and ConvMixer refinement. This advancement enables more accurate environmental monitoring, resource management, and pollution tracking in marine applications.

In [261] they propose UISFormer, a ViT-based model for segmenting aquatic organisms (fish, coral) in complex underwater aquaculture scenes. It addresses challenges like limited data via CutStitch augmentation and enhances feature fusion using RFEM (Residual Feature Enhancement) and MFRF (Multiscale Feature Residual Fusion) modules. Evaluated on three datasets (FishData, Underwater Images Segmentation dataset better known as UISD, large-scale fish data), UISFormer outperforms CNNs and hybrid models (e.g., 92.20% MIoU on UISD). They performed practical tests in Shandong marine ranches to confirm its robustness for real-world aquaculture monitoring.

As seen before, SAMs are large-scale, prompt-driven segmentation models used to perform general-purpose image segmentation without task-specific training. They are trained on billions of masks across diverse images, enabling strong zero-shot performance on unseen objects and domains. Its architecture combines a powerful image encoder (ViT) with a lightweight prompt encoder and a mask decoder. In [262] they introduce UIIS10K, a large underwater instance segmentation dataset to date, containing 10,048 images across 10 categories. They also propose UWSAM (Segment Anything Model Guided Underwater Instance Segmentation), a method that leverages MG-UKD (Mask GAT-based Underwater Knowledge Distillation) to transfer knowledge from SAM into a lightweight ViT-Small encoder, using GAT-based (Graph Attention Network- based) feature reconstruction. They also design EUPG (End-to-End Underwater Prompt Generator), an automatic prompt generation mechanism that enables end-to-end segmentation without relying on external object detectors. The approach achieves a mAP value of 44.6 on UIIS10K for UWSAM-Teacher and 38.7 mAP for UWSAM-Student. Additionally, the UWSAM-Student model demonstrates efficiency (47M parameters) and robustness against common underwater challenges such as turbidity and occlusion.

A more detailed account of the methods cited can be found in

Table 13. Application of segmentation networks in underwater images.

Ref	Dataset	Problem Type	Algorithms	Metric	Value (Highest)
[242]	ImageNet and Fish4Knowledge	Fish classification	Classification convolution autoencoder (CCAE)	Accuracy	73.75% and 99.28%
[243]	Images containing virtual underwater scenery	Underwater object semantic segmentation	Deep encoder–decoder network called SegNet	MIoU/mean accuracy	87%/94%
[244]	Self-created	Semantic segmentation of tilapia fish body parts	Encoder–decoder based on the SegNet approach	MIoU	60-85%
[245]	Self-made underwater image dataset	Underwater species Semantic segmentation	DeepLabv3 +	MIoU	64.65%
[247]	MS-COCO dataset for pre-training and images acquired by sonar (self-created)	Underwater object detection and semantic segmentation	Mask RCNN	For detection: Precision/recall/mAP For segmentation: AP/${AP}_{50}$	For detection: 95.73%/97.21%/96.97% For segmentation: 53.23%/95.15%
[248]	DeepFish dataset	Underwater fish segmentation	A modified version of the U-Net	SCCE loss/SCF loss/Sparse Categorical Crossentropy Accuracy (SCCA)/IoU	0.053/3.48/98.81%/87.6%
[249]	“Coralfish” and “cavefish” dataset obtained from YouTube videos	Fish detection and classification	U-Net	F1-score/mIoU	95.04%/88.19%
[250]	LoVe Observatory dataset	Sponge size and behaviour tracking	U-Net	Pearson’s r	0.98
[251]	Dataset collected using submerged action cameras mounted on a baited underwater remote video (BRUV) rig.	Roman seabream detection and tracking	Mask R-CNN	${mAP}_{50}/{mAP}_{75}$	81.45%/80.28%
[252]	Images collected from multiple existing datasets	Bottom water, seafloor/obstacles, and fish segmentation	DeepLabV3+ and U-Net-based model variations	IoU/Hausdorff distance (HD)	78.76%/19.17
[253]	Fish4Knowledge and Real deep-sea fish images	Underwater fish segmentation	ResNet50 backbone with an enhanced Feature Pyramid Convolutional Architecture (PAFE)	F1-score/MioU/Pixel Accuracy (PA)	90.1%/95.1%/92.1%
[254]	UIIS and USIS10K	Underwater Fish Instance Segmentation for smart fisheries	AASNet framework (GELAN Backbone from YOLOv9)	mAP/AP/${AP}_{75}$	31.7%/49.5%/35.1%
[257]	DeepFish (training), Seagrass and YouTube-VOS dataset	Underwater fish segmentation	CoaT Transformer backbone: Combines Conv-Attentional Image Transformer (CAIT) and Co-Scale Feature Attention Network (CFAN)	${\mathcal{J}\&\mathcal{F}}_{mean}$/${\mathcal{J}}_{mean}/{\mathcal{J}}_{recall}/{\mathcal{F}}_{mean}/{\mathcal{F}}_{recall}$/	YouTube-VOS: 63.3%/63.9%/74.0%/62.7%/69.6%
[260]	SUIM dataset	Underwater Image Semantic Segmentation	SwinConvMixerUNet	mIoU	84.83%
[261]	FishData, UISD, Large-scale fish dataset	Underwater image segmentation	UISFormer	mIoU/Accuracy/CPA (Category Pixel Accuracy)/F1-score	Large-scale Fish Data: 96.7%/99.34%/98.97%/98.32%
[262]	UIIS10K (proposed), UIIS, USIS10K	Underwater instance segmentation	UWSAM Framework	Bounding Box metrics (${mAP}^b$/${AP}_{50}^b$/${AP}_{75}^b$) Mask metrics (${mAP}^s$/${AP}_{50}^s$/${AP}_{75}^s$)	UWSAM-Teacher on USIS10K: ${mAP}^b$ = 45.8%/${AP}_{50}^b$ = 64.1%/${AP}_{75}^b$ = 55.1%) Mask metrics (${mAP}^s$ = 46.0%/${AP}_{50}^s$ = 61.7%/${AP}_{75}^s$ = 51.7%)

.

5.2.4. Multimodal Understanding and Reporting

Recent advances in multimodal models have enabled underwater species detection and classification through vision-language integration. Unlike traditional CNN or Transformer-based detectors, these approaches leverage natural-language prompts, zero-shot reasoning, and cross-modal alignment, enabling robust performance even under scarce annotations or unseen species. Vision-Language Models (VLMs) and Large Language Models (LLMs) have thus emerged as a new family of techniques applicable to underwater environments, warranting a dedicated subsection.

The UVLM (Underwater Video Language Model) [263] targets the lack of video–language benchmarks tailored to underwater environments, where visual degradation, complex temporal dynamics, and domain-specific biological knowledge significantly challenge existing Video-Language Models (VidLMs). The authors introduce the first underwater video-language benchmark built through a human–AI collaborative annotation pipeline, combining expert frame-level annotations with GPT-4o–assisted text generation and strict manual verification. The dataset includes 2109 videos covering diverse scenes and 419 marine organism classes and defines 20 subtasks related to biological and environmental understanding. Extensive evaluations show that current VidLMs suffer significant performance drops underwater compared to terrestrial settings. Fine-tuning on UVLM notably improves results, enabling smaller models to approach large proprietary ones, although fine-grained taxonomic reasoning remains challenging. The results also highlight a persistent performance gap on fine-grained taxonomic classification, indicating intrinsic limitations of smaller models in knowledge-intensive underwater tasks.

State-of-the-art multimodal large language models (MLLMs) show strong general vision–language capabilities, yet their effectiveness in fine-grained marine species recognition remains unclear. The work in [264] presents a large-scale diagnostic study revealing that even the best open-source MLLMs perform poorly on fish species identification, achieving below 10% accuracy in zero-shot settings. To investigate the causes of this failure, the authors introduce FishNet++, a comprehensive multimodal benchmark extending FishNet with rich textual, spatial, and morphological annotations. The analysis disentangles errors arising from missing taxonomic knowledge, weak visual domain understanding, and limited fine-grained perception. Experiments show that MLLMs can localize fish reasonably well but struggle with subtle morphological cues critical for species-level recognition. Fine-tuning on FishNet++ substantially improves performance, and explainable training further enhances interpretability, highlighting the need for domain-specific multimodal benchmarks in aquatic science.

This work [265] proposes a cloud–edge framework for real-time semantic understanding in AUVs, combining lightweight underwater image enhancement and frame sampling at the edge with cloud-based inference using the Qwen2.5-VL model. The edge pipeline performs colour correction, dehazing, and white-balance normalization, followed by low-rate frame sampling to meet bandwidth constraints. A latency model incorporating edge, communication, and cloud processing enables end-to-end performance analysis. Simulations across multiple underwater scenarios report stable latencies around 1.0–1.2 s per frame and reliable cloud inference times. The VLM produces natural language scene descriptions capturing objects, spatial relations, and environmental cues, with object detection recall up to 0.94 and hallucination rates as low as 4% depending on the scene. Results indicate that enhancement substantially reduces hallucination and improves descriptive fidelity under turbidity and occlusion.

The work of [266] introduces AquaVLM, the first underwater communication system leveraging mobile Vision-Language Models (VLMs) to generate context-rich, semantically meaningful messages for scuba divers. It combines multimodal perception (images and sensor data) with hierarchical, intent-guided message generation. To address the lack of underwater knowledge, a diving conversation dataset was created, and the model was trained for message generation, response generation, and message retrieval. It includes error-resilient fine-tuning to handle corrupted acoustic transmissions and was validated with a VR simulator and an iOS prototype. Lake experiments show high semantic similarity (>90%) and low error rates (<3%), outperforming traditional OFDM-based (Orthogonal Frequency Division Multiplexing) methods.

Ref. [267] introduces AquaOV255, a large and detailed underwater segmentation dataset, with over 20,700 images across 255 categories, addressing the limited diversity of previous benchmarks. The authors combine five more datasets to create UOVSBench, the first unified benchmark for open-vocabulary underwater segmentation. To handle domain shifts from terrestrial VLMs, they propose Earth2Ocean, a training-free adaptation framework. The method incorporates two key modules: a Geometric guided Visual Mask Generator (GMG) leveraging DINO (Distillation with No Labels) based self-similarity priors to correct CLIP (Contrastive Language-Image Pre-Training) visual features, and a Category visual Semantic Alignment (CSA) module that enhances textual embeddings using multimodal LLM reasoning and underwater specific templates. Without underwater fine-tuning, it improves average mIoU by 6–8 points while remaining efficient. Ablations show both geometric and semantic modules are crucial for recognizing fine-grained and rare underwater categories, enabling practical marine monitoring applications.

FishDetectLLM [268] is a fish detection framework using a lightweight multimodal LLM, extending TinyLLaVA (a smaller version of the Large Language and Vision Assistant model) to perform both classification and bounding box prediction. It treats fish detection as a visual question answering task, combining visual features with textual reasoning, unlike traditional CNN/Transformer detectors. A multi-turn instruction dataset is created from FishNet (94,532 images, 17,357 species) linking taxonomy descriptions with bounding boxes. The model achieves high classification accuracies (up to 99.09% at the class level) and 84.2 mAP50, outperforming YOLO, DINO, Faster/Mask RCNN, CLIP, and LLaVA, while generalizing to unseen species. Limitations include inference speed, with future work aimed at model compression and broader ecological integration.

This study [269] addresses the limitations of current Vision-Language Models (VLMs), such as CLIP, in fine-grained coral species recognition, where domain-specific morphology and hierarchical taxonomy play crucial roles. The authors introduce HSCR16K, a large-scale hierarchical stony coral dataset containing 16,659 images from 16 species, annotated from kingdom to species and enriched with expert-crafted morphological descriptions. Building on this dataset, they propose CORAL Adapter, a lightweight plug-and-play module designed to augment CLIP with coral-specific knowledge via two complementary components: a morphological adapter that captures texture, structure, and colour cues, and a taxonomic adapter that encodes biological hierarchy relations. The model is optimized via a combination of cross entropy, similarity alignment, and a reversed Jensen–Shannon divergence to balance prior CLIP knowledge with newly learned coral-specific representations. The method demonstrates strong robustness in recognizing unseen bleaching-prone species and transferring across ocean regions. Overall, the work showcases how structured biological knowledge can significantly enhance VLMs for ecological monitoring and coral health assessment.

Table 14. Applications of VLMs and LLMs in underwater images.

Ref	Dataset	Problem Type	Algorithms	Metric	Value (Highest)
[263]	UVLM (proposed), WebUOT (re-annotated)	Underwater video-language understanding	General VidLMs with supervised fine-tuning (VideoLLaMA3, Qwen2.5VL)	Overall accuracy	Qwen2.5VL-72B: 75.49% VideoLLaMA3-7B fine-tuned: 73.04% (+10.34)
[264]	FishNet++ (proposed), FishNet	Fine-grained fish species recognition (open-vocabulary classification), detection, keypoint localization	CLIP, BioCLIP, SigLIP, Qwen2.5-VL, Gemma-3, GPT-4o, YOLO-based baselines	Accuracy/IoU50/IoU90	GPT-4o: 17.9%/YOLO-12: 95.2%; Qwen2.5-VL: 91.5%/YOLO-12: 35.2%; Qwen2.5-VL: 26.7% (Frequent Species)
[265]	Simulated underwater videos	Real-time VLM semantic scene analysis for AUVs	Qwen2.5-VL (72B)	Object Detection Recall (ODR)/Spatial Relationship Accuracy (SRA)/Hallucination Rate /Output Accuracy	0.94/0.91/0.04/0.88
[266]	Five public scuba diving videos captured in different locations and simulated sensor data	Context-aware message generation & recovery for diver communication (mobile VLM)	MobileVLM-3B	Purpose-Align Rate /Semantic Similarity (received vs. original message)	80%/90%
[267]	AquaOV255, UOVSBench (AquaOV255 + USIS16K, SUIM, MAS3K, USIS10K, DUT-USEG)	Training-free open-vocabulary segmentation in underwater scenes	Earth2Ocean (GMG + CSA; transfers terrestrial VLMs to underwater)	mIoU	55.24
[268]	FishNet + LLaVA-1.5 datasets; extra 1100 DeepFish images	LLM-based detection and classification	FishDetectLLM (TinyLLaVA + SigLIP + StableLM-2; instruction conversations)	Accuracy	FishNet: 99.09%
[269]	HSCR16K	Fine-grained coral recognition (species/genera); zero-/few-shot VLM adaptation	CORAL-Adapter (morphological + taxonomic adapters on CLIP)	Accuracy	77.27%

provides a more detailed overview of the methods referenced.

6. Discussion

The application of deep learning (DL) to the analysis of marine species imagery has grown rapidly over the past decade, driven by advances in computer vision and the increasing availability of underwater and above-water image data. These methods hold significant potential for biodiversity monitoring, fisheries management, and ecological research, yet the field faces unique challenges due to the variability of aquatic environments and the limited availability of labelled datasets. The following research questions guide this study’s review of existing literature and methods:

RQ1: Advanced deep learning architectures

DL applications in marine imagery have leveraged a variety of architectures. Regarding image enhancement and restoration, the most widely used architectures are GANs and improved U-Net versions. CNN-based models such as ResNet, DenseNet, and VGGNet are widely used for classification, while region-based methods (e.g., Faster R-CNN) and real-time detectors (YOLO, SSD) address detection tasks. Segmentation is addressed using specific architectures such as Mask R-CNN. Vision Transformer (ViT) models are emerging for complex, cluttered environments and video analysis, benefiting from their ability to model long-range dependencies. Large Vision-Language Models and multimodal LLMs bring zero-/few-shot flexibility to underwater species analysis, but their operational costs and reliability constraints are significant. Model scale has become a fundamental aspect of the architecture of multimodal language and vision models used to study underwater environments. The approaches reviewed range from relatively compact models such as MobileVLM (≈3 billion parameters) to mid-scale architectures such as VideoLLaMA3-7B, and even very large models such as Qwen2.5-VL-72B. Beyond the contrast with traditional CNN-based architectures, such as VGG16 (with approximately 138 million parameters), the gap within the multimodal family itself is particularly pronounced, raising significant questions regarding efficiency, implementation feasibility and fair comparison across studies. Although larger models generally offer more robust reasoning without the need for fine-tuning, their high computational consumption and inference costs represent substantial limitations for real-time underwater applications or those with limited resources.

RQ2: Datasets and acquisition methods

Research draws on both public datasets, such as Fish4Knowledge or LifeCLEF, and custom collections targeting specific taxa or habitats. Image acquisition methods include underwater cameras mounted on ROVs and AUVs, diver-operated photography, surface-level imaging for mammals, and, in some cases, artificially created environments in tanks. Video data is often decomposed into frames for training. It is worth noting the repeated use of certain datasets, such as Fish4Knowledge. Several studies published prior to 2018 report very high and closely clustered accuracy values (above 95%), suggesting that this dataset may not offer sufficient discriminatory power to distinguish meaningfully between methods. This highlights a significant limitation of the extensive use of a specific dataset over an extended period and underscores the need for new datasets that better capture environmental variability and pose a greater challenge to modern DL models.

RQ3: Evaluation metrics and performance trends

Performance is assessed using metrics such as accuracy, precision, recall, F1-score, mAP, and IoU. CNN classifiers often exceed 85% accuracy on clear imagery, while detection in turbid or cluttered conditions yields more variable results (mAP 50–80%). Transformer-based models show superior performance in complex scenes, especially with large datasets, due to their enhanced contextual reasoning. It should be noted that, as discussed in the dataset, a baseline performance close to saturation can also limit the interpretability of evaluation metrics.

RQ4: Limitations, challenges, and future directions

Key challenges include limited labelled datasets, inconsistent image quality, class imbalance, and poor generalization across domains. One challenge associated with image quality assessment is the lack of real reference images in underwater image enhancement. In real-world aquatic environments, it is generally not easy to obtain an undistorted version of an image, which limits the applicability of full-reference metrics such as PSNR or SSIM. Consequently, many studies resort to reference-free metrics, such as UIQM and UCIQE, to quantify the performance of the enhancement. However, these metrics do not always correlate with perceived visual realism or performance in downstream tasks, such as object detection or species classification. For this reason, several studies supplement objective scores with a subjective visual assessment. These differences complicate comparisons between studies and highlight the need for more robust evaluation methodologies. Future research priorities include creating large, standardized datasets, integrating multimodal detection (e.g., optical imaging and sonar), advancing self-supervised methods, and developing lightweight, green AI models for real-time implementation. Explainable AI is also identified as essential for decision-making and user trust. Despite its promising capabilities across multiple domains, current approaches based on foundational models, VLMs, and LLMs remain limited due to their high computational demands, latency, and difficulty of real-time implementation. Their sensitivity to prompt design and underwater visual artifacts can also introduce instability in practical environments. Regarding future work, it would be interesting to investigate the development of lighter multimodal architectures, strategies for the use of more robust language models, and improved adaptation mechanisms tailored to murky, low-light, or species-rich environments.

The results of this study demonstrate the significant potential of artificial intelligence techniques in addressing the unique challenges associated with underwater imaging and marine species analysis. The application of deep learning-based enhancement algorithms markedly improved image clarity and colour fidelity, mitigating the effects of light absorption, scattering, and turbidity commonly observed in subaquatic environments. These enhancements facilitated more accurate downstream tasks, including species classification, object detection, and segmentation, highlighting the interdependence between image preprocessing and model performance.

Overall, the findings of this study highlight the transformative potential of AI-driven methodologies for marine ecology and conservation, offering scalable solutions for monitoring biodiversity and supporting informed environmental management decisions. The integration of image enhancement and automated species analysis represents a critical step toward real-time, high-fidelity assessment of underwater ecosystems.

7. Conclusions

The aim of this paper is to review the state-of-the-art applied to underwater imaging, examining computer vision techniques, which are usually applied to image preprocessing and to extract image features, and researching machine learning and deep learning approaches, which are used for classification and object detection.

The core of the paper is dedicated to reviewing DL approaches specifically tailored to underwater fish and water-related imagery. It is divided into two main areas: underwater image processing and enhancement, and aquatic species detection, classification, and segmentation. The first part discusses techniques to improve image quality in challenging underwater conditions, while the second part examines how various DL models are used to identify and analyze aquatic life, with detailed examples of model applications.

Compared to other reviews on the topic, this article is distinguished by its comprehensiveness and depth in its coverage of the existing literature. While previous reviews have focused on a limited set of studies, this review includes a broader analysis covering 132 recent and relevant articles, providing a more complete overview of the current state of knowledge. Furthermore, a particular effort has been made to include a greater diversity of sources, considering research from different methodological approaches and geographical contexts. This breadth and diversity not only enrich the discussion but also offer a more robust and balanced perspective, making this article an essential reference for any researcher interested in the topic.