Multi-Module Combination for Underwater Image Enhancement

Abstract

Underwater observation and operation for divers and underwater robots still largely depend on optic methods, such as cameras videos, etc. However, due to the poor quality of images captured in murky waters, underwater operations in such areas are greatly hindered. In order to solve the issue of degraded images, this paper proposes a multi-module combination method (UMMC) for underwater image enhancement. This is a new solution for processing a single image. Specifically, the process consists of five modules. With five separate modules working in tandem, UMMC provides the flexibility to address key challenges such as color distortion, haze, and low contrast. The UMMC framework starts with a color deviation detection module that intelligently separates images with and without color deviation, followed by a color and white balance correction module to restore accurate color. Effective defogging is then performed using a rank-one prior matrix-based approach, while a reference curve transformation adaptively enhances the contrast. Finally, the fusion module combines the visibility and contrast functions with reference to two weights to produce clear and natural results. A large number of experimental results demonstrate the effectiveness of the method proposed in this paper, which shows good performance compared to existing algorithms, both on real and synthetic data.

1. Introduction

With profound changes in the global economic landscape and the in-depth promotion of the concept of sustainable development, the development of marine resources is gradually becoming a new strategic upper hand for which countries are competing. As an important tool for the exploitation of marine resources, marine engineering equipment requires a careful observation and analysis of the underwater environment at every stage, from its initial construction, mid-term maintenance and inspection to its later disassembly [1]. Marine resource surveys require high-resolution images to accurately observe seabed topography and resource distribution, thereby providing essential data support for marine development. Archeologists also rely on clear images to effectively identify and analyze underwater artifacts, which is critical for the protection of underwater cultural heritage in archeological research [2]. Furthermore, captured images are integral to various tasks, including underwater robot navigation, target tracking, and seabed environment monitoring. Underwater image enhancement is therefore essential and has extensive applications across multiple fields.

Underwater environments possess unique characteristics, such as low light levels, turbid water, and object reflections, which present significant challenges for the acquisition and processing of underwater images [3]. The imaging quality of underwater images is susceptible to complex hydrodynamic effects, which are mainly characterized by typical degradation features such as non-smooth noise interference, non-uniform optical blurring and multi-spectral attenuation [4]. Degraded features in underwater images can significantly reduce the localization accuracy of target detection, weaken the robustness of feature recognition, and lead to the accumulation of trajectory drift errors during dynamic tracking. By enhancing underwater images, the quality can be significantly improved, thereby increasing the reliability and utility of underwater vision systems. Underwater image enhancement, a crucial technology at the intersection of computer vision and marine optics, aims to reconstruct high-quality images with physical interpretability. This is achieved by establishing a mapping relationship between degradation models and visual perception, thereby overcoming the limitations imposed by complex underwater environments on the effectiveness of visual systems.

In response to the technical challenges presented by complex underwater imaging environments, researchers have proposed a range of innovative solutions in recent years. One such approach is a filter-based single-image defogging method that utilizes a guided joint bilateral filtering technique for haze removal [5]. The proposed algorithm excels in enhancing underwater images, achieving substantial improvements in both perceptual quality and objective evaluation metrics through effective contrast enhancement and noise suppression. Recent advances in deep learning have dramatically transformed underwater image enhancement methods, facilitating overall quality improvement through an end-to-end learning framework. In particular, the developed enhancement network is trained on a reference dataset and demonstrates exceptional capabilities in image processing [6]. Based on the insights gathered from our predecessors, we propose an architecture that first assesses whether to correct the image for color and white balance, depending on the outcomes of color deviation detection. Subsequently, it performs visibility restoration and contrast enhancement processes, followed by a fusion step to create an enhanced underwater image. Figure 1 illustrates the results of the visualization of the enhancement effect of this method in real and simulated underwater scenes in comparison. The main innovation of this study is reflected in the following dimensions:

• In this paper, we present an algorithm that utilizes color bias detection to identify color bias in images. A white balance technique is then applied to process the color-biased images. This preprocessing step enhances the detection of color bias, thereby improving the accuracy of subsequent image processing tasks.
• We employ a defogging and contrast enhancement algorithm that utilizes a rank-one prior matrix and curve transformation. The clarity and legibility of the underwater image are improved by converting it to the LAB color space, removing fog with the rank-one prior matrix, and enhancing the image’s contrast through a curve transformation.
• The defogged image is combined with a contrast-enhanced image to produce a clearer and more recognizable underwater image.

The structure of this paper is organized as follows. Section 2 is an introduction to related work. Section 3 is divided into six subsections detailing the model structure of UMMC. Section 4 first describes the configuration of the experimental environment and then conducts three aspects of validation: (1) qualitative and quantitative evaluations on the UIEB and SUID datasets, (2) feature point matching experiments to verify the effectiveness of the enhanced images in improving the accuracy of underwater target recognition, and (3) an ablation study to assess the independent contributions of each technical component. Finally, an analysis of algorithm complexity and real-time testing are performed. Section 5 summarizes the research results and discusses future research directions.

2. Related Work

Underwater image enhancement represents a critical research area in marine science and technology, with significant contributions from numerous scholars aimed at facilitating more efficient and accurate ocean exploration.

Contrast Enhancement: Ding et al. [7] proposed a contrast enhancement fusion algorithm based on non-subsampled shearlet transform (NSST) framework. The method conducts a multiscale decomposition of the source image, separating it into low-frequency components and high-frequency sub-bands across multiple directions. Drawing on characteristics of human visual perception, the research team innovatively categorizes the low-frequency components into visually salient and non-salient regions. They effectively enhance the global contrast of the image through a fusion strategy while better preserving the detailed features of different regions. Zhang et al. [8] developed an innovative framework for underwater image enhancement that integrates color correction with dual-range contrast optimization. Their approach employs a sophisticated bi-interval histogram processing technique, which automatically determines an optimal equalization threshold to preserve a natural appearance while preventing over-enhancement. Additionally, the method incorporates an adaptive S-curve transformation to selectively boost local contrast and accentuate fine image details, achieving balanced enhancement across various regions of underwater images. In another study, Zhang et al. [9] tackled the issue of image degradation by implementing color channel correction and detail-preserving contrast enhancement. They designed a specialized attenuation matrix to compensate for inferior color channels.

Image Fusion: Xu et al. [10] designed a novel, unified, and unsupervised end-to-end image network to address various image fusion challenges. Zhang et al. [11] introduced a fast unified image fusion network based on the Proportional Maintenance of Gradients and Intensities (PMGI), which reformulates the image fusion problem as one of maintaining the proportionality of texture and intensity in the source images to accomplish multiple image fusion tasks. Tang et al. [12] developed a semantic-aware real-time image fusion network (SeAFusion) that integrates the image fusion module with a semantic segmentation module. Prince et al. [13] presented a multiple exposure image fusion method designed to merge bracketed exposure captures into a single image, enhancing details across a wide dynamic range.

Underwater Image Enhancement: Sudhanthira et al. [14] proposed a multi-module fusion image enhancement network that integrates the original weights of two images along with correlation weights during processing. Jiang et al. [15] proposed a dual-domain migration-based underwater image enhancement method, which achieves the robust migration of cross-domain features by constructing a progressive feature transformation mechanism from the airborne domain to the underwater domain. Zhou et al. [16] also achieved cross-domain adaptation from the aerial domain to the underwater domain by implementing a multi-feature enhancement mechanism (MFEF). This method utilizes multipath input to extract diverse forms of rich features from multiple viewpoints. Zhang et al. [17] developed a weighted wavelet visual perception fusion-based underwater image enhancement technique, which produces high-quality underwater images by merging high-frequency and low-frequency components from images of varying scales.

While deep learning solutions in existing research have achieved promising results, their performance often relies heavily on large-scale training datasets and substantial computational resources. Traditional methods, as a crucial component of underwater image enhancement research, warrant in-depth study. Building on previous solutions that address individual problems in underwater images, we are inspired to tackle multiple types of challenges in underwater environments through multi-module combination techniques. Therefore, we propose a multi-module combination technique for underwater image enhancement.

3. Proposed Method

This section outlines our methods for enhancing underwater images. First, we present the overall framework of the proposed enhancement technique. Subsequently, each module is described in detail.

3.1. Framework Architecture

Figure 2 illustrates the general framework of the UMMC method, which comprises five fundamental modules. The color deviation detection module is primarily designed for images exhibiting color deviations, although such deviations may not be readily apparent in shallow water images. The color and white balance correction module computes a nonlinear color mapping function to correct the image’s color by employing a k-nearest neighbor strategy. The visibility restoration module utilizes an enhanced rank-one a priori matrix decomposition technique, combined with depth information estimation, to achieve the precise removal of scattering blur [18]. The contrast enhancement module enhances the image’s brightness through grayscale correction and the probability distribution of brightness pixels while also applying gamma correction to automatically improve the image contrast. The image fusion module integrates images processed by the contrast enhancement module with those processed through multiple exposures.

3.2. Color Devviation Detection Module

In turbid water, deep water, or low-light environments, the scattering and wavelength attenuation of suspended particles can lead to color deviations. Different scenes are corrected using various color compensation methods to obtain high-quality underwater images. This process not only conserves computational resources by preventing the unnecessary processing of non-color-biased images, but it also ensures that certain images resemble terrestrial images due to the clarity of the seawater. The algorithm analyzes the features of the image by examining the digital image itself. Specifically, the texture information of the image is incorporated into the projection factor K [19]. A crucial step in detecting color deviation involves converting the input image from the RGB color space to the Lab color space. This conversion process begins by transforming the RGB data into the XYZ color space through a linear transformation, which serves as an intermediate step. The XYZ color space possesses device-independent characteristics, providing a standardized basis for color representation in the subsequent conversion. Once the XYZ values are obtained, the XYZ color space is mapped to the Lab color space through a nonlinear transformation. This mapping aligns more closely with the characteristics of human visual perception and facilitates the subsequent analysis of color deviation.

By analyzing both non-color and color images, Fang et al. [19] found that color bias is closely related to the mean and distribution characteristics of the image’s chromaticity. If the chromaticity distribution in the 2D histogram is more concentrated around the Lab chromaticity coordinates, the image exhibits a color bias. A higher mean value of chromaticity indicates a greater degree of color bias, whereas a lower mean value suggests that the color bias is less pronounced. Consequently, the offset coefficient K is employed to quantify the degree of color deviation in an image.

$$d_a={\displaystyle \frac{{\sum }_{i=1}^H{\sum }_{j=1}^{W}a\left(i,j\right)}{H\times W}}$$

(1)

$$d_b={\displaystyle \frac{{\sum }_{i=1}^H{\sum }_{j=1}^{W}b\left(i,j\right)}{H\times W}}$$

(2)

$$D=\sqrt{d_a^2+d_b^2}$$

(3)

$$M_a={\displaystyle \frac{{\sum }_{i=1}^H{\sum }_{j=1}^W\left|a\left(i,j\right)-d_a\right|}{H\times W}}$$

(4)

$$M_b={\displaystyle \frac{{\sum }_{i=1}^H{\sum }_{j=1}^W\left|b\left(i,j\right)-d_b\right|}{H\times W}}$$

(5)

$$M=M_a\times M_b$$

(6)

The expression for the offset coefficient K is given below:

$$K={\displaystyle \frac{D}{M}}$$

(7)

In the equation above, a and b represent the chromaticity components of the Lab color space, indicating the chromaticity deviation from the mean value. H denotes the height of the image and W denotes the width of the image. D denotes the average color deviation of the image and M denotes the center distance of the image. The offset coefficient K is utilized to assess the presence of color deviation in the image. Consequently, it is essential to establish an appropriate threshold value to determine whether color deviation exists in underwater images. In this section, several underwater images are selected for testing, and the corresponding K values for each image are presented in Figure 3.

The K-value of each image in Figure 3 indicates that a more severe color deviation corresponds to a larger K-value. Images with a K-value of less than 1.0 exhibit the least color deviation. The UIEB dataset [6] comprises a diverse collection of real underwater images sourced from open access websites and various research papers. This dataset includes 890 real underwater images that represent a wide array of water types, such as near-shore, deep-sea, and turbid waters, under different lighting conditions and typical degradation types. The reference images were generated using 12 classical enhancement algorithms and were subjectively evaluated by 50 volunteers to ensure alignment with human visual preferences. Consequently, we utilized the reference images in the UIEB dataset as the basis for calculating the K-value and set the average K-value of 0.9 for the 890 reference images as the reference value. The K-value can be adjusted as necessary. Specifically, in this paper, images with a K-value greater than 0.9 are classified as exhibiting color bias. Conversely, images without color deviation do not undergo compensation by the adaptive color channel.

3.3. Color and White Balance Correction Module

Correcting color distortion caused by lighting conditions can be achieved by selecting an appropriate color balancing algorithm, thereby mitigating the impact of the lighting environment on color performance. The grayscale world algorithm [20] is the most commonly employed method for white balance.

A color-biased image is defined as $I\left(x\right)$. $\widehat{Z}\left(x\right)$ is defined as the output image. The 3D vector $\gamma =\left[\gamma \right(R),\gamma (G),\gamma (B){]}^{\mathrm{\top }}$ is used to represent the color of the scene light source and is known as the color bias vector. $\mathcal{l}$ denotes a simple modification of $\gamma$, $\mathcal{l}=\left[\mathsf{\gamma }\right(\mathrm{G})/\mathsf{\gamma }(\mathrm{G}),1,\mathsf{\gamma }(\mathrm{G})/\mathsf{\gamma }(\mathrm{B}){]}^{\mathrm{\top }}$. Afifi and Brown [12] proposed a polynomial color mapping function to deal with the nonlinearity in $I\left(x\right)$. The expression is given in Equation (8).

$$\widehat{Z}\left(x\right)=r\left(M\right)\phi \left(I\right)$$

(8)

The above formulation is based on the polynomial kernel function proposed by Hong et al. [13]. φ: φ ([R, G, B]^⊤)→[R, G, B, RG, RB, GB, R², G², B², RGB, 1]^⊤ and M is denoted as a 3 × 11 matrix.

$$M=H\mathcal{l}$$

(9)

where H is the correction function [21]. The mapping of the input color clustering to H is derived from the mapping model [22].

3.4. Visibility Restoration Module

Due to the presence of underwater microorganisms and suspended particles, light is both scattered and absorbed as it propagates through the water. This phenomenon can result in images that are characterized by blurriness and darkness, ultimately leading to reduced visibility. This section addresses this issue by designing an underwater image visibility restoration module that employs an image defogging method based on the rank-one transmission prior [18]. This method assumes that the imaged scene is predominantly illuminated by spatially homogeneous light, except in the vicinity of the light source. The thickness of the scattered light is influenced by the depth of the scene and the degree of light transmission. Based on the theory of underwater optical imaging [23], the influence of the water body on light propagation is primarily manifested through three physical effects: the forward scattering effect caused by suspended particles, the backward scattering effect resulting from the water itself, and the direct attenuation effect that occurs during light transmission. The experimental data indicate that when the shooting distance is less than 2 m, the contribution of the forward scattering component to imaging quality decreases to less than 5%, rendering it negligible.

$$I_c\approx D_c+B_c$$

(10)

where $I_c$ is the image taken directly under water, $D_c$ is the direct attenuation component and $B_c$ is the backscattering component, Equation (10) can be further subdivided as follows:

$$I_c\left(x\right)=J_c\left(x\right)t_c\left(x\right)+A_c\left(1-t_c\left(x\right)\right)$$

(11)

In Equation (11), $x$ is the coordinate of any pixel in the image. c is the R, G, and B color channels of an image. $J_c\left(x\right)$ represents the ideal clear image. $A_c\in {\mathbb{R}}^{1\times 1\times 3}$ represents the ambient background light intensity in the R, G, and B channels, $t_c{\in \mathbb{R}}^{m\times n\times 3}$ and $t_c\left(x\right)$ represent the transmittance of each pixel. The value of this is related to the absorption and scattering of the light waves by seawater in the scene, and the expression is as follows:

$$t_c\left(x\right)=e^{-\beta \left(\lambda \right)d\left(x\right)}$$

(12)

β(λ) is the attenuation coefficient, including the effect of absorption of the light wave scattering effect. Since the attenuation coefficients of underwater light are different for different frequencies, they vary with the color channel. $d\left(x\right)$ is the scene depth.

The transmittance of the portion of ambient scattered light that affects imaging is described in Equation (11) by $1-t_c\left(x\right)$, where $\tilde{t}$ is used to denote the transmission map. Transmittance $\tilde{t}$ is strongly correlated with the scattered light, and if the spectrum of the scattered light is given, then the transmittance can be expressed as their correlation coefficient. Assuming that L₁ denotes an image of an ideal outdoor scene, image I is the image to be processed of size m × n, and the transmittance map $\tilde{t}\left(x\right)$ ∈ ${\mathbb{R}}^{1\times 3}$ forms the rank-one prior matrix $\tilde{T}\in {\mathbb{R}}^{r\times 3}\left(r=m\times n\right)$. $\tilde{t}\left(x\right)+t_c\left(x\right)=\left(\mathrm{1,1},1\right)$. The relation between the transmittance map $\tilde{\mathrm{t}}$(x) and the spectrum is

$$\tilde{t}\left(x\right)=C\left(x\right)\mathit{b}$$

(13)

where $C\in {\mathbb{R}}^r$ is the vector of coefficients, $\mathit{b}\in {\mathbb{R}}^{1\times 3}$ is the transport basis of $\tilde{t}$, and $\tilde{t}\in {\mathbb{R}}^{m\times n\times 3}$. The modulo expansion of $\tilde{\mathrm{t}}$ is the matrix $\tilde{T}\in {\mathbb{R}}^{3\times r}$ with the expression:

$$\tilde{T}={\left(\mathit{c}\mathit{b}\right)}^{\mathrm{\top }}$$

(14)

where $\mathit{c}$ is the column main order vector of $C$. $\tilde{T}$ is the rank-one matrix and the first-order prior on the scattering map.

With the introduction of the rank-one prior, the traditional physical model of optical imaging is defined:

$$L_1\left(x\right)={\displaystyle \frac{I\left(x\right)-\omega \tilde{t}\left(x\right)A}{\max \left(1-\omega \tilde{t}\left(x\right),t_0\right)}}$$

(15)

$$\tilde{t}\left(x\right)=⟨I\left(x\right),S_{nu}⟩\cdot S_{nu}$$

(16)

$$S_u^c={\displaystyle \frac{1}{\mid \mathsf{\Omega }\mid }}\sum _{x\in \mathsf{\Omega }}{I}_c\left(x\right),c\in \left\{R,G,B\right\}$$

(17)

where $A$ is the ambient light, $\mathsf{\Omega }$ is the domain of pixel positions with mn elements, $\omega \in \left(\mathrm{0,1}\right)$ is an introduced constant relaxation parameter and $t_0$ is the lower bound for the stabilization calculation. Note that $\tilde{t}\left(x\right)+t_c\left(x\right)=(1,1,1)$, ${\forall }_x\in \mathsf{\Omega }$. In this section, $t_0$ is set to 0.001. $S_{nu}$ is the result of normalizing $S_u^c$, which can approximate the direction of transmission, and $L_1\left(x\right)$ is the output image. The physical model performs well in image defogging and deblurring tasks. The reduction in scattered light in underwater images results in clearer images. As shown in Figure 4, it is the processing of the visibility restoration module.

3.5. Comparative Enhancement Module

Underwater image quality is frequently compromised by complex optical environments, which are characterized by non-uniform light distribution, reduced detail contrast, and blurring effects caused by the scattering of suspended particles. The contrast enhancement module improves color saturation and contrast, thereby highlighting the textural details of the image. This enhancement facilitates subsequent analysis and processing.

The AGCWD [24] algorithm dynamically adjusts image luminance through gamma correction while incorporating a weighted probability distribution of pixel intensities, thereby effectively improving the visibility of underexposed regions while maintaining preserving natural image appearance. Assume that X represents $I_c$ or $\widehat{Z}\left(x\right)$ and that its luminance l varies in the range of $l_{min}$ and $l_{max}$. The dynamic range of the image X(i, j) luminance gray scale I is $l_{min}$ and $l_{max}$, where $l_{min}$ is the smallest gray scale order. The probability density function of the gray scale pixel distribution of a statistical image:

$$pdf\left(l\right)={\displaystyle \frac{nl}{MN}}$$

(18)

where MN is the total number of pixels and nl is the number of pixels corresponding to the gray scale n.

The optimal tuning of the probability density function (pdf) is achieved by utilizing cdf and the normalized gamma transform curve while preserving the statistical properties of the original histogram. Specifically, the probability density function is initially calculated based on the statistical histogram from the previous step. Subsequently, the weighted distribution (WD) function is introduced to correct the histogram, ultimately resulting in the optimized weighted probability density function:

$${pdf}_w\left(l\right)=\mathit{max}\left(pdf\right)\times {\left({\displaystyle \frac{pdf\left(l\right)-\mathit{min}\left(pdf\right)}{\mathit{max}\left(pdf\right)-\mathit{min}\left(pdf\right)}}\right)}^{\alpha }$$

(19)

A lower gamma parameter produces a more significant adjustment and can take the value [0.1, 0.5], which can be taken as 0.5 as a rule of thumb to give a smoother function curve. We calculate the cumulative distribution function (cdf) using A to carry out the normalization. Using the weighted probability density function calculated from Equation (19), the smoothed cumulative distribution function is derived:

$${cdf}_s\left(l\right)=\sum _{{l=l}_{min}}^{l_{max}}{\displaystyle \frac{{pdf}_w\left(f\right)}{\sum {pdf}_w}}$$

(20)

$$T\left(l\right)=\left(l_{max}-l_{min}\right)\ast {\left({\displaystyle \frac{l}{l_{max}-l_{min}}}\right)}^{\gamma }$$

(21)

$$\gamma =1-{cdf}_s\left(l\right)\ast P$$

(22)

T(l) is the transform function. Adaptive gamma correction uses the image cumulative distribution function information cdf of Equation (20), where P denotes the adaptive parameter, set to 1, with a range of values [0.5, 1].

$$L_2\left(x\right)=T\left(X\left(i,j\right)\right)$$

(23)

Mapping the luminance values of each pixel point of the image gives the contrast-enhanced image $L_2\left(x\right)$.

3.6. Fusion Module

In this section, a multi-scale feature fusion strategy is employed to co-fuse the visibility-restored processed image with the contrast-enhanced image. A multi-exposure image fusion method, which utilizes adaptive weights that reflect relative pixel intensities and global gradients, aims to fully leverage multiple sources of information and construct more effective fusion weights. This method innovatively introduces a dual weighting mechanism that incorporates local weights based on pixel intensities and global gradients. This approach optimally fuses visibility restoration images with contrast enhancement images [25].

3.6.1. Weighted Design Based on Pixel Intensity

The pixel intensity weighting design computes pixel intensity weights derived from pixel intensity values, which are utilized to emphasize the brightness information across images captured with varying exposures.

The pixel intensity-based weighting design regulates the weights to correspond with the overall brightness of the image, considering the presence of more well-exposed pixels when there is a significant difference between the input image and its neighboring exposed images. Consequently, the first weight is denoted as

$$w_{1,n}\left(x\right)=\exp \left(-{\displaystyle \frac{{\left(L_n\left(x\right)-\left(1-m_n\right)\right)}^2}{2{\sigma }_n^2}}\right)$$

(24)

In Equation (24),

$${\sigma }_n=\left\{\begin{array}{cc}2\alpha \left(m_{n+1}-m_n\right)& n=1\\ \alpha \left(m_{n+1}-m_{n-1}\right)& 1<n<N\\ 2\alpha \left(m_n-m_{n-1}\right)& n=N\end{array}\right.$$

(25)

N is the number of images in a set of multiple exposure images. $L_n\left(x\right)$ is the pixel intensity of the nth image. $m_n$ denotes the average of the pixel intensities of the nth image. When $L_n\left(x\right)$ is close to $1-m_n$, the weight is significant, denoted by $\exp \left(-{\left(L_n\left(x\right)-\left(1-m_n\right)\right)}^2\right)$. The assignment takes into account the exposure levels of the input image and its neighboring images. ${\sigma }_n$ varies with different levels of brightness differences. The authors of the original article proposed setting α to 0.75, a value that was also validated by An et al. [26] in underwater images. Consequently, we have also set the value of α to 0.75. $w_{1,n}\left(x\right)$ represents the first weight that encodes the relative luminance.

3.6.2. Weighted Design Based on Global Gradient

Traditional methods primarily assign fusion weights based on the local gradient magnitude in multi-exposure image fusion. While this approach effectively preserves the detailed features of high-contrast regions, it may overlook the exposure quality of low-gradient areas. An innovative weight assignment strategy based on cumulative histogram analysis evaluates exposure quality by analyzing the distribution characteristics of pixel intensities. Specifically, the differential properties of the cumulative histogram can reveal pixel distribution patterns under varying exposure conditions. In underexposed images, the cumulative histogram displays a steep upward trend in the low-intensity interval, indicating a significant presence of saturated pixels in this region. Conversely, in moderately exposed images, the cumulative histogram exhibits a more gradual increase, reflecting a more uniform distribution of pixel values. When a pixel is situated in an interval with a lower rate of change in the cumulative histogram, it suggests that the intensity value is statistically rare, thus identifying it as a quality exposure region. Consequently, these observations were encoded as weights:

$$w_{2,n}\left(x\right)={\displaystyle \frac{{\mathrm{Grad}}_n{\left(L_n\left(x\right)\right)}^{-1}}{\sum _{n=1}^N{\mathrm{Grad}}_n{\left(L_n\left(x\right)\right)}^{-1}+ϵ}}$$

(26)

To prevent the denominator from becoming zero, a small positive value, denoted as ϵ, is added to it. ${\mathrm{Grad}}_n\left(·\right)$ denotes the gradient of the cumulative histogram, which is called the global gradient. In the field of image processing, the global gradient characterizes the gradient features of similar regions across a large area of the image, in contrast to the local gradient, which only reflects variations in the vicinity of a specific pixel. This global analysis method captures the structural features and spatial distribution patterns of an image more comprehensively. During the weight calculation process, local gradient weights must be effectively combined with global gradient weights, and their coordination is achieved through normalization. The final weight calculation formula can be expressed as follows:

$$W_n\left(x\right)={\displaystyle \frac{w_{1,n}\left(x\right)\times w_{2,n}\left(x\right)}{\sum _{n=1}^N{w}_{1,n}\left(x\right)\times w_{2,n}\left(x\right)+ϵ}}$$

(27)

$W_n\left(x\right)$ denotes the normalization result of the two weights for each image. Therefore, the final fusion result of the model is shown in Equation (28):

$$F\left(x\right)=W_1\left(x\right)L_1\left(x\right)+W_2\left(x\right)L_2\left(x\right)$$

(28)

4. Experiment

We first present the experimental details of the UMMC underwater image enhancement method in the experimental section. Next, we conduct experiments using various datasets to compare our approach with mainstream methods. We perform both quantitative and qualitative analyses based on comparative tests conducted on real and synthetic underwater images. Subsequently, a keypoint matching algorithm is executed [27]. Finally, the operational performance of the proposed method is validated through runtime testing.

4.1. Experimental Details

The primary objective of this section is to compare the proposed underwater image enhancement algorithm with ten widely used underwater image enhancement algorithms. In this experiment, we utilized underwater image datasets, including the UIEB dataset [6], the SUID dataset [28], and the EUVP dataset [29].

The UIEB dataset [6] serves as a benchmark for real underwater imagery and comprises 890 sets of paired data, along with 60 sets of unreferenced challenging samples. This dataset encompasses a wide range of typical underwater imaging conditions, including natural illumination, artificial light sources, and mixed lighting scenarios. The construction process for the reference image synthesizes the outputs of 12 classical enhancement algorithms, selecting the enhancement that best aligns with human visual perception through subjective evaluation. Additionally, the SUID synthetic dataset [28] systematically simulates 900 sets of underwater images, varying in turbidity levels and types of degradation. The EUVP datasets [29] are a large number of high-quality underwater images captured by high-resolution camera equipment and specialized underwater photography.

We randomly selected 100 pairs of images from the UIEB dataset, which included reference images, and 30 pairs of images from the SUID dataset for both quantitative and qualitative evaluations. Additionally, we selected 50 images from the EUVP dataset to conduct runtime duration tests.

4.1.1. Comparative Methods

We compare UMMC with seven UIE methods to validate our superior performance. The methods include an underwater image enhancement method based on an attention mechanism and adversarial autoencoder (UW-AAE) [30], an underwater image enhancement method by correcting color distortion, enhancing contrast, and enriching details (EUICCCLF) [31], a technique focused on minimizing color loss and enhancing local adaptive contrast (MLLE) [32], a method for optimizing contrast enhancement through piecewise color correction and dual prior (PCDE) [33], an underwater image enhancement convolutional neural network model based on underwater scene prior (UWCNN) [34], which we designate as Type I enhancement, an image enhancement technique that separates content and style (UIESS) [35] and a Bayesian Retinex algorithm for enhancing a single underwater image using multi-order gradient priors of reflectance and illumination (BRUIE) [36].

4.1.2. Metrics

For the UIEB dataset, we employed the Natural Image Quality Evaluator (NIQE) [37], the Underwater Color Image Quality Evaluator (UCIQE) [38], and the Non-Reference Underwater Image Quality Measurement (UIQM) [39] assessment metrics. For real underwater images (the UIEB dataset), the selected no-reference quality evaluation metrics are UCIQE and UIQM, both of which exhibit a positive correlation with image quality. In contrast, the NIQE metric shows a negative correlation. Together, these three metrics reflect the degree of consistency between enhancement results and human visual perception. For synthetic underwater images (the SUID dataset), a full-reference evaluation system is employed. In this system, PSNR [40] quantifies the pixel-level similarity between the enhanced image and the reference image, while SSIM [41] assesses the degree of match between the two images in terms of structural features.

4.2. UIEB Dataset Evaluation

This section is evaluated for a randomly selected set of images from a dataset with reference images. Out of 100 randomly selected pairs of images, underwater images of different underwater scenes are chosen to be presented in the subjectivity assessment to go through the enhancement characteristics of different methods.

The output depicted in Figure 5 demonstrates that the UIESS algorithm obviously improves the image brightness and improves the original color bias but introduces a slight color distortion. The MLLE algorithm successfully resolves the color cast issue and restores the image’s colors effectively, but the brightness is excessively high, resulting in a loss of detail. UW-AEE handles blue-green tinted images well, but it is not as effective in terms of defogging effects and boosting contrast. EUICCCLF has the problem of color loss. With UWCNN, it can be clearly seen that the enhancement is poor. PCDE has the problem of detail loss, and BRUIE has the problem of insufficient color compensation and local darkness. Our algorithm outperforms others in processing images with varying color shades, improving brightness and visibility. However, the saturation enhancement is somewhat excessive and requires further optimization.

Table 1. An objective assessment of real underwater benchmarks with different algorithms. The best result is in red, whereas the second-best result is in green.

Methods	Metric
	NIQE	UCIQE	UIQM
UW-AAE	15.705	0.789	115.897
EUICCCLF	15.406	0.704	116.363
MLLE	12.771	2.475	114.243
PCDE	14.085	1.453	126.915
UWCNN	15.328	0.897	69.118
UIESS	16.896	1.059	99.772
BRUIE	12.970	0.812	115.0823
UMMC	12.612	2.532	124.220

presents the evaluation results for the real-world dataset. As indicated in the table, our method outperforms all other approaches. This demonstrates that our method effectively compensates for the color absorption in underwater images while preserving more information.

4.3. SUID Dataset Evaluation

In this section, we present and conduct quantitative experiments on 30 randomly selected pairs of images, choosing generated underwater images with different color shifts, turbidity, and low light in comparison to standard images in a subjectivity assessment of the 30 images.

As illustrated in Figure 6, UMMC and MLLE had the best overall performance in terms of visual representation in the SUID dataset. Compared to MLLE, UMMC handles the foggy and turbid images with average results, but the images are more detailed. EUICCCLF shows unwanted color distortion. PCDE makes the images too dark overall. UIESS images show non-yellow tones overall. BRUIE removes the fog better, with noise and localized over-darkness in some of the images. The UW-AAE enhancement is mediocre and does not effectively deal with color casts and only slightly improves them. UWCNN effectively improved the blue-green bias, but new color problems arose and it did not defog well.

Table 2. An objective assessment of synthetic underwater images with different algorithms. The best result is in red, whereas the second-best result is in green.

Methods	Metric
	PSNR	SSIM
UW-AAE	9.679	0.361
EUICCCLF	16.125	0.786
MLLE	18.010	0.752
PCDE	14.703	0.636
UWCNN	16.162	0.792
UIESS	16.071	0.866
BRUIE	17.422	0.758
UMMC	18.258	0.824

shows the evaluation of the SUID synthetic dataset, and the results for both data demonstrate the high visual quality of the processing effect of our method. It is also verified that in the subjective assessment, our enhancement results in richer details and a reduction in the loss of structural information compared to other images.

4.4. Keypoint Matching

By applying the keypoint matching algorithm to underwater images, we can extract rich feature point information that significantly enhances target detection and recognition. A higher number of feature points in image matching indicates a greater wealth of information contained within the image, thereby improving the algorithm’s performance.

In this section, an underwater image is randomly selected, and the original image is rotated by 180 degrees to align the feature points with those of the original image. The results of feature point matching using different algorithms are presented in Figure 7. As can be seen from

Table 3. The image feature point matching results for each algorithm are presented. The best result is in red, whereas the second-best result is in green.

Methods	Left Match Point	Right Match Point
Raw	1110	1128
UW-AAE	2367	2341
EUICCCLF	4149	4216
MLLE	3132	3104
PCDE	4031	4054
UWCNN	258	233
UIESS	1637	1599
BRUIE	4260	4260
UMMC	3562	3474

, the results of the keypoint matching experiment show that the method of this paper does not achieve good results in this experiment, which is in the middle of the range of various methods.

4.5. Ablation Study

We conducted ablation experiments to systematically validate the role of each module in the proposed methodology. We progressively removed or replaced key components of the model using the control variable method, including the following: (a) white balance and visibility restoration only (WBVR), (b) white balance and contrast enhancement only (WBCE), (c) white balance only (WB), (d) contrast enhancement only (CE), (e) visibility restoration only (VR), and (f) the complete method (UMMC). Figure 8 presents a visual comparison of these approaches. The experimental results indicate that CE significantly improves the brightness of the image, while WB effectively reduces color deviation; however, it also contributes to the overall darkening of the image. Both VR and WBVR tend to produce oversaturation, which can lead to a loss of detail. Additionally, under uneven lighting conditions, the visibility of dark details is enhanced, but the lighter areas become overexposed, resulting in color distortion. Black artifacts may appear at the junctions of visible content and darker regions, highlighting a limitation of the visibility restoration module (VR) when applied to non-uniformly illuminated images. WBVR corrects color bias to a certain extent compared to VR, underscoring the importance of WB. After fusing with contrast-enhanced images, UMMC compensates for the deficiencies of VR, further demonstrating the superiority of the UMMC full model. In contrast, WBCE enhances image details without equally compensating for color accuracy. UMMC effectively integrates WBVR and WBCE, resulting in images characterized by uniform colors and rich details. The results in

Table 4. An underwater image quality evaluation of different variants of the presented method. The best result is in red, whereas the second-best result is in green.

Methods	Metric
	NIQE	UCIQE	UIQM
WBVR	15.254	0.782	104.244
WBCE	14.332	0.633	104.375
WB	16.417	0.703	100.291
CE	12.979	1.757	113.581
VR	12.778	0.784	104.663
UMMC	12.612	2.532	124.220

demonstrate the effectiveness of UMMC.

4.6. Parameters and FLOPs

In this study, the computational performance of each method was systematically evaluated. The experimental configurations are as follows: the hardware platform consists of an Intel Core i7-13620H processor (4.9 GHz) with 16 GB of RAM. Fifty test images, each measuring 256 × 256 pixels, were randomly selected from the EUVP dataset. All methods were tested on the same computer, which was equipped with an Intel^® Core™ i7-13620H CPU operating at 4.9 GHz and 16 GB of RAM. EUICCCLF, MLLE, PCDE, BRUIE, and our method were implemented in MATLAB (R2023b), while the UIESS and UWCNN methods were developed using PyTorch (1.12.0), and the UW-AEE method was implemented in TensorFlow (1.14.0).

Table 5. Comparison of parameters, FLOPs and runtime on different UIE methods. The best result is in red, whereas the second-best result is in green.

Methods	Platform	#Param.	FLOPs	Time (s)	PSNR	SSIM
UW-AAE	TensorFlow	148.77 M	2805.34 G	6.97	17.12	0.85
EUICCCLF	MATLAB	-	-	0.02	16.82	0.71
MLLE	MATLAB	-	-	0.08	16.26	0.64
PCDE	MATLAB	-	-	0.16	15.13	0.60
UWCNN	PyTorch	0.04 M	2.61 G	0.12	11.08	0.30
UIESS	PyTorch	4.2 M	26.35 G	0.43	23.37	0.73
BRUIE	MATLAB	-	-	0.15	15.13	0.60
UMMC	MATLAB	-	-	0.07	18.03	0.87

presents a comparison of the average runtimes for the different UIE methods. The experimental results indicate that our method demonstrates superior performance in UIE experiments.

4.7. Summary

Through the above experiments, we found that UMMC achieved good results overall in the six experimental tests. For underwater image enhancement, we successfully eliminated color bias while largely preserving image quality. At the same time, we also found that it is easy to overcompensate when the degree of color bias is small, and for foggy images, it does not work as well as it should, and the processing effect is poor. The modeling algorithm needs to be adjusted in the future. UMMC runs faster while ensuring quality. The significance of each component of UMMC and its overall performance is further illustrated through the ablation experiment. However, the feature point matching test did not achieve the desired results; the model needs to be further adjusted for analysis.

5. Conclusions

In this paper, a multi-module combination of underwater image enhancement algorithm UMMC is proposed to address the problems of the color distortion, low visibility and low brightness of underwater images. Our approach involves the development of a new network designed to improve enhancement performance through five key modules: the color detection module, the color and white balance correction module, the visibility restoration module, the contrast enhancement module, and the image fusion module. After experimental validation, the method proposed in this paper effectively addresses the issues of color cast and low brightness, making it highly suitable for enhancing real underwater images. Furthermore, our model not only improves image quality but also operates with a faster runtime.

Although UMMC has achieved significant results, there are still several limitations that need to be addressed. For the issue of supersaturation in images with low levels of chromatic aberration, as well as in foggy images that are typically processed, it is essential to optimize algorithms and explore new solutions in the future. In addition, the setting of K is also limiting to some extent, and as the next step, we will look for new ways to address this shortcoming such as introducing an adaptive threshold that depends on the image statistics, K. This optimization will be particularly beneficial in addressing a wider range of underwater image challenges. In the future, we plan to adopt a combined approach with deep learning to explore real-time processing and optimization in low-resource environments.