A New Biorthogonal Spline Wavelet-Based K-Layer Network for Underwater Image Enhancement

Abstract

Wavelet decomposition is pivotal for underwater image processing, known for its ability to analyse multi-scale image features in the frequency and spatial domains. In this paper, we propose a new biorthogonal cubic special spline wavelet (BCS-SW), based on the Cohen–Daubechies–Feauveau (CDF) wavelet construction method and the cubic special spline algorithm. BCS-SW has better properties in compact support, symmetry, and frequency domain characteristics. In addition, we propose a K-layer network (KLN) based on the BCS-SW for underwater image enhancement. The KLN performs a K-layer wavelet decomposition on underwater images to extract various frequency domain features at multiple frequencies, and each decomposition layer has a convolution layer corresponding to its spatial size. This design ensures that the KLN can understand the spatial and frequency domain features of the image at the same time, providing richer features for reconstructing the enhanced image. The experimental results show that the proposed BCS-SW and KLN algorithm has better image enhancement effect than some existing algorithms.

1. Introduction

Underwater imaging introduces unique challenges that necessitate the exploration of more advanced wavelet-based solutions. Underwater images are often compromised by complex environmental factors, including lighting conditions, water quality, and scattering, leading to blurred images and color distortion. Notably, there are significant differences in the attenuation rates of the different wavelengths of light in water, longer wavelengths, such as red light, attenuate more rapidly than shorter wavelengths, like blue or green light. This discrepancy results in a pronounced color cast in underwater images, which typically exhibit a blue or green hue.

The underwater environment’s lighting, combined with the scattering effects of ambient light on plankton and suspended particles, exacerbates the blurring of underwater images. Hence, there is a pressing need for techniques capable of analysing the depth, details, and texture information of images to discern their detailed features accurately. Moreover, underwater images often feature complex backgrounds and biological elements. Adjacent regions within an image may exhibit significantly different feature information due to variations in the structure and physical location, while non-adjacent regions might share similar features [1,2]. This complexity demands a nuanced approach to underwater image analysis, highlighting the critical role of sophisticated wavelet-based methodologies in addressing these challenges.

Physics-based methods construct models using the physical and optical characteristics of images captured underwater [3]. These approaches examine the physical processes responsible for degradation, such as color distortion or scattering, and aim to correct them to enhance underwater images. However, a singular physics-based model may not encompass the diverse array of intricate physical and optical factors inherent in underwater environments. This limitation results in inadequate generalization and may yield outcomes characterized by either excessive or insufficient enhancement.

Wavelet analysis is a potent tool for signal processing, playing an instrumental role in underwater image enhancement. While wavelet decomposition is known for its ability to process image signals across multiple scales and to analyse signals in the frequency domain [4,5]. Traditional wavelets, such as Haar, have been applied across a variety of vision tasks for decomposing visual signals into single or multiple layers, aiming to reduce noise and enhance the contrast of these signals. However, when faced with the complexities inherent to underwater images, particularly in terms of multi-level decomposition and the capture of high-frequency details, the limitations of these traditional wavelets become apparent. This has led to a reevaluation of the trajectory of wavelet theory, with particular emphasis on the contributions of Mallat’s multi-resolution analysis [6]. This reassessment serves as a catalyst for rethinking the design of wavelet functions, thereby more effectively addressing the specific enhancement needs of underwater images. By refining wavelet functions to cater to the unique challenges posed by underwater environments, such as blurring, color cast, and detail loss, this approach seeks to develop more sophisticated and effective methods for underwater image enhancement.

Additionally, spline wavelets usually exhibit less of a ringing effect, which helps to prevent the introduction of unnecessary artifacts in underwater image enhancement tasks. This provides important inspiration for us to explore the combination of wavelet decomposition with deep learning-based image enhancement tasks. As shown in Figure 1, this is our decomposition of underwater images based on BCS-SW. After the underwater image is decomposed by BCS-SW, four sub-band images, ca, cd, ch, and cv, are obtained by conducting convolution and downsampling. Most deep neural network methods that incorporate wavelet analysis tend to introduce Haar wavelets into the network; we aim to explore the integration of more complex wavelets with deep neural networks to better combine the advantages of deep neural networks with wavelet analysis. Our contributions to this work can be summarized as follows:

1. We propose a new BCS-SW based on the CDF wavelet construction method and the cubic special spline algorithm. BCS-SW is a compactly supported, symmetric spline wavelet. Both BCS-SW and its corresponding dual wavelet are constructed for image decomposition and reconstruction, respectively, based on a multi-resolution analysis and two-scale equations. BCS-SW demonstrates superior performance in the frequency domain signal extraction, particularly for complex signals, providing a more versatile and efficient approach for interpreting local features and color degeneration in underwater images;

2. We propose a K-layer network (KLN) based on the BCS-SW for underwater image enhancement. Specifically, the KLN utilizes K-layer wavelet decomposition on underwater images to extract features across multiple frequency domains. Each layer of decomposition is paired with convolution-based layers, each tuned to a distinct scale. These convolution-based layers enhance the network’s comprehension of the images’ spatial characteristics, facilitating more stable frequency domain signals;

3. We devise qualitative and quantitative experiments on multiple underwater image datasets, showcasing the effectiveness of the BCS-SW and KLN in underwater image enhancement. Our work serves as an important reference for the integration of complex wavelet transforms into deep neural networks, demonstrating the potential of this approach.

Figure 1. Visualization of the decomposed underwater original and ground truth images by the biorthogonal cubic special spline wavelet (BCS-SW). After the underwater image is decomposed by BCS-SW, ca is the approximate information of the original image, cd is the diagonal information of the original image, ch is the horizontal information of the original image, and cv is the vertical information of the original image.

2. Related Work

For spline wavelet algorithms, Khan et al. [7] pioneered the formulation of B-spline wavelet packets, alongside the creation of their corresponding dual wavelet packets, delving into the exploration of the inherent properties of B-spline wavelet packets. Cohen et al. [8] proposed the Cohen–Daubechies–Feauveau (CDF) wavelet family, and this wavelet family uses the compact support set spline function as the parent wavelet and has the properties of compact support set and orthogonal, which becomes an important milestone in the study of spline wavelet. Olkkonen et al. [9] introduced the shift-invariant gamma spline wavelet transform for the tree-structured subscale analysis of asymmetric signal waveforms and systems with asymmetric impulse responses. Building on this work,  Tavakoli and Esmaeili [10] subsequently orchestrated the development of biorthogonal multiple knot B-spline (MKBS) scaling functions, along with the inception of multiple knot B-spline wavelet (MKBSW) basis functions, thereby enriching the repository of the tools available in wavelet analysis. Compared to Haar wavelets and Daubechies wavelets, MKBSW offers superior smoothness and continuity, which make them perform better in accurately approximating and analysing continuous signals.

For wavelet analysis applied to image processing, the advantages include its ability to provide multi-resolution analysis, efficient compression, and effective denoising capabilities. However, the disadvantages include higher computational complexity, issues with boundary effects, and difficulties in choosing the appropriate wavelet basis. Spline wavelet transform has been applied to various image processing tasks, such as image super-resolution and denoising proposed by Huang et al. [11] and Kang et al. [12]. A new spatially adaptive method for recovering noisy blurred images proposed by Banham and Katsaggelos [13], which is particularly effective in producing crisp deconvolution while suppressing noise in the flat regions of the image. In the realm of underwater image enhancement, several physically based methods utilize the discrete wavelet transform (DWT) to decompose the image and process it in the frequency domain. For instance, Sree Sharmila et al. [14] proposed a novel image resolution enhancement method based on the combination of DWT and stabilized wavelet transform (SWT), employing the histogram shift shaping method for wavelet decomposition to enhance the contrast and resolution of the image. Singh et al. [15] employed a discrete wavelet transform-based interpolation technique for resolution enhancement. Ma and Oh [16] introduced a dual-stream network based on Haar wavelets [17], which effectively tackles color cast and enhances blurry details in underwater images. Huang et al. [11] introduced a convolutional neural network (CNN) approach based on wavelets, capable of ultra-resolving very low-resolution face images, as small as 16 × 16 pixels, to larger versions at multiple scaling factors (2×, 4×, 8×, and even 16×) within a unified framework.

For deep neural networks applied to underwater image processing, their capability primarily depends on the quality of the training dataset and the capacity of the network. Introducing wavelet analysis into deep neural networks can effectively accelerate the network’s processing of image features and enhance the network’s understanding of images. Due to the robustness and generalization capabilities of deep neural networks in underwater image enhancement tasks, a significant body of work has emerged focusing on leveraging these networks. Perez et al. [18] proposed employing CNN for underwater image enhancement. They trained the CNN using image restoration techniques to achieve an end-to-end transformation model between a hazy image and its corresponding clear image. Wang et al. [19] introduced a CNN-based method named UIE-Net, which is trained on two tasks: color correction and haze removal. This unified training approach enables the network to learn powerful feature representations for both tasks simultaneously. To enhance the extraction of intrinsic features within local blocks, their learning framework incorporates a pixel perturbation strategy, significantly improving convergence speed and accuracy. Goodfellow et al. [20] introduced generative adversarial nets (GANs), presenting a novel methodology for training generative models. As research into GANs deepens, underwater image enhancement increasingly becomes a task of transforming between the underwater domain and the enhancement domain. Li et al. [21] proposed Water-GAN to generate a large training dataset comprising corresponding depth, in-air color images, and realistic underwater images. Water-GAN’s generator is responsible for synthesizing real and depth images into an underwater image, while Water-GAN’s discriminator classifies the real images from the synthesized ones. Moreover, Fabbri et al. [22] proposed UGAN, GANs specifically designed to enhance the quality of underwater images. The model’s design objective is to improve the visibility and clarity of underwater images through adversarial training, addressing the challenges posed by the absorption and scattering of light in underwater environments.

Although deep neural networks (DNNs) have demonstrated strong capabilities in underwater image processing, especially in image enhancement, object detection, and classification, there are some obvious limitations in their application. Underwater images are often heavily affected by noise and perturbations, and deep neural networks may exhibit some vulnerability when processing such data, especially if the network has not been trained on data containing high noise. We propose a new deep neural network K-layer network for underwater image processing based on BCS-SW. In the experimental part, we also compare it with other cutting-edge underwater image processing techniques to show the advantage of the KLN network over other methods.

3. Methodology

The current research that has more combinations with deep neural networks is mostly the Haar wavelets because of the orthogonality of Haar wavelets. Although they have the advantages of simple computation and easy implementation, they also have some limitations, such as a lack of smoothness, resulting in a loss of low-frequency information, and sensitivity to noise, and may not be as good as spline wavelet transform in processing complex images and continuous signals. Hence, we propose a novel spline wavelet combined with deep neural networks to process underwater images.

Inspired by compactly supported spline wavelets [23], which provide a better approximation of various images and its wavelet basis functions can be flexibly adjusted according to the needs of applications. We propose a new biorthogonal special cubic special spline wavelet (BCS-SW) for underwater processing in this work. Furthermore, we also propose a new deep neural network, the K-layer network (KLN), for underwater image enhancement based on BCS-SW.

3.1. A New Biorthogonal Cubic Special Spline Wavelet (BCS-SW)

3.1.1. Cubic Special Spline Algorithm

In our work, BCS-SW is based on the cubic special spline algorithm proposed by Chen and Cai [24]. They proposed a novel spline algorithm and provided various representations of cubic splines with different compact supports. In this paper, we selected one of the cubic splines with the smallest compact support, as shown in Equation (1), and on the basis of this spline, we derived a new class of spline wavelet algorithms following the CDF method of wavelet construction.

$$S(t)=\frac{451}{3}{\beta }_3(t)-\frac{256}{3}({\beta }_3(t-\frac{1}{16})+{\beta }_3(t+\frac{1}{16}))+\frac{64}{3}({\beta }_3(t-\frac{1}{8})+{\beta }_3(t+\frac{1}{8})),$$

(1)

and ${\beta }_3(t)$ is the cubic B-spline:

$${\displaystyle {\beta }_3(t)=\sum _{i=0}^4\frac{{(-1)}^i}{3!}\left(\begin{array}{c}4\\ i\end{array}\right){\left(t+2-i\right)}^3·\varpi \left(t+2-i\right),t\in R,}$$

(2)

where $\varpi (t)$ is the unit step function

$$\varpi (t)=\left\{\begin{array}{cc}0,\hfill & t<0,\hfill \\ 1,\hfill & t\ge 0.\hfill \end{array}\right.$$

(3)

$S(t)$ comes out from a linear combination of the normalized and the shifted B-splines of the same order. Consequently, $S(t)$ can inherit nearly all the favourable properties of ${\beta }_3(t)$, including analyticity, central symmetry, local support, and high-order smoothness. Moreover, $S(t)$ can directly interpolate the provided data without the need to solve coefficient equations, a capability that B-spline lacks.

The Fourier transform expressions of $S(t)$:

$$\widehat{S}(\omega )=\left(\frac{451}{3}-\frac{512}{3}cos\frac{\omega }{16}+\frac{64}{3}cos\frac{\omega }{8}\right){\left(\frac{sin\frac{\omega }{2}}{\frac{\omega }{2}}\right)}^4.$$

(4)

The spline $S(t)$ and the Fourier transform $\widehat{S}(\omega )$ are separately plotted in Figure 2.

Figure 2. Analysis of the cubic special spline $S(t)$. (a) The graph of the $S(t)$. (b) The graph of the Fourier transform $\widehat{S}(\omega )$.

3.1.2. Constructing Biorthogonal Cubic Special Spline Wavelet (BCS-SW)

Chui [25] and Graps [26] have proved B-spline ${\beta }_m(t)$ is the scale function of the corresponding multi-resolution analysis. $S(t)$ is formed by the linear combination of B-spline ${\beta }_3(t)$ translation and expansion. Therefore, we can naturally deduce the following conclusion.

The subspaces $V_j^3$ are generated by $S(t)$ binary dilation and integer translation, as follows:

$$V_j^3=\overline{span}\left\{2^{\frac{j}{2}}S\left(2^{j}t-k\right),k\in Z\right\},j\in Z,$$

(5)

where ${\left\{V_j^3\right\}}_{j\in \mathbf{Z}}$ forms a general multi-resolution analysis (GMRA) in $L^2(\mathbf{R})$, called spline multi-resolution analysis. $g(t)$ is the corresponding scaling function. According to the theory of wavelet construction, $S(t)$, as a scale function, can construct a new wavelet $\psi (t)$. Let $S^{*}(t)$ be the dual scaling function of $S(t)$ and ${\psi }^{*}(t)$ be the dual wavelet of $\psi (t)$, then their corresponding low-pass filters are:

$$H(\omega )=\frac{1}{2}\sum _{n=N_1}^{N_2}{h}_n{\mathrm{e}}^{-\mathrm{i}n\omega },H^{*}(\omega )=\frac{1}{2}\sum _{n=L_1}^{L_2}{h}_n^{*}{\mathrm{e}}^{-\mathrm{i}n\omega }.$$

(6)

And high-pass filters are:

$$G(\omega )=\frac{1}{2}\sum _{k=1-L_2}^{1-L_1}{g}_k{\mathrm{e}}^{-\mathrm{i}k\omega },G^{*}(\omega )=\frac{1}{2}\sum _{k=1-N_2}^{1-N_1}{g}_k^{*}{\mathrm{e}}^{-\mathrm{i}k\omega },$$

(7)

where $N_1,N_2,L_1,L_2$ are all integers, $N_2-N_1+1$ and $L_2-L_1+1$ are the lengths of $H(\omega )$ and $H^{*}(\omega )$, respectively, and $g_k={(-1)}^k{h}_{1-k}$, $g_k^{*}={(-1)}^k{h}_{1-k}^{*}$. All coefficients are real coefficients.

We also construct a new class of compactly supported wavelets based on CDF. We are aware that wavelets with compact supports exist as long as the two-scale sequence of the related scaling function is finite. In the paper, we set $H(\omega )$ and $H^{*}(\omega )$ as odd-length and the support set is symmetric at about 0. The vanishing moment order of $H(\omega )$ and $H^{*}(\omega )$ are N and $N^{*}$, respectively, and they can also have the following representation:

$$H(\omega )=cos{(\frac{\omega }{2})}^{2N}Q(cos(\omega )),$$

(8)

$$H^{*}(\omega )=cos{(\frac{\omega }{2})}^{2N^{*}}{Q}^{*}(cos(\omega )),$$

(9)

where $Q(cos(\omega ))$, $Q^{*}(cos(\omega ))$ are the polynomials of $cos(\omega )$.

Let

$${\displaystyle P(si{n}^2(\frac{\omega }{2}))=Q(cos(\omega ))\overline{Q^{*}(cos(\omega ))},}$$

(10)

and when ${\displaystyle y=si{n}^2(\frac{\omega }{2})}$, we also have:

$$P(y)=\sum _{n=0}^{L-1}\left(\genfrac{}{}{0pt}{}{L-1+n}{n}\right)y^n,$$

(11)

where $L=N+N^{*}$.

From the time domain expression of the two-scale equation corresponding to $S(t)$, the low-pass filter of $\psi (t)$ in the frequency domain can be obtained as follows:

$$H(\omega )=\frac{\widehat{S}(2\omega )}{\widehat{S}(\omega )}=\frac{{\displaystyle 451-512cos\frac{\omega }{8}+64cos\frac{\omega }{4}}}{{\displaystyle 451-512cos\frac{\omega }{16}+64cos\frac{\omega }{8}}}co{s}^4(\frac{\omega }{2}).$$

(12)

From Equations (8), (10) and (12), we can obtain $N=2$, and

$$Q(cos(\omega ))=\frac{{\displaystyle 451-512cos\frac{\omega }{8}+64cos\frac{\omega }{4}}}{{\displaystyle 451-512cos\frac{\omega }{16}+64cos\frac{\omega }{8}}},$$

(13)

$$Q^{*}(cos(\omega ))=\overline{\frac{{\displaystyle P(si{n}^2(\frac{\omega }{2}))}}{Q(cos(\omega ))}}.$$

(14)

When $N=2$, L takes different values, for example, $L=4,5,6,7$, we can obtain multiple corresponding $N^{*}$. Bring these values into Equations (11), (9) and (14), by taking the inverse Fourier transform as in Algorithm 1, we can obtain multiple groups $h_n,h_n^{*}$ of the corresponding low-pass filter coefficients of the new biorthogonal spline wavelet in Table 1. Considering the symmetry of the coefficients, we only give $n=0,1,2,3,...$, as shown in Table 1. In practical application, the corresponding odd coefficients can be symmetrically selected for image processing.  

Algorithm 1 BCS-SW Filter Algorithm

Input:  L: sum of the vanishing moment order of $H(\omega )$ and $H^{*}(\omega )$ by Equations (8) and (9), and $L=7$;  $P(sin())$: defined by Equation (11);  $\omega$: axis angel;  n: integer, the subscript of the low-pass filter coefficient; Output:  $h_n$: the set of corresponding low-pass filter coefficients of $H(\omega )$;  $h_n^{*}$: the set of corresponding low-pass filter coefficients of $H^{*}(\omega )$;   1: function: inverse_fourier_transform$\left\{f\right\}$;   2:   $nm=0:1:\mathrm{size}(f,2)-1$; $j=\sqrt{-1}$;   3: for $i=1:1:size(nm,2)$ do   4:      $ff=f.\ast exp(j\ast n{m}_i\ast \omega )$;   5:      $\omega =-\pi :0.01:\pi$;   6:      $h_{ni}=(1/(2\pi ))\ast real(trapz(\omega ,ff))$; //trapz(): computes the integral   7: end for   8: return: $h_n$.     9: for $n=1:1:size(f,2)-1$ do 10:      $f(k,:)=Q(cos(w))$; // $Q(cos(w))$ is defined by Equation (13) 11:      $f^{*}(k,:)=Q^{*}(cos(w))$; // $Q^{*}(cos(w))$ is defined by Equation (14) 12:      $h_n$ = inverse_fourier_transform(f); 13:      $h_n^{*}$ = inverse_fourier_transform($f^{*}$); 14: end for 15: return: $h_n$ and $h_n^{*}$.

Table 1. The low-pass filter coefficients $h_n,h_n^{*}$, $(n=0,1,2...)$ of the BCS-SW and its dual wavelet.

N	${\mathit{h}}_{\mathit{n}}$	${\mathit{N}}^{*}$	${\mathit{h}}_{\mathit{n}}^{*}$
2	1417/1969, 737/1491, 389/2760, 5/907, −1/1415, 1/6154, −1/19,642, 1/51,496…	2	813/731, 63/298, −419/1066, 91/1301, 81/992, −8/273, 3/689, −1/647, 1/1482, −1/3011, 1/5619…
		3	330/317, 38/141, −157/398, −3/488, 137/972, −13/669, −17/1051, 7/1205, −1/1437, 1/4657, −1/12,080 …
		4	645/643, 96/319, −231/593, −45/733, 143/838, 3/610, −3/83, 7/1030, 5/1491, −1/819, 1/8087…
		5	483/493, 319/994, −589/1535, −65/634, 93/502, 23/749, −25/496, 1/369, 3/308, −1/479, −1/1397…

Due to $g_n={(-1)}^n{h}_{1-n},g_n^{*}={(-1)}^n{h}_{1-n}^{*}$, from the data in Table 1, we can calculate the corresponding $g_n,g_n^{*}$, the high-pass filter coefficients of the $\psi (t)$ and ${\psi }^{*}(t)$. The filter bank in the frequency domain is $\left\{H(\omega ),G(\omega ),H^{*}(\omega ),G^{*}(\omega )\right\}$, the decomposition and reconstruction processes use two different sets of filters, respectively. It was decomposed with $\left\{h_n^{*}\right\}$ and $\left\{g_n^{*}\right\}$, the reconstruction uses a different pair of filters $\left\{h_n\right\}$ and $\left\{g_n\right\}$. Because of this, we make $\left\{h_n^{*}\right\}$ and $\left\{g_n^{*}\right\}$ wavelet decomposition filters, and $\left\{h_n\right\}$ and $\left\{g_n\right\}$ wavelet synthesis filters.

3.2. K-Layer Network

Compared to traditional wavelets, the BCS-SW combines the smoothness of the spline function with the localized accuracy of wavelet analysis [27]. In underwater image processing, due to the influence of water and the scattering of light, the image is often affected by noise and blur. BCS-SW is able to better capture smooth parts of the image, helping to reduce noise and blur. BCS-SW provides better frequency localization. This means that in underwater image processing, the spline wavelet can better capture the local details and structural information of the image. BCS-SW generally has better time–frequency localization characteristics and can better adapt to the different scale and frequency features present in underwater images. And its gradient is more accurate and less prone to be lost, which is beneficial for deep neural networks.

In this section, we propose the K-layer network (KLN) based on BCS-SW. BCS-SW decomposition offers significant advantages for underwater image enhancement tasks and separates high- and low-frequency information from images, and it can effectively remove or reduce noise caused by suspended particles in underwater images. In the KLN network, the input image undergoes K-layer decomposition, the encoder part has two branches: one branch uses wavelets for feature decomposition, and the other branch uses convolution for feature decomposition. Decoder part: the middle tensor of the same shape of the decoder part is spliced upsampling, the upsampling process of decoding, and the output of the previous step is used as the input of the next step, which speeds up the network’s processing of features.

Specifically, the filters of decomposition and reconstruction of BCS-SW are truncated, let $L=7,N=2,N^{*}=5$, $\left\{h_n\right\}$ selects 7 numbers that are symmetric about 0, in the same way, the length of the $\left\{g_n\right\}$ also takes 7; and $\left\{h_n^{*}\right\}$ selects 21 numbers that are symmetric about 0, similarly, the length of $\left\{g_n^{*}\right\}$ is also 21.

As shown in Figure 3, the KLN decomposes underwater image X based on BCS-SW to get four wavelet coefficients in each layer, for example, $c_1$, $d_1^h$, $d_1^v$, and $d_1^d$, in the first layer, $c_1$ is the approximate information of the original image, $d_1^d$ is the diagonal information of the original image, $d_1^h$ is the horizontal information of the original image, $d_1^v$ is the vertical information of the original image. In subsequent layers, the low-frequency coefficient $c_j$ of the previous layer is decomposed to get new wavelet coefficients following Equation (15):

$$\begin{array}{cc}\hfill c_{j+1}(n_1,n_2)& =\sum _{k_1}\sum _{k_2}{h}_n^{*}(2n_1-k_1)h_n^{*}(2n_2-k_2)c_j(k_1,k_2),\hfill \\ \hfill d_{j+1}^h(n_1,n_2)& =\sum _{k_1}\sum _{k_2}{h}_n^{*}(2n_1-k_1)g_n^{*}(2n_2-k_2)c_j(k_1,k_2),\hfill \\ \hfill d_{j+1}^v(n_1,n_2)& =\sum _{k_1}\sum _{k_2}{g}_n^{*}(2n_1-k_1)h_n^{*}(2n_2-k_2)c_j(k_1,k_2),\hfill \\ \hfill d_{j+1}^d(n_1,n_2)& =\sum _{k_1}\sum _{k_2}{g}_n^{*}(2n_1-k_1)g_n^{*}(2n_2-k_2)c_j(k_1,k_2),\hfill \end{array}$$

(15)

where $c_j(k_1,k_2)$ represents the low-frequency coefficient obtained in the jth layer. $j+1$th wavelet decomposition is carried out by the previous jth low-frequency coefficient. Specifically, the low-pass filter lod and the high-pass filter hid filter each row of $c_j$ and sample at intervals, and then filter each column of $c_j$ and sample at intervals with them, respectively, to obtain the $j+1$th layer wavelet coefficients: $c_{j+1}(n_1,n_2)$, $d_{j+1}^h(n_1,n_2)$, $d_{j+1}^v(n_1,n_2)$, $d_{j+1}^d(n_1,n_2)$. We also use Equation (16) to represent this process.

$$c_k,d_k^h,d_k^v,d_k^d=\left\{\begin{array}{cc}W(X),\hfill & k=1,\hfill \\ W(c_{k-1}),\hfill & k>1,\hfill \end{array}\right.$$

(16)

where $W()$ represents the wavelet decomposition based on BCS-SW. At the same time, the KLN will also perform K convolutions on the input underwater image X.

$$x_k=F_{conv}(x_{k-1}),k>0,$$

(17)

where $x_0=X$, $x_k$ represents the features obtained in the kth convolution layer. Equation (17) represents $x_k$, which comes from the convolution processing of $x_{k-1}$, which is the convolution result of the previous layer.

Figure 3. The structure of a K-layer network based (KLN) on BCS-SW. Our method overview involves training on a synthetic dataset. We obtain sub-band images with multiple frequency bands through discrete wavelet transform (DWT). This process effectively decouples color cast and blurry details in underwater images.

The KLN reconstructs the enhanced underwater images as follows Equation (18):

$$r_i=\left\{\begin{array}{cc}{F}_{upconv}(x_k,c_k,d_k^h,d_k^v,d_k^d),\hfill & i=1,\hfill \\ F_{upconv}(r_{i-1},x_{k+1-i},c_{k+1-i},d_{k+1-i}^h,d_{k+1-i}^v,d_{k+1-i}^d),\hfill & i>1,\hfill \end{array}\right.$$

(18)

where $F_{upconv}()$ represents reconstructing the enhanced image based on upsampling and convolution, $r_k$ is the enhanced underwater image, and also is the output of the KLN. We use the L1 loss, L2 loss, SSIM loss, and Perceptual loss for reconstructing enhanced images. Because we used more loss functions and spent a longer time on training, we also helped to obtain good experimental results, as shown:

$${Loss}_{reconstruct}={Loss}_{ssim}(r_k,gt)+{Loss}_{perceptual}(r_k,gt)+|r_k{-gt|}_1+{\parallel r_k-gt\parallel }^2,$$

(19)

where $gt$ is the ground truth enhanced image.

The KLN leverages the Adam optimization method, which is renowned for its efficiency and effectiveness in handling large datasets with high-dimensional parameter spaces. This method significantly enhances the convergence rates by adapting the learning rates based on the estimations of the first and second moments of the gradients, making it ideal for complex models like the KLN. In our implementation, the ‘concat’ command plays a crucial role, as it is employed to perform the concatenation step during the model’s layer fusion process. Specifically, this command facilitates the merging of features from different layers, which is vital for preserving and integrating diverse spatial and contextual information across the network. This technique not only enriches the model’s feature representation but also boosts its overall performance by enabling more comprehensive learning from multiple perspectives within the data.

4. Experiments

4.1. Implementation Details

In this section, we will introduce the quantitative experiments and qualitative experiments of the BCS-SW and KLN. We verify the effectiveness of the BCS-SW and KLN on various underwater datasets. In particular, the KLN is trained under the Pytorch. The training is performed on an NVIDIA A100-PCIE-40GB GPU, we use the Adam optimizer with a batch size of 25 and a learning rate of $1\times {10}^{-4}$. The training set, which includes 5000 images, is the combination of 800 images from UIEBD proposed by Li et al. [28], 3200 images from LSUI proposed by Peng et al. [29], and 1000 images from UIQS proposed by Liu et al. [30]. The test set includes UIEBD90 and UIQS. To evaluate the ability of the BCS-SW and KLN, we introduce the non-reference underwater images metrics: UCIQE proposed by Yang and Sowmya [31] and UIQM proposed by Panetta et al. [32], and the full-reference metrics: PCQI proposed by Wang et al. [33]. UCIQE evaluates the quality of the underwater images by calculating the color intensity, saturation, and contrast of the underwater images. The larger the value, the better the quality of the underwater image. UIQM evaluates the quality of the underwater images by calculating the brightness, contrast, and saturation of the underwater images. The larger the value, the better the quality of the underwater image. PCQI evaluates the quality of the enhanced image by calculating the contrast difference between the enhanced image and the ground truth image in a localized area. The larger the value, the more similar the enhanced image is to the ground truth image.

We first compared the performance of the BCS-SW, Haar, Bior3.5 and DB2 wavelet in underwater image denoising and enhancement tasks on the UIEBD and LSUI underwater image datasets, as shown in Section 4.2. And then, we compare the KLN with other underwater image enhancement methods, including UDCP proposed by Drews et al. [34], GDCP proposed by Peng et al. [35], Ucolor proposed by Li et al. [36], MLLE proposed by Zhang et al. [37], TOPAL proposed by Jiang et al. [38], WWPF proposed by Zhang et al. [39], U-shape proposed by Peng et al. [29], and UIEBD and UIQS proposed by Liu et al. [30], as shown in Section 4.3.

4.2. BCS-SW vs. Other Wavelets in Underwater Image Related Tasks

All existing wavelet-based deep learning network models predominantly utilize Haar wavelets. The newly proposed BCS-SW has better local properties in the frequency domain than Haar and can capture local features in images better. It can represent complex image structures and features more accurately and maintain image details and contours better during image reconstruction, with continuity and smoothness. This means that more continuous and smoother results can be produced during image reconstruction, in contrast to the mutability of the transformation results of Haar, which may result in image reconstruction results with jagged edges.

The process of image approximation information reconstruction after one layer decomposition based on the BCS-SW is: the low-frequency information, horizontal high-frequency information, vertical high-frequency information, and diagonal high-frequency information are obtained after image wavelet decomposition. The wavelet decomposed the first layer of the image, reconstructed the low-frequency information of the first layer of decomposition, obtained the general appearance of the original image, and compared the general appearance of these images. As shown in Figure 4 and Table 2, the experimental outcomes demonstrate that the BCS-SW surpasses the Haar wavelet in terms of PSNR proposed by Korhonen and You [40] and SSIM proposed by Hore and Ziou [41], indicating a better performance in preserving the original image’s quality and structural integrity.

Figure 4. The reconstruction images of approximation information after 1 layer decomposition using BCS-SW transform and Haar wavelet transform. And (a–d) are the original images of the four underwater images and processed by wavelet transform. The first column is the original underwater images; the second column is the reconstructed image after haar wavelet transform; the third column is the reconstructed images after BCS-SW wavelet transform.

Table 2. The PSNR and SSIM of approximation information reconstruction after 1 layer decomposition using Haar and BCS-SW. The 1st best results are in bold. ↑: The higher, the better.

Image	Haar		BCS-SW
	PSNR	SSIM	PSNR ↑	SSIM ↑
Figure 4a	31.7752	0.9922	32.7565	0.9938
Figure 4b	36.6892	0.9820	40.8069	0.9928
Figure 4c	40.3920	0.9979	42.1529	0.9986
Figure 4d	27.8036	0.9121	30.3055	0.9477

As shown in Figure 5 and Table 3, we add Gaussian noise to the underwater images, and the wavelet threshold denoising method is adopted. Firstly, the noisy image is decomposed in two layers by the wavelet, and the wavelet coefficient is processed by threshold; that is, the wavelet coefficient greater than (or) less than a certain threshold is processed, and the original image is reconstructed though using the processed results by Haar, Bior3.5, DB2, and BCS-SW, respectively. Because of the BCS-SW’s representation capabilities, it usually produces better denoising results. The coefficients of the BCS-SW transform are generally easier to distinguish between signal and noise, and are therefore more suitable for removing noise from images. In Table 3, the comparison of PSNR and SSIM indicates that, compared to other wavelets, the images processed by the BCS-SW are the closest to their corresponding unaltered images captured on land.

Figure 5. The figures of four underwater images of BCS-SW, Haar, Bior3.5, and DB2 wavelet denoising. And (a–d) are the original images, denoising with Gaussian noise and denoised by various wavelets of the four underwater images. The first column is the original images, the second column is with Gaussian noise, and columns 3–6 are denoising images using Haar, Bior3.5, DB2, and BCS-SW.

Table 3. The PNSR and SSIM of denoising results of four underwater images with Gaussian noise intensity of 0.02 using different wavelets. The 1st best results are in bold. ↑: The higher, the better.

Image	Haar		Bior3.5		DB2		BCS-SW
	PSNR	SSIM	PSNR	SSIM	PSNR	SSIM	PSNR ↑	SSIM ↑
Figure 5a	22.00	0.6166	21.80	0.5366	22.89	0.6664	23.36	0.6911
Figure 5b	22.11	0.5597	21.89	0.5521	23.12	0.6232	23.76	0.6547
Figure 5c	24.62	0.6159	23.53	0.5824	25.92	0.7137	27.00	0.7484
Figure 5d	22.31	0.5626	21.83	0.5335	23.19	0.6205	23.85	0.6528

In this section, we also show that when multi-layer wavelets decompose the underwater image, the decomposed signal based on a spline wavelet has lower noise, and the partially reconstructed image is closer to the original underwater image.

4.3. KLN vs. Other Underwater Image Enhancement Algorithms

As shown in Figure 6, the test results on UIEBT90 demonstrate significant improvements in the background quality of the underwater images processed by the KLN. Particularly in the images in the first and third rows, a clearly visible change in background color can be observed when compared with other processing methods. More importantly, this enhancement in background color not only allows for the revelation of more details in the background but also makes the subjects in the images more prominent, with color restoration becoming more natural and realistic. The second row of images showcases scenes abundant with fish. After the background improvement by the KLN, the contrast between the background and the fish becomes more pronounced, making the target objects within the images more discernible. Additionally, the texture information of the fish is more clearly displayed in the images. As shown in Table 4, although the scores in PCQI, UIQM, and UCIQE are not the highest, the gap from the highest value is minimal. Other methods might perform optimally in certain metrics, for example, the Ucolor method achieves the highest score in UIQM, but its scores in UCIQE and PCQI are lower than our method’s scores. Furthermore, compared with other methods, the images processed by our method achieve the highest PSNR and SSIM values. Considering all factors, our model presents the best overall performance.

Figure 6. The KLN vs. other underwater image enhancement methods on UIEBD. The PCQI of the KLN is the best, meaning that the processing method has achieved good results in improving the perceived color and quality of the underwater images.

Table 4. The mean UIQM, UCIQE, PCQI, PSNR, and SSIM scores of different methods on UIEBD90. The best results are in bold. ↑: The higher, the better.

	Original	CLAHE (1994)	UCDP (2013)	GDCP (2018)	Ucolor (2021)	MLLE (2022)	TOPAL (2022)	U-Shape (2023)	WWPF (2023)	Semi-UIR (2023)	OURS
UIQM ↑	2.4745	2.7409	2.0180	2.0995	3.0305	1.9561	2.8994	3.0141	2.3900	2.9503	2.8546
UCIQE ↑	0.5031	0.5527	0.5860	0.6141	0.5709	0.6216	0.5726	0.5748	0.6341	0.6188	0.6089
PCQI ↑	—	1.2036	0.9324	1.0161	1.1033	1.2242	1.1377	1.0866	1.2187	1.1704	1.2516
PSNR ↑	—	23.9048	14.0771	15.5725	21.5026	15.3689	22.2745	21.9905	15.8602	19.3214	25.6162
SSIM ↑	—	0.9114	0.6379	0.7581	0.8984	0.5848	0.9028	0.8528	0.6345	0.8005	0.9538

As shown in Figure 7, a detailed examination of the images in the first and second rows of the collection, especially the starfish at their centres, reveals a significant color difference. These starfish should display a full orange-red hue in their natural state, but after being processed by different methods, their color rendition varies. Compared to the more subdued and greyish-orange appearance of the starfish processed by other methods, those processed by the KLN exhibit a more vivid, rich, and natural orange-red color. This stark contrast not only illustrates the KLN’s remarkable capability in restoring and enhancing the red spectrum colors of underwater images but also highlights its superiority in color authenticity. Further analysis of the data in Table 5 shows that the KLN ranks just below WWPF in the UCIQE metric, demonstrating its strong performance in improving the overall color quality of the underwater images. Notably, the KLN’s performance in UIQM and PCQI is also very close to the best, validating its efficacy in maintaining image color saturation, contrast, and brightness, while also emphasizing its exceptional ability to preserve image details and texture. These comprehensive performances make the KLN one of the best choices among various underwater image enhancement technologies for overall effectiveness.

Figure 7. The KLN vs other underwater image enhancement methods on UIQS. The UCQIE of the KLN is the 2nd best, showing that the processing method has a good effect on improving the color quality of the underwater images.

Table 5. The mean UIQM, UCIQE, and PCQI scores of different methods on UIQS. The 1st and 2nd best results are in bold and underline, respectively. ↑: The higher, the better.

	Original	CLAHE (1994)	UCDP (2013)	GDCP (2018)	Ucolor (2021)	MLLE (2022)	TOPAL (2022)	U-Shape (2023)	WWPF (2023)	Semi-UIR (2023)	OURS
UIQM ↑	2.4641	2.6795	2.1315	2.4002	2.9963	2.4112	2.8644	2.9705	2.6591	2.9582	2.7684
UCIQE ↑	0.4321	0.4761	0.5128	0.5467	0.5317	0.5829	0.5004	0.5456	0.5958	0.5667	0.5897
PCQI ↑	—	1.2268	1.1041	1.2127	1.2121	1.3304	1.0899	1.2035	1.3002	1.2911	1.2616

When comparing our model with two other deep learning models, we observed significant differences in the number of parameters and computational complexity (FLOPS), as shown in Table 6. Our model has the highest number of parameters (57.24 M) and FLOPS (207.8 G), indicating that its network structure is the most complex and suitable for tasks that require synthesizing a large amount of global and local information. In contrast, the U-shaped model performs tasks with fewer parameters (31.6 M) and FLOPS (26.11 G), which may be more effective in simple or real-time applications. The UIR model operates with an extremely low number of parameters (1.68 M) but relatively high FLOPS (36.44 G), reflecting its efficient use of parameters, making it suitable for environments with limited computational resources but that require high processing capabilities.

Table 6. The Flops (G) and total parameters of our method with others.

	U-Shape	Semi-UIR	OURS
Flops (G)	26.11	36.44	207.8
Total Parameters (M)	31.6	1.68	57.24

5. Conclusions

In this work, we propose the BCS-SW and introduce the KLN model based on the BCS-SW. We substantiate the effectiveness of both the BCS and the KLN through the theoretical analysis in Section 3. Further, in Section 4, we demonstrate that that BCS-SW surpasses other wavelets in terms of image decomposition, denoising, and reconstruction capabilities. Additionally, we validate the effectiveness of the KLN for underwater image processing tasks. Our approach provides a practical methodology that serves as a benchmark for integrating wavelet and deep learning techniques in underwater image enhancement and related efforts. However, our integration of wavelet processing in the network leads to a higher computational demand compared to other methods, thus limiting our ability to further expand the depth and breadth of the KLN to enhance its performance. In future research, we will focus on how to conveniently incorporate wavelet analysis into deep neural networks. Moving forward, we aim to continue exploring strategies to address image processing challenges through the fusion of wavelet and deep learning technologies.