Wave-DIP: Unsupervised Image Decomposition Fusing Wavelet Multi-Scale Representation and Deep Image Prior

Abstract

To address the issues of texture residuals and structural detail loss often encountered in traditional image decomposition methods, this paper proposes an unsupervised decomposition model that integrates the Wavelet Transform with Deep Image Prior (DIP). Leveraging the multi-scale and multi-directional characteristics of the Wavelet Transform, the model carefully models the structural information of the cartoon component. Meanwhile, capitalizing on the unsupervised learning advantages of Deep Image Prior and incorporating low-rank constraints, it accurately extracts texture details. The model is solved via the Alternating Direction Method of Multipliers (ADMM). Experimental results on multiple test images demonstrate that, compared with existing methods, the proposed model achieves a more thorough separation of image structure and texture, yielding high-quality visual decomposition performance.

1. Introduction

Image decomposition is a fundamental research topic in the fields of image processing and computer vision. Its goal is to decompose an observed image $f$ into two components with distinct morphological characteristics: a cartoon component $u$ (also known as the structural component) and a texture component $v$. The cartoon component describes the large-scale structural information in the image, such as smooth regions and sharp edges; whereas the texture component contains small-scale oscillatory details, such as fabric textures and periodic patterns. This decomposition technique has extensive application value in areas like object recognition, image segmentation, and image inpainting.

Research on image decomposition can be traced back to the development of variational methods. The ROF model proposed by Rudin, Osher, and Fatemi [1] pioneered the paradigm of image processing based on Total Variation (TV). This model decomposes an image into a cartoon component in the space of functions of bounded variation and an oscillatory component in the $L_2$ space. The TV regularization term imposes a constraint on the $L_1$ norm of the image gradient, enabling smoothing while preserving edges. However, it tends to produce piecewise constant results, easily losing fine texture details. To overcome this limitation, Meyer [2] introduced the $G$ norm, which is more suitable for describing oscillatory patterns, to replace the $L_2$ norm, establishing a more accurate texture modeling framework. However, due to the inclusion of the $L_{\infty }$ norm in the $G$ norm, its numerical solution faces challenges. Vese and Osher [3] subsequently used the $L_p$ norm to approximate the Meyer model, proposing the VO model, which is straightforward to solve numerically. Osher, Sole, and Vese [4] further considered the special case of $p=2$, proposing the OSV model based on TV. Aujol et al. [5,6] systematically studied energy function spaces suitable for different types of textures, providing theoretical guidance for parameter selection in variational decomposition models.

With the advancement of research, scholars have attempted to introduce more mathematical tools to improve decomposition performance. Schaeffer and Osher [7] first introduced low-rank priors into texture description, proposing a cartoon-texture separation model based on sparse low-rank decomposition by imposing nuclear norm constraints on image patches. Ono et al. [8] adopted block-wise nuclear norms to characterize texture components, constructing a decomposition model capable of handling various degradation types. Zhang et al. [9] combined the TV norm and global nuclear norm to represent cartoon and texture components, respectively, proposing a decomposition model based on low-rank texture priors that performs excellently when images possess well-defined global structures. In addition, methodologies such as multi-scale decomposition [10,11,12,13], dictionary learning [14] and wavelet transforms [15] have continuously enriched the theoretical framework of image decomposition. As a classic multi-resolution analysis tool, the wavelet transform can decompose an image into sub-bands of different scales and directions. Its coefficients possess natural sparsity. The low-frequency sub-bands capture the general structure of the image, while the high-frequency sub-bands contain detailed information in various directions. This multi-scale and multi-directional decomposition approach is particularly suitable for describing texture components with oscillatory characteristics. However, although combining low-rank priors with traditional models solves some texture separation problems, it remains limited by the expressive power of hand-crafted regularizers.

To overcome the expressive limitations of traditional hand-crafted regularizers, researchers have begun to shift towards data-driven deep learning methods. However, supervised learning approaches typically require a large amount of paired training data, which is particularly difficult to obtain for image decomposition tasks. This is because clean ground truth for cartoon-texture separation is often unavailable in real-world scenarios. In recent advances in image decomposition, unsupervised and self-supervised learning methods have continued to achieve significant breakthroughs. Liang et al. [16] proposed Fusion from Decomposition, which achieves multi-modal image fusion through self-supervised decomposition. Its core unsupervised component separation strategy provides a new perspective for cartoon-texture decomposition. Liu et al. [17] proposed CoCoNet, which achieves finer feature characterization in image component separation by coupling contrastive learning with multi-level feature integration. These studies collectively indicate that unsupervised deep learning is becoming an important development direction in this field.

The Deep Image Prior (DIP) [18] has emerged as a powerful alternative to data-driven methods, demonstrating that the structure of a convolutional neural network itself acts as a sufficient prior for image restoration tasks like denoising and inpainting. Unlike traditional deep learning, DIP requires no external training data, relying instead on the network’s implicit bias towards natural image statistics. For image decomposition, this characteristic is particularly valuable: the network’s inherent tendency to fit structured patterns allows it to effectively separate components with high self-similarity (such as textures) from complex mixtures. The “Double-DIP” framework proposed by Gandelsman et al. [19] systematically applied this idea to image decomposition for the first time. By coupling multiple DIP networks to generate different image layers respectively, and imposing reconstruction constraints and separation losses, this method is capable of achieving various decomposition tasks such as image dehazing, foreground/background segmentation, and watermark removal under unsupervised conditions. This work verified the universality and effectiveness of DIP in image decomposition. Kim et al. [20] further integrated deep variational priors into the traditional TV-$L_1$ model. Through a Plug-and-Play approach, they used convolutional neural networks to learn the prior distribution of structural images, successfully realizing the distinction between high-amplitude details and structural edges. Zhou et al. [21] proposed a structure and texture-aware decomposition method. By using deep neural networks to uniformly optimize the decomposition objective function, they ingeniously constructed a self-supervised learning mechanism, achieving model training and optimization without the need for paired ground truth labels. Cascarano et al. [22] introduced spatially adaptive weighted TV regularization into the DIP framework, solved it via ADMM, and adaptively updated local regularization parameters using image gradient information, achieving results that are better than those of standard DIP in image restoration tasks. Nevertheless, these methods often require pre-trained networks or rely on specific forms of variational models. Addressing these issues, Guennec et al. [23] proposed a joint structure-texture modeling framework, which performs joint regularization on structural and texture components as a whole. By embedding deep neural networks through a Plug-and-Play framework, it can still effectively generalize to natural images after training on synthetic data, providing new insights for overcoming the limitations of traditional decomposition models.

Building on the advantages of DIP, Xu et al. [24] proposed a decomposition model that combines a low-rank prior with DIP. This model uses an adaptive weighting mechanism to better preserve edge information and achieve good decomposition results. However, the TV regularizer is essentially a first-order sparse constraint based on spatial gradients. It tends to generate piecewise constant cartoon components, which can easily cause staircasing effects in smooth regions. It also has a limited ability to represent complex textures with multi-scale and directional features. When image textures show periodic or fine structures, the TV term might mistakenly treat some texture as structure, leaving it in the cartoon part, or over-smooth the image, leading to the loss of texture details.

Meanwhile, the combination of wavelet transforms and deep learning has attracted widespread attention. For instance, Nguyen et al. [25] proposed combining sparse low-rank priors with DIP, utilizing the 2D discrete wavelet transform to obtain sparse representations, and achieved excellent results in hyperspectral image denoising. In the field of image restoration, the multi-wavelet guided deep prior method proposed by Zhang et al. [26] obtained stronger prior information by integrating the structural representation capability of wavelet transforms with the learning ability of deep networks. Recently, the WTConv wavelet convolutional layer proposed by Fogel et al. [27] achieved a very large receptive field by stacking wavelet decompositions, while also enhancing the response to low-frequency information, further verifying the unique advantage of wavelet transforms in representing low-frequency structures. Ramamonjisoa et al. [28] applied wavelet decomposition to single-image depth prediction, demonstrating that wavelet coefficients can be learned without direct supervision and can significantly reduce the computational cost of the decoder. Recently, the interpretable deep image decomposition framework proposed by Gao et al. [29] improved the interpretability of decomposition results while ensuring model generalization by combining hierarchical Bayesian modeling with deep learning. In addition, some studies have begun to explore combining multi-scale decomposition with unsupervised learning, such as the multi-branch autoencoder structure proposed by Günaydın and Sen [30], which decomposes images into different components like smooth, detail, and residual, further verifying the potential of combining multi-scale decomposition with unsupervised learning.

Inspired by research on wavelet-domain ADMM deep networks and multi-wavelet guided deep priors, this paper proposes introducing wavelet transforms into the DIP framework. We replace the original TV regularizer with the magnitude of wavelet coefficients to build a new image decomposition model. This model aims to use the ability of wavelets to distinguish between structure and texture to better describe the sparse structure of the cartoon component. This allows for a more thorough separation of texture from the image, effectively alleviating the staircasing effects and texture residuals caused by traditional TV regularizers.

It should be noted that the target tasks of the aforementioned wavelet-based deep prior methods are fundamentally different from the cartoon-texture decomposition addressed in this paper. Moreover, in those existing frameworks, wavelets typically serve as a pre-processing transform or as part of the network architecture, rather than as an explicit sparse constraint to replace TV regularization. Furthermore, PnP frameworks rely on external pre-trained models, whereas this paper employs a single-image-specific, untrained DIP. To the best of our knowledge, no prior work has incorporated the sparse regularization of wavelet coefficients as a direct replacement for TV within a DIP + low-rank decomposition framework to solve the cartoon-texture disentanglement problem. This paper fills this gap and, on this basis, designs a global adaptive weighting strategy in the wavelet domain.

The remainder of this paper is organized as follows: Section 2 introduces the related works that are closely associated with the model proposed in this paper. Section 3 presents our proposed model and provides the algorithm for solving this new model. Section 4 describes the numerical experimental setup and comparative experimental results. Section 5 presents the conclusions.

2. Related Work

The variational methods for image decomposition can be traced back to the ROF model proposed by Rudin, Osher, and Fatemi [1], which formulates the image decomposition problem as:

$$\underset{u}{\min }{\displaystyle {\int }_{\Omega }\left|\nabla u\right|}dx+\lambda {{\displaystyle {\int }_{\Omega }\left|f-u\right|}}^{2}dx,$$

(1)

among them, the first term is the Total Variation (TV) regularization term, which achieves edge-preserving smoothing by penalizing image gradients; the second term is the fidelity term, ensuring the similarity between the decomposition result and the original image. The ROF model constrains the cartoon component in the space of functions of bounded variation, while the texture component lies in the $L_2$ space. Although this model can effectively preserve edges, it tends to over-smooth fine textures by treating them as noise.

To better describe oscillating textures, Meyer [2] introduced the $G$ norm, which is more suitable for handling textures, to replace the $L_2$ norm, and proposed an improved model:

$$\underset{u}{\min }{\displaystyle {\int }_{\Omega }\left|\nabla u\right|}dx+\lambda {‖v‖}_G,$$

(2)

where ${‖v‖}_G$ is defined as the norm on the function space satisfying $v=\nabla \cdot g$ with $\left|g\right|\in L_{\infty }$. However, the numerical solution of the $G$ norm is challenging. While subsequent studies have attempted to approximate or improve this, a significant advancement was made by Zhang et al. [9]. They effectively combined the TV norm and the global nuclear norm to represent cartoon and texture components, respectively, proposing a decomposition model based on low-rank texture priors.

$$\underset{u,v}{\min }\tau ‖Du‖+\mu {‖v‖}_{*}+{\displaystyle \frac{1}{2}}{‖u+v-f‖}_2^2,$$

(3)

this model can obtain cleaner texture extraction results than traditional methods when the image has a good global structure. However, the low-rank model makes strong assumptions about the regularity of textures and thus still has limitations when dealing with complex, aperiodic textures.

Distinct from traditional deep learning methods for images, Ulyanov et al. [18] proposed the Deep Image Prior (DIP) method. This method leverages the inductive bias inherent in the structure of an untrained convolutional neural network as image prior information to solve various image inverse problems, such as denoising, super-resolution, and image inpainting. The corresponding model is formulated as follows:

$$\underset{\theta }{\arg \min }{\displaystyle \frac{1}{2}}{‖H{f}_{\theta }(z)-Y‖}_2^2,$$

(4)

in the above formula, $f_{\theta }(\bullet )$ represents a Convolutional Neural Network (CNN) generator, $H$ is a linear degradation operator, $Y$ denotes a natural image, and $z$ is a randomly initialized input vector. The core of this method lies in modeling the image to be reconstructed as a differentiable function of the randomly initialized neural network parameters, fitting only to a single degraded image during the optimization process. This strategy allows the model to operate without relying on large-scale training datasets, achieving relatively good reconstruction results to a certain extent.

Leveraging the advantages of DIP, Xu et al. [24] proposed a decomposition model combining low-rank priors with DIP. This model uses DIP to generate the cartoon component $u=T_{\theta }\left(z\right)$, employs the low-rank norm ${‖v‖}_{*}$ to constrain the texture part, and introduces a weighted TV regularization term to impose gradient sparsity constraints on the cartoon component:

$$\underset{\theta ,v}{\min }{\displaystyle \sum _{i=1}^N{\tau }_i}\Vert {(D{T}_{\theta }(z))}_i{\Vert }_1+\mu {‖v‖}_{*}+{\displaystyle \frac{1}{2}}{‖T_{\theta }(z)+v-f‖}_2^2,$$

(5)

where $N$ is the image size. This model effectively preserves edge information through an adaptive weighting mechanism, achieving good decomposition results.

It is worth noting that the proposed “wavelet-domain global adaptive weighting” strategy shares conceptual similarities with adaptive representation learning methods that have emerged in other fields. Rezaei et al. [31] applied deep reinforcement learning to image hashing, where an adaptive bit selection mechanism dynamically retains the most informative hash bits and directly optimizes retrieval metrics—an idea similar to our approach of dynamically adjusting the regularization strength based on wavelet coefficient sparsity. Yang et al. [32] addressed the problem of data scarcity in industrial soft sensing by designing a self-modified dynamic domain adaptation framework that adaptively adjusts its strategy to improve cross-condition prediction robustness. Jiang et al. [33] employed pre-trained large language models and multi-modal generative models to synthesize defect images, using large-scale external generative priors to compensate for the lack of data in few-shot scenarios. Although the target tasks of these works differ from our cartoon-texture decomposition, they share a common core idea: relying on adaptive or generative priors to compensate for insufficient learning under limited information. Our DIP-based framework, which requires no pre-training or external data and achieves unsupervised decomposition solely through the network’s inherent structural prior and wavelet-domain adaptive weighting, can be regarded as a concrete instance within this broader paradigm.

3. New Model and Algorithm

The total variation regularization term in the spatial domain of Equation (5) is essentially a first-order sparse constraint on gradients, lacking the ability to distinguish scale and direction. Therefore, it is highly likely to misidentify fine textures as structures (leaving residuals) or produce staircasing effects in smooth regions. To thoroughly address this inherent spatial limitation, this paper fully leverages the two major advantages of the wavelet transform: multi-scale frequency separation and directional sparse representation. Furthermore, we propose a global adaptive weighting strategy in the wavelet domain—dynamically calculating weight factors based on the global sparsity of the wavelet coefficients of the current structural component (inversely proportional to the coefficient energy), thereby adaptively adjusting the regularization strength according to the image content. Specifically, using the $L_1$ norm of the wavelet coefficients as the new structural regularization term to replace the traditional TV constraint, the wavelet decomposition separates the image into low-frequency approximations and multiple high-frequency detail subbands, naturally decoupling the cartoon and texture components in the transform domain. The sparse distribution of coefficients enhances the difference between structure and texture, while the adaptive weighting effectively suppresses texture residuals while protecting structural edges. This achieves a more thorough and flexible decomposition than traditional TV models without the need for pre-trained networks. The new model established from this is as follows:

$$\underset{\theta ,v}{\min }{\displaystyle {\int }_{\Omega }\alpha \left(x\right)}\left|W{T}_{\theta }\left(z\right)\right|dx+\mu {‖v‖}_{*}+{\displaystyle \frac{\lambda }{2}}{‖T_{\theta }\left(z\right)+v-f‖}_2^2,$$

(6)

where $f$ is the observed degraded image, $W\left(\bullet \right)$ denotes the wavelet transform, $\theta$ represents the weight parameters of the deep neural network, and $\alpha \left(x\right)$ is the adaptive regularization parameter that can be dynamically adjusted according to the image content. $T_{\theta }\left(z\right)$ is the output of the deep network with a fixed random vector $z$ as input, representing the structural component (cartoon part) of the image. $v$ is the texture component to be restored, and $\mu ,\lambda$ are regularization parameters.

Compared with existing frameworks based on traditional TV models and DIP-based weighted TV, the main improvements of the proposed model lie in the following aspects: First, wavelet domain sparse regularization is used to replace spatial domain TV regularization—traditional TV relies solely on the first-order sparsity of gradients, lacking scale and directional discrimination capabilities, which easily leads to texture residuals or staircasing effects. In contrast, the wavelet transform decomposes the image into multi-scale and multi-directional subbands, and its $L_1$ norm constraint naturally fits the oscillatory characteristics of textures, thereby achieving a more thorough separation of structure and texture. Second, a wavelet domain global adaptive weighting strategy is designed. Unlike the fixed regularization parameters in existing methods, this paper dynamically calculates weight factors based on the global sparsity of the wavelet coefficients of the structural component (inversely proportional to the coefficient energy), enabling the regularization strength to adaptively adjust according to the image content. This effectively suppresses texture residuals while protecting structural edges. Third, it continues the unsupervised learning paradigm of DIP, requiring no pre-trained networks or paired data. By embedding the wavelet transform as an explicit sparse prior, it compensates for the shortcomings of ordinary DIP networks, where implicit priors are insufficient for representing complex textures. This enhances the model’s expressive power for multi-scale and directional features, thereby achieving more favorable generalization performance on natural images compared to existing methods.

Next, we present the optimization algorithm for the proposed model. First, by letting $u=T_{\theta }\left(z\right)$, we obtain the augmented Lagrangian function as follows:

$$L={\displaystyle \frac{\lambda }{2}}{‖T_{\theta }\left(z\right)-\left(f-v\right)‖}^2+{\displaystyle {\int }_{\Omega }\alpha \left(x\right)}\left|Wu\right|dx+\mu {‖v‖}_{*}+{\displaystyle \frac{\beta }{2}}{‖T_{\theta }\left(z\right)-u+b‖}_2^2.$$

(7)

To facilitate the solution of Equation (7), we adopt the ADMM, formulated as follows:

$${\theta }^{k+1}=\arg \underset{\theta }{\min }{\displaystyle \frac{\lambda }{2}}{‖T_{\theta }\left(z\right)-\left(f-v^k\right)‖}_2^2+{\displaystyle \frac{\beta }{2}}{‖T_{\theta }\left(z\right)-u^k+b^k‖}_2^2,$$

(8)

$$u^{k+1}=\arg \underset{u}{\min }{\displaystyle {\int }_{\Omega }\alpha \left(x\right)}\left|Wu\right|dx{\displaystyle \frac{\beta }{2}}{‖u-T_{{\theta }^{k+1}}(z)-b^k‖}_2^2,$$

(9)

$$v^{k+1}=\arg \underset{v}{\min }\mu {‖v‖}_{*}+{\displaystyle \frac{\lambda }{2}}{‖T_{{\theta }^{k+1}}\left(z\right)-f+v‖}_2^2,$$

(10)

$$b^{k+1}=b^k+T_{{\theta }^{k+1}}(z)-u^{k+1}.$$

(11)

For Equation (8), we use the Adam optimizer [34] for iteration.

For the solution of Equation (9), we first handle the adaptive parameter $\alpha \left(x\right)$. Utilizing the UPEN principle [35], we dynamically calculate the weight factor in each ADMM iteration based on the global sparsity of the wavelet coefficients of the current structural component, enabling the regularization strength to adaptively adjust according to the image content. Specifically, in the $k+1$-th iteration, the weight factor is calculated as follows:

$$\alpha {(x)}^{k+1}={\displaystyle \frac{\lambda }{2}}{\displaystyle \frac{{‖T_{{\theta }^{k+1}}(z)+v^k-f‖}_2^2}{{‖(W(T_{{\theta }^{k+1}}(z)))‖}_1}},$$

(12)

this formula dynamically adjusts the regularization weight by calculating the global energy of the wavelet coefficients of the current reconstructed image. Specifically, when the image $x$ contains rich structures or textures (i.e., the $L_1$ norm of the wavelet coefficients is large), the denominator increases, causing the weight factor $\alpha (x)$ to decrease. This reduces the regularization strength to protect details. Conversely, when the image tends to be smooth (i.e., the $L_1$ norm of the wavelet coefficients is small), the weight factor $\alpha (x)$ increases, enhancing noise suppression capabilities and thereby achieving content-adaptive sparse constraints.

Substituting the adaptive weights into Equation (9) and then applying coefficient-wise soft-thresholding to the wavelet coefficients of $T_{{\theta }^{k+1}}(z)+b^k$, yields the closed-form solution of Equation (9).

$$u^{k+1}=W^{-1}\left({\mathcal{S}}_{{\alpha }^{k+1}/\beta }(W(T_{{\theta }^{k+1}}(z)+b^k))\right),$$

(13)

where ${\mathcal{S}}_{\tau }(x)=\max (\left|x\right|-\tau ,0)\cdot \mathrm{sign}(x)$ is the soft thresholding operator. In this way, we achieve spatial adaptive weighting of the wavelet coefficients, thereby extracting the structural components of the image more flexibly.

For Equation (10), using the singular value thresholding shrinkage operator [36], the solution is obtained as:

$$v^{k+1}={\mathcal{S}}_{\mu /\lambda }\left(f-T_{{\theta }^{k+1}}\left(z\right)\right)=U\mathrm{shrink}(\Sigma ,\mu /\lambda )V^T,$$

(14)

where $\mathcal{S}$ is the soft thresholding operator, $\mathrm{shrink}(\Sigma ,\mu )=\mathrm{sign}(\Sigma )\cdot \max \left\{\right|\Sigma |-\mu ,0\}$, and $f-T_{\theta }(z)=U\Sigma V^T$ is its singular value decomposition.

Based on the above discussion, the Algorithm 1 for the new model is as follows:

Algorithm 1. The Algorithm Proposed in This Paper

Input: Initial values of $f,W,u,v,\theta$ and selected parameters $\tau ,\alpha ,\beta ,\mu ,\lambda$

for $k=0\to K$ do:

Calculate ${\theta }^{k+1}$ via (8)

Calculate $u^{k+1}$ via (13)

Calculate $v^{k+1}$ via (14)

Update Lagrange multiplier $b^{k+1}$ via (11)

Output: Decomposed components $u$ and $v$

4. Experiments

In this section, we will discuss the experimental setup and use different test images to verify the effectiveness of the new model. The test images are shown in Figure 1. Among them, Figure 1a is the Face image, Figure 1b is the Leg image, Figure 1c is the Word image, Figure 1d is the Table image, Figure 1e is the Monster image, Figure 1f is the TomJerry synthetic image, Figure 1g is the Barbara RGB natural image, and Figure 1h is the Bishapur RGB image.

4.1. Wavelet Selection

In this experiment, the wavelet transform module selects the db2 wavelet basis function, and the decomposition level is set to two layers. The db2 wavelet belongs to the Daubechies orthogonal wavelet family, possessing compact support, orthogonality, and one vanishing moment. Among these, the vanishing moment enables the wavelet transform to effectively filter out smooth backgrounds and enhance oscillatory texture representations, which closely matches the design goal of this paper to replace TV constraints with wavelet domain sparse regularization; orthogonality and compact support ensure the accuracy of the decomposition and computational efficiency. Controlling the decomposition level at two layers aims to balance multi-scale feature extraction capability with computational complexity, avoiding computational redundancy and edge artifacts caused by overly deep decomposition.

4.2. Parameter Selection

In our algorithm, the maximum number of iterations is set to 300, and 300 Adam iterations are used in each outer iteration to approximately solve for the network parameters $\theta$ in subproblem (8). The global weight $\alpha (x)$ in the wavelet domain is updated adaptively, and the hyperparameters involved in the model, namely $\mu$ (nuclear norm regularization parameter), $\beta$ (ADMM penalty parameter), and $\lambda$ (data fidelity term weight), need to be specified in advance. Based on experimental experience, we find that when the parameters are set within the following ranges, relatively optimal reconstruction results can be obtained through manual adjustment for different test images: $0.01<\mu <10$, $0<\beta <0.1$, and $\lambda$ is typically set to the order of one or ten. In practical applications, fine-tuning can be performed within the above ranges according to the image noise level and texture complexity.

4.3. Experimental Results

To verify the effectiveness of the new model, we compared the proposed method with some representative image decomposition methods. The comparison methods include: the variational model-based [9,24], the method proposed in [37], the rolling guidance filter-based method [38], the parameter-free unrolling network of [39] which builds upon the Low Patch Rank model to learn decomposition parameters from data while achieving fast inference and good generalization to natural images, the neural-network-based method of [21], and the ref. [24] model based on DIP and low-rank decomposition. Among them, Ref. [9] uses a combination of total variation and nuclear norm to characterize structure and texture; Ref. [35] constructs a decomposition model based on Sobolev space; Ref. [38] achieves scale-aware decomposition via a rolling guidance filter; Ref. [39] learns to separate structure and texture by unrolling an iterative optimization scheme without requiring manual parameter tuning; Ref. [21] learns to separate structure and texture by training a neural network; Ref. [24] introduces weighted TV and low-rank constraints within the DIP framework to achieve the separation of structure and texture. In contrast, this paper introduces the wavelet transform into the DIP framework, replacing the traditional total variation constraint with a sparse regularization term of wavelet coefficients. Without the need for pre-trained networks, it utilizes the multi-scale characteristics of wavelets to more finely extract the structural information of the cartoon component. In the experiments, all comparison methods were manually tuned to obtain their best decomposition results. Through comparison, it can be seen that the proposed method outperforms existing methods in terms of the thoroughness of structure and texture separation, edge detail preservation capability, and overall visual effect.

Figure 2 presents the decomposition results of the Leg image. It can be observed that the cartoon component in Figure 2a suffers from over-smoothing, with relatively blurred edge details. Although the edge sharpness in Figure 2b is somewhat improved, obvious texture residues still remain in the structural layer. Figure 2c achieves a relatively better overall processing effect for structure and texture; however, striped texture interference is still visible in the leg region. In Figure 2d, although the structure is free of texture contamination, it is overly smooth, losing some fine structural details. In contrast, Figure 2e, obtained by the method proposed in this paper, is capable of achieving more precise separation of structure and texture, yielding a better overall decomposition effect.

Figure 3 presents the decomposition results of the Table image. It can be observed that the cartoon components in Figure 3a,b still contain residual texture lines and details, failing to achieve complete separation. Although the decomposition in Figure 3c is relatively thorough in most areas, texture is still mixed into the cartoon component in local regions such as the table corners. Figure 3d shows the decomposition result of the proposed method, where the separation of structure and texture is cleaner, structural edges are preserved intact, texture extraction is more precise, and the overall decomposition effect is more ideal.

Figure 4 presents the relevant decomposition results. Among them, the cartoon components of Figure 4a–c all exhibit texture residuals in the eye region of the Monster image; especially in Figure 4b, the cartoon component shows considerable texture. Furthermore, the texture component of Figure 4c is mixed with color information that should belong to the cartoon component. In contrast, the proposed new model in Figure 4d achieves better decomposition performance in both the cartoon and texture components.

The cartoon component in Figure 5a exhibits an over-smoothed trend, failing to effectively preserve detailed contours. The texture extraction results in Figure 5b,c are suboptimal, with redundant interference information remaining, thus failing to achieve pure texture extraction. In contrast, the structural layer in Figure 5d performs better; not only are the edge contours clear and distinguishable, but the structural regions are also clean and tidy without obvious texture mixture, making the decomposition result more consistent with expectations.

Figure 6 presents the decomposition results of the Barbara RGB natural image. As can be seen from the results, in the cartoon component of Figure 6a, texture lines and details still remain in the edge areas of the table, indicating that structure and texture have not been completely separated. The cartoon component of Figure 6b suffers from texture residue; for instance, significant texture remains in the character’s scarf area and the table corner. The decomposition effect of the cartoon component in Figure 6c is similarly suboptimal; not only are the contours of the character’s eyes and scarf area blurred, but there is also obvious texture residue at the table corner. Figure 6d shows the decomposition result of the method proposed in this paper. Compared with the above methods, this result achieves an efficient separation of structure and texture. It not only completely preserves the detailed features of structural edges but also accurately extracts texture information, demonstrating better overall decomposition performance.

The leg edges in Figure 7a–c all appear blurred, failing to clearly reveal structural details. Although Figure 7b,c show good processing results in the facial region, there is a loss of structure in the curtain area at the top right corner. In contrast, the edge processing accuracy of the structural map in Figure 7d is significantly improved; various types of structural information are effectively extracted to the structural layer, and the overall decomposition effect meets expectations.

4.4. Parameter Sensitivity Analysis

To evaluate the influence of the key hyperparameter $\lambda$ (data fidelity weight) on the decomposition results, while avoiding unnecessary quantitative metrics due to the lack of ground truth, we adopt a visual comparison approach. The word image is selected as the test sample. All other hyperparameters are fixed, and only λ is varied (taking values 0.1, 5, 10, 50). Figure 8 shows the resulting cartoon and texture components under different $\lambda$ values.

From Figure 8, one can observe that when $\lambda =0.1$, the cartoon component retains significant texture information, and the texture component is overly sparse. When $\lambda =50$, the texture component is contaminated by some structural edges, and the cartoon component exhibits slight over-smoothing. When $\lambda \in \left[5,\left.10\right]\right.$, the visual quality of the decomposition results is similar; all effectively separate structure from texture while preserving edges well. This observation indicates that the model is insensitive to $\lambda$ within the range $\lambda \in \left[5,\left.10\right]\right.$, and the decomposition results are stable. Therefore, manually selecting $\lambda$ within this range is reasonable, and satisfactory decomposition can be achieved without rigorous numerical scanning. Similar conclusions are drawn for other hyperparameters (e.g., the nuclear norm parameter $\mu$ and the ADMM penalty parameter $\beta$): within the reported parameter ranges, the visual quality remains stable. Due to space limitations, they are not presented here.

4.5. Ablation Study

To separately verify the contributions of wavelet regularization and adaptive weighting to the decomposition performance, we designed and compared the following three model variants. Baseline model: The model of Xu et al. [24], which adopts TV regularization and low-rank constraints, without wavelet transform or adaptive weighting. We have already compared the baseline model with the full model in Figure 2, Figure 3, Figure 4, Figure 5, Figure 6 and Figure 7; the phenomenon that its cartoon component contains significant texture residue is not repeated here; wavelet only: Based on the baseline model, this variant replaces the TV regularization with the $L_1$ norm of wavelet coefficients, but does not employ adaptive weighting (i.e., uses a fixed weight). This variant is used to evaluate the contribution of replacing TV with wavelet; full model: The proposed method in this paper, i.e., wavelet $L_1$ norm + global adaptive weighting.

Figure 9 shows the decomposition results of the three models on the Face image. From the cartoon components, it can be seen that Variant A effectively alleviates the staircase effect, but some texture information still remains in the cartoon component, for example, on the character’s scarf and in the upper-right corner of the image, where texture residues of various shapes appear. The full model further removes these texture residues while preserving the sharpness of structural edges. A similar trend can be observed from the texture components: Variant A extracts finer textures, while the full model further excludes structural information that has been mixed in.

The above results demonstrate that wavelet regularization is superior to TV regularization in capturing multi-scale texture features, and that the global adaptive weighting can further suppress texture residue and improve decomposition quality. Together, they achieve the optimal decomposition performance.

4.6. Robustness Analysis

To evaluate the stability of the proposed model with respect to random initialization and stochasticity in the optimization process, we fixed all hyperparameters and ran the model five times on the representative Barbara image, changing only the random seed. Since cartoon-texture decomposition lacks ground truth, we followed the common practice in this field and used visual comparison as the primary evaluation criterion. The experimental results show that, across the five runs, the obtained cartoon and texture components exhibit no noticeable visual differences (Figure 10 shows only one representative result). This observation is consistent with the finding reported in the original DIP work [18]: despite random initialization and stochastic perturbations during optimization, the visual quality of the network output remains relatively stable. Therefore, the proposed model is not sensitive to random initialization and possesses good robustness.

4.7. Runtime Comparison

To evaluate the computational efficiency of the proposed method, we ran the baseline model (Xu et al. [24]) and our full model on images of size 256 × 256 under the same hardware environment. Due to space limitations, only the average runtime from start to convergence (300 outer iterations with 300 Adam updates per inner iteration) is reported. The results are shown in

Table 1. Runtime comparison of different models on 256 × 256 images.

Model	Average Runtime (s)
Baseline (Xu et al. [24])	186.42
Full model (proposed)	427.14

.

As can be seen from Table 1, the runtime of the full model is approximately 2.17 times that of the baseline. The additional time overhead mainly comes from the forward and inverse wavelet transforms and the calculation of the adaptive weighting in each iteration. In terms of algorithmic complexity, both the wavelet transform and the soft-thresholding operation have a complexity of O(N) (where N is the number of pixels), which is much lower than the computational cost of the DIP network itself. Therefore, given the significant improvement in decomposition quality achieved by the proposed method, this computational cost is acceptable in practical applications (e.g., offline image processing, medical image analysis, etc.).

5. Conclusions

This paper proposes an unsupervised image decomposition model fusing wavelet transform and DIP. Under the framework of DIP combined with low-rank decomposition, this method introduces a sparse regularization term of wavelet coefficients to replace the traditional total variation constraint, utilizing the multi-scale characteristics of wavelets to more finely characterize the structural information of the cartoon component. Experimental results demonstrate that, compared with existing methods, this model can separate texture and structure more thoroughly. It effectively removes texture residuals while better preserving edge details, achieving high-quality visual decomposition results.