A Fast Nonlinear Sparse Model for Blind Image Deblurring

Abstract

Blind image deblurring, which requires simultaneous estimation of the latent image and blur kernel, constitutes a classic ill-posed problem. To address this, priors based on $L_2$, $L_1$, and $L_p$ regularizations have been widely adopted. Based on this foundation and combining successful experiences of previous work, this paper introduces $L_N$ regularization, a novel nonlinear sparse regularization combining the $L_p$ and $L_{\mathrm{\infty }}$ norms via nonlinear coupling. Statistical probability analysis demonstrates that $L_N$ regularization achieves stronger sparsity than traditional regularizations like $L_2$, $L_1$, and $L_p$ regularizations. Furthermore, building upon the $L_N$ regularization, we propose a novel nonlinear sparse model for blind image deblurring. To optimize the proposed $L_N$ regularization, we introduce an Adaptive Generalized Soft-Thresholding (AGST) algorithm and further develop an efficient optimization strategy by integrating AGST with the Half-Quadratic Splitting (HQS) strategy. Extensive experiments conducted on synthetic datasets and real-world images demonstrate that the proposed nonlinear sparse model achieves superior deblurring performance while maintaining completive computational efficiency.

1. Introduction

Recent advances in computer vision have intensified the research focus on image processing, particularly through significant contribution to the development of image deblurring, a fundamental component of low-level image processing. In the context of space-invariant blur kernel modeling, a blurred image y can be mathematically represented as:

$$B=I\otimes k-n$$

(1)

here B denotes the blurred image, $I$ denotes the sharp image corresponding to B, ⊗ denotes the convolution operator, $k$ denotes the blur kernel, and $n$ denotes the inevitable additional noise. Image deblurring encompasses two distinct categories: non-blind deblurring, where the kernel $k$ is known, and blind deblurring, where the kernel $k$ is unknown. This research addresses the latter.

Blind deblurring algorithms aim to direct the optimization process toward the desired solution, with traditional optimization approaches employing regularization terms to enhance constraints on both the latent image $I$ and blur kernel $k$, facilitating convergence to the appropriate blur kernel and sharp latent image. The standard formulation of blind deblurring is expressed as:

$$(I,k)=\underset{I,k}{\arg \min }F(I,k)+\alpha R_1(I)+\beta R_2(k),$$

(2)

where $F(I,k)$ denotes the data fidelity term; $R_1\left(I\right)$ and $R_2\left(I\right)$ denote the regularization terms for the latent image $I$ and blur kernel $k$, respectively.

Regarding the regularization term $R_1\left(I\right)$, researchers have developed numerous effective image priors, with gradient sparsity-based priors receiving extensive attention and implementation. For instance, Xu et al. [1] introduced an $L_0$ norm-based image smoothing algorithm and subsequently expanded $L_0$ regularization to the field of image deblurring [2], revealing that optimizing $L_0$ regularization presents a Nondeterministic Polynomial (NP)-hard problem, rendering direct solutions impractical. To address this limitation, Xu et al. [2] developed an unnatural distribution to approximate the $L_0$ regularization solution. Concurrently, the compressed sensing community typically addressed this challenge by relaxing the $L_0$ norm to one of convex optimization. Earlier approaches commonly replaced the $L_0$ norm with that of $L_1$ [3,4]; however, the limited sparsity of the $L_1$ norm results in performance that fails to match that of $L_0$ regularization in image processing applications.

To address this issue, Wang et al. [3,4,5,6] introduced a series of norm-ratio-based regularizations, e.g., $L_1/L_2$ and $L_1/L_{\infty }$, establishing that $L_1/L_{\infty }$ could effectively approximate the L₀ norm [7]. Given that the L₁ norm represents a special case of $L_p$ norm where p = 1, we extend Wang et al.’s work by generalizing $L_1/L_{\infty }$ regularization into a broader framework: $L_p/L_{\infty }$ (0 < p < 1) regularization (termed nonlinear sparse regularization, denoted as $L_N$ regularization). To evaluate the effectiveness of $L_N$ regularization, we perform a sparsity analysis of various regularizations using gradient distribution statistics, as presented in Figure 1. The results indicate that the proposed $L_N$ regularization achieves superior sparsity, offering preliminary evidence of its advantage over $L_1/L_{\infty }$ regularization.

Due to its high sparsity, the $L_0$ gradient prior exhibits strong filtering capabilities against harmful artifacts in blurred images, although this restricts its performance when processing detail-rich images. To overcome this limitation, researchers have developed an $L_0+X$ deblurring prior framework that combines the $L_0$ gradient prior with complementary prior terms, a hybrid approach which aims to filter detrimental artifacts effectively while preserving fine image structures. Current X-prior terms frequently incorporate various patch-based image priors [9,10,11,12,13]; however, these methods encounter challenges due to their high computational complexity, excessive resource consumption, and low operational efficiency. In response, Chen et al. [14] introduced an enhanced sparse model utilizing the $L_1$ gradient prior as the X-term, substantially improving computational speed. Building on this work, we incorporate the proposed $L_N$ regularized gradient prior as the X-term and combine it with the $L_0$ gradient prior, developing a novel fast nonlinear sparse model.

The main contributions of this paper are as follows:

• We propose a novel nonlinear sparse regularization ($L_N$) that nonlinearly couples the $L_p$ norm with the $L_{\infty }$ norm.
• An Adaptive Generalized Soft-Thresholding (AGST) algorithm is developed to optimize the $L_N$ regularization problem.
• Building upon $L_N$-regularization, we design a novel nonlinear sparse model for blind deblurring and develop an efficient optimization algorithm based on AGST and HQS.

The rest of this paper is organized as follows: Section 2 provides a comprehensive review of existing deblurring methods; Section 3 introduces the proposed nonlinear sparse regularization and the corresponding fast nonlinear sparse model; Section 4 experimentally evaluates the model on synthetic datasets and real-world blurred images; Section 5 analyzes ablation studies in order to validate the components of our fast nonlinear sparse model, including runtime performance tests; and Section 6 gives our conclusion.

2. Related Work

Generally, all existing methods can be classified into two categories: optimization-based and deep-learning-based methods. This section provides an overview of these distinct methodologies.

2.1. Optimization-Based Methods

Optimization-based image deblurring algorithms originated from the Richardson-Lucy method [15,16]. Subsequently, various approaches emerged, including the Gaussian mixture model (Fergus et al. [17]) and a fast $L_2$ norm-based deblurring method (Cho et al. [18]). However, $L_2$ regularization demonstrates limited effectiveness in blur kernel estimation due to its insufficient sparsity; therefore, a sparse regularized prior is urgently needed to meet the performance requirements of image deblurring. Yang et al. [19] and Candes et al. [20] proposed several $L_1$-norm methods; however, these approaches failed to satisfy the sparsity requirements of the deblurring prior term.

To obtain superior restoration performance, Daniele et al. [21] tried an $L_p$ (0 < p < 1) gradient prior. Since the kernel estimation based on $L_p$ sparse regularization is a non-convex problem, this leads to a hard solution of $L_p$ optimization. To address this problem, Daniele et al. [21] transformed the non-convex $L_p$ optimization problem into a quadratic optimization problem by taking the log function of the $L_p$ regularization term. Gasso et al. [22] and Zou et al. [23] extended Iteratively Reweighted $L_1$ minimization (IRL1) [20] to the non-convex problem domain of $L_p$ minimization. Rao and Kreutz-Delgado [24] proposed an Iteratively Reweighted Least Squares (IRLS) approach to $L_p$ minimization; and She et al. [25] proposed the Iteratively Thresholding Method (ITM), which was only suitable for unconstrained problems. In 2013, Zuo et al. [26] proposed a GST operator to solve the $L_p$ minimization problem, and following this, Zuo et al. [27] applied $L_p$ regularization to the field of image deblurring.

In pursuit of enhanced norm sparsity during the deblurring process, research focus shifted toward the L₀ norm. Xu et al. [1] introduced an $L_0$ image smoothing method in 2011, and building upon this, Xu et al. [2] developed an $L_0$ gradient prior by applying $L_0$ norm constraints to image gradients, enhancing kernel estimation and large-scale optimization. Extensive research has demonstrated that the generalized $L_0$ sparse gradient prior can effectively extract strong edges, and Xu et al.’s [2] groundbreaking work inspired numerous deblurring methods based on $L_0$ regularization: Pan et al. [28] implemented $L_0$ regularized intensity and gradient prior for text image deblurring, and Li et al. [29] applied $L_0$ norm to constrain the blur kernel intensity.

Despite the $L_0$-regularization prior’s proven effectiveness in removing harmful artifacts from images and widespread adoption in blind deblurring, it often underperforms when processing images with complex structural details. To address this limitation, researchers developed the $+X$ paradigm, combining the $L_0$ gradient prior with supplementary image priors. A representative example is the Dark Channel Prior (DCP) developed by Pan et al. [9], who discovered that sharp images exhibited sparser dark channels compared to blurred ones, combining the DCP with the $L_0$ gradient prior and achieving good results. Similarly, Yan et al. [30] extended the applicability of the DCP by combining it with the bright channel prior. Additionally, Eqtedaei et al. [31] developed a deblurring prior based on the difference between local maximum and minimum pixel values within an image region, developing two distinct deblurring algorithms utilizing $L_1$ and $L_0$ regularization, respectively.

The aforementioned image patch-based priors rely on overlapping patches, which substantially increases their computational complexity. To address this limitation, researchers have explored non-overlapping patches as an alternative approach. Notable examples include the Patch-wise Minimum Pixel (PMP) prior proposed by Wen et al. [11] and the Patch-wise Maximum Gradient (PMG) prior developed by Xu et al. [13]. These non-overlapping patch priors demonstrate significant computational acceleration while maintaining restoration accuracy; however, despite their improved efficiency, these methods still require individual path processing. Additionally, many patch-based priors necessitate the introduction of large, sparse matrices during optimization, consuming considerable computational resources and reducing algorithmic efficiency.

On the other hand, edge detection-based deblurring algorithms have emerged as a viable technical approach. Joshi et al. [32] implemented direct detection and prediction of latent sharp edges to enhance blur kernel estimation; Cho et al. [18] integrated bilateral filtering, shock filtering, and edge gradient thresholding for salient edges prediction; Xu et al. [33] developed a two-phase robust kernel estimation framework with an effective edge selection strategy; and Pan et al. [34] proposed a self-adaptive edge selection algorithm, while Liu et al. [35] implemented a surface-aware approach. Although explicit edge prediction methods demonstrate validity in blind deblurring, they remain dependent on heuristic filters. These methods therefore tend to amplify noise, potentially compromising the deblurring process and producing over-sharpened images as, furthermore, natural images do not consistently contain salient edges. Additionally, some scholars have begun exploring the integration of learning mechanisms with traditional optimization frameworks [36,37,38].

2.2. Learning-Based Methods

In the last decade, the rapid advancement of deep learning technology has prompted researchers to investigate its applications in image deblurring tasks.

In 2015, Sun et al. [39] pioneered the application of Convolutional Neural Networks (CNNs) to non-uniform image deblurring, marking an early successful integration of deep learning techniques. Subsequently, numerous CNN-based methods have emerged. For instance, Chakrabarti et al. [40] modified initial network layer connectivity using multi-frequency decomposition; Gong et al. [41] proposed a CNN-based approach for direct motion flow estimation from blur kernels; Ren et al. [42] designed a Maximum a Posteriori (MAP) deep learning hybrid framework, utilizing dual-branch networks for the alternating optimization of latent sharp images and blur kernels. Feng et al. [43] proposed Ghost-UNet, incorporating lightweight sub-networks for enhanced computational efficiency while preserving feature representation capacity; Mao et al. [44] developed a Residual Fast Fourier Transform with Convolution Block (ResFFT-Conv) module; and Mou et al. [45] transformed the Proximal Gradient Descent algorithm into a learnable deep architecture.

Beyond CNNs, other neural network architectures, including Recurrent Neural Networks (RNNs), Generative Adversarial Networks (GANs), and Feed-Forward Networks (FNNs), have demonstrated success in image deblurring. Zhang et al. [46] developed a Hybrid Deblur Net incorporating RNNs for non-uniform deblurring; Wang et al. [5] proposed a real-time deblurring algorithm utilizing GANs; and Kong et al. [47] developed a Frequency domain-based Self-Attention Solver (FSAS) to address the limitations of FFNs in image deblurring.

Despite their superior deblurring capabilities, neural networks face two significant limitations: (1) Their substantial data dependency means they require extensive training samples for optimal performance, resulting in generalization failures with distributionally shifted data. (2) Computational requirements are significant during both training and inference phases, particularly for architectures with numerous parameters.

3. Proposed Method

In this section, we present our improved sparse regularization and develop an effective deblurring algorithm based on this model.

3.1. Definition of Nonlinear Sparse Regularization

The nonlinear sparse regularization ($L_N$) is defined as:

$$||\cdot |{|}_N={\displaystyle \frac{||\cdot |{|}_p}{||\cdot |{|}_{\infty }}}$$

(3)

Given a corrupted signal $\mathcal{A}$, we assume the latent sharp signal $\mathcal{B}$ is sparse. With a basic quadratic penalty, the objective energy function can be written as:

$${\mathcal{B}}^{\ast }=\underset{\mathcal{B}}{\arg \min }\Vert \mathcal{A}-\mathcal{B}{\Vert }_2^2+\lambda \Vert \mathcal{B}{\Vert }_N$$

(4)

where $\lambda$ denotes the regularization parameter, and $k$ denotes the iteration level. Connecting to the definition of $L_N$ in (Equation (3)), Equation (4) can be rewritten as:

$${\mathcal{B}}^{\ast }=\underset{\mathcal{B}}{\arg \min }\Vert \mathcal{A}-\mathcal{B}{\Vert }_2^2+\lambda {\displaystyle \frac{\Vert \mathcal{B}{\Vert }_p}{\Vert \mathcal{B}{\Vert }_{\infty }}}$$

(5)

where ${\mathcal{B}}^{\ast }$ denotes the result after this iteration level, while $\mathcal{B}$ denotes the result of the previous iteration level. The ${||·||}_{\infty }$ notation represents the infinity norm, which is mathematically defined as the maximum absolute value of a matrix, expressed as ${||·||}_{\infty }=\mathrm{m}\mathrm{a}\mathrm{x}\{|·|\}$. Building on previous successful practices [3,4,6,7], we utilize the infinity norm ($\Vert \mathcal{B}{\Vert }_{\infty }$) of the signal from the previous iteration as a weighting factor to adjust the regularization parameter. To facilitate the solution of our model, we decompose the signals $\mathcal{A}$ and $\mathcal{B}$ into a series of independent subproblems:

$${{\mathcal{B}}_i}^{\ast }=\underset{{{\mathcal{B}}_i}^k}{\arg \min }{\left|{\mathcal{A}}_i-{\mathcal{B}}_i\right|}_2^2+\lambda {\displaystyle \frac{{\left|{\mathcal{B}}_i\right|}^p}{\Vert \mathcal{B}{\Vert }_{\infty }}},$$

(6)

where $i$ denotes the location of an element. The nonlinear sparse regularization term in Equation (6) presents a non-convex optimization problem, and following successful experiments, we are able to transform $L_N$ optimization into $L_p$ optimization with an adaptively adjustable regularization parameter. Based on the GST algorithm widely employed for $L_p$ norm optimization, we further develop an AGST algorithm. To illustrate the nonlinear sparse regularization, we abstract the $L_N$-related component in Equation (6) as a function:

$$F({\mathcal{A}}_i,{\mathcal{B}}_i)={({\mathcal{A}}_i-{\mathcal{B}}_i)}^2+\lambda {\displaystyle \frac{|{\mathcal{B}}_i{|}^p}{\Vert \mathcal{B}{\Vert }_{\infty }}}.$$

(7)

The curves of function $F$ under different values of variable ${\mathcal{A}}_i$ are displayed in the following figure:

The curves in Figure 2 illustrate the existence of a threshold $\tau$: when ${\mathcal{A}}_i$ falls below this threshold, the minimum value of the function in Equation (7) occurs at ${\mathcal{B}}_i=0$; when ${\mathcal{A}}_i$ exceeds this threshold, the function reaches its minimum at a non-zero value. These properties indicate that the threshold $\tau$ satisfies the following condition:

$$F(\tau ,{\mathcal{B}}_{i\tau })=F(0,{\mathcal{B}}_{i\tau }),$$

(8)

$$F’(\tau ,{\mathcal{B}}_{i\tau })=0.$$

(9)

Here, ${\mathcal{B}}_{i\tau }$ denotes the variable ${\mathcal{B}}_i$ when the function in Equation (7) achieves its non-zero minimum, and $F’$ denotes the first-order derivative of $F$. These conditions yield the following solutions:

$${\mathcal{B}}_{i\tau }={({\displaystyle \frac{\lambda (1-p)}{\Vert \mathcal{B}{\Vert }_{\infty }}})}^{{\displaystyle \frac{1}{2-p}}},$$

(10)

$$\tau ={({\displaystyle \frac{\lambda (1-p)}{\Vert \mathcal{B}{\Vert }_{\infty }}})}^{{\displaystyle \frac{1}{2-p}}}+{\displaystyle \frac{\lambda p}{2\Vert \mathcal{B}{\Vert }_{\infty }}}{({\displaystyle \frac{\lambda (1-p)}{\Vert \mathcal{B}{\Vert }_{\infty }}})}^{{\displaystyle \frac{p-1}{2-p}}}.$$

(11)

As shown in Equation (11), since $\Vert \mathcal{B}{\Vert }_{\infty }$ varies with the optimization target $\mathcal{B}$, which directly relates to the threshold $\tau$, the proposed AGST algorithm adapts its thresholding dynamically according to the intrinsic characteristics of input variables. The solution ${\mathcal{B}}_i^{\ast }$ is expressed as:

$${{\mathcal{B}}_i}^{\ast }=AGST({\mathcal{A}}_i,\mathcal{B},\lambda ,p).$$

(12)

The workflow of AGST is outlined in Algorithm 1.

Algorithm 1: The Adaptive Generalized Soft-Thresholding algorithm

input: ${\mathcal{A}}_i$, $\mathcal{B}$, $\lambda$, p, J   $\tau ={({\displaystyle \frac{\lambda (1-p)}{\Vert \mathcal{B}{\Vert }_{\infty }}})}^{{\displaystyle \frac{1}{2-p}}}+{\displaystyle \frac{\lambda p}{2\Vert \mathcal{B}{\Vert }_{\infty }}}{({\displaystyle \frac{\lambda (1-p)}{\Vert \mathcal{B}{\Vert }_{\infty }}})}^{{\displaystyle \frac{p-1}{2-p}}}$   if$|{\mathcal{A}}_i|\le \tau$     ${{\mathcal{B}}_i}^{\ast }=0$   else     $b^{(0)}=|{\mathcal{A}}_i|$     for t = 1, 2, …, J         $b^{(t)}=|{\mathcal{A}}_i|-{\displaystyle \frac{\lambda p}{\Vert \mathcal{B}{\Vert }_{\infty }}}{(b^{(t-1)})}^{p-1}$         $t\leftarrow t+1$     end     ${{\mathcal{B}}_i}^{\ast }=\mathrm{sgn}({\mathcal{A}}_i)b^{(J)}$   end Output: ${\mathcal{B}}_i^{\ast }$

3.2. Deblurring Model and Optimization

This subsection describes the proposed deblurring model and its optimization procedure. In our formulation, we employ the $L_2$-norm, commonly utilized in traditional algorithms, to regularize the fidelity term and the blur kernel term. For image-related regularization terms, building upon previous successful approaches [9,10,11,13,14], we combine the $L_N$ gradient prior with the widely adopted $L_0$ gradient prior, thereby constructing a novel fast nonlinear sparse model. The complete model is expressed as:

$$\underset{I,k}{\min }||I\otimes k-B|{|}_2^2+\alpha ||\nabla I|{|}_N+\beta \Vert \nabla I{\Vert }_0+\gamma ||k|{|}_2^2$$

(13)

where ∇ denotes the gradient operators in vertical and horizontal dimensions (i.e., $=\{{\nabla }_h,{\nabla }_v\}$); $\alpha$, $\beta$, and $\gamma$ denote the weight parameters. We solve Equation (13) by alternatively updating $I$ and $k$ with the other one held fixed. The sub-problems referring to $I$ and $k$ are given by:

$$\underset{I}{\min }||I\otimes k-B|{|}_2^2+\alpha ||\nabla I|{|}_N+\beta \Vert \nabla I{\Vert }_0,$$

(14)

$$\underset{k}{\min }||I\otimes k-B|{|}_2^2+\gamma ||k|{|}_2^2,$$

(15)

3.2.1. Updating Latent Image $I$

The latent image $I$ is updated while keeping the kernel $k$ fixed. Since Equation (14) presents a highly non-convex problem, the HQS method is employed. Two auxiliary variables, $u$ and $g$, are introduced to represent $I$ in the second and third terms of Equation (14), respectively, transforming Equation (14) into:

$$\underset{I,u,g}{\min }||I\otimes k-B|{|}_2^2+\alpha ||u|{|}_N+\beta \Vert g{\Vert }_0+{\lambda }_1||u-\nabla I|{|}_2^2+{\lambda }_2\Vert g-\nabla I{\Vert }_2^2,$$

(16)

where ${\lambda }_1$ and ${\lambda }_2$ denote the penalty parameters. Similarly to Equation (13), Equation (16) is decomposed into three subproblems associated with $I$, $u$, and $g$:

$$\underset{I}{\min }||I\otimes k-B|{|}_2^2+{\lambda }_1||u-\nabla I|{|}_2^2+{\lambda }_2\Vert g-\nabla I{\Vert }_2^2,$$

(17)

$$\underset{u}{\min }\alpha ||u|{|}_N+{\lambda }_1||u-\nabla I|{|}_2^2,$$

(18)

$$\underset{g}{\min }\beta \Vert g{\Vert }_0+{\lambda }_2\Vert g-\nabla I{\Vert }_2^2.$$

(19)

Solving $\mathit{I}$. Equation (17) represents a classical quadratic optimization problem solvable via Fast Fourier Transform (FFT), with its closed-form solution expressed as:

$$I={\mathcal{F}}^{-1}({\displaystyle \frac{\overline{\mathcal{F}(k)}\cdot \mathcal{F}(B)+{\lambda }_1\overline{\mathcal{F}(\nabla )}\cdot \mathcal{F}(u)+{\lambda }_2\overline{\mathcal{F}(\nabla )}\cdot \mathcal{F}(g)}{\overline{\mathcal{F}(k)}\cdot \mathcal{F}(k)+({\lambda }_1+{\lambda }_2)\overline{\mathcal{F}(\nabla )}\cdot \mathcal{F}(\nabla )}}).$$

(20)

Solving $\mathit{u}$. Based on the definition in Equation (3), Equation (18) is reformulated as:

$$\underset{u}{\min }{\lambda }_1\Vert u-\nabla I{\Vert }_2^2+\alpha {\displaystyle \frac{\Vert u{\Vert }_p}{\Vert \nabla I_{pre}{\Vert }_{\infty }}},$$

(21)

where $I_{pre}$ denotes the $I$ obtained from the previous iteration level. Adopting the AGST algorithm, the solution for variable $u$ is expressed as:

$$u=AGST(\nabla I,\nabla I_{pre},{\displaystyle \frac{{\lambda }_1}{\alpha }},p).$$

(22)

Solving $\mathit{g}$. The objective function for the auxiliary variable $g$ in Equation (19) represents an $L_0$ gradient prior [2]. The solution employs the unnatural distribution method, yielding the closed-form solution for g:

$$g=\left\{\begin{array}{l}\nabla I,|g{|}^2>{\displaystyle \frac{\beta }{{\lambda }_2}}\\ 0,others\end{array}\right..$$

(23)

The principal steps for estimating the latent image $I$ are summarized in Algorithm 2.

Algorithm 2: Latent image estimation

Input Blurred image $B$, initialized $k$ from the coarser level.    $I\leftarrow B$, ${\lambda }_1\leftarrow \alpha$, ${\lambda }_2\leftarrow \beta$    repeat       Calculating $u$ using Equation (22)       Calculating $g$ using Equation (23)       Calculating $I$ using Equation (20)       λ1←2λ1 λ2←2λ2    until λ1 >αmax Output Intermediate latent image $I$.

3.2.2. Updating Blur Kernel k

The objective function for the blur kernel $k$ presents a quadratic optimization problem similar to Equation (15). While Equation (15) emphasizes image intensity information, previous advanced methods [9,13,18,28] demonstrate that blur kernel estimation achieves higher accuracy when based on image gradient. Therefore, Equation (15) is modified into a gradient-based form:

$$\underset{k}{min}\Vert \nabla I\otimes k-\nabla B{\Vert }_2^2+\gamma \Vert k{\Vert }_2^2,$$

(24)

and can be effectively solved through FFT:

$$k={\mathcal{F}}^{-1}({\displaystyle \frac{\overline{\mathcal{F}(\nabla I)}\mathcal{F}(\nabla B)}{\overline{\mathcal{F}(\nabla I)}\mathcal{F}(\nabla I)+\gamma }}).$$

(25)

The essential steps of blur kernel estimation are summarized in Algorithm 3.

Algorithm 3: Blur kernel estimation

Input Blurred image $B$    Initialized $k$ from the previous level of the image pyramid.       while $i\le max\_iter$ do    Estimate $I$ using Algorithm 2       Estimate $k$ using Equation (25) Output Blur kernel $k$

4. Experimental Results

This section presents an evaluation of the proposed method using natural image datasets [8,48,49], real-world images, and a specific domain dataset [50], comparing it with several state-of-the-art methods, including traditional methods and deep learning methods. For all uniform image deblurring experiments, the parameters are set as $\alpha =0.0013$, $\beta =0.0023$, $\gamma =9$, $p=0.8$, $max\_iter=5$, and ${\alpha }_{max}={10}^5$, and for fair comparison, the other algorithms utilize the default settings from the authors’ codes. Throughout the experiments, blur kernel estimation employed different blind deblurring methods, followed by the same non-blind deblurring method as the final step, and the model implementation used MATLAB R2022a with efficiency assessment conducted on an Intel Core i7-11800H CPU with 16GB RAM (Intel Corporation, Santa Clara, CA, USA).

4.1. Natural Images

This subsection demonstrates our method’s performance on two synthetic datasets from Levin et al. [8] and Sun et al. [49]. The restoration results undergo quantitative evaluation using three standard metrics: Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index Measure (SSIM), and cumulative error ratio. The mathematical definitions of each evaluation metric are expressed as follows:

$$\begin{array}{c}PSNR=10\cdot {\log }_{10}({\displaystyle \frac{MA{X}_S^2}{MSE}}),\\ s.t.MSE={\displaystyle \frac{1}{mn}}{\displaystyle \sum _{i=0}^{m-1}}{\displaystyle \sum _{j=0}^{n-1}}[I(i,j)-I_t(i,j){]}^2,\end{array}$$

(26)

$$SSIM(I,S)={\displaystyle \frac{(2{\mu }_I{\mu }_S+C_1)(2{\sigma }_{IS}+C_2)}{({\mu }_I^2+{\mu }_S^2+C_1)({\sigma }_I^2+{\sigma }_S^2+C_2)}},$$

(27)

$$errorratio={\displaystyle \frac{\Vert I_t-I{\Vert }_2^2}{\Vert I_t-I_k{\Vert }_2^2}},$$

(28)

where $I$ denotes the restored image, $I_t$ denotes the reference ground truth image used for quality assessment, and $I_k$ denotes the deblurred image using the real blur kernel. In Equation (26), $m$ and $n$ denote the size of an image, ${MAX}_S$ indicates the maximum pixel value of image $S$, and $MSE$ means the mean squared error. In Equation (27), ${\mu }_I$ and ${\mu }_S$ denote the mean values of images $I$ and $S$, ${\sigma }_I$ and ${\sigma }_S$ denote their standard deviation, ${\sigma }_{\mathit{IS}}$ denotes the covariance between the two images, and $C_1$ and $C_2$ are two constants.

4.1.1. Levin’s Dataset

The initial evaluation utilizes the dataset reported by Levin et al. [8], comprising 32 blurred images generated from 4 images filtered with eight blur kernels, each maintaining a uniform resolution of 255 × 255 pixels. The PSNR, SSIM, and cumulative error ratio metrics of our method is compared with several state-of-the-art methods [9,11,14,30,31,33,51,52,53], with the results presented in Figure 3. As demonstrated in Figure 3a,b, the proposed model achieves superior average PSNR and SSIM metrics (32.234 dB in PSNR and 0.909 in SSIM), surpassing the $L_e$ model by 0.866 dB in PSNR and 0.024 in SSIM, respectively. Additionally, Figure 3c displays the cumulative error ratio curves of the comparative methods, with the results demonstrating that the proposed method consistently outperforms competing approaches, achieving a 90.625% success rate when the error ratio is ≤1.5, and a 100% success rate when the error ratio is ≤2.0.

Figure 4 presents a particularly challenging image—exhibiting a large blur kernel and complex texture details, posing significant difficulties for deblurring algorithms—alongside the restoration results of the compared methods, with corresponding PSNR and SSIM values annotated in the upper-left corner. The proposed method achieves superior kernel estimation accuracy and visual quality, producing the highest PSNR (30.046 dB).

4.1.2. Sun’s Dataset

To expand the comparative analysis, evaluation was conducted on Sun’s dataset [49], containing 640 high-resolution blurred images, using the cumulative error ratio as the comparison metric. Our method was evaluated against several established deblurring methods [9,14,18,31,33,52,54,55,56], with the quantitative results presented in Figure 5. For equitable comparison, an identical non-blind deblurring approach [57] was applied for all competing methods. As the figure illustrates, the proposed method achieves an 87.500% success rate when the error ratio is ≤2, exceeding Chen et al.’s [14] enhanced sparse model, which achieves an 85.469% success rate.

Following an established protocol, a representative case is presented in Figure 6 to illustrate the advantages of the proposed method over comparative approaches, with quantitative metrics annotated in the upper-left corner. The selected example features an interior architectural space with intricate structural details, and the results demonstrate that the proposed method has superior blur kernel accuracy compared to alternative approaches. Notably, the method achieves a 0.489 dB PSNR improvement and 0.011 SSIM gain over the $L_e$ model proposed by Chen et al. [14].

4.2. Specific Images

With Section 4.1 having demonstrated the superior performance of the proposed method on natural images, this subsection presents the results of targeted experiments using representative scenarios from the dataset in [50], specifically evaluating performance on two distinct scenarios: human face images and text images.

4.2.1. Human Face Images

Face image processing constitutes a fundamental research area in this field, with images of faces presenting unique challenges due to their frequent absence of dominant structural information, complicating blur kernel estimation. Figure 7 presents quantitative comparison results of face images from the dataset in [50]. The results indicate the superior PSNR and SSIM metrics (27.638 dB PSNR and 0.860 SSIM) of the proposed method, showing an improvement of 1.337 dB in PSNR and 0.030 in SSIM compared to Chen et al.’s $L_e$ model [14].

The first row of Figure 8 presents a challenging face image example from the dataset [50], while column (b) displays the ground-truth sharp image. This example demonstrates that our method reconstructs the most accurate blur kernel while producing results with minimal ringing artifacts. Following standard practice, the PSNR and SSIM metrics are displayed in the under-left corner, indicating the substantial improvement achieved by the proposed method over the $L_e$ model, obtaining a PSNR of 28.085 dB.

4.2.2. Text Images

Text image processing represents another significant application domain, different from other tasks in that these images predominantly contain two-tones that do not follow the heavy-tailed distribution of natural images, making text images particularly challenging for most deblurring methods. Figure 9 presents the average PSNR and SSIM for text images from the dataset in [50], indicating that our method achieves the highest quantitative evaluation metrics, surpassing the second-highest method by 0.533 dB in PSNR and 0.066 in SSIM. For visual comparison, the second row of Figure 8 illustrates an exemplar text image from the dataset [19], containing abundant image details. The comparative experimental results demonstrate that our method generates reconstructed results with superior detail preservation. The quantitative analysis metrics indicated in the upper-left corner reveal a PSNR improvement of 0.374 dB compared to other methods.

4.3. Comparison Against Deep Learning Methods

The recent decade has witnessed rapid advancement in deep learning, leading to numerous deep learning-based deblurring methods being proposed. To validate the effectiveness of our approach, we performed comparative experiments with several state-of-the-art deep learning models [44,45,47,51,52,54,58,59,60] on dataset proposed by Köhler et al. [48]. The quantitative analysis results, presented in Figure 10, demonstrate that our model achieves superior performance compared to several deep learning approaches in terms of both PSNR and Mean SSIM (MSSIM), surpassing the best-performing deep learning model [45] by 2.237 dB and 0.057, respectively.

For qualitative evaluation, we illustrate a challenging example from the dataset in Figure 11. The results indicate that most deep learning methods encounter difficulties in producing satisfactory restoration results when processing images with large blur kernels. Our method, however, maintains accurate reconstruction quality even in this challenging case, achieving a PSNR of 28.064 dB and an MSSIM of 0.907.

4.4. Real-World Images

Following evaluation on synthetic datasets, we tested our algorithm using real-world blurred images, inputs that present greater randomness and uncertainty compared to synthetic images, thus imposing higher requirements on algorithmic stability and adaptability. Figure 12 illustrates an example of a real-world blurred image, featuring rich structural details and exhibiting relatively complex blur kernel size and motion trajectory characteristics. To ensure fair comparison, we applied the same non-blind image deblurring algorithm [8] with identical parameter settings throughout all experiments, and the other methods’ restoration parameters for other methods were configured using the combinations published by their respective authors. The results demonstrate that our proposed fast nonlinear sparse model achieves the most accurate blur kernel estimation among all compared methods, producing a final restored image with superior detail preservation and minimal ringing artifacts.

5. Analysis and Discussion

In this section, we analyze a series of ablation experiments to systematically validate the effectiveness of our proposed fast nonlinear sparse model, accompanied by comprehensive discussions on key parameter influences and computational efficiency.

5.1. The Effectiveness of the Fast Nonlinear Sparse Model

This section presents a series of ablation experiments conducted to validate the effectiveness of the proposed fast nonlinear sparse model, evaluating various combinations of regularization norms commonly employed in deblurring methods, with quantitative results illustrated in

Table 1. Quantitative analysis of the ablation experiment.

	$\frac{\Vert \cdot {\Vert }_1}{\Vert \mathbf{\cdot }{\Vert }_2}$	$\frac{\Vert \cdot {\Vert }_1}{\Vert \mathbf{\cdot }{\Vert }_{\mathit{\infty }}}$	$\frac{\Vert \cdot {\Vert }_{\mathit{p}}}{\Vert \cdot {\Vert }_{\mathit{\infty }}}$	$\Vert \cdot {\Vert }_0$	$\frac{\Vert \cdot {\Vert }_1}{\Vert \cdot {\Vert }_2}+\Vert \cdot {\Vert }_0$	$\frac{\Vert \cdot {\Vert }_1}{\Vert \cdot {\Vert }_{\mathit{\infty }}}+\Vert \cdot {\Vert }_0$	$\frac{\Vert \cdot {\Vert }_{\mathit{p}}}{\Vert \cdot {\Vert }_{\mathit{\infty }}}+\Vert \cdot {\Vert }_0$
Average PSNR	28.481	30.829	31.420	31.555	31.739	31.646	32.234
Average SSIM	0.811	0.884	0.894	0.889	0.895	0.895	0.909

. Specifically, columns 1–3 present the quantitative deblurring evaluations for three distinct regularization method–$L_1/L_2$, $L_1/L_{\infty }$, and $L_p/L_{\infty }$; column 4 presents the average PSNR and SSIM obtained by standalone $L_0$-regularization; and columns 5–7 demonstrate the corresponding performance when these norm-ratio-based regularizations are coupled with the $L_0$ norm.

The comparative analysis reveals three significant findings: First, within columns 1–3, the proposed nonlinear sparse regularization ($L_p/L_{\infty }$) exhibits substantial advantages over other norm-ratio-based regularizations. Compared to the $L_1/L_{\infty }$ prior, the proposed $L_p/L_{\infty }$ prior achieves an average PSNR improvement of 0.591 dB, indicating superior performance over other nonlinear coupled norm priors. Second, the $L_0$ coupled versions (columns 5–7) achieve markedly better performance than their standalone counterparts (columns 1–3). Incorporating the $L_0$ prior yields an average improvement of 1.629 dB in PSNR and 0.038 in SSIM, which demonstrates that the $L_0$ regularization prior significantly enhances algorithm performance. Third, the results in the last four columns indicate that combining nonlinear coupled norm priors with the $L_0$ prior enhances the deblurring algorithm’s performance more effectively than using the $L_0$ prior alone, achieving PSNR improvements of 0.184 dB, 0.091 dB, and 0.679 dB, respectively. Furthermore, the proposed fast nonlinear sparse model, integrating the $L_N$ prior with the $L_0$ prior, achieves optimal deblurring performance with a 32.234 dB PSNR and 0.909 SSIM.

5.2. Effect of Main Parameters

The proposed fast nonlinear sparse model incorporates four key parameters: $\alpha$, $\beta$, $\gamma$, and $p$. The parameter optimization process consists of two stages: first, a two-dimensional grid search for the key parameters, $\alpha$ and $\beta$, utilizing the average PSNR as the selection criterion, and second, a separate search grid for parameter $p$, given its relative independence, as illustrated in Figure 13c.

First, based on existing empirical knowledge, we construct a grid search for parameters $\alpha$ and $\beta$ over the interval [$1\times {10}^{-3}$, $10\times {10}^{-3}$] with a step size of $1\times {10}^{-3}$, as illustrated in Figure 13a, where the boxed coordinate indicates the position yielding the maximum average PSNR. This primary grid serves to identify the approximate optimal ranges for $\alpha$ and $\beta$. Subsequently, we perform a secondary grid search within the identified optimal range from the initial screen, employing a refined step size of ${10}^{-4}$ (Figure 13b), enabling the precise determination of the final parameter values.

Parameter stability is essential for robust optimization-based deblurring, and that of the fast nonlinear sparse model was evaluated using blur kernel similarity (Figure 14). Parameters $\alpha$ and $\beta$ demonstrate stable performance across the refined grid range illustrated in Figure 13b, with kernel similarity variances of $5.62\times {10}^{-5}$ and $1.78\times {10}^{-7}$, respectively. All three parameters show minimal fluctuations in their similarity curves, confirming stable performance within reasonable ranges. Parameter $\gamma$ exhibits particularly low sensitivity, maintaining stable kernel similarity across its 3–20 operating range (Figure 14c).

5.3. Runtime Analysis

Computational efficiency serves as a crucial metric for evaluating deblurring algorithm performance, whit shorter durations indicating higher efficiency. In this experiment, we assess the computational time of our algorithm across three distinct image resolutions (255 × 255, 600 × 600, 800 × 800 pixels), with testing conditions controlled by maintaining a fixed blur kernel size of 27 × 27 throughout the trials. The comprehensive timing results are presented in

Table 2. Comparison of runtime performance (in seconds) across three different resolutions for various methods.

	255 × 255	600 × 600	800 × 800
Pan et al. [9]	63.652	327.051	659.056
Wen et al. [11]	10.229	22.034	59.721
Xu et al. [13]	9.321	35.021	82.246
Chen et al. [10]	33.041	179.554	327.993
Eqtedaei et al. [31]	27.079	114.625	198.729
Ours	4.657	20.871	35.443

.

We compared runtime results between our method and several classical patch-based deblurring approaches [9,10,11,13,31]. As demonstrated in Table 2, non-overlapping patch priors [11,13] show substantial efficiency improvements over overlapping patch design priors [9,10,31], with our method achieving an approximately 50% runtime reduction compared to standard non-overlapping patch-based methods. This enhancement results from its pixel-wise optimization strategy, which eliminates patch extraction requirements in each iteration.

5.4. Limitations

In the previous sections, we evaluated the superior performance and computational efficiency of our proposed nonlinear sparse model compared to existing state-of-the-art methods using both synthetic datasets and real-world images. However, our model still exhibits certain limitations. First, due to its pixel-wise computation approach during optimization, it demonstrates poor resistance to salt-and-pepper noise; as shown in Figure 15 and Figure 16, restoration performance deteriorates significantly when such noise is present. Additionally, our model performs weakly when handling locally blurred images, such as that in Figure 17.

6. Conclusions

This paper introduces a novel nonlinear sparse regularization ($L_N$-regularization) based on the nonlinear coupling of the $L_p$ and $L_{\infty }$ norms, and to facilitate effective optimization, an AGST algorithm is developed. Through the integration of the $L_N$ and $L_0$ regularization priors, this research establishes a fundamentally new fast nonlinear sparse model. Statistical analyses demonstrate that $L_N$ regularization achieves optimal sparsity. Comprehensive experiments on synthetic datasets and real-world blurred images validate that our fast nonlinear sparse model delivers superior deblurring performance. Quantitative results show that the proposed model achieves approximately 1 dB higher PSNR and 0.04 better SSIM values compared to state-of-the-art optimization-based deblurring methods, further reducing the computational time by 50% compared to conventional patch-based approaches due to its pixel-wise optimization strategy.