Adaptive Optics Retinal Image Restoration Using Total Variation with Overlapping Group Sparsity

Abstract

Adaptive optics (AO)-corrected retina flood illumination imaging technology is widely used for investigating both structural and functional aspects of the retina. Given the inherent low-contrast nature of original retinal images, it is necessary to perform image restoration. Total variation (TV) regularization is an efficient regularization technique for AO retinal image restoration. However, a main shortcoming of TV regularization is its potential to experience the staircase effects, particularly in smooth regions of the image. To overcome the drawback, a new image restoration model is proposed for AO retinal images. This model utilizes the overlapping group sparse total variation (OGSTV) as a regularization term. Due to the structural characteristics of AO retinal images, only partial information regarding the PSF is known. Consequently, we have to solve a more complicated myopic deconvolution problem. To address this computational challenge, we propose an ADMM-MM-LAP method to solve the proposed model. First, we apply the alternating direction method of multiplier (ADMM) as the outer-layer optimization method. Then, appropriate algorithms are employed to solve the ADMM subproblems based on their inherent structures. Specifically, the majorization–minimization (MM) method is applied to handle the asymmetry OGSTV regularization component, while a modified version of the linearize and project (LAP) method is adopted to address the tightly coupled subproblem. Theoretically, we establish the complexity analysis of the proposed method. Numerical results demonstrate that the proposed model outperforms the existing state-of-the-art TV model across several metrics.

1. Introduction

Adaptive optics (AO) is an optoelectronic technique that compensates for wavefront distortion in optical systems. This technique was initially developed in the field of astronomical observation [1,2] and later applied to retinal imaging. AO retinal imaging is a widely used technique to explore the structure and function of the living retina [3]. By compensating for wavefront distortions during real-time imaging, AO enables us to examine the architecture of the retina at a cellular scale. However, AO retinal images inherently contain information from both in-focus and out-of-focus planes, leading to resolution degradation in AO flood images. Therefore, accurate interpretation of these images requires advanced post-processing techniques. From a mathematical perspective, retinal imaging can be modeled as a three-dimensional (3D) convolution. When the real object demonstrates translational invariance along the optical axis, the original 3D model can be simplified to a two-dimensional (2D) representation.

In conventional 2D image restoration, the point spread function (PSF) is generally fully known or treated as completely unknown [4]. However, AO retinal image restoration presents a distinctive scenario where the true PSF (and the blurring matrix A) is only partially known. This intermediate regime between fully known and completely unknown PSFs is formally classified as a mildly blind or myopic deconvolution problem. Specifically, the global PSF of AO retinal images can be represented through a model that combines unknown parameters and a few PSFs. This combination tends to create coupled terms, making the problem more complex. Therefore, this poses a challenge to the existing methods for AO retinal image restoration problems [5]. The central difficulty lies in the inherent composite nature of the global PSF, which requires simultaneous parameter estimation during image restoration.

The mathematical formulation of myopic deconvolution for AO retinal imaging can be expressed as

$$\mathit{d}=A\left(\mathit{w}\right)\mathit{x}+\eta ,$$

where $\mathit{d}\in {\mathbb{R}}^{n^2}$ denotes an observed image with noise, $\eta \in {\mathbb{R}}^{n^2}$ represents additive noise, and $\mathit{x}\in {\mathbb{R}}^{n^2}$ represents unknown real images. $A\left(\mathit{w}\right)\in {\mathbb{R}}^{n^2\times n^2}$ is the blurring matrix defined by PSF, and $\mathit{w}\in {\mathbb{R}}^p$ is an unknown parameter in the blurring matrix $A\left(\mathit{w}\right)\in {\mathbb{R}}^{n^2\times n^2}$. $A\left(\mathit{w}\right)$ is constructed as a linear combination of p known blurring matrices $A_j$, formulated as

$$A\left(\mathit{w}\right)=\sum _{j=1}^p{w}_j{A}_j=w_1{A}_1+w_2{A}_2+\dots +w_p{A}_p.$$

Regularization methods play a central role in solving such deconvolution problems. As a classical technique in image denoising, total variation (TV) regularization [6] effectively balances noise suppression and detail preservation through its edge-preserving diffusion mechanism. TV regularization has demonstrated robust performance across various noise environments, including Gaussian noise, Poisson noise, and other complex noise scenarios [7,8,9,10]. To address the AO retinal image denoising challenge, Chen et al. developed a novel model incorporating isotropic TV regularization [5]

$$\begin{array}{c}\left\{\begin{array}{c}{\displaystyle \underset{\mathit{x}\in {\mathcal{C}}_{\mathit{x}},\mathit{w}\in {\mathcal{C}}_{\mathit{w}}}{min}}\Phi \left(\mathit{x},\mathit{w}\right)={\displaystyle \frac{\mu }{2}}\parallel A\left(\mathit{w}\right){\mathit{x}-\mathit{d}\parallel }_2^2+{\displaystyle \sum _{i=1}^{n^2}}{\parallel D_i\mathit{x}\parallel }_2,\hfill \\ \mathrm{s}.\mathrm{t}.{\displaystyle \sum _{j=1}^p}{w}_j=1,\hfill \end{array}\right.\end{array}$$

where ${\mathcal{C}}_{\mathit{x}}=\{\mathit{x}\mid x_i\ge 0\mathrm{for}i=1,\dots ,n^2\},{\mathcal{C}}_{\mathit{w}}=\{\mathit{w}\mid w_j\ge 0\mathrm{for}j=1,\dots ,p\}.$${\displaystyle \frac{\mu }{2}}{\parallel A\left(\mathit{w}\right)\mathit{x}-\mathit{d}\parallel }_2^2$ denotes the data fidelity term. ${\displaystyle \sum _{i=1}^{n^2}}{\parallel D_i\mathit{x}\parallel }_2$ denotes the isotropic TV regularizer, which has symmetry properties or rotational invariance. $\mu >0$ denotes the weighting parameter to balance the two terms in the objective function, $w_j$ satisfies certain constraints. The TV model achieves simultaneous optimization of image restoration and unknown parameter estimation.

However, the primary limitation of TV regularization lies in its tendency to produce step artifacts. To address this issue, numerous enhanced regularization methods have been developed, including fractional-order TV (FOTV) [11,12], higher-order TV (HOTV) [13,14,15], non-local TV (NLTV) [16,17], total generalized variation (TGV) [18,19], total variation with overlapping group sparse (OGSTV) [20,21,22], and so on. Among them, Selesnick et al. proposed OGSTV [22] as an extension of TV, which has been successfully applied to image denoising in recent years by considering the group sparsity feature of the signal derivatives and utilizing the correlation of neighboring gradient values to retain the image detail information, which effectively reduces the step artifacts.

Specifically, the OGSTV regularizer divides the image gradient field into overlapping $K\times K$ pixel blocks (K is a fixed constant) and extends them in all directions centered on the pixel blocks. This ensures that every gradient is jointly constrained by multiple overlapping groups. Mathematically, for the two-dimensional gradient operators, $D_{\left(1\right)}^{+}\mathit{u}$ and $D_{\left(2\right)}^{+}\mathit{u}$, OGSTV constructs corresponding overlapping group matrices ${\mathit{u}}_{i,j,K}$ and computes the sum of $L_2$ norm of gradients within each group, enforcing the global consistency of local smoothness. The OGSTV regularizer has been widely applied to different noise removal scenarios. Liu et al. effectively eliminated salt-and-pepper noise using OGSTV [23]. Yin et al. enhanced impulse noise suppression by integrating OGSTV with an $L_0$ norm data fidelity term [24]. Li et al. achieved high-quality restoration of Gaussian noise-degraded images through a hybrid framework combining generalized nonconvex nonsmooth fourth-order total variation with OGS regularization [25].

Inspired by the AO retinal image model proposed by Chen et al. in [5], and the better restoration effect of OGSTV in image restoration, we propose the image myopic deconvolution model with OGSTV regularization term

$$\begin{array}{c}\left\{\begin{array}{cc}{\displaystyle \underset{\mathit{x}\in {\mathcal{C}}_{\mathit{x}},\mathit{w}\in {\mathcal{C}}_{\mathit{w}}}{min}}\hfill & \Phi \left(\mathit{x},\mathit{w}\right)={\displaystyle \frac{\mu }{2}}{\parallel A\left(\mathit{w}\right)\mathit{x}-\mathit{d}\parallel }_2^2+\phi (▽\mathit{x})\hfill \\ \mathrm{s}.\mathrm{t}.& {\displaystyle \sum _{j=1}^p}{w}_j=1,\hfill \end{array}\right.\end{array}$$

(1)

where $\phi (▽\mathit{x})$ denotes the OGSTV regularization function. This is a non-convex model. To the best of our knowledge, there have been no attempts to apply OGSTV to AO retinal imaging to simultaneously perform image restoration and parameter estimation in the coupled problem.

Since the proposed model is non-convex, we adopt the alternating direction method of multipliers (ADMM) as the outer-layer optimization strategy. While ADMM was originally designed for 2-block convex optimization problems, recent theoretical advances have demonstrated its empirical effectiveness in handling non-convex objective functions or non-convex sets [26,27,28,29,30,31]. This motivates our application of ADMM to the AO retinal image restoration task. In the concrete steps, the computational architecture employs the ADMM as the outer-layer optimization framework. According to the special structure in our proposed AO retinal image restoration model, we also use the majorization–minimization (MM) method [24] and the linearize and project (LAP) method [32]. Among them, the MM method deals with the OGSTV regularizer that appears during each ADMM iteration, while the LAP method solves the tightly coupled subproblem.

The main contributions of this paper are outlined below:

(i)

We propose a new myopic deconvolution model with an OGSTV regularization term arising from AO retinal image restoration. The proposed model achieves superior image restoration and enhanced parameter estimation accuracy.

(ii)

To address the nonconvex nature of the model and the tight coupling between variables $\mathit{x}$ and $\mathit{w}$, we propose an ADMM-MM-LAP method.

(iii)

We conduct comprehensive numerical experiments to evaluate the performance of our proposed OGSTV model.

The following outlines the organization of this paper. First, we propose an AO retinal imaging model with the OGSTV regularization term in Section 2. In Section 3, we present the iterative framework of ADMM and provide a concise overview of the MM method and the LAP method. Then, we propose the ADMM-MM-LAP method. Theoretically, we also analyze the computational complexity. Section 4 provides the numerical experiments, and Section 5 offers the concluding remarks.

2. AO Retinal Image Restoration Model with OGSTV Regularization

In this section, we construct a myopic deconvolution model with overlapping group sparse total variation (OGSTV) regularization. First, we introduce the discrete gradient operator $▽\mathit{x}$ and the overlapping group sparsity (OGS) function and provide the definition of the OGSTV regularization term. Subsequently, we propose a new myopic deconvolution model with an OGSTV regularization term for AO retinal image restoration.

2.1. The First-Order Difference Operator $▽\mathit{x}$

First, we provide the definition of the first-order finite difference. For a gray image $\mathit{x}\in {\mathbb{R}}^{n^2}$ under periodic boundary conditions, the first-order difference operator $\nabla :{\mathbb{R}}^{n^2}\to {\mathbb{R}}^{n^2\times 2n^2}$ is defined as

$${(▽\mathit{x})}_i=\left(\begin{array}{c}{\left(D_{\left(1\right)}^{+}\mathit{x}\right)}_i,{\left(D_{\left(2\right)}^{+}\mathit{x}\right)}_i\end{array}\right),$$

(2)

where $D_{\left(1\right)}^{+}\mathit{x}$ and $D_{\left(2\right)}^{+}\mathit{x}$ are the first-order forward differences of $\mathit{x}$ in the horizontal and vertical directions, respectively. Specifically, we have

$${\left(D_{\left(1\right)}^{+}\mathit{x}\right)}_i:=\left\{\begin{array}{cc}{x}_{i+n}-x_i,\hfill & \mathrm{if}1\le i\le n(n-1),\hfill \\ x_{\mathrm{mod}(i,n)}-x_i,\hfill & \mathrm{else},\hfill \end{array}\right.$$

and

$${\left(D_{\left(2\right)}^{+}\mathit{x}\right)}_i:=\left\{\begin{array}{cc}{x}_{i+1}-x_i,\hfill & \mathrm{if}\mathrm{mod}(i,n)\ne 0,\hfill \\ x_{i-n+1}-x_i,\hfill & \mathrm{else},\hfill \end{array}\right.$$

where mod$(·,·)$ denotes the residual function, ${\mathit{x}}_i$ stands for pixel of i in the image, i = 1, …, $n^2$.

2.2. OGSTV and Proposed Model

For a given vector $\mathit{y}\in {\mathbb{R}}^n$, the group K-point is defined as

$${\mathit{y}}_{i,K}=[\mathit{y}\left(i\right),\dots ,\mathit{y}(i+K-1)]\in {\mathbb{R}}^K,$$

where ${\mathit{y}}_{i,K}$ represents a block of size K starting at index i. A widely used group sparse regularization [22] is formulated as

$$\varphi \left(\mathit{y}\right)=\sum _i{\left[\sum _{k=0}^{K-1}{\left|\mathit{y}(i+k)\right|}^2\right]}^{1/2}.$$

In the 2D scenario, the $K\times K$-point group of an image $\mathit{g}\in {\mathbb{R}}^{n\times n}$ is given by

$$\begin{array}{c}\tilde{\mathit{g}}{(i,j)}_K\\ =\left[\begin{array}{cccc}\mathit{g}(i-m_1,j-m_1)& \mathit{g}(i-m_1,j-m_1+1)& \dots & \mathit{g}(i-m_1,j+m_2)\\ \mathit{g}(i-m_1+1,j-m_1)& \mathit{g}(i-m_1+1,j-m_1+1)& \dots & \mathit{g}(i-m_1+1,j+m_2)\\ ⋮& ⋮& \ddots & ⋮\\ \mathit{g}(i+m_2,j-m_1)& \mathit{g}(i+m_2,j-m_1+1)& \dots & \mathit{g}(i+m_2,j+m_2)\end{array}\right]\\ \in {\mathbb{R}}^{K\times K},\end{array}$$

with $m_1=\lfloor {\displaystyle \frac{K-1}{2}}\rfloor ,m_2=\lfloor {\displaystyle \frac{K}{2}}\rfloor$, where $\lfloor x\rfloor$ represents the maximum integer that is less than or equal to x. The center of $\tilde{\mathit{g}}{(i,j)}_K$ is $(i,j).$ Let $\mathit{g}{(i,j)}_K$ denote a vector formed by stacking the K columns of matrix $\tilde{\mathit{g}}{(i,j)}_K$, i.e., $\mathit{g}{(i,j)}_K=$$\tilde{\mathit{g}}{(i,j)}_K(:)$. Subsequently, the OGS function for the grayscale image $\mathit{g}$ is defined as

$$\varphi \left(\mathit{g}\right)=\sum _{i,j=1}{\parallel \mathit{g}{(i,j)}_K\parallel }_2.$$

(3)

Thus, combining (2) and (3), the OGSTV regularization term can be represented as

$$\phi \left(▽\mathit{x}\right)=\varphi \left(D_{\left(1\right)}^{+}\mathit{x}\right)+\varphi \left(D_{\left(2\right)}^{+}\mathit{x}\right).$$

(4)

We would like to point out that when $K=1$, the OGSTV regularization term is a traditional anisotropic TV functional with asymmetry.

Subsequently, the proposed myopic deconvolution model incorporating the OGSTV regularization term can be expressed as

$$\begin{array}{c}\left\{\begin{array}{cc}{\displaystyle \underset{\mathit{x}\in {\mathcal{C}}_{\mathit{x}},\mathit{w}\in {\mathcal{C}}_{\mathit{w}}}{min}}\hfill & \Phi \left(\mathit{x},\mathit{w}\right)={\displaystyle \frac{\mu }{2}}{\parallel A\left(\mathit{w}\right)\mathit{x}-\mathit{d}\parallel }_2^2+\phi (▽\mathit{x})\hfill \\ \mathrm{s}.\mathrm{t}.\hfill & {\displaystyle \sum _{j=1}^p}{w}_j=1,\hfill \end{array}\right.\end{array}$$

(5)

where ${\mathcal{C}}_{\mathit{x}}=\{\mathit{x}\mid x_i\ge 0\mathrm{for}i=1,\dots ,n^2\},{\mathcal{C}}_{\mathit{w}}=\{\mathit{w}\mid w_j\ge 0\mathrm{for}j=1,\dots ,p\}$, $A\left(\mathit{w}\right)={\sum }_{j=1}^p{w}_j{A}_j$ and $\phi (▽\mathit{x})$ denotes the OGSTV regularization term.

Moreover, to limit the ‘sum of weights’ to be close to 1, a suitable regularization term is introduced on w. Specifically, a quadratic penalty function $S\left(\mathit{w}\right)$ is introduced, which for sufficiently large $\xi$ effectively guarantees that the weights $w_j,j=1,\dots ,p$ are summed equal to 1. Consequently, we propose the following model:

$$\underset{\mathit{x}\in {\mathcal{C}}_{\mathit{x}},\mathit{w}\in {\mathcal{C}}_{\mathit{w}}}{min}{\displaystyle \frac{\mu }{2}}{\parallel A\left(\mathit{w}\right)\mathit{x}-\mathit{d}\parallel }_2^2+\phi (▽\mathit{x})+S\left(\mathit{w}\right),$$

(6)

where the regularization $S\left(\mathit{w}\right)$ is defined as

$$S\left(\mathit{w}\right)={\displaystyle \frac{\xi }{2}}{({\mathit{e}}^{\top }\mathit{w}-1)}^2,$$

where $\xi >0$ denotes a weighting parameter, and $\mathit{e}\in {\mathbb{R}}^p$ denotes the all-ones vector.

3. Optimization Schemes

In this section, we employ the alternating direction method of multipliers (ADMM) as an outer-layer optimization framework to solve the proposed model (6). The ADMM subproblems are efficiently solved by two strategies: the majorization–minimization (MM) method is applied to address the OGSTV component, while a modified version of the linearize and project (LAP) method is implemented to address the tightly coupled subproblem. Based on the strategies above, we propose an ADMM-MM-LAP method.

3.1. ADMM Scheme

Considering the structure of the model (6), we apply ADMM as an outer-layer algorithm to solve it. By introducing the artificial vector $\mathit{v}$, we rewrite (6) in the following equivalent optimization problem

$$\left\{\begin{array}{c}{\displaystyle \underset{\mathit{x},\mathit{w},\mathit{v}}{min}}{\displaystyle \frac{\mu }{2}}\left|\right|A\left(\mathit{w}\right)\mathit{x}-{\mathit{d}\left|\right|}_2^2+\phi \left(\mathit{v}\right)+S\left(\mathit{w}\right)+{\delta }_{{\mathcal{C}}_x}\left(\mathit{x}\right)+{\delta }_{{\mathcal{C}}_w}\left(\mathit{w}\right)\hfill \\ \mathrm{s}.\mathrm{t}.\mathit{v}=▽\mathit{x},\hfill \end{array}\right.$$

(7)

where ${\delta }_{\mathcal{C}}(·)$ represents the indicator function

$${\delta }_{\mathcal{C}}\left(s\right)=\left\{\begin{array}{cc}0,& s\in \mathcal{C},\\ \infty ,& s\notin \mathcal{C}.\end{array}\right.$$

The augmented Lagrangian function for (7) is given by

$$\begin{array}{cc}& L(\mathit{x},\mathit{w},\mathit{v},\alpha )\hfill \\ \hfill =& {\displaystyle \frac{\mu }{2}}{\parallel A\left(\mathit{w}\right)\mathit{x}-\mathit{d}\parallel }_2^2+S\left(\mathit{w}\right)+{\delta }_{{\mathcal{C}}_x}\left(\mathit{x}\right)+{\delta }_{{\mathcal{C}}_w}\left(\mathit{w}\right)+\phi \left(\mathit{v}\right)-{\alpha }^T(\mathit{v}-▽\mathit{x})+{\displaystyle \frac{\rho }{2}}{\parallel \mathit{v}-▽\mathit{x}\parallel }_2^2,\hfill \end{array}$$

(8)

where $\alpha$ represents the Lagrangian multiplier and $\rho >0$ represents the penalty parameter.

The main idea of ADMM for solving (7) is to alternately update variables by minimizing the augmented Lagrangian function (8). Then, we treat $\mathit{v}$ as a block of variables and $(\mathit{x},\mathit{w})$ as another block of variables. Given an initial point $({\mathit{v}}^{\mathbf{0}},{\mathit{x}}^{\mathbf{0}},{\mathit{w}}^{\mathbf{0}},{\alpha }^{\mathbf{0}})$, ADMM iteratively solves the following $\mathit{v}$-subproblem and $(\mathit{x},\mathit{w})$-subproblem

$$\left\{\begin{array}{c}\begin{array}{cc}\hfill {\mathit{v}}^{k+1}& =arg\underset{\mathit{v}}{min}L({\mathit{x}}^k,{\mathit{w}}^k,\mathit{v},{\alpha }^k),\hfill \\ \hfill ({\mathit{x}}^{k+1},{\mathit{w}}^{k+1})& =arg\underset{\mathit{x},\mathit{w}}{min}L(\mathit{x},\mathit{w},{\mathit{v}}^{k+1},{\alpha }^k),\hfill \\ \hfill {\alpha }^{k+1}& ={\alpha }^k+\rho \left(▽{\mathit{x}}^{k+1}-{\mathit{v}}^{k+1}\right).\hfill \end{array}\hfill \end{array}\right.$$

First, the $\mathit{v}$-subproblem can be formulated as

$$\begin{array}{cc}\hfill {\mathit{v}}^{k+1}=& arg\underset{\mathit{v}}{min}{\displaystyle \frac{\rho }{2}}{∥\mathit{v}-▽{\mathit{x}}^k∥}_2^2-{\alpha }^T(\mathit{v}-▽{\mathit{x}}^k)+\phi \left(\mathit{v}\right)\hfill \\ \hfill =& arg\underset{\mathit{v}}{min}{\displaystyle \frac{\rho }{2}}{\parallel \mathit{v}-(▽{\mathit{x}}^k+{\displaystyle \frac{{\alpha }^k}{\rho }})\parallel }_2^2+\phi \left(\mathit{v}\right).\hfill \end{array}$$

(9)

Second, for the $(\mathit{x},\mathit{w})$-subproblem,

$$({\mathit{x}}^{k+1},{\mathit{w}}^{k+1})=arg\underset{\mathit{x},\mathit{w}}{min}{\displaystyle \frac{\mu }{2}}{∥A\left(\mathit{w}\right)\mathit{x}-\mathit{d}∥}_2^2+S\left(\mathit{w}\right)-{\alpha }^T({\mathit{v}}^{k+1}-▽\mathit{x})+{\displaystyle \frac{\rho }{2}}{∥{\mathit{v}}^{k+1}-▽\mathit{x}∥}_2^2,$$

let

$$C(\mathit{x},\mathit{v},\alpha )=\phi \left(\mathit{v}\right)-{\alpha }^T(\mathit{v}-▽\mathit{x})+{\displaystyle \frac{\rho }{2}}{\parallel \mathit{v}-▽\mathit{x}\parallel }_2^2,$$

and define

$$\widehat{\Psi }(\mathit{x},\mathit{w},\mathit{v},\alpha ):={\displaystyle \frac{\mu }{2}}{\parallel A\left(\mathit{w}\right)\mathit{x}-\mathit{d}\parallel }_2^2+S\left(\mathit{w}\right)+C(\mathit{x},\mathit{v},\alpha ).$$

Then, the ($\mathit{x},\mathit{w}$)-subproblem can be formulated as

$$\begin{array}{c}({\mathit{x}}^{k+1},{\mathit{w}}^{k+1})=\underset{\mathit{x}\in {\mathcal{C}}_x,\mathit{w}\in {\mathcal{C}}_w}{min}\widehat{\Psi }(\mathit{x},\mathit{w},{\mathit{v}}^{k+1},{\alpha }^k).\end{array}$$

(10)

Finally, we update the Lagrangian multiplier

$${\alpha }^{k+1}={\alpha }^k+\rho \left(▽{\mathit{x}}^{k+1}-{\mathit{v}}^{k+1}\right).$$

(11)

Next, we will solve these two subproblems according to their respective structural characteristics.

3.2. The Majorization–Minimization Algorithm (MM)

To tackle the $\mathit{v}$-subproblem (9), we adopt the MM method in [24]. Before providing the solution to (9), we specifically explore a more generalized problem formulation, i.e.,

$$\underset{\mathit{v}}{min}\{F\left(\mathit{v}\right)={\displaystyle \frac{1}{2}}\parallel \mathit{v}-{\mathit{v}}_0{\parallel }_2^2+{\displaystyle \frac{1}{\rho }}\phi \left(\mathit{v}\right)\},\mathit{v}\in {\mathbb{R}}^{n^2},$$

(12)

where ${\mathit{v}}_0=▽{\mathit{x}}^k+{\displaystyle \frac{{\alpha }^k}{\rho }}$ and the function $\phi (·)$ is given by (4). Rather than directly optimizing the complicated problem (9), we employ the MM method to iteratively solve it. Specifically, we simply look for a sequence of convex functions: $G\left(\mathit{v},{\mathit{v}}^k\right),k=0,1,2,\dots$. The principle underlying the MM method is that each $G(\mathit{v},{\mathit{v}}^k)$ serves as an upper bound on $F\left(\mathit{v}\right)$ for any $\mathit{v}$ and ${\mathit{v}}^k$ and attains $F\left(\mathit{v}\right)$ at $G(\mathit{v},\mathit{v})=F\left(\mathit{v}\right)$. The updated iteration yields a reduced cost. However, the complexity within the MM framework lies in constructing an appropriate function $G(\mathit{v},{\mathit{v}}^k)$.

Notice that the inequality

$${\displaystyle \frac{1}{2}}\left({\displaystyle \frac{1}{{\parallel \mathit{u}\parallel }_2}}{\parallel \mathit{v}\parallel }_2^2+{\parallel \mathit{u}\parallel }_2\right)\ge {\parallel \mathit{v}\parallel }_2,$$

(13)

for all $\mathit{v}$ and $\mathit{u}\ne 0$ with equality when $\mathit{u}=\mathit{v}$. By plugging each group of $\phi \left(\mathit{v}\right)$ into (13) and aggregating the results, we obtain a majorizor of $\phi \left(\mathit{v}\right)$. Let

$$T(\mathit{v},\mathit{u})={\displaystyle \frac{1}{2}}\sum _{i,j=1}^n\left[{\displaystyle \frac{1}{\parallel {\mathit{u}}_{i,j,K}{\parallel }_2}}\parallel {\mathit{v}}_{i,j,K}{\parallel }_2^2+{\parallel {\mathit{u}}_{i,j,K}\parallel }_2\right]$$

with $T(\mathit{v},\mathit{u})\ge \phi \left(\mathit{v}\right),T(\mathit{u},\mathit{u})=\phi \left(\mathit{u}\right)$, provided that $\parallel {\mathit{u}}_{i,j,K}{\parallel }_2\ne 0$. By collating, $T(\mathit{v},\mathit{u})$ can be reformulated as

$$T(\mathit{v},\mathit{u})={\displaystyle \frac{1}{2}}{\parallel \Omega \left(\mathit{u}\right)\mathit{v}\parallel }_2^2+C\left(\mathit{u}\right),$$

(14)

where $C\left(\mathit{u}\right)$ represents a constant, which is independent on $\mathit{v}$, and $\Omega \left(\mathit{u}\right)$ denotes a diagonal matrix whose diagonal elements are defined as

$${[\Omega \left(\mathit{u}\right)]}_{l,l}=\sqrt{{\displaystyle \sum _{i,j=-m_1}^{m_2}}{\left[{\displaystyle \sum _{k_1,k_2=-m_1}^{m_2}}{\left|{\mathit{u}}_{r-i+k_1,t-j+k_2}\right|}^2\right]}^{-{\displaystyle \frac{1}{2}}}},$$

(15)

with $l=(r-1)n+t$, $\forall r,t=1,2,\dots ,n$, the entries of $\Omega \left(\mathit{u}\right)$ can be efficiently computed via convolution operation.

Then, the majorizer of $F\left(\mathit{v}\right)$ can be expressed as

$$G\left(\mathit{v},\mathit{u}\right)={\displaystyle \frac{1}{2}}\parallel \mathit{v}-{\mathit{v}}_0{\parallel }_2^2+{\displaystyle \frac{1}{\rho }}T(\mathit{v},\mathit{u})={\displaystyle \frac{1}{2}}\parallel \mathit{v}-{\mathit{v}}_0{\parallel }_2^2+{\displaystyle \frac{1}{2\rho }}{\parallel \Omega \left(\mathit{u}\right)\mathit{v}\parallel }_2^2+\mu C\left(\mathit{u}\right).$$

We have $G(\mathit{v},\mathit{u})\ge F\left(\mathit{v}\right)$ for any $\mathit{u}$, $\mathit{v}$, and $G(\mathit{u},\mathit{u})=F\left(\mathit{u}\right)$. In order to obtain the optimal $\mathit{v}$ from $F\left(\mathit{v}\right)$, the MM iterations can be performed as

$${\mathit{v}}^{k+1}={\left(I+\mu \Omega {\left({\mathit{v}}^k\right)}^{\top }\Omega \left({\mathit{v}}^k\right)\right)}^{-1}{\mathit{v}}_0,k=0,1,\dots ,$$

(16)

where I represents the identity matrix of the same dimension as $\Omega \left({\mathit{v}}^k\right)$. We would like to point out that the inversion of $I+\mu \Omega {\left({\mathit{v}}^k\right)}^{\top }\Omega \left({\mathit{v}}^k\right)$ can be computed efficiently, as it requires only straightforward computations of its components. To sum up, Algorithm 1 shows the iterative format of the MM method for (12).

Algorithm 1 The MM method for (12)

1: Input: ${\mathit{v}}_0\in {\mathbb{R}}^{n^2}$. 2: Output: Set $\mathit{v}={\mathit{v}}^{k+1}$. 3: Initialize: ${\mathit{v}}^0={\mathit{v}}_0$, $k=0$, $\rho$, group size K, maximum iterations N. 4: Iterates 5: Calculate $\Omega \left({\mathit{v}}^k\right)$ according to (15). 6: Calculate ${\mathit{v}}^{k+1}$ according to (16). 7: $k=k+1.$ 8: until iteration N is satisfied.

3.3. The Linearize and Project (LAP)

To solve the $(\mathit{x},\mathit{w})$-subproblem (10), we employ a modified version of the linearize and project (LAP) method. The original LAP framework is by Herring et al. [32] for solving coupled optimization problems, incorporating fundamental principles derived from the Gauss–Newton method. By using linearization and projection steps to deal with coupled variables, this method can effectively deal with tightly coupled optimization problems and complex problems with pointwise boundary constraints.

The main step of the LAP method is to linearize the residuals first, and then eliminate a set of variables by projection operation to obtain the approximation problem. The variables can be classified into two distinct groups: active set variables and inactive set variables. Subsequently, the update steps $\delta \mathit{x}$ and $\delta \mathit{w}$ can be calculated within both sets. First, we formulate the set of feasible indicators as

$$\mathcal{N}:=\left\{q\in \mathbb{N}|\left[\begin{array}{c}{\mathit{x}}_q\\ {\mathit{w}}_q\end{array}\right]\ge 0\right\}.$$

The active set and inactive set are defined as

$$\begin{array}{cc}\hfill \mathcal{A}& :=\left\{q\in \mathbb{N}|\left[\begin{array}{c}{\mathit{x}}_q\\ {\mathit{w}}_q\end{array}\right]=\mathbf{0}\right\},\hfill \\ \hfill \mathcal{I}& :=\mathcal{N}\setminus \mathcal{A},\hfill \end{array}$$

respectively.

Then, the variables can be divided into active set variables $\left[\begin{array}{c}{\mathit{x}}_{\mathcal{A}}\\ {\mathit{w}}_{\mathcal{A}}\end{array}\right]$, and inactive set variables $\left[\begin{array}{c}{\mathit{x}}_{\mathcal{I}}\\ {\mathit{w}}_{\mathcal{I}}\end{array}\right]$. The update step can be represented as $\left[\begin{array}{c}\delta {\mathit{x}}_{\mathcal{A}}\hfill \\ \delta {\mathit{w}}_{\mathcal{A}}\hfill \end{array}\right]$ and $\left[\begin{array}{c}\delta {\mathit{x}}_{\mathcal{I}}\hfill \\ \delta {\mathit{w}}_{\mathcal{I}}\hfill \end{array}\right]$, respectively.

The computation method of $\delta {\mathit{x}}_{\mathcal{I}}$ and $\delta {\mathit{w}}_{\mathcal{I}}$ is similar to the unconstrained situation. It follows the idea of Newton’s method, the update steps $\left[\begin{array}{c}\delta \mathit{x}\\ \delta \mathit{w}\end{array}\right]$ are given by

$$\left[\begin{array}{c}\delta \mathit{x}\\ \delta \mathit{w}\end{array}\right]=-{▽}_{\mathit{x},\mathit{w}}^2\widehat{\Psi }{\left(\mathit{x},\mathit{w},{\mathit{v}}^{k+1},{\alpha }^k\right)}^{-1}{▽}_{\mathit{x},\mathit{w}}\widehat{\Psi }\left(\mathit{x},\mathit{w},{\mathit{v}}^{k+1},{\alpha }^k\right).$$

That is,

$$\left[\begin{array}{c}\delta \mathit{x}\\ \delta \mathit{w}\end{array}\right]=-{\left[\begin{array}{cc}{\mathbf{J}}_{\mathit{x}}^{\top }{\mathbf{J}}_{\mathit{x}}+{\displaystyle \frac{1}{\mu }}{▽}_{\mathit{x}}^{2}C\left(\mathit{x},{\mathit{v}}^{k+1},{\alpha }^k\right)& {\mathbf{J}}_{\mathit{x}}^{\top }{\mathbf{J}}_{\mathit{w}}\\ {\mathbf{J}}_{\mathit{w}}^{\top }{\mathbf{J}}_{\mathit{x}}& {\mathbf{J}}_{\mathit{w}}^{\top }{\mathbf{J}}_{\mathit{w}}+{▽}_{\mathit{w}}^{2}S\left(\mathit{w}\right)\end{array}\right]}^{-1}\left[\begin{array}{c}{\mathbf{J}}_{\mathit{x}}^{\top }\mathit{r}+{\displaystyle \frac{1}{\mu }}{▽}_{\mathit{x}}C\left(\mathit{x},{\mathit{v}}^{k+1},{\alpha }^k\right)\\ {\mathbf{J}}_{\mathit{w}}^{\top }\mathit{r}+{▽}_{\mathit{w}}S\left(\mathit{w}\right)\end{array}\right],$$

where ${▽}_x\mathit{C}(\mathit{x},{\mathit{v}}^{\mathit{k}+\mathbf{1}},{\alpha }^{\mathit{k}})={▽}^T{\alpha }^k-\rho {▽}^T({\mathit{v}}^{k+1}-▽\mathit{x})$ and ${▽}_x^2\mathit{C}(\mathit{x},{\mathit{v}}^{\mathit{k}+\mathbf{1}},{\alpha }^{\mathit{k}})=\rho {▽}^T▽$. ${\mathbf{J}}_{\mathit{x}}$ and ${\mathbf{J}}_{\mathit{w}}$ denote the Jacobian matrices with respect to the $\mathit{x}$ block and the $\mathit{w}$ block, respectively. However, it should be noted that the variables must be projected onto the inactive set. To be specific, the value of $\delta {\mathit{x}}_{\mathcal{I}}$ at the present iteration point $(\mathit{x},\mathit{w})$ can be computed by

$$\widehat{\mathit{M}}\delta {\mathit{x}}_{\mathcal{I}}=\widehat{\mathit{b}},$$

(17)

$$\begin{array}{cc}\hfill \widehat{\mathit{M}}:=& {\widehat{\mathbf{J}}}_{\mathit{x}}^{\top }\left(\mathit{I}-{\widehat{\mathbf{J}}}_{\mathit{w}}{\left({\widehat{\mathbf{J}}}_{\mathit{w}}^{\top }{\widehat{\mathbf{J}}}_{\mathit{w}}+{▽}_{\mathit{w}}^2\widehat{S}\left(\mathit{w}\right)\right)}^{-1}{\widehat{\mathbf{J}}}_{\mathit{w}}^{\top }\right){\widehat{\mathbf{J}}}_{\mathit{x}}+{\displaystyle \frac{1}{\mu }}{▽}_{\mathit{x}}^2\widehat{C}\left(\mathit{x},{\mathit{v}}^{k+1},{\alpha }^k\right),\hfill \\ \hfill \widehat{\mathit{b}}:=& -{\widehat{\mathbf{J}}}_{\mathit{x}}^{\top }\left(\mathit{I}-{\widehat{\mathbf{J}}}_{\mathit{w}}{\left({\widehat{\mathbf{J}}}_{\mathit{w}}^{\top }{\widehat{\mathbf{J}}}_{\mathit{w}}+{▽}_{\mathit{w}}^2\widehat{S}\left(\mathit{w}\right)\right)}^{-1}{\widehat{\mathbf{J}}}_{\mathit{w}}^{\top }\right)\mathit{r}-{\displaystyle \frac{1}{\mu }}{▽}_{\mathit{x}}\widehat{C}\left(\mathit{x},{\mathit{v}}^{k+1},{\alpha }^k\right)\hfill \\ & +{\widehat{\mathbf{J}}}_{\mathit{x}}^{\top }{\widehat{\mathbf{J}}}_{\mathit{w}}{\left({\widehat{\mathbf{J}}}_{\mathit{w}}^{\top }{\widehat{\mathbf{J}}}_{\mathit{w}}+{▽}_{\mathit{w}}^2\widehat{S}\left(\mathit{w}\right)\right)}^{-1}{▽}_{\mathit{w}}\widehat{S}\left(\mathit{w}\right),\hfill \end{array}$$

where ${\widehat{\mathbf{J}}}_{\mathit{x}},{\widehat{\mathbf{J}}}_{\mathit{w}},{▽}_{\mathit{x}}\widehat{C},{▽}_{\mathit{x}}^2\widehat{C},{▽}_{\mathit{w}}\widehat{S}$, ${▽}_{\mathit{w}}^2\widehat{S}$ denote ${\mathbf{J}}_{\mathit{x}},{\mathbf{J}}_{\mathit{w}},{▽}_{\mathit{x}}C,{▽}_{\mathit{x}}^{2}C,{▽}_{\mathit{w}}S$, ${▽}_{\mathit{w}}^{2}S$ restricted to the inactive set, respectively. The reduced problem (17) does not require a high-precision solution. For instance, in [32], a stopping tolerance of ${10}^{-1}$ is employed for solving the reduced problem iteratively. Once the value of $\delta {\mathit{x}}_{\mathcal{I}}$ has been obtained through the solving process, the value of $\delta {\mathit{w}}_{\mathcal{I}}$ can then be calculated by

$$\delta {\mathit{w}}_{\mathcal{I}}=-{\left({\widehat{\mathbf{J}}}_{\mathit{w}}^{\top }{\widehat{\mathbf{J}}}_{\mathit{w}}+{▽}_{\mathit{w}}^2\widehat{S}\left(\mathit{w}\right)\right)}^{-1}\left({\widehat{\mathbf{J}}}_{\mathit{w}}^{\top }{\widehat{\mathbf{J}}}_{\mathit{x}}\delta {\mathit{x}}_{\mathcal{I}}+{\widehat{\mathbf{J}}}_{\mathit{w}}^{\top }\mathit{r}+{▽}_{\mathit{w}}\widehat{S}\left(\mathit{w}\right)\right).$$

(18)

For the active set, $\delta {\mathit{x}}_{\mathcal{A}}$ and $\delta {\mathit{w}}_{\mathcal{A}}$ can be obtained through

$$\left[\begin{array}{c}\delta {\mathit{x}}_{\mathcal{A}}\hfill \\ \delta {\mathit{w}}_{\mathcal{A}}\hfill \end{array}\right]=-\mu \left[\begin{array}{c}{\tilde{\mathbf{J}}}_{\mathit{x}}^{\top }\mathit{r}\\ {\tilde{\mathbf{J}}}_{\mathit{w}}^{\top }\mathit{r}\end{array}\right]-\left[\begin{array}{c}{▽}_{\mathit{x}}\tilde{C}\left(\mathit{x}+\delta {\mathit{x}}_{\mathcal{A}},{\mathit{v}}^{k+1},{\alpha }^k\right)\\ {▽}_{\mathit{w}}\tilde{S}\left(\mathit{w}\right)\end{array}\right],$$

(19)

where ${\tilde{\mathbf{J}}}_{\mathit{x}},{\tilde{\mathbf{J}}}_{\mathit{w}},{▽}_{\mathit{x}}\tilde{C}$, ${▽}_{\mathit{w}}\tilde{S}$ denote the projection of ${\mathbf{J}}_{\mathit{x}},{\mathbf{J}}_{\mathit{w}},{▽}_{\mathit{x}}C$, ${▽}_{\mathit{w}}S$ onto the active set, respectively.

Subsequently, $\delta \mathit{x}$ and $\delta \mathit{w}$ can be computed as a combination of $\delta {\mathit{x}}_{\mathcal{A}}$, $\delta {\mathit{w}}_{\mathcal{A}}$, $\delta {\mathit{x}}_{\mathcal{I}}$, and $\delta {\mathit{w}}_{\mathcal{I}}$ in the following equation

$$\left[\begin{array}{c}\delta \mathit{x}\hfill \\ \delta \mathit{w}\hfill \end{array}\right]=\left[\begin{array}{c}\delta {\mathit{x}}_{\mathcal{I}}\hfill \\ \delta {\mathit{w}}_{\mathcal{I}}\hfill \end{array}\right]+\theta \left[\begin{array}{c}\delta {\mathit{x}}_{\mathcal{A}}\hfill \\ \delta {\mathit{w}}_{\mathcal{A}}\hfill \end{array}\right],$$

(20)

the appropriate parameter $\theta$ can be selected based on [33].

After that, the update steps are integrated through the projection gradient descent method, with the active set and inactive set being iteratively updated to guarantee the point-wise boundary constraints. This comprehensive approach enables the effective handling of large-scale optimization problems with tightly coupled variables and point-wise boundary constraints. The LAP method is used to solve the $(\mathit{x},\mathit{w})$-subproblem inexactly. The residual is

$$\left[\begin{array}{c}{\eta }_{\mathit{x}}^{k+1}\\ {\eta }_{\mathit{w}}^{k+1}\end{array}\right]:={▽}_{(\mathit{x},\mathit{w})}\widehat{\Psi }(\mathit{x},\mathit{w},\mathit{v},\alpha ).$$

For the stopping criterion, let

$$\left[\begin{array}{c}{\eta }_{\mathit{x}}^{k+1}\\ {\eta }_{\mathit{w}}^{k+1}\end{array}\right]\le {\epsilon }_{k+1},$$

where ${\left\{{\epsilon }_{k+1}\right\}}_{k=0}^{\infty }$ is a non-negative sequence satisfying ${\displaystyle \sum _{k=0}^{\infty }}{\epsilon }_{k+1}<\infty$.

In summary, from the above analysis, we provide the iteration scheme for solving (12) in Algorithm 2.

Algorithm 2 The LAP method for (12)

1: Input: $({\mathit{x}}^0,{\mathit{w}}^0)\in {\mathbb{R}}^{n^2}\times {\mathbb{R}}^p$. Let $k=0$. 2: Output: $({\mathit{x}}^k,{\mathit{w}}^k)$. 3: Iterates 4: Apply the LAP method to compute $\delta {\mathit{x}}_{\mathcal{I}}$ and $\delta {\mathit{w}}_{\mathcal{I}}$ according to (17) and (18), respectively. 5: Apply the projection gradient descent method to compute $\delta {\mathit{x}}_{\mathcal{A}}$ and $\delta {\mathit{w}}_{\mathcal{A}}$ according to (19). 6: Combine $\delta {\mathit{x}}_{\mathcal{A}},\delta {\mathit{w}}_{\mathcal{A}},\delta {\mathit{x}}_I,\delta {\mathit{w}}_I$ according to (20). 7: Compute ${\mathit{x}}^{k+1}$ and ${\mathit{w}}^{k+1}$ with a projected Armijo line search. 8: Update the active set $\mathcal{A}$ and the inactive set $\mathcal{I}$. 9: $k:=k+1$. 10: until the termination condition is satisfied.

3.4. ADMM-MM-LAP Method

Based on the representations and analysis above, we present the iterative scheme of the ADMM-MM-LAP method for solving the AO retinal image model (7) with the OGSTV regularization term in Algorithm 3.

Algorithm 3 ADMM-MM-LAP method for (7)

1: Input: ${\mathit{v}}_0,{\mathit{x}}_0,{\mathit{w}}_0$ and parameters $\mu >0,\rho >0$, group size K. 2: Output: ${\mathit{v}}^k,{\mathit{x}}^k,{\mathit{w}}^k,{\alpha }^k$. 3: Initialize: ${\mathit{x}}_0=f,k=0,{\alpha }^0=\mathbf{0}$. 4: Apply Algorithm 1 to compute ${\mathit{v}}^{(k+1)}$. 5: Apply Algorithm 2 to compute (${\mathit{x}}^{(k+1)},{\mathit{w}}^{(k+1)}$). 6: Update the multipliers using (11). 7: $k:=k+1$. 8: until the termination condition is satisfied.

3.5. Complexity Analysis

In this subsection, the computational cost generated at each step of Algorithm 3 is considered, and the computational complexity analysis is given.

First, the $\mathit{v}$-subproblem can be solved by Algorithm 1. Each iteration needs to compute the diagonal matrix $\Omega \left({\mathit{v}}^k\right)$, and the computational complexity of each iteration is $O\left(n\right)$ by calculating diagonal elements through convolution operation. Then, the matrix inverse calculation can be computed efficiently by simple element-by-element calculation, and the overall computational complexity remains $O\left(n\right)$. Moreover, the computational cost is also related to the group size K, and K is a constant. Therefore, based on the above analysis, the complexity of $\mathit{v}$-subproblem is $O\left(n\right)$.

Second, the $(\mathit{x},\mathit{w})$-subproblem can be efficiently solved using Algorithm 2, whose computational cost comprises several main subcomponents, such as $\delta {\mathit{x}}_{\mathcal{I}},\delta {\mathit{w}}_{\mathcal{I}},\delta {\mathit{x}}_{\mathcal{A}},\delta {\mathit{w}}_{\mathcal{A}}$. The computation primarily involves three core components: matrix–vector product, vector norm computation, and conjugate gradient method. Then, we proceed with a detailed step-by-step complexity analysis. First, for computing the update step $\delta {\mathit{x}}_{\mathcal{I}}$, it is necessary to calculate the product of matrix ${\displaystyle \frac{1}{\mu }}{▽}_{\mathit{x}}^2\widehat{C}\left(\mathit{x},{\mathit{v}}^{k+1},{\alpha }^k\right)+{\widehat{\mathbf{J}}}_{\mathit{x}}^{\top }\left(\mathit{I}-{\widehat{\mathbf{J}}}_{\mathit{w}}{\left({\widehat{\mathbf{J}}}_{\mathit{w}}^{\top }{\widehat{\mathbf{J}}}_{\mathit{w}}+{▽}_{\mathit{w}}^2\widehat{S}\left(\mathit{w}\right)\right)}^{-1}{\widehat{\mathbf{J}}}_{\mathit{w}}^{\top }\right){\widehat{\mathbf{J}}}_{\mathit{x}}$, vectors $-{\widehat{\mathbf{J}}}_{\mathit{x}}^{\top }\left(\mathit{I}-{\widehat{\mathbf{J}}}_{\mathit{w}}{\left({\widehat{\mathbf{J}}}_{\mathit{w}}^{\top }{\widehat{\mathbf{J}}}_{\mathit{w}}+{▽}_{\mathit{w}}^2\widehat{S}\left(\mathit{w}\right)\right)}^{-1}{\widehat{\mathbf{J}}}_{\mathit{w}}^{\top }\right)\mathit{r}$ and ${\widehat{\mathbf{J}}}_{\mathit{x}}^{\top }{\widehat{\mathbf{J}}}_{\mathit{w}}{\left({\widehat{\mathbf{J}}}_{\mathit{w}}^{\top }{\widehat{\mathbf{J}}}_{\mathit{w}}+{▽}_{\mathit{w}}^2\widehat{S}\left(\mathit{w}\right)\right)}^{-1}{▽}_{\mathit{w}}\widehat{S}\left(\mathit{w}\right)$, respectively. Due to the Jacobi matrix ${\mathbf{J}}_{\mathit{x}}=A{\left(\mathit{w}\right)}^{\top }$ and each PSF matrix, $A_i$ supports vector multiplication via the Fast Fourier Transform (FFT), the computational complexity of multiplying ${\mathbf{J}}_{\mathit{x}}$ with a vector is $O\left(n^{2}logn\right)$. Similarly, multiplying ${\mathbf{J}}_{\mathit{w}}$ with a vector also achieves $O\left(n^{2}logn\right)$ complexity by FFTs. So the matrix–vector product involving ${\widehat{\mathbf{J}}}_{\mathit{x}}^{\top }\left(\mathit{I}-{\widehat{\mathbf{J}}}_{\mathit{w}}{\left({\widehat{\mathbf{J}}}_{\mathit{w}}^{\top }{\widehat{\mathbf{J}}}_{\mathit{w}}+{▽}_{\mathit{w}}^2\widehat{S}\left(\mathit{w}\right)\right)}^{-1}{\widehat{\mathbf{J}}}_{\mathit{w}}^{\top }\right){\widehat{\mathbf{J}}}_{\mathit{x}}$ has the same $O\left(n^{2}logn\right)$ complexity. Moreover, the computational cost for the matrix–vector product of ${\displaystyle \frac{1}{\mu }}{▽}_{\mathit{x}}^2\widehat{C}\left(\mathit{x},{\mathit{v}}^{k+1},{\alpha }^k\right)$ is $O\left(n^2\right)$. Furthermore, the operations involving the expressions $-{\widehat{\mathbf{J}}}_{\mathit{x}}^{\top }\left(\mathit{I}-{\widehat{\mathbf{J}}}_{\mathit{w}}{\left({\widehat{\mathbf{J}}}_{\mathit{w}}^{\top }{\widehat{\mathbf{J}}}_{\mathit{w}}+{▽}_{\mathit{w}}^2\widehat{S}\left(\mathit{w}\right)\right)}^{-1}{\widehat{\mathbf{J}}}_{\mathit{w}}^{\top }\right)\mathit{r}$ and ${\widehat{\mathbf{J}}}_{\mathit{x}}^{\top }{\widehat{\mathbf{J}}}_{\mathit{w}}{\left({\widehat{\mathbf{J}}}_{\mathit{w}}^{\top }{\widehat{\mathbf{J}}}_{\mathit{w}}+{▽}_{\mathit{w}}^2\widehat{S}\left(\mathit{w}\right)\right)}^{-1}{▽}_{\mathit{w}}\widehat{S}\left(\mathit{w}\right)$ can be computed via FFTs with a complexity of $O\left(n^{2}logn\right)$. Thus, the overall computational cost of $\delta {\mathit{x}}_{\mathcal{I}}$ is $O\left(n^{2}logn\right)$. Similarly, for $\delta {\mathit{w}}_{\mathcal{I}},\delta {\mathit{x}}_{\mathcal{A}},\delta {\mathit{w}}_{\mathcal{A}}$, the computations of their matrix–vector products also demand acceleration by FFTs, such as ${\widehat{\mathbf{J}}}_{\mathit{w}}^{\top }{\widehat{\mathbf{J}}}_{\mathit{x}}\delta {\mathit{x}}_{\mathcal{I}}$, ${\widehat{\mathbf{J}}}_{\mathit{w}}\mathit{r}$,${\tilde{\mathbf{J}}}_{\mathit{x}}^{\top }\mathit{r},{\tilde{\mathbf{J}}}_{\mathit{w}}^{\top }\mathit{r}$, etc. Therefore, the computational cost of calculating $\delta {\mathit{w}}_{\mathcal{I}},\delta {\mathit{x}}_{\mathcal{A}},\delta {\mathit{w}}_{\mathcal{A}}$ is $O\left(n^{2}logn\right)$.

In addition, update steps $\delta \mathit{x}$ and $\delta \mathit{w}$ mainly involve scalar–vector multiplication and vector summation calculation, while the projection Armijo line search involves the calculations of ${▽}_{\mathit{x},\mathit{w}}\widehat{\Psi }\left(\mathit{x},\mathit{w},{\mathit{v}}^{k+1},{\alpha }^k\right)$ and the inner product. The computational cost is $O\left(n^2\right)$. In summary, the $(\mathit{x},\mathit{w})$-subproblem complexity is $O\left(n^{2}logn\right)$.

Finally, in the Lagrangian multiplier update step, the computational cost is dominated by the calculations of $\rho {\mathit{v}}^{k+1}$ and $\rho ▽{\mathit{x}}^{k+1}$, each requiring $O\left(n^2\right)$ operations. Therefore, we can conclude the following theorem that the computational complexity per iteration of Algorithm 3 is $O\left(n^{2}logn\right)$.

Theorem 1. 

The computational complexity of Algorithm 3 is $O\left(n^{2}logn\right)$.

4. Numerical Experiments

In this section, we present the numerical experimental results of the proposed model, which incorporates the OGSTV regularization term for myopic deconvolution of AO retinal images. All experimental results were implemented using MATLAB 2020a on a PC with 16.0 GB RAM and AMD Ryzen 5 4600U with Radeon Graphics at 2.10 GHz.

The restoration performance is evaluated by five principal metrics, including the relative errors of $\mathit{x}$ and $\mathit{w}$, signal-to-noise ratio $\left(\mathrm{SNR}\right)$, peak signal-to-noise ratio $\left(\mathrm{PSNR}\right)$, and structural similarity index $\left(\mathrm{SSIM}\right)$. Specifically, the metrics are defined as follows:

• ${\mathrm{RE}}_{\mathrm{x}}={\displaystyle \frac{{∥\mathit{x}-{\mathit{x}}^{*}∥}_2}{{∥{\mathit{x}}^{*}∥}_2}}$, where ${\mathit{x}}^{*}$ denotes the true image and $\mathit{x}$ denotes the restored image.
• ${\mathrm{RE}}_w={\displaystyle \frac{{∥\mathit{w}-{\mathit{w}}^{*}∥}_2}{{∥{\mathit{w}}^{*}∥}_2}}$, where ${\mathit{w}}^{*}$ is the real parameter, and $\mathit{w}$ is the calculated parameter.
• $\mathrm{SNR}=10·{log}_{10}{\displaystyle \frac{\parallel {\mathit{x}}^{*}-\tilde{\mathit{x}}{\parallel }_2^2}{\parallel {\mathit{x}}^{*}{-\mathit{x}\parallel }_2^2}}$, where ${\mathit{x}}^{*}$ denotes the true image, $\tilde{\mathit{x}}$ denotes the mean intensity value of ${\mathit{x}}^{*}$, and $\mathit{x}$ denotes the restored image.
• $\mathrm{PSNR}=10·{log}_{10}{\displaystyle \frac{N_p}{\parallel {\mathit{x}}^{*}{-\mathit{x}\parallel }_2}}$, where $N_p$ refers to the size of the image, ${\mathit{x}}^{*}$ and $\mathit{x}$ denote the true image and the restored image, respectively.
• $\mathrm{SSIM}={\displaystyle \frac{(2{\mu }_{\mathit{x}}{\mu }_{{\mathit{x}}^{*}}+{\mathrm{C}}_1)(2\sigma +{\mathrm{C}}_2)}{({\mu }_{\mathit{x}}^2+{\mu }_{{\mathit{x}}^{*}}^2+{\mathrm{C}}_1)({\sigma }_{\mathit{x}}^2+{\sigma }_{{\mathit{x}}^{*}}^2+{\mathrm{C}}_2)}}$, where ${\mathrm{C}}_1$ and ${\mathrm{C}}_2$ are constants. $\sigma$ denotes the covariance between the images $\mathit{x}$ and ${\mathit{x}}^{*}$. ${\mu }_{{\mathit{x}}^{*}}$, ${\sigma }_{{\mathit{x}}^{*}}$, and ${\mu }_{\mathit{x}}$, ${\sigma }_{\mathit{x}}$ denote the average and standard variance of the AO retinal image ${\mathit{x}}^{*}$ and $\mathit{x}$, respectively.

The quantitative evaluation of retinal image restoration requires multiple metrics. Although PSNR provides computationally efficient distortion measurements via mean square error (MSE), its correlation with clinical diagnostic relevance remains limited due to the non-linear nature of human visual perception. In contrast, SSIM demonstrates superior alignment with subjective evaluations by modeling luminance, contrast, and structural correlations. For the comprehensive analysis, we employ both metrics alongside relative errors to ensure mathematical rigor and parameter estimation.

The parameters are carefully configured through experimental verification. In the OGSTV model, the group size K is set to 3 for ensuring restoration performance [12]. We set the parameter $\xi =100$ so that the obtained parameter meets ${\displaystyle \sum _{j=1}^p}{\omega }_j=1$. The parameter $\rho$ is chosen as 9. In order to guarantee the non-negative sequence ${\left\{{\epsilon }_{k+1}\right\}}_{k=0}^{\infty }$ satisfying ${\displaystyle \sum _{k=0}^{\infty }}{\epsilon }_{k+1}<\infty$, we choose ${\epsilon }_{k+1}={\displaystyle \frac{1}{b{(k+1)}^2}}$, with $b>0$. We implement a unified stopping criterion

$${\displaystyle \frac{|\widehat{\Psi }({\mathit{x}}^{k+1},{\omega }^{k+1},{\mathit{v}}^{k+1},{\alpha }^{k+1})-\widehat{\Psi }({\mathit{x}}^k,{\mathit{w}}^k,{\mathit{v}}^k,{\alpha }^k)|}{|\widehat{\Psi }({\mathit{x}}^k,{\mathit{w}}^k,{\mathit{v}}^k,{\alpha }^k)|}}<\epsilon ,$$

where $\widehat{\Psi }({\mathit{x}}^{k+1},{\omega }^{k+1},{\mathit{v}}^{k+1},{\alpha }^{k+1})$ represents the function value at the $(k+1)$-th iteration. The maximum number of iterations for the ADMM is set to 50. For the MM method, the inner iteration number N is set to 10.

In the context of AO retinal imaging, we use the TV model combined with the ADMM-LAP algorithm as the benchmark method [5]. In the following numerical experiments, we present quantitative comparisons between the proposed OGSTV model and the classical TV model, including relative errors for both reconstructed images and estimated parameters, as well as SNR, PSNR, and SSIM.

4.1. Example 1

• Problem Setting

For the myopic deconvolution problem of AO retinal images, $p=2$ is established in this example. The global PSF is synthesized by combining two distinct components: a focused PSF and a defocused PSF. The test image is a $256\times 256$ segment extracted from an AO retinal image. The test problem is constructed using the regularization toolbox IR Tools [34]. It can produce different BlurLevel indicators (‘mild’, ‘medium’, ‘severe’), and simulate spatially invariant Gaussian blur and out-of-focus blur using the $PRblurgauss$ function and $PRblurdefocus$ function in the IR Tools, respectively. For example, the $PRblurgauss$ function simulates a spatially invariant Gaussian blur to construct a PSF, while the $PRblurdefocus$ function simulates a spatially invariant out-of-focus blur for another PSF. In the example, three cases are designed. One PSF is fixed as a spatially invariant Gaussian blur generated by the $PRblurgauss$ function with a ‘mild’ blur level, whereas the other PSF is a composite of two components: a ‘mild’ Gaussian blur from $PRblurgauss$, and a defocus blur from $PRblurdefocus$ with three distinct levels (‘mild’, ‘medium’, and ‘severe’). Next, we choose the parameter $\mu =6\times {10}^5$ and set $\epsilon$ in the stopping criterion to $\epsilon =0.05$. Gaussian noise is added via the $PRnoise$ function and the noise level is 0.01. The true parameter is set to ${\omega }^{★}$ = [0.3; 0.7]. We choose the initial value of the parameter ${\omega }^0$ = [0.5; 0.5]. The initial guess $x^0$ is a random image with $256\times 256$ pixels.

• Experimental results

As an example, when the BlurLevel of $PRblurdefocus$ is set to ‘medium’, comparing the image restoration effects of our proposed OGSTV model and the classical TV model, it can be seen from Figure 1c,d that both models restored the images relatively clearly. It is not easy to judge which one of the OGSTV model and the TV model restores better by the eye only. To show the differences more clearly, Figure 1e–h presents an enlarged fragment of the corresponding images. In Figure 1g, the image recovered by the OGSTV model exhibits enhanced clarity and contrast, while the background in Figure 1h demonstrates a coarser background texture and has more residual noise. This indicates that the restored image obtained by the OGSTV model is more similar to the true image, and its restoration effect is better than that of the TV model.

For a quantitative comparison of Example 1, Figure 2 and

Table 1. The numerical results for Example 1.

BlurLevel	Model	${\mathbf{RE}}_{\mathbf{x}}$	${\mathbf{RE}}_{\mathbf{w}}$	SNR	PSNR	SSIM	Time (s)
‘mild’	OGSTV	1.61 $\times {\mathbf{10}}^{-\mathbf{1}}$	6.01 $\times {\mathbf{10}}^{-\mathbf{2}}$	8.86	25.24	0.72	33.65
	TV	2.98 $\times {10}^{-1}$	2.21 $\times {10}^{-1}$	3.50	19.88	0.52	21.46
‘medium’	OGSTV	1.33 $\times {\mathbf{10}}^{-\mathbf{1}}$	4.48 $\times {\mathbf{10}}^{-\mathbf{2}}$	10.47	26.86	0.81	35.60
	TV	1.43 $\times {10}^{-1}$	6.81 $\times {10}^{-2}$	9.87	26.25	0.78	31.29
‘severe’	OGSTV	7.52 $\times {\mathbf{10}}^{-\mathbf{2}}$	4.38 $\times {\mathbf{10}}^{-\mathbf{2}}$	15.46	31.84	0.92	37.53
	TV	1.17 $\times {10}^{-1}$	4.39 $\times {10}^{-2}$	11.60	27.99	0.85	50.11

The use of bold indicates that the numerical results are better.

enumerate the ${\mathrm{RE}}_{\mathrm{x}}$, ${\mathrm{RE}}_w$, SNR, PSNR, and SSIM of the restored images. In Figure 2, we plot the curve of ${\mathrm{RE}}_{\mathrm{x}}$ and ${\mathrm{RE}}_w$ changing with the number of iterations. We can see that the relative errors of the OGSTV model in the recovered image and estimating parameters are all lower than those of the TV model. It shows that our OGSTV model simultaneously enhances image restoration accuracy and improves parameter recovery precision. This validates our OGSTV model’s capability to outperform the TV model in AO retinal image processing.

In Table 1, the data comparison between the two models under three Blurlevels is detailed. Our OGSTV model outperforms the TV model in the key metrics. Specifically, our proposed OGSTV achieves consistently lower relative errors ${\mathrm{RE}}_{\mathrm{x}}$ and ${\mathrm{RE}}_{\mathrm{w}}$ than the TV model, reflecting its enhanced precision in reconstructing AO retinal images and parameter estimation. Simultaneously, the OGSTV model’s accuracy is further corroborated by its superior performance in SNR, PSNR, and SSIM levels. Although the OGSTV model exhibits slightly higher computational time compared to the TV model, the disparity remains minor. More critically, the OGSTV model recovers AO retinal images of better quality within a similar computation time. Notably, under the ‘severe’ $BlurLevel$, the OGSTV achieves a $35.7\%$ lower ${\mathrm{RE}}_{\mathrm{x}}$ and a $33.2\%$ higher SNR compared to the TV model. It indicates that the OGSTV model demonstrates increasing gaps in computationally demanding cases, and the image restoration effect is getting better. Therefore, it again shows that the OGSTV model outperforms the TV model in this test problem.

4.2. Example 2

• Problem Setting

The myopic deconvolution problem of AO retinal imaging is also considered, and the image size is 256 × 256. In this example, we build a combined PSF using $PRblurgauss$ with a ‘mild’ BlurLevel and $PRblurdefocus$ with three BlurLevels (‘mild’, ‘medium’, ‘severe’) in IR Tools. The other PSF is constructed using only $PRblurgauss$ with a ‘mild’ BlurLevel. We choose the parameter $\mu =5\times {10}^5$ and set the stopping criterion with $\epsilon =0.05$. Add Gaussian noise with a noise level of 0.01. Let the real parameter be ${\omega }^{★}$ = [0.3; 0.7]. To initiate the process, we select an arbitrary $256\times 256$ image as the initial value $x^0$ and select the initial value of the parameter ${\omega }^0$ = $[{\omega }_1^0;{\omega }_2^0]\in {\mathbb{R}}^2$, where ${\omega }_1^0,{\omega }_2^0$ are constants in [0, 1] and satisfy ${\omega }_1^0+{\omega }_2^0=1.$

• Experimental results

The experimental results are consistent with the results of Example 1. In this example, the BlurLevel of $PRblurdefocus$ is ‘mild’. We still need to observe the enlarged fragments of the two models in Figure 3. It can be observed that the background region in Figure 3g retains more complete texture details with significantly reduced residual noise levels. The images in Figure 3 further indicate that the image restored by our proposed OGSTV model resembles the true image more closely.

Figure 4 shows the relative errors of the reconstructed image and the estimated parameters against the number of iterations. As shown in Figure 4, although the TV model achieves a marginally lower image’s relative error at the fourth iteration, the OGSTV model demonstrates superior stability. Under identical stopping criteria, the OGSTV model attains a significantly smaller final error after additional iterations. Notably, the parameter estimation error of the OGSTV model converges to a negligible magnitude.

In

Table 2. The numerical results for Example 2.

BlurLevel	Model	${\mathbf{RE}}_{\mathbf{x}}$	${\mathbf{RE}}_{\mathbf{w}}$	SNR	PSNR	SSIM	Time (s)
‘mild’	OGSTV	1.58 $\times {\mathbf{10}}^{-\mathbf{1}}$	3.12 $\times {\mathbf{10}}^{-\mathbf{2}}$	8.11	24.86	0.68	16.12
	TV	3.70 $\times {10}^{-1}$	2.01 $\times {10}^{-1}$	0.74	17.48	0.41	21.34
‘medium’	OGSTV	1.10 $\times {\mathbf{10}}^{-\mathbf{1}}$	1.58 $\times {\mathbf{10}}^{-\mathbf{2}}$	11.30	28.05	0.80	17.11
	TV	1.63 $\times {10}^{-1}$	3.70 $\times {10}^{-1}$	7.85	24.59	0.72	49.06
‘severe’	OGSTV	1.13 $\times {\mathbf{10}}^{-\mathbf{1}}$	4.06 $\times {\mathbf{10}}^{-\mathbf{1}}$	11.05	27.79	0.89	45.83
	TV	1.70 $\times {10}^{-1}$	8.23 $\times {10}^{-1}$	7.48	24.22	0.77	30.19

The use of bold indicates that the numerical results are better.

, a detailed comparison of the data obtained from the two methods under three distinct Blurlevels is presented. The OGSTV model demonstrates consistent superiority over the TV model in both image reconstruction quality and parameter estimation accuracy across all tested conditions. Meanwhile, the OGSTV framework achieves significantly higher values in key quantitative metrics, including SNR, PSNR, and SSIM. Although the TV model exhibits marginally shorter computational times in some blur scenarios, the OGSTV model maintains a better balance between computational efficiency and restoration precision.

4.3. Example 3

• Problem Setting

In this example, we examine the scenario where the parameter p is set to 3. We use the same AO retina image as in Example 1, with a portion of 256 × 256. In Example 3, the first PSF is constructed by $PRblurgauss$ (’mild’). The second PSF is a combination of $PRblurgauss$ (’mild’) and $PRblurdefocus$ (’medium’), and the third PSF is a combination of $PRblurgauss$ (’mild’) and $PRblurdefocus$ (‘mild’, ‘medium’, and ‘severe’) combination. The parameter $\mu$ is chosen as $\mu =6\times {10}^5$, and the parameter $\epsilon$ in the stopping criterion is set to $\epsilon =0.01$. We add Gaussian noise with a noise level of 0.01. Let the real parameter ${\omega }^{*}$ = $[{\omega }_1^{*},{\omega }_2^{*},{\omega }_3^{*}]\in {\mathbb{R}}^3$ be a random quantity with ${\omega }_i^{*}\ge 0,{\omega }_1^{*}+{\omega }_2^{*}+{\omega }_3^{*}=1.$ We select a random $256\times 256$ image to serve as the initial value $x^0$. The initial value of the parameter is choosen as ${\omega }^0=[{\omega }_1^0;{\omega }_2^0;{\omega }_3^0],$ where ${\omega }_1^0,{\omega }_2^0,{\omega }_3^0$ are constants in [0, 1] and satisfy ${\omega }_1^0+{\omega }_2^0+{\omega }_3^0=1.$

• Experimental results

In Figure 5, we show the corresponding resulting image when the BlurLevel of $PRblurdefocus$ is ‘severe’. A larger p means that the AO image denoising is also more complex. In Example 3, the image restoration is significantly more computationally challenging than the previous two examples. By observing the images, unlike Example 1 and 2, we do not necessitate observing enlarged images to evaluate the image restoration. Just by observing Figure 5c–d, it is obvious that the images recovered by the OGSTV model are closer to the real image than those recovered by the TV model. This demonstrates the superior denoising capability of the OGSTV model, particularly when processing images with intricate structures.

In Figure 6, we plot the relative errors against the number of iterations for both models. As in Example 1 and 2, the OGSTV model outperforms the TV model in both the image restoration and parameter estimation tasks. In addition, the TV model terminates the operation at the fifth iteration, while the OGSTV model can perform more iterations and generate smaller relative error values in each iteration.

Table 3. The numerical results for Example 3.

BlurLevel	Model	${\mathbf{RE}}_{\mathbf{x}}$	${\mathbf{RE}}_{\mathbf{w}}$	SNR	PSNR	SSIM	Time (s)
‘mild’	OGSTV	1.30 $\times {\mathbf{10}}^{-\mathbf{1}}$	8.64 $\times {\mathbf{10}}^{-\mathbf{2}}$	10.70	27.09	0.78	111.65
	TV	1.32 $\times {10}^{-1}$	1.81 $\times {10}^{-1}$	10.54	26.93	0.77	93.94
‘medium’	OGSTV	1.38 $\times {\mathbf{10}}^{-\mathbf{1}}$	1.97 $\times {\mathbf{10}}^{-\mathbf{1}}$	10.20	26.59	0.76	548.61
	TV	3.17 $\times {10}^{-1}$	7.04 $\times {10}^{-1}$	2.97	19.35	0.52	381.43
‘severe’	OGSTV	1.51 $\times {\mathbf{10}}^{-\mathbf{1}}$	2.28 $\times {\mathbf{10}}^{-\mathbf{1}}$	9.43	25.81	0.73	378.33
	TV	3.10 $\times {10}^{-1}$	3.28 $\times {10}^{-1}$	3.14	19.52	0.52	225.78

The use of bold indicates that the numerical results are better.

presents the data comparison between the two models at three different Blurlevels. The results indicate a correlation between increased blur and higher computational complexity, as evidenced by the rise in processing time across both models. Although the TV model has some advantages in terms of computation time, its key indexes are significantly degraded under high blur conditions. In contrast, the OGSTV model showed significant advantages, with better performance in relative error, SNR, PSNR, and SSIM, which fully verified the validity and reliability of the proposed OGSTV model in AO retinal image restoration. In conclusion, our proposed OGSTV model outperforms the TV model in AO retinal image restoration in all test cases.

4.4. Performance Comparison

To evaluate the performance differences between the proposed OGSTV model and the TV model, we show the PSNR variations of both models during the iterative process across the above three examples. The corresponding curves of PSNR versus iteration numbers are shown in Figure 7. The results demonstrate that our proposed OGSTV model exhibits significant convergence and achieves higher PSNR levels compared to the TV model.

Moreover, we would like to point out that the OGSTV model requires substantial computational resources for processing high-resolution images. Consequently, in the aforementioned simulations, OGSTV exhibited longer processing times compared to TV in some examples. Future research will prioritize the development of lightweight variants of OGSTV to reduce the computational cost while preserving the reconstruction accuracy.

5. Conclusions

In this paper, we propose a myopic deconvolution model with OGSTV regularization. To efficiently solve the proposed model, we propose an ADMM-MM-LAP method. Specifically, ADMM is applied as an outer-layer optimization method, while the MM method and the LAP method are applied as internal optimization methods, respectively. Theoretically, we give the complexity analysis of the ADMM-MM-LAP method. Numerical experiments demonstrate that the proposed OGSTV model is superior to the existing state-of-the-art TV model. In our future work, we plan to work with more real clinical AO retinal datasets and compare our algorithm with some algorithms developed for high-performance AO systems like the Large Binocular Telescope.