Estimating the Transfer Functions of Optical Imaging Systems from Their Degraded Images by Optimization and Global Search Algorithms

Abstract

Optical imaging is among the safest and most highly impactful biomedical imaging modalities. Aberration in optical imaging systems leads to distorted images. This distortion is almost nonlinear and hence affects the relative size, intensity and appearance of image details. Image aberration has many types, with some or all of them able to be imposed on the image based on the quality of the imaging system and/or surrounding conditions. Many approaches have been introduced to remove or minimize aberration from optical images. If the transfer function of an imaging system and the function of the noise added during the imaging process are known, then an ideal image can be obtained from the image produced by this system. The point spread function (PSF) of an optical imaging system is the image it produces for a point object. PSF is the observable form of the transfer function. The transfer function itself is the exit pupil function or typically the system aberration. The nonlinearity and multiplicity of the aberration imposed on the image, together with the added noise, make it difficult to obtain the transfer function from the degraded images. In this work, optimization and global search techniques are utilized in an iterative image restoration algorithm to estimate the transfer function and restore the image. The proposed technique updates an initially suggested solution of transfer function by optimizing the aberration coefficients. The final solution of the transfer function and hence the PSF is reached when the optimum restored image is obtained. The proposed algorithm is validated by a set of 12 images degraded by different combinations of aberration patterns. Many statistical metrics are used to assess the performance of the proposed algorithm, resulting in a 100% success rate with SSIM equals 1 and a MAE range from 0 to $1\times {10}^{-8}$.

1. Background and Literature Review

In the medical field, optical imaging remains a vital imaging tool for many biological and physiological studies. A main issue in optical imaging is the aberration imposed on the images by the imaging system, which leads to image distortion. The aberration comes from the limitations of the design of optical components, the surrounding conditions and/or the physical characteristics of the imaged objects [1,2]. The lens itself is an imaging system, although sophisticated systems often have many components. Therefore, in optical imaging, the lens is generally used to refer to the imaging system.

A collimated beam has a planer wavefront that keeps its shape when it propagates through a homogeneous and uniform medium. Passing through a non-planar optical element or lens, this wavefront gets distorted by aberration. Aberration is due to the deviation of rays, departing from a single point on the object, from their expected paths. As a result, rays from a single point on the object reach the sensor, or the image plane, at different points with different intensities. And hence, point on the object is not represented by a point on the image but rather by a shape; however, the point itself can still be observed in this shape on the image. If this is applied to all points of the object, intuitively, it leads to the formation of an image distorted by the rays’ deviations [3,4].

The image plane is the plane on which the best image can be formed. The image is fully described by the spatial distribution of amplitude and phase. However, the observable image is the spatial distribution of light intensities simulating the distribution of light intensities coming from the object. Taking into account that light intensity is given by the squared amplitude of the beam, denote the 2D object and the image functions by $f(x,y)$ and $g(x,y)$ respectively. An ideal system produces an ideal image $g\left(x,y\right)=f(x,y)$. As a system, the output of the optical imaging system is given by

$$g\left(x,y\right)=h\left(x,y\right)\ast f\left(x,y\right)+n\left(x,y\right)$$

(1)

where $h\left(x,y\right)$ and $n(x,y)$ are the system transfer function and the noise added to the image by the environment, respectively. In the frequency domain, it is given by

$$G\left(u,v\right)=H\left(u,v\right)·F\left(u,v\right)+N\left(u,v\right)$$

(2)

The formula of the produced image in Equation (1) comes from the fact that each point on the object sends a cone of light covering the full extent of the lens. This means that each point on the lens collects (i.e., adds) rays from all points on the object and sends them to different points on the image plane. A cone of rays emanates from one point on the object, passes the lens and is focused onto the image plane. If the system is ideal, it focuses this cone to a single point on the image plane and hence produces an ideal image. However, nothing created by humans is ideal, and the cone is focused to a shape (e.g., a spot) rather than a point. This happens with cones from each point on the object, which leads to an overall distorted image. The noise is added to the image by the surrounding conditions. This definition means that the image is the convolution of the object function and the transfer function plus the noise, as described by Equation (1).

So, the transfer function $h(x,y)$ is basically the image of a point source if no noise is added. The point source is represented by a delta or an impulse function, and hence the transfer function is also called the impulse response of the system. Therefore, for a point source,

$$\begin{gathered} Pobj=\delta (x,y) \\ PSF=I\left(x,y\right)={\left|h(x,y)\right|}^2 \end{gathered}$$

(3)

where $Pobj$ is a point object given by the delta or impulse function and $PSF$ (Point Spread Function) is its observable image function given by the intensity distribution $I(x,y)$ over the space of the image. The transfer function $h(x,y)$ is the spatial distribution of light amplitude and phase. To clarify, consider a planar uniform wavefront passing through an optical element or system that becomes deformed due to the lens shape into a non-planar wavefront that reaches the sensor with different intensities, i.e., non-uniform. The shape of the lens pushes rays to pass slower or faster than each other with slightly different deviations, leading to an aberrated wavefront. As a result, the aberration pattern is the deviation or spatial error function between the planar wavefront and its deformed wavefront. This wavefront deviation or aberration pattern is denoted by $w(x,y)$ and $w(r,\theta )$ in the Cartesian and polar coordinates, respectively. In the frequency domain, the aberration pattern is described by the phase differences between the shape of the original and the deformed wavefronts, $\theta \left(u,v\right)$. Consequently, it is more easily studied in the frequency domain [5]. The lens aberration is imposed to any imaged object either it is a point or any other shape and is denoted by

$$Abrrn=\theta \left(u,v\right)$$

(4)

where $\theta$ is the phase difference and $\left(u,v\right)$ are the spatial frequencies in the $x$ and $y$ directions, respectively.

To decrease the lens aberration, a stop aperture is placed after the lens, as shown in Figure 1. This allows only paraxial rays to pass to the image plane. Paraxial rays undergo less deviations than peripheral rays. The stop aperture is called the exit pupil. In all, the aperture multiplies the wavefront by a zero-one function outside and inside its circumference, respectively. This is just an apodization function and is given by

$$\left(x,y\right)=\left\{\begin{array}{c}1 0\le r\le 1\\ 0 r>1\end{array}\right. , r=\sqrt{x^2+y^2}$$

(5)

where $r$ is the radius of the pupil. The aberration and, consequently, the transfer function is typically measured after the exit pupil, and that’s why it is called the pupil function. For a circular aperture, the pupil function is given in the polar coordinate by

$$P\left(r,\theta \right)=A\left(r,\theta \right)e^{-j2\pi w\left(r,\theta \right)}=H(r,\theta )$$

where $A(r,\theta )$, $w\left(r,\theta \right)$ and $H(r,\theta )$ are the notations of the apodization function, the aberration pattern and the transfer functions in the polar coordinates, respectively.

From the above analysis and as provided in [6], the transfer function in the frequency domain is given by

$$H\left(u,v\right)=A\left(u,v\right)e^{-j\theta \left(u,v\right)}$$

(6)

Different optical elements impose different aberration patterns. For a single lens, the point object appears as a shape in the image. For a 3D image, this shape may be a sphere, an ellipsoid, etc. For a 2D image, the point object may appear in the image as a spot, a comma, etc., as shown in Figure 1. As an example, if the transfer function is a Gaussian spot, then the narrower the spot, the sharper the PSF and the better the lens. That is why the image of a point object is called the Point-Spread Function (PSF), as it indicates how spread the image of a point is [6].

However, the mathematical foundation of aberration in the spatial domain is done starting from the fact that the aberration is the wavefront deviation. Aberration has different forms or patterns. Practically, all patterns are imposed on the image but with different weights according to quality of the optical elements used. So, the overall aberration is the summation of nonlinear patterns with different weights, and hence it is not easy to fit it in a systematic mathematical function. In 1934, Frits Zernike, the famous Dutch physicist, introduced the sequence of Zernike polynomials. These polynomials are orthogonal over a unit circle. Zernike reported the great similarity between these polynomials and aberration patterns for the circular exit pupil and [7]. Decomposing the aberration to its different patterns, these patterns are also orthogonal over a unit circle. Therefore, the aberration wavefront $w(\rho ,\theta )$ is the summation of Zernike terms with different coefficients representing the weights of different aberration patterns. Details of different Zernike polynomials and different aberration types are provided in [7,8,9]. A lens or an optical imaging system imposes multiple types of aberration on the produced image [10]. The wavefront aberration is given in the Cartesian and polar coordinates by

$$w\left(x,y\right)={\sum }_{k=1}^i{C}_k{Z}_k(x,y)$$

(7)

$$w\left(r,\theta \right)=\sum _{k=1}^i{C}_k{Z}_k\left(r,\theta \right)$$

(8)

where $Z_k$ is the kth Zernike term, $C_k$ is the coefficient of the kth term and $i$ is the number of Zernike terms used to describe the aberration as accurately as possible. The aberration types are usually sorted in triangular outlines called aberration pyramid. On the apex, zeroth order which has only one pattern called piston. The first order has two patterns: tilt-x and tilt-y. The second order has three patterns: astigmatism $\pm 45°$ and defocus. The third order has four patterns: trefoil-x, trefoil-y, coma-x and coma-y, and so on [10].

Theoreticaaly, the number of Zernike terms $i=\infty$, but usually few types or even one type of aberration is dominant for a lens or a system. These dominant types have the greatest coefficients and are enough to be defined for aberration correction [10]. In [10], the authors provide the details of 15 types of aberration representing the first five orders.

Table 1. Some common aberration types.

($\mathit{k}$, Order)	${\mathit{Z}}_{\mathit{k}}\left(\mathit{r},\mathit{\theta }\right)$	Aberration Type	Aberration Pattern
	${\mathit{Z}}_{\mathit{k}}\left(\mathit{x},\mathit{y}\right)$
(0, 0)	1	Piston
	1
(1, 1)	$\rho sin sin \theta$	Tilt-y
	y
(8, 3)	$(3{\rho }^3-2\rho )cos\theta$	Coma-x
	$3x(x^2+y^2)-2x$
(4, 2)	$(2{\rho }^2-1)$	Defocus
	${2(x}^2+y^2)-1$
(12, 4)	$(6{\rho }^4-6{\rho }^2+1)$	Primary Spherical
	$6{\left(x^2+y^2\right)}^2-6\left(x^2+y^2\right)+1$

presents 5 of these 15: piston, tilt or prism along the x-direction, coma in the x-direction, defocus and primary spherical with their formulae in the polar and Cartesian coordinates. Some other patterns of these 15 are used in this work and are presented in Table 2.

From Equations (6) and (7) and over the pupil area,

$$h\left(x,y\right)=a\left(x,y\right)w\left(x,y\right)$$

(9)

$$H\left(u,v\right)=A\left(u,v\right)e^{-j2\pi W\left(u,v\right)}$$

(10)

To improve the optical image, the researchers provided many techniques to remove or minimize the aberrations. A single optical element or system can impose one or more aberration types on the produced image. The imposed aberration types are usually unknown and have different coefficients. This makes it too difficult to remove aberration, but the good news is that one or two types of imposed aberration are dominant according to the values of their coefficients. Trials are usually done to remove such dominant aberration types in a process called aberration correction [3,4,5,6].

From Equation (6), the system transfer function is the exit pupil function with aberration representing the phase of such function. Therefore, knowing the transfer function, the aberration can be removed using the inverse function. However, practical handling is not always that simple because of the nonlinearity of aberration as well as the noise added to images during capture [9].

A lot of research has been conducted to reduce the aberration of optical images, either by digital image processing or using the adaptive optics kits aligned with the optical imaging system. The use of adaptive optics, either regarding the size or alignment, is expensive and adds complexity to the system.

In the field of digital image processing, many researchers have introduced computational methods to estimate the original image from the aberrated image with or without knowing the imposed aberrations. The new approaches use a deconvolution algorithm to deblur the aberrated images [11,12]. If the transfer function of the imaging system is known, the reconstructed image is simply obtained by deconvolution of the aberrated image with the transfer function. The advantage of using post-processing by digital algorithms can be extended to enhance the resolution and contrast of the image and to denoise the image with no cost. Unfortunately, the transfer system is almost unknown. Using blind deconvolution approaches suggested by many studies is not guaranteed to give an accurate image reconstruction [13].

Many modern approaches to computational aberration correction are based on deep learning (DL) methods. For example, Li trained a DL model with a set of optical lenses and their corresponding images [14,15]. Their system takes the aberrant image and the PSF map as inputs and outputs a corrected image. In another example of DL-based aberration correction, Eboli reported a “blind” optical aberration compensation technique in which chromatic aberrations were corrected using a convolutional neural network trained to minimize residuals between the red/green and blue/green planes [16]. More recently, Gong reported a DL-based approach in which the physical properties of optical lenses were incorporated into an aberration correction network [17].

While the most recent examples of aberration correction incorporate a physics-informed approach, these techniques have largely avoided differentiable optical models or the discussion of fundamental aberration theory with Zernike coefficients. Liaudat proposed such a model in which the wavefront itself is considered, not merely the pixels of the image [18]. Another recent example in aberration correction that considers Zernike coefficients is the work of Sauniere, in which the team used a DL approach to extract Zernike coefficients from a low-resolution PSF image [19]. These DL approaches are limited in that they require advanced computing power. There remains a gap in the literature for advanced aberration correction methods that do not rely on DL. Such an analytical system would speed up aberration correction, a proposition that would be useful in many areas where optical imaging is employed. To test the validity of the proposed method, a reference high-quality image was used, a degraded version was obtained by convolving the reference image by a known aberration pattern, obtaining both the restored image and the test pattern. Through visual investigation, the restored method was identical to the reference image, and the induced aberration pattern was identical to the imposed testing pattern.

2. Methods

As the observable image of a point object, the PSF is commonly used to assess the quality of the imaging system because it describes how a point object is represented (or get spread) in the produced image. From Equation (3), the PSF is the square of the modulus of the system’s transfer function. As a result, the transfer function $h(x,y)$—and consequently the PSF—differs from system to system and basically from one optical element to another based on the quality of the materials, fabrication methods and alignment. It is worth mentioning that the Fourier transform of the PSF is the OTF, abbreviated from Optical Transfer Function, while MTF or $H(u,v)$ is the Fourier transform of $h\left(x,y\right)$ [20].

Image improvement is mostly done digitally by applying image processing algorithms to the images. In most cases, image processing is applied while the transfer functions are unknown. Filtering techniques are applied to the images for many reasons, including restoration. Theoretically, image restoration means obtaining the ideal image $f(x,y)$ from a given distorted image $g(x,y)$. Practically, the ideal image cannot be obtained but rather an estimate of it that must be improved as much as possible. Therefore, image restoration leads to aberration correction. Image restoration can be done by digital filtering in the spatial domain to obtain an estimated image $\widehat{f}(x,y)$ or in the frequency domain to estimate $\widehat{F}(u,v)$ and then converting it to the spatial domain. It is usually recommended to apply digital filters in the frequency domain to remove the complexity of convolution in the spatial domain [21].

The Wiener filter is known for its efficiency in image restoration and obtaining the spatial deformation exerted on an image while taking noise into consideration. The Wiener filtering process in the frequency domain is given by

$$\widehat{F}\left(u,v\right)=\left\{\begin{array}{c}{\displaystyle \frac{H^{\ast }(u,v)G\left(u,v\right)}{{\left|H\left(u,v\right)\right|}^2+K(u,v)}} \left|H\left(u,v\right)\right|\ge T\\ 0 \left|H\left(u,v\right)\right|<T\end{array}\right.$$

(11)

where $H^{\ast }(u,v)$ is the complex conjugate of $H(u,v)$, $T$ is a thresholding small value applied on $H\left(u,v\right)$ to avoid the infinity mathematical result, and $K(u,v)$ is the noise-to-signal power ratio calculated from the prior knowledge of both the noise $n\left(x,y\right)$ and the original object $f(x,y)$. $K(u,v)$ can be empirically selected in case of no prior knowledge of the original image or object and noise. It can also be set to zero in case of the absence of noise [22]. In the field of digital image processing, the system transfer function is called the degradation function because an image produced by a system is degraded by the effect of its transfer function.

The phase diversity method, described in [22], is a phase retrieval technique. Phase retrieval means obtaining the aberration pattern that is distorting or degrading the image, allowing the original image to be restored. The phase diversity technique is based on capturing multiple images of the same scene such that one of them is in-focus and the others are degraded with known aberration patterns. A mathematical model for the aberration is initially assumed, and the algorithm runs iteratively to minimize the difference between the model and the real data. This difference represents a merit function. The smaller the merit function indicator, the closer the updated aberration pattern is to the known real one, which occurs while the algorithm is converging. In [23], the authors utilized the phase diversity method to restore the original image as well as the imposed aberration by a modified Wiener filter. This sort of Wiener filter receives two versions of degraded images, $g_1(x,y)$ and $g_2(x,y)$ of the same object. In most cases, the researcher has one degraded image $g(x,y)$ and the aim is to restore its original image. In [23], the authors used the given image $g(x,y)$ as $g_1(x,y)$ and obtained $g_2(x,y)$ by further degrading $g(x,y)$ by a known aberration pattern. This process was done iteratively to get more improved estimates $\widehat{f}(x,y)$. In the frequency domain, this restoration process is given by

$$\widehat{F}\left(u,v\right)=\left\{\begin{array}{c}{\displaystyle \frac{H_1^{\ast }\left(u,v\right)G_1\left(u,v\right)+H_2^{\ast }\left(u,v\right)G_2\left(u,v\right)}{{\left|H_1\left(u,v\right)\right|}^2+{\left|H_2\left(u,v\right)\right|}^2+K(u,v)}} \left|H_1\left(u,v\right)\right|\ge T,\left|H_2\left(u,v\right)\right|\ge T, \\ 0 \left|H_1\left(u,v\right)\right|<T, \left|H_2\left(u,v\right)\right|<T\end{array}\right.$$

(12)

where $G_1\left(u,v\right)$and $G_2\left(u,v\right)$ are the Fourier transform of $g_1(x,y)$ and $g_2(x,y)$ or the two versions of $g(x,y)$; $H_1\left(u,v\right)$ and $H_2\left(u,v\right)$ are the corresponding MTF or degradation functions. For the optical imaging systems, MTF is given by the apodization and the aberration as driven above in Equation (6). The process is done iteratively until reaching the best solution. Initial solutions for $H_1\left(u,v\right)$and $H_2\left(u,v\right)$ are randomly suggested and, consequently, a first solution of $\widehat{F}\left(u,v\right)$ is obtained. While iterating, the transfer functions $H_1\left(u,v\right)$and $H_2\left(u,v\right)$ are updated and consequently $\widehat{F}\left(u,v\right)$. The iterations are executed in a converging fashion, until reaching the optimum solution.

An objective function representing the difference between the estimated and the given image is calculated to check the convergence of the process. According to the value of the objective function, the values of the MTFs are updated and, consequently, so is the estimated image.

In previous works, to optimize the solution, the steepest descent and the conjugate gradient optimizers were used. These optimizers were gradient-based local search methods susceptible to numerous local minima in the objective function [24,25].

In this work, the MTFs are just the Zernike terms of coefficients changing during the iterations. According to the value of the objective function, the coefficients of the Zernike terms composing the transfer functions are updated and, consequently, the values of $H_1\left(u,v\right)$, $H_2\left(u,v\right)$ and $\widehat{F}\left(u,v\right)$. Instead of using the gradient-based local optimization, we used the global Particle Swarm Optimization (PSO) method. PSO is a population-based global metaheuristic optimizer that uses a swarm of particles to simultaneously explore the entire Zernike coefficient space, eliminating the entrapment of local-minima.

Using an array $C$ of values representing the coefficients of the Zernike terms, PSO explores the bounded Zernike coefficient space to minimize the objective function until reaching the optimum coefficient. The optimum Zernike coefficient leads to the optimum solution of $\widehat{F}\left(u,v\right)$, by which the optimum solution of $\widehat{H}\left(u,v\right)$ can be estimated as follows:

$$\begin{gathered} \widehat{F}\left(u,v\right)=G\left(u,v\right)\widehat{H}\left(u,v\right) \\ \therefore \widehat{H}\left(u,v\right)={\displaystyle \frac{\widehat{F}\left(u,v\right)}{G\left(u,v\right)}} \end{gathered}$$

The process ends at this point or after completing the pre-set number of iterations.

The objective function used in this work is the grand sum error representing the difference between the estimated image and the given image. The grand sum error objective function is given by

$$E\left(C\right)=sum\left({E_1\left(C\right)}^2+{E_2\left(C\right)}^2\right)$$

(13)

where $E_1\left(C\right)$ and $E_1\left(C\right)$ are the grand summation of the squared pixel-wise difference between the corresponding estimate and the degraded images. Squaring the difference is done to overcome the sign issue. So, for a Zernike coefficient $C$, the grand sum errors are given by

$$\begin{gathered} E_1\left(C\right)=g_1\left(x,y\right)-F^{-1}\left\{H_1\left(u,v\right)\widehat{F}\left(u,v\right)\right\} \\ {E}_2\left(C\right)=g_2\left(x,y\right)-F^{-1}\left\{H_2\left(u,v\right)\widehat{F}\left(u,v\right)\right\} \end{gathered}$$

According to Equation (13), the optimum solution is obtained at the optimum value $C$ at which Equation (13) gives the minimum value of $E\left(C\right)$. For a limited number of executions, the range of the Zernike coefficients $C$ lies between a lower bound $lb$ and an upper bound $ub$, and a function tolerance, $FUNTOL$, is set to reach the minimum objective error such that

$$min E\left(C\right) , lb\le C\le ub$$

Figure 2 shows a flow chart of the proposed method where the execution stops at the $i^{th}$ iteration if the following condition is satisfied:

$$E\left(C_i\right)-E\left(C_{i+1}\right)<FINTOL OR i=max$$

3. Implementation and Testing

To test the validity of this work, the high-quality microscopic image shown in Figure 3 was used as a reference or an original image $f(x,y)$. Considering a zero-noise function, a degraded or an input image $g(x,y)$ was obtained by convolving the reference image by a known transfer function as follows:

$$g\left(x,y\right)=h\left(x,y\right)\ast f\left(x,y\right)$$

(14)

$$G\left(u,v\right)=H\left(u,v\right)F\left(U,v\right)$$

(15)

Over the area of the pupil where the apodization function equals 1, form Equation (10) was substituted into Equation (15):

$$\begin{gathered} G\left(u,v\right)=AF\left(u,v\right)e^{-j2\pi W\left(u,v\right)} \\ W\left(u,v\right)=F\left\{w\left(x,y\right)\right\} \end{gathered}$$

(16)

According to the phase diversity method [6,23], the aberration function $\theta \left(u,v\right)$ and hence $W(u,v)$ can be chosen as one aberration pattern or a combination of multiple aberration patterns and then be imposed on the reference image.

The same method was followed to obtain two versions $g_1(x,y)$ and $g_2(x,y)$ from the input image $g(x,y)$ by degrading it using two known aberration patterns, $w_1(x,y)$ and $w_2(x,y)$, respectively. The algorithm was then applied to obtain the optimum estimate $\widehat{F}\left(u,v\right)$ and the corresponding aberration pattern $W\left(u,v\right)$.

The generated test images where the reference image was degraded by a spherical aberration pattern to get the input image $g(x,y)$ are shown in Figure 3. For simplicity, the first known aberration pattern used to get $g_1(x,y)$ was the piston type which resulted in the same image. The second aberration pattern used to get $g_2(x,y)$ was a quadratic term particularly a defocus aberration, which is commonly used for this purpose. The quadratic form of the defocus aberration provides diversity over the full spatial frequency range, making it a robust choice for phase retrieval [6,23]. So, from Table 1,

$$\begin{gathered} w\left(x,y\right)=6{\left(x^2+y^2\right)}^2-6\left(x^2+y^2\right)+1 \\ g\left(x,y\right)=f(x,y)\ast \left(6{\left(x^2+y^2\right)}^2-6\left(x^2+y^2\right)+1\right) \\ {w}_1\left(x,y\right)=1 \\ {g}_1\left(x,y\right)=g\left(x,y\right)\ast w_1\left(x,y\right) \\ {g}_1(x,y)=g(x,y) \\ {W}_2\left(x,y\right)={2(x}^2+y^2)-1 \\ {g}_1\left(x,y\right)=g\left(x,y\right)\ast \left({2(x}^2+y^2)-1\right) \end{gathered}$$

The workflow of the proposed algorithm was implemented from the above equations and applied to gray-level images and gray-level aberration patterns. Figure 3 shows a diagram representing the workflow of the proposed method.

The number of Zernike modes $Zn$ used to validate the methods was 37 modes. For the 37 Zernike modes, the PSO initialized a swarm of $S =2Zn= 74$ particles at random positions drawn uniformly from the range $[0,1]$ of $C_0$, where 0 is the lower bound $lb$ or minimum $C_0$ and 1 is the upper bound $ub$ or maximum $C_0$. The upper bound $ub =1$ for the contributing or active Zernike terms, but $ub = 0$ for the zero-contribution Zernike modes. This reduced the effective search dimensionality. The initial guess of the starting position of one particle of the active Zernike modes $x_0$ was tried for three values $x_0=0.0, 0.5, 1.0$—i.e., $x_0=lb, midpoint, ub$—with three independent random seeds to test the robustness of the algorithm. In each of the nine runs, the PSO converged to the exact global optimal solution, confirming the complete robustness to the choice of initial particle position.

All methods were implemented in MATLAB 2025a running on a 64-bit operating system. The hardware environment consisted of an Intel^® Core™ i7-9850H CPU (@ 2.60 GHz) processor with 32.0 GB of RAM.

The particle swarm optimization, PSO, was implemented using MATLAB’s particleswarm function from the Global Optimization Toolbox to search for the optimal Zernike coefficients that minimize the error function E (C). At each iteration, each particle updates its velocity and position guided by its personal best solution and the global best solution found by the entire swarm. The algorithm was configured with MaxIterations = 1000 and FunctionTolerance = 10⁻¹⁰. Parallel evaluation across all 74 particles enabled via MATLAB’s Parallel Computing Toolbox (UseParallel = true). Unlike gradient-based local solvers such as Sequential Quadratic Programming (SQP) or fmincon, PSO requires no gradient information and is inherently resistant to local minima, making it well-suited to the multi-modal E (C) landscape illustrated in Figure 4.

4. Assessment of the Proposed Method

To assess the performance and success rate of the proposed framework, a set of synthetically aberrated images was fed to the framework. The images and the aberration patterns used to degrade them are presented in Table 2. The Structural Similarity Index Measure (SSIM) was used to compare the estimated image to the reference image. The Mean Absolute Error (MAE) was used to compare the obtained set of aberration coefficients to those used for degradation as a reference.

The SSIM measures the quality of the visually perceived image by comparing its contrast, brightness and structural information to those of the reference image. The image statistics are usually used to express its brightness, contrast and conformation [26]. So, the SSIM is given by

$$SSIM(f,\widehat{f})={\displaystyle \frac{\left(2{\mu }_f{\mu }_{\widehat{f}}+b_1\right)\left(2{\sigma }_{f\widehat{f}}+b_2\right)}{\left({\mu }_f^2+{\mu }_{\widehat{f}}^2+b_1\right)\left({\sigma }_f^2+{\sigma }_{\widehat{f}}^2+b_2\right)}}$$

where ${\mu }_f, {\mu }_{\widehat{f}}, { \sigma }_f$and ${\sigma }_{\widehat{f}}$ are the local means and standard deviations of the search windows in the reference and the estimated image; ${\sigma }_{f\widehat{f}},$is the cross covariance of a window in $f$ and its corresponding window in $\widehat{f}$; and $b_1$and $b_2$ are two variables used to stabilize the division with weak denominators. These two variables are commonly given in terms of the dynamic range of the image’s brightness. The SSIM ranges from zero to one, and the higher the SSIM, the greater the similarity between the reference and the estimated image is.

The MAE calculates the average of the absolute differences between a reference and an estimated set of values [27]. Hence, the MAE suits the case of comparing the set of the estimated aberration coefficients and those used to degrade the image. The closer the value of the MAE to zero, the greater the similarity between the two compared sets of coefficients, and hence the more similar the aberration patterns themselves are. The MAE of this case is given by

$$MAE={\displaystyle \frac{1}{K}}\sum _{k=1}^{k=K}\left|C_k-{\widehat{C}}_k\right|$$

(17)

where $C_{k }$and ${\widehat{C}}_k$ are the kth aberration coefficients of the reference and estimated patterns, respectively, and $K$ is the number of coefficients.

5. Results and Discussion

The proposed method for the transfer function estimation utilized a PSO algorithm for optimization and a global search instead of the local search used with the phase diversity method. PSO overcomes the problem of local minima entrapment, which is commonly faced while using the local gradient descent search. During the search algorithm, the local optimizer may give wrong results due to the existence of many local minima. Figure 4 demonstrates real examples of the presence of local minima for the 1D and 2D Zernike coefficient spaces, respectively. The multi-start points approach used by PSO provides a global search because the entire space is searched simultaneously by a swarm of particles, overcoming the local minima entrapment [23,24,25].

Moreover, the PSO-based method is entirely data-driven and doesn’t require any prior knowledge of the signal and noise power spectrum. This increases the robustness of the proposed method [23,24,25].

A third advantage of the use of PSO is that it provides a provable optimum solution of the spectrum function $\widehat{F}(u,v)$ that gives the least-square error at each PSO iteration. This is done through achieving the restoration and identification sub-problems in a unified framework instead of solving them sequentially as in most of the phase diversity variants [23,24,25].

In this work, we used 37 Zernike modes to cover a wide range of possible aberration patterns that may affect the optical images, starting from the lowest or 0th order to the beginning of the 8th order. For this number of Zernike modes, a swarm of 74 particles was used to simultaneously explore the entire Zernike coefficient space.

Generally speaking, PSO has no formal guarantee of convergence in the strict mathematical sense. However, for a compact search space of [0, 1]^nx with a continuous objective function error $E\left(C\right)$, PSO is about 100% probable to converge to a near-global optimum regardless of the initial conditions [26,27]. The convergence of the proposed method is shown on the left side of Figure 5.

As shown in Figure 3, the testing image was obtained by degrading the reference image by the spherical aberration pattern

$$w\left(r,\theta \right)=0.44(6{\rho }^4-6{\rho }^2+1)$$

The coefficient of the optimum estimated aberration pattern was also 0.44, the SSIM equaled 1, and the MAE equaled 0, which means the restored image was identical to the reference one, and the estimated aberration pattern was identical to the reference degradation pattern.

To calculate the success rate of the proposed method, we applied the algorithm to 12 images degraded by different known aberration pattern(s): namely, (a) defocus, (b) 45° primary astigmatism, (c) secondary astigmatism-x, (d) primary coma-y, (e) a combination of secondary coma-x and secondary coma-y, (f) primary spherical, (g) secondary spherical, (h) a combination of defocus and 45° primary astigmatism, (i) a combination of defocus and primary coma-y, (j) a combination of defocus and primary spherical, (k) a combination of defocus, 45°primary astigmatism, primary coma-x and primary coma-y, and (l) a combination of defocus, primary coma-y, primary spherical, and 45°primary astigmatism.

The preset number of iterations was 1000 for all cases, but the proposed method utilizing PSO converged in less than 130 iterations, as shown on the left side of Figure 5. for case L. The results of these 12 cases are presented in Table 2. The results include the reference patterns, their coefficients, the degraded images, the estimated aberration coefficients and the performance assessment metrics. For more confirmation, besides the SSIM and the MAE, we also calculated the peak signal-to-noise ratio (PSNR) of the estimated image, the root mean square error (RMSE) for the estimated image and the aberration coefficients. As shown in Table 2, for a precision of four decimal points, the reference and estimated aberration coefficients were identical. The SSIM for all cases equaled 1, which means the highest degree of similarity between the restored and the reference images. These results were confirmed by the values of the RMSE, which is given by $iRMSE$ in Table 2; the values ranged from 329 $\times {10}^{-8}$ to 0. The MAE equaled $1\times {10}^{-8}$ for combinations of four aberration patterns and equaled 0 for other cases, which reflects the ultimate similarity between the estimated and reference coefficients. These results were also confirmed by the RMSE, which is denoted as $pRMSE$ in Table 2, and the values ranged from 0 to $3\times {10}^{-8}$, confirming the very high similarity. For more confirmation of these promising results, the PSNR of the estimated images were calculated to range from 109.65–243.72 dB, which indicates high quality of the restored images.

For more discussion about the importance of using PSO for optimization, its convergence was investigated through the change of the objective error function $E \left(C\right)$ against the number of iterations, as shown on the left side of Figure 5, for case L with four aberration patterns. The objective error function reached zero in about 130 iterations. The right side of Figure 5 shows the evolution of the estimated Zernike coefficients during the iteration process for case L. The solid curves represent the recovered coefficients, while the dashed horizontal lines indicate the corresponding true values, showing the close convergence of all coefficients to their ground-truth values after about 50 iterations. The rapid convergence caused by the use of PSO led to faster execution times. As presented in the rightmost column of Table 2, the execution time ranged from 35.55 s to 166.70 s according to the number and order of the imposed aberration patterns. This reflects the complexity added when treating images degraded by greater numbers and higher orders of aberration patterns. Moreover, the execution times reported in this work are much shorter than the times needed by classical phase diversity methods with local gradient descent, which may exceed multiple hours.

Analyzing the fault tolerance is an important aspect to reach a solid conclusion [28]. A sensitivity analysis for the function tolerance, FUNTOL, was conducted on case l as one of the complicated cases. Loose tolerances (10⁻², 10⁻³) led to premature convergence ($iRMSE = 0.0004$), while 10⁻⁴ and tighter consistently achieved RMSE = 0. A function tolerance of 10⁻⁴ is recommended, as it achieved the correct solution in 166.7 s versus 190.0 s for 10⁻¹⁰ with a 14% reduction in runtime while keeping its accuracy. Although the maximum number of iterations was preset to 1000, the optimum solutions were reached in less than 130 iterations.

Table 2. The results of algorithm execution at FUNTOL of 10⁻⁴ to the 12 different cases, where N/T are the number and type of aberration patterns used to degrade the reference image and obtain an image $g\left(x,y\right), C$ and $\widehat{C}$ are the reference and estimated aberration coefficient(s), $iRMSE$ and $pRMSE$ are the RMSE of the image and the aberration patterns, respectively, and $Time$ is the execution time in seconds.

N/T	$\mathit{C}$	$\mathbf{P} \hspace{1em}$	$\mathit{g}(\mathit{x},\mathit{y})$	$\widehat{\mathit{C}}$	$\mathit{S}\mathit{S}\mathit{I}\mathit{M}$	$\mathit{M}\mathit{A}\mathit{E}$	$\mathit{P}\mathit{S}\mathit{N}\mathit{R}$	$\mathit{i}\mathit{R}\mathit{M}\mathit{S}\mathit{E}$	$\mathit{p}\mathit{R}\mathit{M}\mathit{S}\mathit{E}$	$\mathit{T}\mathit{i}\mathit{m}\mathit{e}$
(a) 1/Defocus	0.5200			0.5200	1	0	109.6598	$329\times {10}^{-8}$	0	38.75
(b) 1/45° Primary Astigmatism-x	0.4600			0.4600	1	0	173.7725	0	0	39.18
(c) 1/Secondary Astigmatism-y	0.2200			0.2200	1	0	166.9769	0	0	38.77
(d) 1/Primary Coma-y	0.7100			0.7100	1	0	188.4843	0	0	38.20
(e) 1/Secondary Coma-x + Secondary Coma-y	0.8900, 0.8900			0.8900, 0.8900	1	0	176.4338	0	0	43.299
(f) 1/Primary Spherical	0.4400			0.4400	1	0	243.7258	0	0	35.55
(g) 1/Secondary Spherical	0.3800			0.3800	1	0	140.5378	$1\times {10}^{-8}$	0	44.50
(h) 2/Defocus and 45° Primary Astigmatism	0.6700, 0.6900			0.6700, 0.6900	1	0	143.1842	$7\times {10}^{-8}$	0	66.60
(i) 2/Defocus + Primary Coma-y	0.5100, 0.5200			0.5100, 0.5200	1	0	214.322	0	0	61.89
(j) 2/Defocus + Primary Spherical	0.7700, 0.9200			0.7700, 0.9200	1	0	175.7244	0	0	56.52
(k) 4/Defocus + 45°Primary Astigmatism + Primary Coma-x + Primary Coma-y	0.4000, 0.9400, 0.6300, 0.6300			0.4000, 0.9400, 0.6300, 0.6300	1	$1\times {10}^{-8}$	144.1946	$6\times {10}^{-8}$	$3\times {10}^{-8}$	109.97
(L) 4/Defocus + Primary Coma-y + Primary Spherical + Secondary Astigmatism-x	0.1100, 0.5100, 0.8700, 0.3200			0.1100, 0.5100, 0.8700, 0.3200	1	1 $\times {10}^{-8}$	137.6549	$13\times {10}^{-8}$	$3\times {10}^{-8}$	166.70

Table 2. The results of algorithm execution at FUNTOL of 10⁻⁴ to the 12 different cases, where N/T are the number and type of aberration patterns used to degrade the reference image and obtain an image $g\left(x,y\right), C$ and $\widehat{C}$ are the reference and estimated aberration coefficient(s), $iRMSE$ and $pRMSE$ are the RMSE of the image and the aberration patterns, respectively, and $Time$ is the execution time in seconds.

N/T	$\mathit{C}$	$\mathbf{P} \hspace{1em}$	$\mathit{g}(\mathit{x},\mathit{y})$	$\widehat{\mathit{C}}$	$\mathit{S}\mathit{S}\mathit{I}\mathit{M}$	$\mathit{M}\mathit{A}\mathit{E}$	$\mathit{P}\mathit{S}\mathit{N}\mathit{R}$	$\mathit{i}\mathit{R}\mathit{M}\mathit{S}\mathit{E}$	$\mathit{p}\mathit{R}\mathit{M}\mathit{S}\mathit{E}$	$\mathit{T}\mathit{i}\mathit{m}\mathit{e}$
(a) 1/Defocus	0.5200			0.5200	1	0	109.6598	$329\times {10}^{-8}$	0	38.75
(b) 1/45° Primary Astigmatism-x	0.4600			0.4600	1	0	173.7725	0	0	39.18
(c) 1/Secondary Astigmatism-y	0.2200			0.2200	1	0	166.9769	0	0	38.77
(d) 1/Primary Coma-y	0.7100			0.7100	1	0	188.4843	0	0	38.20
(e) 1/Secondary Coma-x + Secondary Coma-y	0.8900, 0.8900			0.8900, 0.8900	1	0	176.4338	0	0	43.299
(f) 1/Primary Spherical	0.4400			0.4400	1	0	243.7258	0	0	35.55
(g) 1/Secondary Spherical	0.3800			0.3800	1	0	140.5378	$1\times {10}^{-8}$	0	44.50
(h) 2/Defocus and 45° Primary Astigmatism	0.6700, 0.6900			0.6700, 0.6900	1	0	143.1842	$7\times {10}^{-8}$	0	66.60
(i) 2/Defocus + Primary Coma-y	0.5100, 0.5200			0.5100, 0.5200	1	0	214.322	0	0	61.89
(j) 2/Defocus + Primary Spherical	0.7700, 0.9200			0.7700, 0.9200	1	0	175.7244	0	0	56.52
(k) 4/Defocus + 45°Primary Astigmatism + Primary Coma-x + Primary Coma-y	0.4000, 0.9400, 0.6300, 0.6300			0.4000, 0.9400, 0.6300, 0.6300	1	$1\times {10}^{-8}$	144.1946	$6\times {10}^{-8}$	$3\times {10}^{-8}$	109.97
(L) 4/Defocus + Primary Coma-y + Primary Spherical + Secondary Astigmatism-x	0.1100, 0.5100, 0.8700, 0.3200			0.1100, 0.5100, 0.8700, 0.3200	1	1 $\times {10}^{-8}$	137.6549	$13\times {10}^{-8}$	$3\times {10}^{-8}$	166.70

6. Conclusions and Future Work

In conclusion, we have proposed a robust method for the estimation of the transfer function of optical imaging systems by obtaining the deformation of wavefront a.k.a. the aberration imposed on the image. The proposed algorithm utilizes particle swarm optimization while restoring the reference image by Wiener filtering. This method showed its superiority with respect to the classic phase diversity methods based on local gradient search, since it converges rapidly and produces accurate results. Since this study is limited by the use of synthetic data, we are planning to build an optical setup to take different images under different conditions with different lens parameters to generalize the concept proved in this work. Different deep learning methods will be investigated and utilized with the experimental data we will collect. Also, the proposed method can be applied to images produced by planar x-ray and other imaging modalities to estimate their transfer functions other than aberration.