Python Non-Uniform Fast Fourier Transform (PyNUFFT): An Accelerated Non-Cartesian MRI Package on a Heterogeneous Platform (CPU/GPU)

Abstract

A Python non-uniform fast Fourier transform (PyNUFFT) package has been developed to accelerate multidimensional non-Cartesian image reconstruction on heterogeneous platforms. Since scientific computing with Python encompasses a mature and integrated environment, the time efficiency of the NUFFT algorithm has been a major obstacle to real-time non-Cartesian image reconstruction with Python. The current PyNUFFT software enables multi-dimensional NUFFT accelerated on a heterogeneous platform, which yields an efficient solution to many non-Cartesian imaging problems. The PyNUFFT also provides several solvers, including the conjugate gradient method, ℓ1 total variation regularized ordinary least square (L1TV-OLS), and ℓ1 total variation regularized least absolute deviation (L1TV-LAD). Metaprogramming libraries have been employed to accelerate PyNUFFT. The PyNUFFT package has been tested on multi-core central processing units (CPUs) and graphic processing units (GPUs), with acceleration factors of 6.3–9.5× on a 32-thread CPU platform and 5.4–13× on a GPU.

1. Introduction

Fast Fourier transform (FFT) is an exact fast algorithm to compute the discrete Fourier transform (DFT) when data are acquired on an equispaced grid. In certain image processing fields however, the frequency locations are irregularly distributed, which obstructs the use of FFT. The alternative non-uniform fast Fourier transform (NUFFT) algorithm offers fast mapping for computing non-equispaced frequency components.

Python is a fully-fledged and well-supported programming language in data science. The importance of Python language can be seen in the recent surge of interest in machine learning. Developers have increasingly relied on Python to build software, taking advantage of its abundant libraries and active community. Yet the standard Python numerical environments lack a native implementation of NUFFT packages, and the development of an efficient Python NUFFT (PyNUFFT) may fill a gap in the image processing field. However, Python is notorious for its slow execution, which hinders the implementation of an efficient Python NUFFT.

During the past decade, the speed of Python has been greatly improved by numerical libraries with rapid array manipulations and vendor-provided performance libraries. However, parallel computing using a multi-threading model cannot easily be implemented in Python. This problem is mostly due to the global interpreter lock (GIL) of the Python interpreter, which allows only one core to be used at a time, while the multi-threading capabilities of modern symmetric multiprocessing (SMP) processors cannot be exploited.

Recently, general-purpose graphic processing unit (GPGPU) computing has allowed an enormous acceleration by offloading array computations onto graphic processing units, which are equipped with several thousands of parallel processing units. This emerging programming model may enable an efficient Python NUFFT package by circumventing the limitations of the GIL. Two GPGPU architectures are commonly used, i.e. the proprietary Compute Unified Device Architecture (CUDA, NVIDIA, Santa Clara, CA, USA) and Open Computing Language (OpenCL, Khronos Group, Beaverton, OR, USA). These two similar schemes can be ported to each other, or can be dynamically generated from Python codes [1].

The current PyNUFFT package was implemented and has been optimized for heterogeneous systems, including multi-core central processing units (CPUs) and graphic processing units (GPUs). The design of PyNUFFT aims to reduce the runtime while maintaining the readability of the Python program. Python NUFFT contains the following features: (1) algorithms written and tested for heterogeneous platforms (including multi-core CPUs and GPUs); (2) pre-indexing to handle multi-dimensional NUFFT and image gradients; (3) the provision of several nonlinear solvers.

2. Materials and Methods

The PyNUFFT software was developed on a 64-bit Linux system and tested on a Windows system. PyNUFFT has been re-engineered to improve its performance using the PyOpenCL, PyCUDA [1] and Reikna [2] libraries. Figure 1 illustrates the code generation and the memory hierarchy on multi-core CPUs and GPUs. The single precision floating point (FP32) version has been released under dual MIT License and GNU Lesser General Public License v3.0 (LGPL-3.0) [3], which may be used in a variety of projects.

2.1. PyNUFFT: An NUFFT Implementation in Python

The execution of PyNUFFT proceeds through three major stages: (1) scaling; (2) oversampled FFT; and (3) interpolation. The three stages can be formulated as a combination of linear operations:

$$\begin{array}{c}\hfill \mathbf{A}=\mathbf{V}\mathbf{F}\mathbf{S}\end{array}$$

(1)

where $\mathbf{A}$ is the NUFFT, $\mathbf{S}$ is the scaling, $\mathbf{F}$ is the FFT, and $\mathbf{V}$ is the interpolation. A large interpolator size can achive great accuracy for different kernels, at the cost of high memory usage and lower execution speed. To improve the performance, a smaller kernel is preferred. It was previously shown that the min–max interpolator [4] achieves accurate results in a kernel size of 6–7. In a min–max interpolator, the scaling factor and the kernel are designed to minimize the maximum error in k-space.

The design of the three-stage NUFFT algorithm is illustrated in Figure 2A. To save on data transfer times, variables are created on the device global memory.

Currently, multi-coil computations are realized in loop mode or batch mode. The loop mode is robust but the computation times are proportional to the number of parallel coils. In batch mode, the variables of multiple coils are created on the device memory. Thus, the use of batch mode can be restricted by the available memory of the device.

2.1.1. Scaling

Scaling was performed by in-place multiplication (cMultiplyVecInplace). The complex multi-dimensional array was offloaded to the device.

2.1.2. Oversampled FFT

This stage is composed of two steps: (1) zero padding, which copies the small array to a larger array; and (2) FFT. The first step recomputes the array indexes on-the-flight with logical operations. However, GPUs are specialized in floating-point arithmetic and it is noted that matrix reshaping is not efficiently supported on GPUs. Thus, it is better to replace matrix reshaping with other GPU-friendly mechanisms.

Here, a pre-indexing procedure is implemented to avoid matrix reshaping on the flight, and the cSelect subroutine copies array1 to array2 according to the pre-indexes order1 and order2 (See Figure 3). This pre-indexing avoids multidimensional matrix reshaping on the flight, thus greatly simplifying the algorithm for GPU platforms. In addition, pre-indexing can be generalized to multidimensional arrays (with size $N_{in}$, $N_{out}$).

Once the indexes (inlist, outlist) are obtained, the input array can be rapidly copied to a larger array. No matrix reshaping is needed during iterations.

The inverse FFT (IFFT) is also based on the same subroutines of the oversampled FFT, but it reverses the order of computations. Thus, an IFFT is followed by array copying.

2.1.3. Interpolation

While the current PyNUFFT includes the min–max interpolator [4], other kernels can also be used. The scaling factor of the min–max interpolator is designed to minimize the error of off-grid samples [4]. The interpolation kernel is stored as the Compressed Sparse Row matrix (CSR) format for the C-order (row-major) array. Thus, the indexing is accommodated for C-order, and the cSparseMatVec subroutine can quickly compute the interpolation without matrix reshaping. The cSparseMatVec routine is optimized to exploit the data coalescence and parallel threads on the heterogeneous platforms [5], which also adopt the 6-floating-point operations per second (FLOPS) two-sum algorithm [6]. The warp in CUDA (the wavefront in OpenCL) controls the size of the workgroups in the cSparseMatVec kernel. Note that the indexing of the C-ordered array is different from the F-order Fortran array (column-major) implemented in MATLAB.

Gridding is the conjugate transpose of the interpolation, which also uses the same cSparseMatVec subroutine.

2.1.4. Adjoint PyNUFFT

Adjoint NUFFT reverses the order of the forward NUFFT. Each stage is the conjugate transpose (Hermitian transpose) of the forward NUFFT:

$$\begin{array}{c}\hfill {\mathbf{A}}^{\mathbf{H}}={\mathbf{S}}^{\mathbf{H}}{\mathbf{F}}^{\mathbf{H}}{\mathbf{V}}^{\mathbf{H}}\end{array}$$

(2)

which is also illustrated in Figure 4.

2.1.5. Self-Adjoint NUFFT (Toeplitz)

In iterative algorithms, the cost function is used to represent data fidelity:

$$\begin{array}{ccc}\hfill \mathbf{J}\left(\mathbf{x}\right)& =& {∥\mathbf{Ax}-\mathbf{y}∥}_2^2\hfill \end{array}$$

(3)

The minimization of the cost function finds the solution at $J\left(x\right)=0$, which leads to the normal equation composed of interpolation ($\mathbf{A}$) and gridding (${\mathbf{A}}^{\mathbf{H}}$):

$$\begin{array}{c}\hfill \frac{\partial \mathbf{J}\left(\mathbf{x}\right)}{\partial \mathbf{x}}={\mathbf{A}}^{\mathbf{H}}\mathbf{A}\mathbf{x}-{\mathbf{A}}^{\mathbf{H}}\mathbf{y}=0\end{array}$$

(4)

Thus, precomputing the ${\mathbf{A}}^{\mathbf{H}}\mathbf{A}$ can improve the runtime efficiency [7]. See Figure 5 for the software implementation.

2.2. Solver

The NUFFT provides solvers to restore multidimensional images (or one-dimensional signals in the time domain) from the non-equispaced frequency samples. A great number of reconstruction methods exist for non-Cartesian image reconstruction. These methods are usually categorized into three families: (1) density compensation and adjoint NUFFT; (2) least square regression in k-space; and (3) iterative NUFFT.

2.2.1. Density Compensation and Adjoint NUFFT

The sampling density compensation method introduces a tapering function $\mathbf{w}$ (sampling density compensation function), which can be calculated from the following stable iterations [8]:

$$\begin{array}{c}\hfill {\mathbf{w}}_{i+1}={\mathbf{w}}_i\oslash \left({{\mathbf{AA}}^{\mathbf{H}}\mathbf{w}}_i\right)\end{array}$$

(5)

where the element-wise division (⊘) compensates for the over-estimation or under-estimation of the current ${\mathbf{w}}_i$, and ${\mathbf{w}}_{i+1}$. It is noted that the element-wise division tends to make the denominator closer to one. Once $\mathbf{w}$ is prepared, the sampling density compensation method can be calculated by:

$$\begin{array}{c}\hfill \mathbf{x}={\mathbf{A}}^{\mathbf{H}}(\mathbf{y}\odot \mathbf{w})\end{array}$$

(6)

Here, the element-wise multiplication operator ⊙ multiplies the data ($\mathbf{y}$) by the sampling density compensation function ($\mathbf{w}$).

2.2.2. Least Square Regression

Least square regression is a solution to the inverse problem of image reconstruction. Consider the following problem:

$$\begin{array}{c}\hfill \widehat{\mathbf{x}}=\underset{\mathbf{x}}{\mathrm{arg\; min}}{∥\mathbf{y}-\mathbf{A}\mathbf{x}∥}_2^2\end{array}$$

(7)

The solution $\widehat{\mathbf{x}}$ is estimated from the above minimization problem. Due to the enormous memory requirements of the large NUFFT matrix $\mathbf{A}$, iterative algorithms are more frequently used.

• Conjugate gradient method (cg): For Equation (7) there is the two-step solution:

  $$\begin{array}{ccc}\hfill \widehat{\mathbf{k}}& =& \underset{k}{\mathrm{arg\; min}}{∥\mathbf{y}-\mathbf{Vk}∥}_2^2\hfill \end{array}$$

  (8)

  Once $\widehat{\mathbf{k}}$ has been solved, the inverse of $\mathbf{FS}$ can be computed by the inverse FFT and then divided by the scaling factor:

  $$\begin{array}{ccc}\hfill \widehat{\mathbf{x}}& =& {\mathbf{S}}^{-\mathbf{1}}{\mathbf{F}}^{-\mathbf{1}}\widehat{\mathbf{k}}\hfill \end{array}$$

  (9)

  The most expensive step is to find $\widehat{\mathbf{k}}$ in Equation (8). The conjugate gradient method is an iterative solution when the sparse matrix is a symmetric Hermitian matrix:

  $$\begin{array}{ccc}\hfill {\mathbf{V}}^{\mathbf{H}}\mathbf{y}& =& {\mathbf{V}}^{\mathbf{H}}\mathbf{Vk},solve\mathbf{k}\hfill \end{array}$$

  (10)
  Then each iteration generates the residue, which is used to compute the next value. The conjugate gradient method is also provided for heterogeneous systems.
• Other iterative methods for least-squares problems: Scipy [9] provides a variety of least square solvers, including Sparse Equations and Least Squares (lsmr and lsqr), biconjugate gradient method (bicg), biconjugate gradient stabilized method (bicgstab), generalized minimal residual method (gmres), and linear solver restarted generalized minimal residual method (lgmres). These solvers are integrated into the CPU version of PyNUFFT.

2.2.3. Iterative NUFFT Using Variable Splitting

Iterative NUFFT reconstruction solves the inverse problem with various forms of image regularization. Due to the large size of the interpolation and FFT, iterative NUFFT is computationally expensive.

Here, PyNUFFT is also optimized for iterative NUFFT reconstructions in heterogeneous systems.

• Pre-indexing for fast image gradients: Total variation in basic image regularization has been extensively used in image denoising and image reconstruction. The image gradient is computed using the difference between adjacent pixels, which is represented as follows:

  $$\begin{array}{c}\hfill {\nabla }_{i}x=\mathbf{x}(\dots ,a_i+1,\dots )-\mathbf{x}(\dots ,a_i,\dots )\end{array}$$

  (11)

  where $a_i$ is the index of the i-th axis. Computing the image gradient requires image rolling, followed by a subtraction of the original image and the rolled image. However, multidimensional image rolling in heterogeneous systems is expensive and PyNUFFT adopts pre-indexing to save runtime. This pre-indexing procedure generates the indexes for the rolled image, and the indexes are offloaded to heterogeneous platforms before initiating the iterative algorithms. Thus, image rolling is not needed during the iterations. The acceleration of this pre-indexing method is demonstrated in Figure 3, in which pre-indexing makes the image gradient run faster on the CPU and GPU.
• ℓ1 total variation-regularized ordinary least square (L1TV-OLS): The ℓ1 total variation regularized reconstruction includes piece-wise smoothing into the reconstruction model.

  $$\begin{array}{c}\hfill \mathbf{x}=\underset{\mathbf{x}}{\mathbf{arg\; min}}(\mathbf{\mu }{∥\mathbf{y}-\mathbf{Ax}∥}_{\mathbf{2}}^{\mathbf{2}}+\mathit{\lambda }\mathit{TV}\left(\mathbf{x}\right))\end{array}$$

  (12)

  where $\mathit{TV}\left(\mathbf{x}\right)$ is the total variation of the image:

  $$\begin{array}{c}\hfill \mathit{TV}\left(\mathbf{x}\right)=\mathbf{\Sigma }|{\nabla }_{\mathbf{x}}\mathbf{x}|+|{\nabla }_{\mathbf{y}}\mathbf{x}|\end{array}$$

  (13)

  Here, the ${\nabla }_{\mathbf{x}}$ and ${\nabla }_{\mathbf{y}}$ are directional gradient operators applied to the image domain along the $\mathbf{x}$ and $\mathbf{y}$ axes. Equation (12) is solved by the variable-splitting method, which has already been developed [10,11,12].
  The iterations of L1TV-OLS are explicitly shown in Algorithm 1.

  Algorithm 1: The pseudocode for the ℓ1 total variation-regularized ordinary least square (L1TV-OLS) algorithm

• ℓ1 total variation-regularized least absolute deviation (L1TV-LAD): Least absolute deviation (LAD) is a statistical regression model which is robust to non-stationary noise distribution [13]. It is possible to solve the ℓ1 total variation-regularized problem with the LAD cost function [14]:

  $$\begin{array}{c}\hfill \mathbf{x}=\underset{\mathbf{x}}{\mathbf{arg\; min}}(\mathbf{\mu }{∥\mathbf{y}-\mathbf{Ax}∥}_{\mathbf{1}}+\mathit{\lambda }\mathit{TV}\left(\mathbf{x}\right))\end{array}$$

  (14)

  where $\mathit{TV}\left(\mathbf{x}\right)$ is the total variation of the image. Note that LAD is the ℓ1 norm of the data fidelity. The iterations of L1TV-LAD are as follows:
  Note that the shrinkage function (shrink) in Algorithm 2 can be quickly solved on the CPU as well as on heterogeneous systems.

  Algorithm 2: The pseudocode for the ℓ1 total variation-regularized least absolute deviation (L1TV-LAD)

• Multi-coil image reconstruction: In multi-coil regularized image reconstruction, the self-adjoint NUFFT in Equation (4) is extended to multi-coil data:

  $$\begin{array}{ccc}\hfill {\mathbf{A}}_{\mathbf{multi}}^{\mathbf{H}}{\mathbf{A}}_{\mathbf{multi}}\mathbf{x}& =& \sum _{{i}}^{\mathit{Nc}}\mathit{conj}\left({\mathbf{c}}_{\mathit{i}}\right)\odot \left({\mathbf{A}}^{\mathbf{H}}\mathbf{A}({\mathbf{c}}_{\mathit{i}}\odot \mathbf{x})\right)\hfill \end{array}$$

  (15)

  where the coil-sensitivities (${\mathbf{c}}_{\mathit{i}}$) of multiple channels multiply each channel before the NUFFT ($\mathbf{A}$) and after the adjoint NUFFT (${\mathbf{A}}^{\mathbf{H}}$). Sensitivity profiles can be estimated either using the magnitude of smoothed images divided by the root-mean-squared image, or using the dedicated eigenvalue decomposition method [15]. See Figure 6 for a visual example of estimation of coil sensitivity profiles.

2.2.4. Iterative NUFFT Using the Primal-Dual Type Method

In Cartesian magnetic resonance imaging (MRI), the $\mathbf{K}$ matrix of the ℓ2 subproblem can be quickly (and exactly) solved by the diagonal matrix in the Fourier domain, i.e. the ${\mathbf{F}}^{-\mathbf{1}}\mathbf{KF}$ is strictly diagonal. The diagonal matrix is very convenient for compressed sensing MRI on the Cartesian grid [16]. In some non-Cartesian k-spaces, however, the ${\mathbf{F}}^{-\mathbf{1}}\mathbf{KF}$ matrix is not strictly diagonal, which makes variable-splitting methods prone to numerical errors. These errors may accumulate during the image reconstructions, causing instabilities.

Alternatively, the primal-dual hybrid gradient type of algorithm [17,18] exists as a simpler solution to the ℓ1-ℓ2 regularized problems. The alternative primal-dual hybrid gradient algorithms eliminate the ℓ2 subproblem, which conquers one of the major shortcomings of the ℓ1-ℓ2 problems.

• Generalized basis pursuit denoising algorithm (GBPDNA)
  Here, the generalized basis pursuit denoising algorithm (GBPDNA) [19] is implemented, which reads as the following Algorithm 3.
  In Algorithm 3, the $\mathbf{A}$ is the NUFFT in Equation (1). ${\overline{\mathbf{u}}}^k,{\mathbf{v}}^k,{\mathbf{z}}^k,{\mathbf{w}}^k$ are intermediate variables initiated as zeros. $\theta$ is the step size: $0<\theta \le 1$. The notation $\mathbf{R}$ is the generic regularization term, which can be the image itself (as in the fast iterative shrinkage-thresholding algorithm (FISTA) [20]), or linearly transformed images. In the next section, the total variation-like regularization terms are explicitly formulated. ${\tau }_1,{\tau }_2$ satisfy ${\tau }_1{∥\mathbf{A}∥}^2<1$ and ${\tau }_2{∥\mathbf{R}∥}^2<1$.

  Algorithm 3: The pseudocode for the generic generalized basis pursuit denoising algorithm (GBPDNA) algorithm

  $P_{\lambda }$ is the simple convex projection function applying to each component:

  $$P_{\lambda }\left(w_i\right)=\left\{\begin{array}{cc}\frac{w_i}{|w_i|}\lambda ,\hfill & |w_i|>\lambda \hfill \\ w_i,\hfill & |w_i|\le \lambda \hfill \end{array}\right.$$

  (16)
  $Q_{\mathbf{f},ϵ}$ is the constrained function, defined as follows:

  $$Q_{\mathbf{y},ϵ}\left(\mathbf{v}\right)=\left\{\begin{array}{cc}\mathbf{y}+\frac{\mathbf{v}-\mathbf{y}}{\left|\mathbf{v}-\mathbf{y}\right|}ϵ,\hfill & \left|\mathbf{v}-\mathbf{y}\right|>ϵ\hfill \\ \mathbf{v},\hfill & \left|\mathbf{v}-\mathbf{y}\right|\le ϵ\hfill \end{array}\right.$$

  (17)
• Matrix form of total variation-like regularizations
  The piece-wise constant total variation regularization has been used in image denoising and image reconstruction. Total generalized variation [21] introduces the piece-wise smooth model and the additional variables $v_x$ and $v_y$ to mitigate the stair-casing artifacts. Such modification allows total generalized variation to include the divergence and the curl of the image.
  The total variation can be written as a compact block sparse matrix. The compact form of anisotropic total variation is as follows:

  $$\begin{array}{c}\hfill \mathbf{R}=\left[\begin{array}{c}{\mathbf{G}}_{\mathbf{x}}\\ {\mathbf{G}}_{\mathbf{y}}\end{array}\right]\end{array}$$

  (18)

  which yields the explicit anisotropic TV using GBPDNA:

  Algorithm 4: The pseudocode for the GBPDNA algorithm for anisotropic total variation

  Similarly, the explicit isotropic TV using GBPDNA is:

  Algorithm 5: The pseudocode for the GBPDNA algorithm for isotropic total variation

  Similar to total variation, the total generalized variation regularization can be expressed in a matrix form:

  $$\begin{array}{c}\hfill R=\left[\begin{array}{ccc}{\mathbf{G}}_{\mathbf{x}}& -\mathbf{I}& 0\\ {\mathbf{G}}_{\mathbf{y}}& 0& -\mathbf{I}\\ 0& {\mathbf{G}}_{\mathbf{y}}& -{\mathbf{G}}_{\mathbf{x}}\\ 0& {\mathbf{G}}_{\mathbf{x}}& {\mathbf{G}}_{\mathbf{y}}\end{array}\right]\end{array}$$

  (19)

  where the above variables are modified to match the size of the regularization term.

2.3. Applications to Brain MRI

A Periodically Rotated Overlapping ParallEL Lines with Enhanced Reconstruction (PROPELLER) k-space [22] is used in the simulation study. Non-Cartesian data are retrospectively generated from a fully sampled 3T brain MRI template [23]. In conventional methods, data are regridded to a 512 × 512 k-space by linear and cubic spline interpolation methods. The gridded data are processed by IFFT. Three PyNUFFT algorithms were compared with the conventional IFFT-based method. The matrix size = 512 × 512, oversampled ratio = 2, and kernel size = 6. Parameters of L1TV-OLS and L1TV-LAD are: $\mu$ = 1, $\lambda$ = 1, maximum iteration number = 500.

2.4. 3D Computational Phantom

The pre-indexing mechanism of PyNUFFT allows multi-dimensional NUFFT and reconstruction to be carried out. In this simulation study, a three-dimensional (3D) computational phantom [24] (Figure 7A) was used to generate data which are randomly scattered in the 3D k-space (Figure 7B). The image parameters were: matrix size = 64 × 64 × 64, subsampling ratio = 57.7%, oversampled grid = 1×, and kernel size = 1. The parameters of L1TV-OLS and L1TV-LAD were $\mu$ = 1, $\lambda$ = 0.1; the number of outer iterations was 500.

2.5. Benchmark

PyNUFFT was tested on a Linux system. All the computations were completed with a complex single precision floating point (FP32). The configurations of CPU and GPU systems were as follows.

• Multi-core CPU: The CPU instance (m4.16xlarge, Amazon Web Services) was equipped with 64 vCPUs (Intel E5 2686 v4) and 61 GB of memory. The number of vCPUs could be dynamically controlled by the CPU hotplug functionality of the Linux system, and computations were offloaded onto the Intel OpenCL CPU device with 1 to 64 threads. The single thread CPU computations were carried out with Numpy compiled with the FFTW library [25]. PyNUFFT was executed on the multi-core CPU instance with 1–64 threads. The PyNUFFT transforms were offloaded to the OpenCL CPU device and were executed 20 times. The runtimes required for the transforms were compared with the runtimes on the single-thread CPU.
  Iterative reconstructions were also tested on the multi-core CPU. The matrix size was 256 × 256, and kernel size was 6. The execution times of the conjugate gradient method, ℓ1 total variation regularized reconstruction and the ℓ1 total variation-regularized LAD were measured on the multi-core system.
• GPU: The GPU instance (p2.xlarge, Amazon Web Services) was equipped with 4 vCPUs (Intel E5 2686 v4) and one Tesla K80 (NVIDIA, Santa Clara, CA, USA) with two GK210 GPUs. Each GPU was composed of 2496 parallel processing cores and 12 GB of memory. Computations were preprocessed and offloaded onto one GPU by CUDA or OpenCL APIs. Computations were repeated 20 times to measure the average runtimes. The matrix size was 256 × 256, and kernel size was 6.
  Iterative reconstructions were also tested on the K80 GPU, and the execution times of the conjugate gradient method, ℓ1 total variation regularized reconstruction and ℓ1 total variation regularized LAD on the multi-core system were compared with the iterative solvers on the CPU with one thread.
• Comparison between PyNUFFT and Python nfft: It was previously shown that the min–max interpolator yields a better estimation of DFT than the Gaussian kernel does for a kernel size of less than 8 [4]. Thus, a similar testing was carried out and the amplitudes of 1000 randomly scattered non-uniform locations of PyNUFFT and nfft were compared to the amplitudes of the DFT. The input 1D array length was 256, and the kernel size was 2–7.
  The computation times of PyNUFFT and Python nfft were also measured on a single CPU core. This testing used a Linux system equipped with an Intel Core i7-6700HQ running at 2.6–3.1 GHz (Intel, Santa Clara, CA, USA) with a system memory of 16 GB.
• Comparison between PyNUFFT and gpuNUFFT : This testing used a Linux system equipped with an Intel Core i7-6700HQ running at 2.6–3.1 GHz (Intel, Santa Clara, CA, USA), 16 GB of system memory, and an NVIDIA GeForce GTX 965M (945 MHz) (NVIDIA, Santa Clara, CA, USA) with 2 GB of video memory (driver version 387.22), using the CUDA toolkit version 9.0.176. The gpuNUFFT was compiled with FP16. The parameters of the testing were: image size = 256 × 256, oversampling ratio = 2, kernel size = 6. The GPU version of PyNUFFT was executed using the identical parameters to gpuNUFFT. A radial k-space with 64 spokes [26] was used for gpuNUFFT and PyNUFFT.
• Scalability of PyNUFFT: A study of the scalability of PyNUFFT was carried out to compare the runtimes of different matrix sizes and the number of non-uniform locations. The system was equipped with a CPU (Intel Core i7 6700HQ at 3500 MHz, 16 GB system memory) and a GPU (NVIDIA GeForce GTX 965m at 945 MHz, 2 GB device memory).

3. Results

3.1. Applications to Brain MRI

The data fidelity of L1TV-OLS and L1TV-LAD converged between 100 and 500 iterations (Figure 8). Some visual results of MRI reconstructions can be seen in Figure 9. The error of cubic spline was greater than the error (mean-squared error (MSE) = 12.8%) of linear interpolation (MSE = 5.45%). In comparison, NUFFT-based algorithms obtained lower errors than the conventional IFFT and gridding methods. Iterative NUFFT using L1TV-OLS (MSE = 2.40%) and L1TV-LAD (MSE = 2.38%) yielded fewer ripples in the brain structure than in the sampling density compensation method (MSE = 2.87%).

3.2. 3D Computational Phantom

The result of the 3D phantom study can be seen in Figure 7. While conjugate gradient (CG) generated the image volume with artifacts and blurring, ℓ1TV-OLS and ℓ1TV-LAD restored the image details and preserved the edge of the phantom.

3.3. Benchmarks

• Multi-core CPU:

  Table 1. The runtime and acceleration (shown in parentheses) of subroutines and solvers.

  Operations	1 vCPU *	32 vCPUs	PyOpenCL K80	PyCUDA K80
  Scaling(S)	709 µs	557 µs (1.3×)	133 µs (5.3×)	300 µs (2.4×)
  FFT(F)	12.6 ms	2.03 ms (6.2×)	0.78 ms (16.2×)	1.12 ms (11.3×)
  Interpolation(V)	25 ms	2 ms (12×)	4 ms(6×)	4 ms (6×)
  Gridding(VH)	24 ms	2 ms (12×)	5 ms (4.8×)	4.5 ms (5.4×)
  IFFT(FH)	16 ms	3.1 ms (5.16×)	4.0 ms (4×)	3.5 (4.6×)
  Rescaling (SH)	566 µs	595 µs (0.95×)	122 µs (4.64×)	283 µs (2×)
  VHV	4.79 ms	2.62 ms (1.83×)	0.18 ms (26×)	0.15 ms (31×)
  Forward (A)	39 ms	5 ms (7.8×)	6 ms (6.5×)	6 ms (6.5×)
  Adjoint (AH)	38 ms	6 ms (6.3×)	7 ms (5.4×)	6 ms (6.3×)
  Self-adjoint (AHA)	34.7 ms	11 ms (3.15×)	9 ms (3.86×)	8 ms (4.34×)
  Solvers	1 vCPU	32 vCPUs	PyOpenCL K80	PyCUDA K80
  Conjugate gradient	11.8 s	2.97 s (4×)	1.32 s (8.9×)	1.89 s (6.3×)
  L1TV-OLS	14.7 s	3.26 s (4.5×)	1.79 s (8.2×)	1.68 s (8.7×)
  L1TV-LAD	15.1 s	3.62 s (4.2×)	1.93 s (7.8×)	1.78 s (8.5×)

  * vCPU: virtual CPU.

  lists the profile for each stage of forward NUFFT, adjoint NUFFT, and self-adjoint NUFFT (Toeplitz). The overall execution speed of PyNUFFT was faster on the multi-core CPU platform than the single-thread CPU, yet the acceleration factors of each stage varied from 0.95 (no acceleration) to 15.6. Compared with computations on a single thread, 32 threads accelerated interpolation and gridding by a factor of 12, and the FFT and IFFT were accelerated by a factor of 6.2–15.6.
  The benefits of 32 threads are limited for certain computations, including scaling, rescaling, and interpolation gridding (${\mathbf{V}}_{\mathbf{H}}\mathbf{V}$). In these computations, the acceleration factors of 32 threads range from 0.95 to 1.85. This limited performance gain is due to the high efficiency of single-thread CPUs, which leaves limited room for improvement. In particular, the integrated interpolation gridding (${\mathbf{V}}_{\mathbf{H}}\mathbf{V}$) is already 10 times faster than the separate interpolation and regridding sequence. On a single-thread CPU, ${\mathbf{V}}_{\mathbf{H}}\mathbf{V}$ requires only 4.79 ms, whereas the separate interpolation ($\mathbf{V}$) and gridding (${\mathbf{V}}^{\mathbf{H}}$) require 49 ms. In this case, 32 threads only deliver an extra 83% of performance to ${\mathbf{V}}_{\mathbf{H}}\mathbf{V}$.
  Figure 10 illustrates the acceleration on the multi-core CPU against the single thread CPU. The performance of PyNUFFT improved by a factor of 5–10 when the number of threads increased from 1 to 20, and the software achieved peak performance with 30–32 threads (equivalent to 15–16 physical CPU cores). More than 32 threads seem to bring no substantial improvement to the performance.
  Forward NUFFT, adjoint NUFFT and self-adjoint NUFFT (Toeplitz) were accelerated on 32 threads with an acceleration factor of 7.8–9.5. The acceleration factors of iterative solvers (conjugate gradient method, L1TV-OLS and L1TV-LAD) were from 4.2–5.
• GPU: Table 1 shows that GPU delivers a generally faster PyNUFFT transform, with the acceleration factors ranging from 2 to 31.
  Scaling and rescaling have led to a moderate degree of acceleration. The most significant acceleration took place in the interpolation-gridding (${\mathbf{V}}_{\mathbf{H}}\mathbf{V}$) in which GPU was 26–31 times faster than single-thread CPU. This significant acceleration was faster than the acceleration factors for separate interpolation ($\mathbf{V}$, with 6× acceleration) and gridding (${\mathbf{V}}^{\mathbf{H}}$ with 4–4.6× acceleration).
  Forward NUFFT, adjoint NUFFT and self-adjoint NUFFT (Toeplitz) were accelerated on K80 GPU by 5.4–13. Iterative solvers on GPU were 6.3–8.9 faster than single-thread, and about twice as fast as with 32 threads.
• Comparison between PyNUFFT and Python nfft: A comparison between PyNUFFT and nfft (Figure 11) evaluated (1) the accuracy of the min–max interpolator and the Gaussian kernel; and (2) the runtimes of a single-core CPU. The min–max interpolator in PyNUFFT attains a lower error than Gaussian kernel. PyNUFFT also requires less CPU times than nfft, due to the fact that nfft recalculates the interpolation matrix with each nfft or nfft_adjoint call.
• Comparison between PyNUFFT and gpuNUFFT: Figure 12 compares the runtimes of different GPU implementations. In forward NUFFT, the fastest is the PyNUFFT (batch), followed by PyNUFFT (loop) and gpuNUFFT.
  In single-coil adjoint NUFFT, the performance of PyNUFFT (loop), PyNUFFT (batch) and gpuNUFFT is similar. Multi-coil NUFFT increases the runtimes, and the PyNUFFT (loop) is the slowest in the adjoint transform in the case of three coils.
• Scalability of PyNUFFT: Figure 13 evaluates the performance of forward NUFFT and adjoint NUFFT vs. the number of non-uniform locations (M) for different matrix sizes. The condition of M = 300,000 is close to a fully sampled 512 × 512 k-space (with 262,144 samples). The values at M = 0 (y-intercept ) indicate the runtimes for scaling ($\mathbf{S}$) and FFT ($\mathbf{F}$), which change with the matrix size. The slope can be attributed to the runtimes versus M, which is due to interpolation ($\mathbf{V}$) or gridding (${\mathbf{V}}^{\mathbf{H}}$).
  For a large problem size (matrix size = 512 × 512, M = 300,000), GPU PyNUFFT requires less than 10 ms in the forward transform, and less than 15 ms in the adjoint transform. For a small problem size (matrix size = 128 × 128, M = 1000), GPU PyNUFFT requires 830 ns in the forward transform, and 850 ns in the adjoint transform.

4. Discussion

4.1. Related Work

Different kernel functions (interpolators) are available in previous NUFFT implementations, including: (1) the min–max interpoaltor [4]; (2) the fast radial basis functions [27,28]; (3) the least square interpolator [29]; (4) the least mean squared error interpolator [30]; (5) the fast Gaussian summation [31]; (6) the Kaiser–Bessel function [26]; and (7) the linear system transfer function or inverse reconstruction [32,33].

The NUFFTs have been implemented in different programming languages: (1) MATLAB (Mathworks Inc., MA, USA) [4,26,30,34]; (2) C++ [35]; (3) CUDA [26,35]; (4) Fortran [31]; and (5) OpenACC using a PGI compiler (PGI Compilers & Tools, NVIDIA Corporation, Beaverton, OR, USA) [36]. Several Python implementations of NUFFT came to our attention during the preparation of this manuscript. There are one-dimensional non-equispaced fast Fourier transform (nfft package in pure Python) and the multidimensional Python nfft (Python wrapper of the nfft C-library). A python based MRI reconstruction toolbox (mripy) based on nfft was accelerated using the Numba compiler. NUFFT has been accelerated on single and multiple GPUs. Fast iterative NUFFT using the Kaiser–Bessel function was accelerated on a GPU with total variation regularization [35] and total generalized variation regularization [26]. A real-time inverse reconstruction was developed in Sebastinan et al. [37] and Murphy et al. [38], as a pre-gridding procedure saves interpolation and gridding during iterations. The patent of Nadar et al. [39] describes a custom multi-GPU buffer to improve the memory access for image reconstruction with a non-uniform k-space.

In addition to NUFFT, iterative DFT can also be accelerated on GPUs [40,41].

4.2. Discussions of PyNUFFT

The current PyNUFFT has the advantage of high portability between different hardware and software systems. The NUFFT transforms (forward, adjoint, and Toeplitz) have been accelerated on multi-core CPUs and GPUs. In particular, the benefits of fast iterative solvers (including least square and iterative NUFFT) have been shown in the results of benchmarks. The image reconstruction times (with 100 iterations) for one 256 × 256 image are less than 4 s on a 32-thread CPU platform and less than 2 s on a GPU platform.

The current PyNUFFT has been tested with computations using single precision floating numbers (FP32). However, the number of double precision floating point (FP64) units on the GPU is only a fraction of the number of FP32 units, which reduces the performance with FP64 and slows down the performance of PyNUFFT.

In the future, a GPU NUFFT library written in pure C would allow researchers to use the power hardware accelerators in different high-level languages. However, the innate complexity of heterogeneous platforms tends to lower the portability of software, which results in considerable efforts for development and testing. Recently, computer scientists have proposed several emerging GPGPU initiatives to simplify the task, such as the Low Level Virtual Machine (LLVM), OpenACC, and OpenMP 4.0, and these standards are likely to mature in the next few years.

5. Conclusions

An open-source PyNUFFT package was implemented to accelerate the non-Cartesian image reconstruction on multi-core CPU and GPU platforms. The acceleration factors were 6.3–9.5× on a 32-thread CPU platform and 5.4–13× on a Tesla K80 GPU. The iterative solvers with 100 iterations could be completed within 4 s on the 32-thread CPU platform and within 2 s on the GPU.