Fast Fourier-Based Phase Unwrapping on the Graphics Processing Unit in Real-Time Imaging Applications

Abstract

Numerous imaging techniques measure data that are mathematically wrapped to the finite interval [−π, π], corresponding to the principle value domain of the arctangent function. A wide range of reconstruction algorithms has been developed to obtain the true, unwrapped phase by adding an integral multiple of 2π to each point of the wrapped grid. However, the phase unwrapping procedure is hampered by the presence of noise, phase vortices or insufficiently sampled digital data. Unfortunately, reliable phase unwrapping algorithms are generally computationally intensive and their design often requires multiple iterations to reach convergence, leading to high execution times. In this paper, we present a high-speed phase unwrapping algorithm that is robust against noise and phase residues. By executing the parallel implementation of a single-step Fourier-based phase unwrapping algorithm on the graphics processing unit of a standard graphics card, we were able to reduce the total processing time of the phase unwrapping algorithm to < 5 ms when executed on a 640 × 480-pixel input map containing an arbitrarily high density of phase jumps. In addition, we expand upon this technique by inserting the obtained solution as a preconditioner in the conjugate gradient technique. This way, phase maps that contain regions of low-quality or invalid data can be unwrapped iteratively through weighting of local phase quality.

1. Introduction

Phase maps are generated in many applications in medicine, physics and engineering to quantify a wide variety of different physical properties. In optics, profilometric [1,2] and interferometric [3,4,5] techniques deliver phase maps to measure topological or mechanical properties such as shape, strain and local deformation of materials. Also, beyond the field of optics, phase images are generated by many imaging techniques. In magnetic resonance imaging, susceptibility-weighted phase mapping offers a new form of contrast enhancement between structures with different magnetic susceptibilities [6] and blood flow trajectories can be monitored through velocity encoded phase gradient echo imaging [7]. Remote sensing techniques that rely on synthetic aperture radar generate phase information that is used in a host of geodetic applications to monitor geodynamical phenomena such as tectonic deformation [8], volcanic activity [9] and glacial motion [10,11]. Electron holography recovers the electron phase to produce electrostatic and magnetostatic potential distribution maps of the sample [12,13] and optical holography techniques capture the phase of reflected light beams to measure relative movement or deformation [14,15] of the sample.

A common obstacle in the image processing schemes of the above techniques is the fact that the obtained phase map is defined only to within a single 2π-cycle. That is, the value of each measurement point in the phase map is generally wrapped to the region [−π, π], corresponding to the principle value domain of the arctangent function. Therefore, phase unwrapping needs to be carried out before the phase map can be linked to the actual physical property it represents. The ability to correctly extract the true phase map from the principle value of the phase is severely complicated by the presence of noise and phase vortices [16,17,18]. To this end, a lot of effort has been made to increase the robustness of the phase-unwrapping process. When the image is processed sequentially, the phase unwrapping problem is a cumulative process where the error generated by the correction for a false jump or the miss of a genuine one will propagate throughout the rest of the image. In order to avoid or at least minimize false phase jumps, a multitude of phase unwrapping algorithms have been proposed over the years, including region-growing algorithms [19], discontinuity minimization algorithms [20], minimum Lp-norm [21] and least-squares [22] algorithms, quality-guided [23,24] and network flow [25] algorithms, branch-cut [26] algorithms and flood-fill [27] algorithms. The processing times of these algorithms range from seconds to minutes, and in some cases hours to generate a standard 640 × 480-pixelphase map [28,29]. Generally, the quality of the phase unwrapping methods increases with theirexecution time [30]. This is due to their iterative nature and local multi-pixel dependency, which inhibits parallel implementation in regular coordinate space. If the produced phase map contains minimal noise and is sufficiently well sampled between consecutive phase jumps, optimized scan-line algorithms may be employed to boost the speed of quality guided algorithms [31]. However, the above-mentioned imaging techniques cannot always guarantee such conditions and applications that aspire real-time visualization of their results often find the phase unwrapping step to be the bottleneck in their image processing schemes [32].

Two state-of-the-art Fourier-based phase unwrapping techniques were described by Schofield and Zhu [33] and by Volkov and Zhu [34]. The algorithms were developed in the field of electron holography and were designed specifically for superior stability against noise and residues present in the wrapped phase map. At their core, unlike the above-mentioned algorithms, they are both single-iteration methods that employ a combination of Fourier techniques to perform phase unwrapping in reciprocal space. As integration is performed in Fourier space, the algorithms are path-independent and show good noise-resistance in real-space.

In this manuscript, we show that optimized serial implementation of this Fourier-based phase unwrapping technique results in processing times which are limited to several tens of milliseconds, depending on the size of the input phase map. As these processing speeds are insufficiently fast for many real-time imaging applications, we propose a breakthrough in speed by optimizing the phase unwrapping algorithm for execution on parallel hardware. We demonstrate that the total processing time can be reduced by over an order of magnitude when compared to serial code by offloading the algorithm to a standard commodity graphics board. This way, a common bottleneck that is currently present in many real-time imaging applications that require high-speed phase unwrapping can be eliminated. Furthermore, we show that its parallel implementation preserves the algorithm’s superior performance for noisy images or images with high phase gradients. Additionally, a similar approach will be explored to solve the phase unwrapping problem by representing the 2D wrapped phase map as a set of sparse linear equations and by optimizing the minimization procedure of the preconditioned conjugate gradient (PCG) algorithm in Fourier space. The major advantage of this approach is that it allows the user to weight the iterative PCG solver towards areas of high quality phase data and away from areas within the phase map in which noise is expected to corrupt the phase unwrapping procedure. By providing the algorithm with a local quality map that quantifies pixel validity, convergence of the minimization procedure can be accelerated.

2. Method and Implementation

2.1. Schofield, Volkov and Zhu Phase Unwrapping

The general problem of phase unwrapping boils down to the correct extraction of the true phase $\text{φ}\left(\overrightarrow{r}\right)$ from the wrapped phase ${\text{φ}}_W\left(\overrightarrow{r}\right)$ by constructing an integer number field $k\left(\overrightarrow{r}\right)$ such that (1) becomes an identity for all $\overrightarrow{r}\in {\text{Ω}}_N$:

$$\text{φ}\left(\overrightarrow{r}\right)={\text{φ}}_W\left(\overrightarrow{r}\right)+2\text{π}k\left(\overrightarrow{r}\right)$$

(1)

Although both methods developed by Volkov and Zhu and Schofield and Zhu aim to solve (1) using a set of Fourier techniques, their specific implementations differ. Whereas the Volkov algorithm combines the use of several Fourier transform properties with the symmetrization rule [35] to obtain, in one go, the whole field of integer numbers $k\left(\overrightarrow{r}\right)$, the Schofield algorithm formulates (1) in terms of Laplace operators:

$$k\left(\overrightarrow{r}\right)=\frac{1}{2\text{π}}{\nabla }^{-2}\left[{\nabla }^2\text{φ}\left(\overrightarrow{r}\right)-{\nabla }^2{\text{φ}}_W\left(\overrightarrow{r}\right)\right]$$

(2)

where ${\nabla }^2$ and ${\nabla }^{-2}$ represent the forward and inverse 2-dimensional Laplace operators, respectively. Next, Equation (2) can be solved using fast Fourier techniques for the Laplacians [36]:

$${\nabla }^{2}f\left(x,y\right)=-\frac{4{\text{π}}^2}{MN}IDCT\left\{\left(p^2+q^2\right)DCT\left[f(x,y)\right]\right\}$$

(3)

$${\nabla }^{-2}g\left(x,y\right)=-\frac{MN}{4{\text{π}}^2}IDCT\left\{\frac{DCT\left[g(x,y)\right]}{\left(p^2+q^2\right)}\right\}$$

(4)

given an $M\times N$-pixel sized image with $\left(x,y\right)$ its real space and $\left(p,q\right)$ its Fourier space pixel coordinates. By using discrete cosine transforms (DCT) and inverse discrete cosine transforms (IDCT) instead of general fast Fourier transforms (FFT) [37], the Neumann boundary conditions that require the gradient of $k\left(\overrightarrow{r}\right)$ normal to the original image boundary to vanish is met and the additional memory transfers required by the symmetrization rule can be avoided. As it is our goal to achieve fast execution of the algorithm on parallel hardware, it is best to avoid as many additional memory transfers as possible. Therefore, the design principle of the Schofield algorithm was chosen for further parallel implementation. In order to solve (2) without prior knowledge of the true unwrapped phase $\text{φ}\left(\overrightarrow{r}\right),$ the authors define the complex quantity $P\left(r\right)=exp\left[i{\text{φ}}_W\left(\overrightarrow{r}\right)\right]= exp\left[i\text{φ}\left(\overrightarrow{r}\right)-i2\text{π}k\left(\overrightarrow{r}\right)\right]=exp\left[i\text{φ}\left(\overrightarrow{r}\right)\right]$ and recognize that $Im\left(\left(1/P\right){\nabla }^{2}P\right)={\nabla }^2\text{φ}$ [38], where $Im(\dots )$ indicates the imaginary part. The combination of these identities yields an expression for the Laplacian of the unwrapped phase, using only knowledge of the wrapped phase:

$${\nabla }^2\text{φ}=\cos {\text{φ}}_W{\nabla }^2\left(\sin {\text{φ}}_W\right)-\sin {\text{φ}}_W{\nabla }^2\left(\cos {\text{φ}}_W\right)$$

(5)

Combining Equations (3–5) along with enforcing the constraint that $k\left(\overrightarrow{r}\right)$ must be an integer number results in a single-step solution for Equation (2) and ultimately in a fast and deterministic solution for Equation (1).

2.2. Preconditioned Conjugate Gradient Phase Unwrapping

The Schofield, Volkov and Zhu phase unwrapping technique implicitly requires correct phase data information to be present across the entire wrapped phase map. In many practical applications, this is not the case. For such applications, input data weighting based on user-defined quality masks may be desirable. To this end, we propose to employ the approximate solution obtained with the Schofield, Volkov and Zhu technique as a first estimator of the unwrapped phase and iteratively weigh the obtained solution using a preconditioned conjugate gradient algorithm. The PCG method of 2D-weighted least-squares phase unwrapping is presented in detail in numerous papers [39,40,41,42]. For a thorough treatment of conjugate gradient methods, the reader is referred to Kershaw et al. [43]. First, an approximate solution is obtained using Equations (1–5). This solution is then used as a preconditioned input to an iterative solver which converges towards the exact solution in N or less iterations for an $N\times N$ problem. However, in practice, the preconditioning step greatly increases convergence speed so that the problem may be solved sufficiently well in only a few iterations.

In essence, the weighted PCG algorithm requires iterative minimization of:

$${\text{ε}}^2={\displaystyle \sum }_{i=0}^{M-2}{\displaystyle \sum }_{j=0}^{N-2}{\left({\text{φ}}_{i+1,j}-{\text{φ}}_{i,j}-{\Delta }_{i,j}^x\right)}^2+{\displaystyle \sum }_{i=0}^{M-2}{\displaystyle \sum }_{j=0}^{N-2}{\left({\text{φ}}_{i+1,j}-{\text{φ}}_{i,j}-{\Delta }_{i,j}^y\right)}^2$$

(6)

where ${\text{φ}}_{i,j}$ is the discrete wrapped phase value at coordinate point $(i,j)$ and ${\Delta }_{i,j}^x$ and ${\Delta }_{i,j}^y$ represent the phase differences of the unwrapped phase grid in the x- and y-direction, respectively. This minimization problem can be solved iteratively, as shown in the pseudo-code below:

$$\begin{array}{l}\mathit{f}\mathit{o}\mathit{r}k=0\to MaxIterations\\ Applypreconditioningstep \left(Equations(1-5\right))\\ Updatesolutionusingweightedconjugategradientmethod\\ Checkforconvergence\\ \mathit{i}\mathit{f}Convergencethen\\ Exitloop\\ end \mathit{i}\mathit{f}\\ end\mathit{f}\mathit{o}\mathit{r}\end{array}$$

The weighted conjugate gradient method allows the user to influence the relative importance of different phase values in coordinate space by amplifying or reducing their discrete contribution to the total sum of ${\text{ε}}^2$.

2.3. Parallel Implementation of the Discrete Cosine Transform

In order to prepare the algorithm for optimized execution on parallel hardware, we write the discrete cosine transform as a combination of matrix multiplications. As the DCT is a separable linear transformation, the two-dimensional variant is equivalent to a one-dimensional DCT performed along the first dimension, followed by a one-dimensional DCT along the second dimension. Hence, the two-dimensional DCT and corresponding IDCT for an $M\times N$ input image $A$ and output image $B$ can be defined as [44]:

$$\begin{array}{cc}DCT{\left(A\right)}_{p,q}\hfill & =B_{p,q}\hfill \\ \hfill & ={\text{α}}_p{\text{α}}_q{\displaystyle {\displaystyle \sum }_{m=0}^{M-1}{\displaystyle \sum }_{n=0}^{N-1}}{A}_{m,n}\cos \frac{p\text{π}\left(2m+1\right)}{2M}\cos \frac{q\text{π}\left(2n+1\right)}{2N}\hfill \end{array}$$

(7)

$$\begin{array}{cc}IDCT{\left(B\right)}_{m,n}\hfill & =A_{m,n}\hfill \\ \hfill & ={\displaystyle {\displaystyle \sum }_{p=0}^{M-1}{\displaystyle \sum }_{q=0}^{N-1}}{\text{α}}_p{\text{α}}_q{B}_{p,q}\cos \frac{p\text{π}\left(2m+1\right)}{2M}\cos \frac{q\text{π}\left(2n+1\right)}{2N}\hfill \end{array}$$

(8)

with

$$\{\begin{array}{l}0\le p\le M-1\hfill \\ 0\le q\le N-1\hfill \end{array}\text{ and }\{\begin{array}{c}0\le m\le M-1\hfill \\ 0\le n\le N-1\hfill \end{array}$$

where the normalization factors ${\text{α}}_p$ and ${\text{α}}_q$ are defined as:

$${\text{α}}_p=\{\begin{array}{l}\frac{1}{\sqrt{M}}, p=0\hfill \\ \sqrt{\frac{2}{M}},1\le p\le M-1\hfill \end{array}$$

(9)

$${\text{α}}_q=\{\begin{array}{l}\frac{1}{\sqrt{N}}, q=0\hfill \\ \sqrt{\frac{2}{N}},1\le q\le N-1\hfill \end{array}$$

(10)

If $A$ is a square matrix, the 2D-DCT can be computed as:

$$DCT\left(A\right)=T*A*T\prime$$

(11)

where $T$ represents the square matrix containing the two-dimensional forward DCT coefficients that can be obtained through orthogonalization of Equation (7) [45]. The “*” and “$\prime$” symbols denote matrix multiplication and matrix transposition, respectively. Alternatively, when $A$ is a non-square rectangular matrix containing $M$ rows and $N$ columns, a set of square transformation coefficients $F_L$ (size $M\times M$) and $F_R$ (size $N\times N$) can still be constructed from Equation (7) such that:

$$B=DCT\left(A\right)=F_L*A*F_R$$

(12)

for any $M,N\ne 0$. Likewise, the couple of transformation matrices $I_L$ (size $M\times M$) and $I_R$ (size $N\times N$) that perform the 2D inverse discrete cosine transform can be constructed from Equation (8):

$$IDCT\left(B\right)=I_L*B*I_R$$

(13)

For a succession of DCTs on matrices of any given size, the transformation matrices $F_L$, $F_R$, $I_L$ and $I_R$ need to be determined only once. This way, the calculation of the DCT or IDCT transform of matrix $A$ can be reduced to two matrix multiplications (MM).

3. Numeric and Experimental Results

3.1. Schofield, Volkov and Zhu Phase Unwrapping

The vast majority of computational demands required by the presented phase unwrapping technique are represented by a total of six (three forward and three inverse) discrete cosine transforms. Recently, implementation optimization of the DCT on both parallel and sequential hardware has been the object of study [46,47]. Considerable speed-ups have been reported by implementing the DCT algorithm on the GPU [48,49], without loss of accuracy [50]. However, in previous work [51], it was concluded that the central processing unit (CPU) of a standard personal computer benefits most from the symmetrization rule and FFT-based DCT implementation, whereas, in contrast, the parallel architecture of the graphics processing unit (GPU) clearly favors the matrix multiplication-based approach. To this end, we have implemented the entire phase unwrapping algorithm using highly optimized CPU-code on the one hand, and in parallel code on the GPU on the other hand, each with its respective DCT implementation (CPU-FFT and GPU-MM) that results in faster execution speeds. Both CPU-and GPU-based algorithms were benchmarked on the same machine, consisting of an i5-660 processor (4MB cache, 3.33 Ghz), an NVidia Geforce GTX 770 graphics processing unit and 16 GB of RAM memory. In order to provide a fair comparison between CPU- and GPU-based versions of the presented algorithm, both were implemented using C++ code that was specifically optimized for sequential and parallel execution, respectively. The FFTW-package [52] was used to boost the calculation speed of the FFT and the Intel SSE2-instruction set was fully exploited to maximize the floating point processing capabilities of the CPU. The GPU-code was implemented using NVidia’s compute unified device architecture (CUDA) and was customized for maximum performance on all 1536 present CUDA processing cores. Furthermore, CUDA’s basic linear algebra subroutines (CUBLAS) library was used to optimize the matrix multiplications. Source code was generated and managed in Microsoft Visual Studio and its Visual Profiler with NVidia’s Nsight plug-in was used for benchmarking. Executable code can be downloaded freely at the website of the University of Antwerp [53].

The single precision (4 bytes per pixel value) execution times of both CPU-and GPU-based phase unwrapping methods were averaged over 1000 iterations of the algorithm, for a set of varying image pixel sizes. The results are included in Table 1. In addition, processing times of the isolated DCT and IDCTs, including and excluding memory transfers from host to device and back, are included in the last two columns of Table 1.

Table 1. Processing times of central processing unit (CPU)-and graphics processing unit (GPU)-based phase unwrapping algorithms, executed on phase maps of varying pixel sizes. The last two columns include the corresponding processing times of optimized GPU-accelerated discrete cosine transforms (DCTs) and inverse DCTs (IDCTs).

Image Pixel Size	CPU (FFT) (ms)	GPU (MM) incl. mem. trans. (ms)	GPU (MM) excl. mem. trans. (ms)	Acceleration Factor (×)	DCT/IDCT incl. mem trans. (ms)	DCT/IDCT excl. mem trans. (ms)
256 × 256	9.6	1.6	1.2	6.0	0.5	0.1
512 × 512	40.2	4.1	3.2	9.8	1.3	0.3
640 × 480	48.5	4.9	4.0	9.9	1.5	0.5
1024 × 1024	182.9	20.1	16.7	9.1	5.9	2.5
2048 × 2048	735.9	136.4	128.4	5.4	27.9	19.8

Table 1. Processing times of central processing unit (CPU)-and graphics processing unit (GPU)-based phase unwrapping algorithms, executed on phase maps of varying pixel sizes. The last two columns include the corresponding processing times of optimized GPU-accelerated discrete cosine transforms (DCTs) and inverse DCTs (IDCTs).

Image Pixel Size	CPU (FFT) (ms)	GPU (MM) incl. mem. trans. (ms)	GPU (MM) excl. mem. trans. (ms)	Acceleration Factor (×)	DCT/IDCT incl. mem trans. (ms)	DCT/IDCT excl. mem trans. (ms)
256 × 256	9.6	1.6	1.2	6.0	0.5	0.1
512 × 512	40.2	4.1	3.2	9.8	1.3	0.3
640 × 480	48.5	4.9	4.0	9.9	1.5	0.5
1024 × 1024	182.9	20.1	16.7	9.1	5.9	2.5
2048 × 2048	735.9	136.4	128.4	5.4	27.9	19.8

The second column of Table 1 represents the execution times of the sequential CPU-version of the entire phase unwrapping algorithm whereas the third and fourth columns indicate the corresponding processing times of the parallel GPU-based algorithm, including and excluding the memory transfers of the input image from the host to the device and back, respectively. The GPU-version of the algorithm outperforms the single-core CPU-version by a factor of 5–10 (including memory transfers). This acceleration factor increases linearly as a function of image pixel size up to input maps around 1024 × 1024 pixels where the SSE2 instruction set allows the CPU to increasingly benefit from the inherent parallelism of the single instruction, multiple data (SIMD) computations as well. Nevertheless, the GPU-over-CPU performance gains remain significant. It could be noted that further optimization of the CPU-code for execution on a multi-core CPU with an increasing number of processing cores could reduce this performance difference. However, this approach would ultimately converge to the parallel model. Furthermore, by offloading the workload of the phase unwrapping procedure to the GPU, the CPU is exempt from these calculations and can continue to coordinate other processes in the digital processing pipeline. If we benchmark the individual matrix multiplication-based DCT/IDCTs, we report processing bandwidths ranging between 131 and 204 Megapixels/s if we include the memory transfers from host to device and back. These correspond well to corresponding bandwidths of high-performance GPU-based DCTs reported in literature (141–155 Megapixels/s [48]).

In addition, the GPU processing times without memory transfers from host to device and back are included in the fourth column of Table 1 to include adequate benchmarks for phase unwrapping when the wrapped phase data is already present on GPU memory and does not need to be transferred back to host memory before visualization. As previously stated, the coefficients that are used to calculate the DCT through matrix multiplication are not included in the memory transfer time, as they need to be transferred only once, before the first iteration of the algorithm. Overall, it can be concluded that the GPU implementation of the algorithm is fast enough for most real-time applications, whereas the CPU implementation allows real-time phase unwrapping only on images of low pixel sizes and does not leave the operator with much remaining time for any additional processing.

As the success of the phase unwrapping procedure relies only on the quality of the phase map and not on the nature of the imaging technique itself, the presented algorithm can be used in a variety of applications. Figure 1 contains two such examples, one from the field of electron holography (first row), the second from magnetic resonance imaging (MRI-second row). Figure 1a contains the electrostatic and magnetostatic phase information of a Cobalt nanowire grown directly onto the grid through the use of a focused ion beam [54] and Figure 1d represents the horizontal susceptibility-weighted phase map of a mouse brain acquired with gradient echo phase-sensitive MRI [55].

Figure 1. Phase unwrapping procedure for two phase maps obtained with different imaging modalities (first row (a–c): electron holography, second row (d–f): phase-sensitive magnetic resonance imaging (MRI)). The first column (a,d) shows the input wrapped phase map; the second column (b,e) depicts the integer number of 2π that is added to the respective wrapped phase map to end up with the unwrapped phase map, included in the third column (c,f). Scale bars are included in the third column of subfigures only but apply to the other images in their respective row, also.

Figure 1. Phase unwrapping procedure for two phase maps obtained with different imaging modalities (first row (a–c): electron holography, second row (d–f): phase-sensitive magnetic resonance imaging (MRI)). The first column (a,d) shows the input wrapped phase map; the second column (b,e) depicts the integer number of 2π that is added to the respective wrapped phase map to end up with the unwrapped phase map, included in the third column (c,f). Scale bars are included in the third column of subfigures only but apply to the other images in their respective row, also.

The second column (subfigures b and e) of Figure 1 is included to emphasize the essentially deterministic quality of the presented method. It shows the $k\left(\overrightarrow{r}\right)$-map of integer multiples of 2π, corresponding to the solution of (2) that need to be added to the wrapped phase images of the first column pixel-per-pixel to acquire, finally, the unwrapped phase images (subfigures c and f) of the third column of Figure 1. Although the examples contain high phase gradients, noise and phase residues, the presented method succeeds in unwrapping them correctly. It should be noted, however, that since the boundary conditions are merely approximated and a rounding step is introduced in the solution for the $k\left(\overrightarrow{r}\right)$-map, the presented Fourier-based algorithm may not completely unwrap the phase map when correct phase information is not present in the entire image. For this class of images, additional iterations of the algorithm, possibly in combination with interpolation of the phase map, may be needed.

3.2. Preconditioned Conjugate Gradient Phase Unwrapping

As the Fourier-based phase unwrapping technique implicitly requires a continuous phase distribution of the original, unwrapped phase, the algorithm is known to fail on images containing regions of invalid phase information. To demonstrate this effect, a random wrapped phase map was generated containing discrete regions where phase data was deliberately set to constant zero. The failure of the algorithm can be seen in the second column of Figure 2. The integer field $k\left(\overrightarrow{r}\right)$ of multiples of 2π (first row) shows discontinuous jumps (circled in red) that do not correlate with the actual phase jumps, leading to incorrectly unwrapped phase maps (second row). However, by providing the PCG algorithm with an input mask of regions where we know the data is invalid (second row, first column), the approximate solution to the single-step Fourier algorithm (second row, second column) can be used as preconditioner (IT1) or first estimator in an iterative weighted conjugate gradient process. In the presented example, the minimization procedure of Equation (6) produces a correctly unwrapped phase map within two additional iterations after initial preconditioning, by weighting the intermediate solutions to the conjugate gradient method with a simple binary pixel map. The second to fourth columns of Figure 2 indicate that, in the presented example, the phase misses are reduced from seven instances (IT1) to one miss (IT2) to none (IT3).

Figure 2. The preconditioned conjugate gradient algorithm is able to correctly unwrap a randomly generated phase map containing regions where phase data is set to zero in three iterations (IT 1-3) by weighting the intermediate preconditioned conjugate gradient (PCG) solutions according to a user-defined input quality mask. Using a simple binary mask, the phase misses (circled in red) are reduced from seven to none in three iterations.

Figure 2. The preconditioned conjugate gradient algorithm is able to correctly unwrap a randomly generated phase map containing regions where phase data is set to zero in three iterations (IT 1-3) by weighting the intermediate preconditioned conjugate gradient (PCG) solutions according to a user-defined input quality mask. Using a simple binary mask, the phase misses (circled in red) are reduced from seven to none in three iterations.

Construction of the preconditioner required 4.9 ms for a phase map containing 640 × 480 pixels (see Table 1), and required an additional 2.1 ms per iteration. Therefore, calculation of the third generation integer field $k\left(\overrightarrow{r}\right)$ in the presented example required a total processing time of 9.1 ms. This shows that the PCG-minimization procedure can be used effectively in combination with Fourier-based phase unwrapping to process input phase maps with discernible regions of invalid or corrupt data whilst still retaining real-time processing speeds. Depending on the prevalence of these regions, the user can specify a number of additional iterations which are to be calculated, at an additional processing time penalty per iteration.

Again, executable code of the preconditioned conjugate gradient-based phase unwrapping algorithm can be downloaded freely at the authors’ website [53].

3.3. Real-Time Optical Profilometry

In order to evaluate the presented method qualitatively and to demonstrate its real-time processing capabilities, the phase unwrapping algorithm was implemented in a high-speed optical profilometry setup, similar to the one presented by Huang et al. [56]. A set of three phase-shifted line-patterns is projected onto the target at a rate of 60 projections per second. When observed by a camera with adequate frame rate, placed at an angle with the projection direction, 20 phase maps of 640 × 480 data points can be calculated from the recorded images each second. As the resulting phase values are wrapped to the finite interval [−π, π], phase unwrapping needs to take place before scaling transformations can link them to their respective height values. A single-frame excerpt from such a multi-image recording session (Media 1 [53]) is shown in Figure 3. It can be noticed that the simultaneous phase calculation, phase unwrapping procedure and multiple visualization operations can easily be completed in the provided time window of 50 ms, representing the current limit of the employed projector-camera system.

Figure 3. Single-frame excerpt from a multi-image video recording of real-time optical profilometry measurements of moving pieces of fabric (Media 1). (a) One of three phase-shifted input images. (b) Wrapped phase map. (c) Unwrapped phase map. (d) 3D perspective-view of (c).

Figure 3. Single-frame excerpt from a multi-image video recording of real-time optical profilometry measurements of moving pieces of fabric (Media 1). (a) One of three phase-shifted input images. (b) Wrapped phase map. (c) Unwrapped phase map. (d) 3D perspective-view of (c).

To highlight the robustness of the proposed phase unwrapping algorithm, a combination of highly reflective (right side) and poorly reflective (left side) fabric was chosen as measurement object. The low amount of light reflected from the left side of the object results in a lower local contrast of observed projected line patterns, as can be seen in Figure 3a. This produces a much noisier phase distribution with less fluent phase jumps (Figure 3b), with a shadow effect at the border of both pieces of fabric. Nevertheless, the phase unwrapping algorithm is able to cope with these regions of poor phase quality perfectly as the fabric moves and shifts at reasonably high speeds. The corresponding unwrapped phase map (Figure 3c) shows no phase discontinuities throughout the recording session. An additional 3D perspective-view of (c), including vertex and fragment shading, is included in Figure 3d to allow for better depth perception of the object’s height distribution.

4. Conclusions

By combining novel Fourier-based phase unwrapping techniques with a custom parallel implementation of the two-dimensional discrete cosine transform and by optimizing its execution on the graphics processing unit of a standard low-cost graphics card, we have presented a real-time phase unwrapping algorithm that is robust in the presence of noise and phase vortices. Furthermore, its processing time is dependent only on the input pixel size and not on the phase jump density or noise level present in the image. The single-step and deterministic algorithm can unwrap the phase of input images of relatively large pixel sizes at previously unreported speeds, thereby eliminating a bottleneck that is commonly present in many applications in real-time optical profilometry and interferometry, as well as in other image processing applications in which phase map images are generated at high speeds. The parallel implementation of an iterative PCG-solver allows for additional user control over the convergence procedure of the phase unwrapping procedure.