Deep Learning for Motion Artifact-Suppressed OCTA Image Generation from Both Repeated and Adjacent OCT Scans

Abstract

Optical coherence tomography angiography (OCTA) is a popular technique for imaging microvascular networks, but OCTA image quality is commonly affected by motion artifacts. Deep learning (DL) has been used to generate OCTA images from structural OCT images, yet limitations persist, such as low label image quality caused by motion and insufficient use of neighborhood information. In this study, an attention-based U-Net incorporating both repeated and adjacent structural OCT images in network input and high-quality label OCTA images in training was proposed to generate high-quality OCTA images with motion artifact suppression. A sliding-window correlation-based adjacent position (SWCB-AP) image fusion method was proposed to generate high-quality OCTA label images with suppressed motion noise. Six different DL schemes with various configurations of network inputs and label images were compared to demonstrate the superiority of the proposed method. Motion artifact severity was evaluated by a motion noise index in B-scan (MNI-B) and in en-face (MNI-C) OCTA images, which were specifically defined in this study for the purpose of evaluating various DL models’ capability in motion noise suppression. Experimental results on a nailfold OCTA image dataset showed that the proposed DL method generated the best results with a peak signal-to-noise ratio (PSNR) of 32.666 ± 7.010 dB, structural similarity (SSIM) of 0.926 ± 0.051, mean absolute error (MAE) of 1.798 ± 1.575, and MNI-B of 0.528 ± 0.124 in B-scan OCTA images and a contrast-to-noise ratio (CNR) of 1.420 ± 0.291 and MNI-C of 0.156 ± 0.057 in en-face OCTA images. Our proposed DL approach generated OCTA images with improved blood flow contrast and reduced motion artifacts, which could be used as a fundamental signal processing module in generating high-quality OCTA images from structural OCT images.

1. Introduction

Optical coherence tomography (OCT) is a non-invasive depth-resolved imaging technique that uses low-coherence light to acquire 2D and 3D images with micrometer-level resolution from optical scattering media, exploring the structural features of biological tissues [1,2]. Optical coherence tomography angiography (OCTA) is a functional imaging technique based on OCT, which can rapidly generate high-resolution microvascular network images of tested tissues [3]. OCT and OCTA have been applied to disease detection and diagnosis in various fields such as ophthalmology [4], dermatology [5], neurology [6], and others [7]. OCTA can be used to perform qualitative and quantitative analysis in clinical applications, yielding critical physical or physiological indicators such as vascular density and vessel diameter, which reflect the functional status of tissues [8,9].

The fundamental imaging principle of OCTA is to detect the contrast of signal changes caused by the movement of particles, primarily red blood cells, which allows for the visualization of the microvascular network in biological tissues [10]. The specific imaging process involves scanning a single slow-axis position multiple times and processing the OCT signals among repeated B-scans to distinguish between signals of static tissues and flowing blood [11,12]. However, OCTA exhibits certain technical drawbacks, such as noise and artifacts, which hinder the extraction of the true vascular network and accurate quantification of vasculature information [13]. High-quality OCTA images are of great significance for the precise assessment of clinical diseases and biomedical research. Improvement in OCTA image quality can be achieved by hardware or software-based approaches. Hardware-based methods normally rely on additional sensors to be installed in imaging systems to detect abnormalities in scanning conditions and guide OCTA rescanning, such as eye-tracking technology. However, this approach can significantly increase hardware costs and system complexity [14], and a lot of researchers resort to software-based approaches, such as increasing the number of repeated B-scans or averaging repeated en-face scans. But these approaches increase acquisition time and data size [15,16]. Considering normal OCTA data acquisition already takes several seconds and a longer time would induce extra motion artifacts, it is not favorable to simply increase the data acquisition time in clinical situations. It deserves more research efforts to develop an optimal signal processing method based on available OCTA data acquired in a normal mode to generate high-quality OCTA images.

In recent years, rapid advancements in deep learning (DL) computer vision techniques, particularly the popular convolutional neural networks (CNNs), have found widespread applications in medical image analysis, including classification, segmentation, denoising, and super-resolution. DL has also gradually found applications in the fields of OCT, resulting in significant breakthroughs in various tasks such as retinopathy classification and retinal layer segmentation [17]. In order to enhance the quality of OCTA images, several studies have also used DL to directly process OCTA images. OCTA image enhancement can be achieved through the supervised learning of training a DL model using low-quality OCTA images as input and corresponding high-quality OCTA images as the target. A high-quality label OCTA image could be obtained using different approaches. In studies [18,19,20], a high-quality label en-face OCTA image was obtained using multiple volume-scan averaging. In a study by Liao et al. [21], a low-quality input en-face OCTA image was obtained from a small number of repeated B-scan OCT images at each slow-axis position, while a high-quality label en-face OCTA image was reconstructed from a large number of repeated B-scan OCT images at each slow-axis position. These methods significantly improve en-face OCTA image quality without the need for repeated volume data acquisitions or a large number of repeated B-scan OCT images at each slow-axis position. DL could also be used to enhance the quality of low-resolution OCTA images obtained from low-density spatial sampling with a similar supervised learning scheme using high-resolution OCTA images as the target. Gao et al. [22] proposed a DL network to reconstruct high-resolution OCTA images from low-density sampled OCTA images of the superficial vascular complex. Subsequently, they designed networks for reconstructing high-resolution OCTA images of the intermediate capillary plexus and deep vascular plexus [23]. Using this type of supervised super-resolution DL learning method, OCTA images were reconstructed with enhanced resolution, improved vascular connectivity, and reduced noise. Kim et al. [24] proposed an integrated DL framework with a two-stage adversarial training scheme to reconstruct both high-density sampled and high-quality OCTA images from their corresponding under-sampled and low-quality counterparts. The abovementioned approaches commonly relied on paired high-quality/resolution and low-quality/resolution images from the same subjects for DL model training, which was not easy to obtain in some practical situations. Zhang et al. [25] introduced a frequency-aware inverse-consistent generative adversarial network (GAN) to enhance OCTA image resolution without relying on paired low-resolution and high-resolution images. By leveraging frequency transformation, their model successfully refined high-frequency information while preserving low-frequency information, resulting in the reconstruction of high-quality OCTA images. Considering the essence of a traditional OCTA flow contrast computation algorithm is to produce OCTA images with blood flow information calculated from repeated-scan structural OCT images, deep learning can therefore be directly used for this purpose, transforming the task of OCTA image generation into an image-to-image translation process. DL has been extensively used in natural image translation tasks. Isola et al. [26] proposed conditional GAN (cGAN) to facilitate image translation. This approach involves utilizing the input image as a condition to generate the corresponding output image. Similar endeavors also become common in the medical image field, which include generating polarization-sensitive OCT images from conventional OCT images [27,28], translating between multimodal magnetic resonance (MR) images [29], and converting MR images into computed tomography (CT) images [30].

Using deep learning techniques to generate OCTA images is a promising approach to improving the quality of OCTA images. Lee et al. [31] first used a symmetric autoencoder network to produce a retinal OCTA image from a single B-scan OCT image. This study demonstrated the feasibility of using DL to generate OCTA images from single-scan structural OCT images, though this method was not designed for the purpose of OCTA image quality enhancement. Using a similar scheme, Zhang et al. [32] further proposed a texture-guided U-Net architecture with a customized loss function based on Euclidean distance and successfully enhanced the quality of the generated OCTA images. Li et al. [33] proposed a GAN-based method combining neighborhood OCT image information for the purpose of OCTA image enhancement. They utilized three OCT images at adjacent positions and their corresponding three-label OCTA images to explore neighborhood information for better OCTA image generation. However, this method only utilized a single OCT image at each position as the input and showed limited capability in extracting capillary details. Liu et al. [34,35] trained DL models with high-quality label OCTA images generated from 48 consecutive B-scans at each slow-axis position and used a small number of repeated OCT images at a single position as the input. Due to a higher signal-to-noise ratio of the target image, the quality of the generated OCTA images was significantly improved. Furthermore, their studies compared the results obtained from two to four repetitions of scans as input, and they found that a larger number of repetitions used as input resulted in higher-quality OCTA images. Jiang et al. [36] used repeated-scan OCT images from adjacent positions as input with a label image acquired at a single position. A pseudo-3D architecture was constructed to further integrate input information and improve the accuracy of OCTA reconstruction. This approach led to an enhancement in the quality of OCTA images compared with the approach using input from a single position. The performance of the OCTA image generation algorithm largely depends on the quality of the label images in the training. Recently, there have also been some studies that addressed the problem of imperfect labels in training. Jiang et al. [37] addressed this issue by using a weakly supervised training strategy. Weak label OCTA images were generated from only two or four repeated OCT images of B-scans at a slow-axis position using the traditional flow contrast computation method. Another two or four repeated OCT images scanned at the same slow-axis position but independent of those used for the weak label OCTA image generation were placed as the input for a DL model, which was trained with the Noise2Noise strategy. This method was found to produce better image quality compared with supervised learning using the weak label OCTA images directly as true labels in training. Li et al. [38] proposed a novel translation framework to generate 3D OCTA volumes from 3D OCT volumes using a 3D DL model. The framework primarily consists of a 3D GAN image translation baseline, featuring a 2D segmentation network referred to as the Vessel Promoted Guidance (VPG) module and a 2D GAN named the Heuristic Contextual Guidance (HCG) module. The VPG played a crucial role in enhancing the quality of blood flow within the generated OCTA images, while the HCG module contributed to improving the overall image translation completeness and enhancing blood flow continuity.

A motion artifact is a common type of artifact observed in OCTA imaging, which may typically appear as significantly increased noise of bright pixels in B-scan OCTA images and white lines in en-face OCTA images [13]. Although quite a number of deep learning approaches have been proposed for the generation of OCTA images from structural OCT images, as described in the abovementioned literature, the problem of motion artifacts seems to not be addressed in these studies, which is the focus of this study. Considering motion noise, we proposed using both repeated and adjacent OCT images as the input and utilizing high-quality label OCTA images fused from adjacent OCTA images for the training of deep learning models. We also proposed two dedicated evaluation metrics, a motion noise index (MNI) in B-scan OCTA images (MNI-B) and an MNI in en-face OCTA images (MNI-C), to assess the effect of motion noise suppression and demonstrate the superior performance of our proposed approach. Details of this study are described as follows.

2. Materials and Methods

Figure 1 illustrates the overall flowchart of the method proposed in this paper, which uses a deep learning model to generate high-quality OCTA blood flow images from structural OCT images. We proposed using repeated and adjacent structural OCT images as the input and explored the advantages of high-quality label OCTA images in training to improve the performance of OCTA image generation, particularly its effect on motion artifact suppression. A comparison of methods with respect to other configurations of network input and label image used for training was conducted using an OCTA dataset obtained from a nailfold OCTA imaging experiment. Various evaluation metrics including those specifically proposed in this study were utilized to demonstrate the superior performance of the proposed method, which is described in detail as follows.

2.1. OCTA Data Preparation

The OCTA data used in this study were acquired by a custom-built linear-k spectral domain OCT (SD-OCT) for imaging the nailfold microvasculature [39]. Figure 2a shows a schematic of the SD-OCT system. The system mainly included a superluminescent diode light source (IPSDS1307C-1311, Inphenix Inc., Livermore, CA, USA) with a central wavelength of 1290 nm and a 3 dB bandwidth of 80 nm (1250–1330 nm), a 50:50 fiber coupler (TW1300R5A2, Thorlabs, Inc., Newton, NJ, USA), a linear-wavenumber spectrometer (PSLKS1300-001-GL, Pharostek, Rochester, MN, USA), and a highspeed line scan camera (GL2048L-10A-ENC-STD-210, Sensors Unlimited Inc., Princeton, NJ, USA) running at an A-line acquisition rate of 76 kHz. The OCTA data acquisition mode repeated four B-scans at each slow-axis position, which could be used to extract the blood flow signal in conventional OCTA imaging. A single OCTA volume dataset covering an approximate area of 1 mm × 1 mm with a scan density of 400 × 400 was acquired from the central region of the right ring finger nailfold area of healthy subjects in approximately ten seconds. Sixteen subjects were recruited, and two volume datasets were collected from each subject with the same operators using the same imaging protocols. One volume dataset was not used because of poor image quality, and finally, thirty-one volume datasets were used in this study. The schematic diagram of the OCTA scanning protocols is shown in Figure 2b.

2.2. OCTA Data Preprocessing

2.2.1. Raw Label OCTA Image Reconstruction

For the linear-k SD-OCT system, a Fourier inverse transform was performed on the spectral data, yielding the structural OCT signal. The blood flow signal was extracted from each A-line in the repeated four B-scans at each position, using an eigendecomposition (ED)-based clutter filtering technique [40]. Final B-scan OCT/OCTA images were obtained as 400 × 400 grayscale images. Two-dimensional en-face images were constructed for each OCTA volume dataset using the maximum intensity projection method, enabling a comprehensive observation of the entire blood flow image. Figure 3 shows some typical nailfold B-scan OCT/OCTA images and a typical en-face OCTA image. OCTA images with motion artifacts were quite commonly seen in the raw label OCTA images, which were shown as ubiquitous white noisy points in B-scan images and white lines in en-face images (Figure 3c,d). The low-quality label OCTA images with obvious motion artifacts were expected to affect the training of OCTA image generation; therefore, the following subsection of the high-quality label OCTA image preparation process was presented.

2.2.2. Label OCTA Image Preparation and Image Fusion Experiment

In order to train a deep learning network that can extract blood flow signals and suppress motion artifacts, it is necessary to prepare high-quality label OCTA images. In this study, these high-quality label OCTA images were prepared by fusing adjacent-position OCTA images. A sliding-window correlation-based adjacent-position (SWCB-AP) image fusion method was proposed for this purpose. For each B-scan position, three consecutive B-scan OCTA images were recruited for the fusion, considering that the motion artifact took place in a single B-scan in most cases. The image fusion process is shown in Figure 4, and the main steps of the fusion process are described as follows.

SWCB-AP was conducted with a sliding window method. The window size and the step size were set as 15 × 15 pixels and 5 pixels, respectively. In each window, a correlation coefficient between each pair of images in the three consecutive images was calculated, denoted as ${\mathrm{C}}_{\mathrm{ij}}$:

$${\mathrm{C}}_{\mathrm{ij} }=\frac{{\sum }_{\mathrm{m}}{\sum }_{\mathrm{n}}\left({\mathrm{A}}_{\mathrm{mn}} - \overline{\mathrm{A}}\right)\left({\mathrm{B}}_{\mathrm{mn}} - \overline{\mathrm{B}}\right)}{\sqrt{\left({\sum }_{\mathrm{m}}{\sum }_{\mathrm{n}}{\left({\mathrm{A}}_{\mathrm{mn}} - \overline{\mathrm{A}}\right)}^2\right)\left({\sum }_{\mathrm{m}}{\sum }_{\mathrm{n}}{\left({\mathrm{B}}_{\mathrm{mn}} - \overline{\mathrm{B}}\right)}^2\right)}}$$

(1)

where ${\mathrm{A}}_{\mathrm{mn}}$ and ${\mathrm{B}}_{\mathrm{mn}}$ are the pixel intensity at the pixel position (m, n) of the two comparison window images, $\overline{\mathrm{A}}$ and $\overline{\mathrm{B}}$ are the mean values of the window images, and $\mathrm{i}, \mathrm{j}=1, 2, 3$ are the index for window images of A (Window i) and B (Window j). A fused window image was then constructed by averaging all window images except the ones with motion artifacts using the following fusion rule:

$$\mathrm{Win}=\left\{\begin{array}{cc}0{.3\mathrm{Win}}_1+0{.4\mathrm{Win}}_2+0{.3\mathrm{Win}}_3,& \mathrm{all} \left|{\mathrm{C}}_{\mathrm{ij}}\right| \ge \mathrm{T}, \mathrm{i}, \mathrm{j}=1,2,3 \hfill \\ 0{.5\mathrm{Win}}_1+0{.5\mathrm{Win}}_2, \hfill & \hfill \left|{\mathrm{C}}_{12}\right| \ge \mathrm{T}, \left|{\mathrm{C}}_{23}\right| < \mathrm{T}, \left|{\mathrm{C}}_{13}\right| < \mathrm{T}\\ 0{.5\mathrm{Win}}_2+0{.5\mathrm{Win}}_3, \hfill & \hfill \left|{\mathrm{C}}_{12}\right| < \mathrm{T}, \left|{\mathrm{C}}_{23}\right| \ge \mathrm{T}, \left|{\mathrm{C}}_{13}\right| < \mathrm{T}\\ 0{.5\mathrm{Win}}_1+0{.5\mathrm{Win}}_3, \hfill & \hfill \left|{\mathrm{C}}_{12}\right| < \mathrm{T}, \left|{\mathrm{C}}_{23}\right| < \mathrm{T}, \left|{\mathrm{C}}_{13}\right| \ge \mathrm{T}\\ 0, \hfill & \mathrm{else} \hfill \end{array}\right.$$

(2)

where $\mathrm{Win}$ represents the fused window image, and ${\mathrm{Win}}_1$, ${\mathrm{Win}}_2$, and ${\mathrm{Win}}_3$ are three windows of the same depth and fast-axis positions in three consecutive images with ${\mathrm{Win}}_2$ as the center window. T represents a threshold value of $\left|{\mathrm{C}}_{\mathrm{ij}}\right|,$ ranging from 0 to 1 for the detection of motion frame. Different values of T were experimented with to perform the image fusion operation, and metrics were utilized to assess the OCTA image fusion results to determine the optimal T used in this study. Finally, all the fused window images were added and averaged in the overlapping area to obtain a final full OCTA image of size 400 × 400.

To verify the effectiveness of this method in improving the quality of OCTA images, fusion experiments were conducted using all 31 OCTA volume datasets. The SWCB-AP image fusion method was applied to the OCTA images obtained with the ED algorithm (SWCB-AP-ED). Raw OCTA images were also obtained using other flow contrast computation methods including speckle variance (SV) [41] and intensity differentiation (ID) [42]. Another two fusion methods were also implemented for comparison with the SWCB-AP-ED method. One was the weighted average fusion of the three types of full-scale OCTA images (SV, ID, and ED) at a single position, which is called WA-SP-SV/ID/ED. The second one was the weighted average fusion of adjacent-position full-scale OCTA images obtained with the ED algorithm (WA-AP-ED). The weight coefficients were set as 0.3, 0.4, and 0.3 in both the weighted average fusion methods. The image quality of the raw and fused OCTA images was qualitatively assessed and quantitatively compared using three evaluation metrics (MNI-B, CNR, MNI-C) as described in the subsequent “Evaluation metrics” Subsection.

2.3. OCTA Image Generation Using Deep Learning

2.3.1. Deep Learning NETWORK Architecture

Attention-based U-Net was utilized in this study as the primary deep learning network framework to perform the OCTA image generation task. It was used to extract and analyze the features of structural OCT images and learn the mapping relationship between the dynamic changes in the tissue structural signal in OCT images and the blood flow signal in OCTA images. U-Net [43] is mainly composed of multi-level feature extraction and concatenation with symmetrical skip connections, enabling accurate localization and context information sensing. The network architecture is mainly composed of blocks of convolutional layers, batch normalization (BN) layers, rectified linear unit (ReLU) layers, skip connection, max pooling layers, upsampling layers, and final 1 × 1 convolutional layer. To further enhance network performance, this paper integrated the Convolutional Block Attention Module (CBAM) [44], a channel-spatial mixed attention module, within the backbone U-Net. Figure 5 illustrates the basic network architecture used in this paper, where the CBAM module is added before the first upsampling layer of the U-Net.

2.3.2. Loss Function

The ${\mathrm{L}}_1$ (MAE) loss function, which calculates the average absolute error between the predicted and the true values, can preserve the edge and detailed information of an image without over-smoothing. The structural similarity (SSIM) [45] based on the human visual perception of image quality can quantify the visual similarity between two images and better preserve the contrast and high-frequency details of an image. An ${\mathrm{L}}_1$-SSIM loss function [46], which combines ${\mathrm{L}}_1$ and SSIM loss functions, was used in this paper as the optimization objective for the OCT to OCTA image translation task. Definitions of these loss functions are given as follows:

$$\mathrm{MAE}=\frac{1}{\mathrm{HW}}{\sum }_{\mathrm{i}=1}^{\mathrm{H}}{\sum }_{\mathrm{j}=1}^{\mathrm{W}}\left|{\mathrm{I}}_{\mathrm{ij} }{- \mathrm{J}}_{\mathrm{ij}}\right|,$$

(3)

$$\mathrm{SSIM}=\frac{{2\mathsf{\mu }}_{\mathrm{I}}{\mathsf{\mu }}_{\mathrm{J}}{+\mathrm{C}}_1}{{\mathsf{\mu }}_{\mathrm{I}}^2{+\mathsf{\mu }}_{\mathrm{J}}^2{+\mathrm{C}}_1}*\frac{{2\mathsf{\sigma }}_{\mathrm{IJ} }{+\mathrm{C}}_2}{{\mathsf{\sigma }}_{\mathrm{I}}^2{+\mathsf{\sigma }}_{\mathrm{J}}^2{+\mathrm{C}}_2},$$

(4)

$$\mathrm{Loss}=\mathsf{\alpha }\left(1 - \mathrm{SSIM}\right)+\left(1 - \mathsf{\alpha }\right)\mathrm{MAE},$$

(5)

where, in Equation (3), $\mathrm{I}$ and $\mathrm{J}$ represent the image generated by the network and the reference image, respectively, $(\mathrm{i}, \mathrm{j})$ represents the pixel position, and H and W represent the height and width of the image. In Equation (4), ${\mathsf{\mu }}_{\mathrm{I}}$ and ${\mathsf{\mu }}_{\mathrm{J}}$ represent the mean pixel intensity of the two images I and J, ${\mathsf{\sigma }}_{\mathrm{I}}$ and ${\mathsf{\sigma }}_{\mathrm{J}}$ represent the standard deviations and ${\mathsf{\sigma }}_{\mathrm{IJ}}$ represents their covariance, and ${\mathrm{C}}_1$ and ${\mathrm{C}}_2$ are constants, with default values of 0.01 and 0.03 used in this study. In Equation (5), $\mathsf{\alpha }$ is a weighting constant empirically set as 0.84, which was adopted from [46] for this study.

2.3.3. Models and Training Schemes

In this study, we proposed using both repeated and adjacent OCT scans in the network input and incorporating enhanced high-quality OCTA label images in the model training for the task of structural OCT image to OCTA image conversion with better suppression of motion artifacts. In order to demonstrate the superior performance of our proposed approach, six different experimental schemes were tested in this study according to the arrangements of the network input (structural OCT images) and the label image (OCTA images) used in training. The network input was divided into two types based on whether a single scan (SS) or repeated scans (RS) were used for the slow-axis position and whether OCT images of a single position (SP) or multiple adjacent positions (MP) were used. The label image was divided into two types based on whether fused (F) or non-fused (NF) OCTA images were used, representing high-quality and normal label images utilized in training, respectively. Based on these definitions, six schemes were tested, which were simply named SS-SP-NF, SS-MP-NF, RS-SP-NF, RS-MP-NF, RS-SP-F, and RS-MP-F, respectively. Specifically, “SS-SP-NF” means the input with single-scan OCT images acquired at a single position and training with non-fused conventional OCTA images, and the other five scheme names can be interpreted similarly. Figure 6 shows a schematic diagram of these training schemes with respect to the network input and the label OCTA image used in training.

2.3.4. Implementation

An approximate six-fold cross-validation strategy was used to test the performances of different OCTA generation schemes using the acquired 31 OCTA volume datasets. In each fold, 5 datasets were used for testing, and the remaining 26 datasets were used for training. Therefore, a total of 30 datasets were used for result evaluation. Data augmentation was performed on the training dataset by horizontally flipping each image, doubling the number of training datasets to 52. Each dataset contained 400 different slow-axis positions, and thus, in total, there were 20,800 pairs of B-scan OCT and OCTA images for a single slow-axis position included in the training dataset.

All experiments were conducted using a computer equipped with 2 Intel E5-2678V3 CPUs, 64 GB DRAM memory, and 4 NVIDIA GTX1080 8 GB GPUs, running an Ubuntu operating system where Python 3.6 and PyTorch 1.10.2 deep learning framework were installed. The number of training epochs was set as 40, and the batch size was set as 32 for all models. The Adam [47] optimizer, a stochastic optimization algorithm based on adaptive gradient estimation, was used to train the model parameters. The initial learning rate was set to 0.0001 for all models, and a cosine annealing learning rate decay strategy [48] was used, which smoothly reduced the learning rate during the training and avoided premature convergence or local optima. The testing datasets were also used as the validation datasets to prevent overfitting in DL model training, and all the models were saved at the point of a validation loss minimum for generating OCTA images in the testing datasets. Due to limitations in memory and computing resources, each input B-scan image was resized to 256 × 256 in training and then rescaled to its original size of 400 × 400 in testing using a bilinear interpolation method. The resizing process may affect image generation performance a little. But this process was performed for all comparison schemes and, therefore, fairness could be maintained for comparisons among different DL schemes with the same resizing operation.

2.4. Evaluation Metrics

To quantitatively evaluate the quality of the generated B-scan OCTA images, this study used three reference-based evaluation metrics: the peak signal-to-noise ratio (PSNR), the structural similarity index (SSIM), and mean absolute error (MAE). These metrics were calculated with reference to a label image Y and a predicted image Z. PSNR is defined as:

$${\mathrm{PSNR}=10\log }_{10}\left(\frac{{\left[\mathrm{Max}\left(\mathrm{Y}\right)\right]}^2}{\mathrm{MSE}\left(\mathrm{Y},\mathrm{Z}\right)}\right),$$

(6)

where $\mathrm{Max}\left(\mathrm{Y}\right)$ represents the maximum intensity of the label image Y and $\mathrm{MSE}\left(\mathrm{Y}, \mathrm{Z}\right)$ is the mean squared error between Y and Z, which is defined as follows:

$$\mathrm{MSE}\left(\mathrm{Y},\mathrm{Z}\right)=\frac{1}{\mathrm{HW}}{\sum }_{\mathrm{i}=1}^{\mathrm{H}}{\sum }_{\mathrm{j}=1}^{\mathrm{W}}{\left({\mathrm{Y}}_{\mathrm{ij}}{ - \mathrm{Z}}_{\mathrm{ij}}\right)}^2,$$

(7)

where $(\mathrm{i}, \mathrm{j})$ indicates the pixel position index and (H, W) are the height and width of the image. The definitions of MAE and SSIM have already been given in Equations (3) and (4), respectively.

We also proposed a new no-reference metric, called the motion noise index (MNI) for B-scan (MNI-B), to measure the level of flow intensity compared to the background motion noise across the fast-axis direction in a B-scan OCTA image. MNI-B is calculated as follows: First, the gray level of each column of the B-scan image is averaged to obtain a one-dimensional intensity vector across the fast-axis direction. Subsequently, this intensity vector is divided into multiple consecutive segments based on its peaks and valleys, with each segment containing only one peak and two valleys. Finally, the MNI of each segment is calculated and averaged to obtain an overall mean MNI-B for each B-scan OCTA image, which is defined as follows:

$$\mathrm{MNI}-\mathrm{B}=\frac{1}{\mathrm{N}}{\sum }_{\mathrm{n}=0}^{\mathrm{N}-1}\frac{{\mathrm{v}}_{\mathrm{n}}{+\mathrm{v}}_{\mathrm{n}+1}}{{2\mathrm{p}}_{\mathrm{n}+1}},$$

(8)

where $\mathrm{N}$ is the total number of valley–peak–valley segments, $\mathrm{n}$ is a segment number index, ${\mathrm{p}}_{\mathrm{n}+1}$ represents the segment peak intensity, and ${(\mathrm{v}}_{\mathrm{n}}{, \mathrm{v}}_{\mathrm{n}+1})$ represent the left and right valley intensity of the segment $\mathrm{n}+1$. A schematic diagram of this calculation process is shown in Figure 7. MNI-B lies in the range of 0~1, with a higher value indicating a high motion noise level. Typical OCTA images with low and high levels of motion noise, which can be reflected by small and high values of MNI-B, are shown in Figure 8.

To evaluate the quality of the generated en-face OCTA images, two no-reference metrics were used: one was the contrast-to-noise ratio (CNR) and the other was the motion noise index in the en-face mode (MNI-C), which was proposed in this study, similar to MNI-B. Among them, CNR is a metric that measures the contrast between the vasculature and non-vasculature regions in the image, where a higher CNR value indicates better image quality. CNR is calculated as follows:

$$\mathrm{CNR}=\frac{{\mathsf{\mu }}_{\mathrm{s}} - {\mathsf{\mu }}_{\mathrm{b}}}{\sqrt{{\mathsf{\sigma }}_{\mathrm{s}}^{2 }+{\mathsf{\sigma }}_{\mathrm{b}}^2}},$$

(9)

where ${\mathsf{\mu }}_{\mathrm{s}}$ and ${\mathsf{\mu }}_{\mathrm{b}}$ denote the mean intensity of the vasculature and non-vasculature regions, and ${\mathsf{\sigma }}^2$ represents the variance in intensity within each region. The two regions were differentiated by a localized Bernsen thresholding-based segmentation algorithm with manual adjustment [49] for the vasculature, where the background is defined as the region excluding the vasculature part in an OCTA image.

MNI-C was the second metric proposed in this study that could quantify the severity of motion artifacts, most commonly seen as white lines in an en-face OCTA image. A lower MNI-C indicates better suppression of the motion artifact-related white lines. To calculate MNI-C, a 1D intensity signal $\mathrm{I}\left(\mathrm{n}\right)$ (n is the row index) was firstly calculated by averaging the pixel intensity along the horizontal direction in an en-face image. Then, a 1D gradient signal ${\mathrm{G}}_{\mathrm{I}}\left(\mathrm{n}\right)$was calculated by a difference operation of the 1D intensity vector $\mathrm{I}\left(\mathrm{n}\right)$. MNI-C is calculated as:

$$\mathrm{MNI}-\mathrm{C}=\frac{\max \left(\left|{\mathrm{G}}_{\mathrm{I}}\right|\right)}{{\mathsf{\mu }}_{\mathrm{s}}},$$

(10)

where ${\mathsf{\mu }}_{\mathrm{s}}$ denotes the mean value of the vasculature region and $\max \left(\left|{\mathrm{G}}_{\mathrm{I}}\right|\right)$ is calculated using the mean of the top 20 peak values of the 1D intensity gradient signal ${\mathrm{G}}_{\mathrm{I}}\left(\mathrm{n}\right)$. Figure 9 shows a typical en-face OCTA image with significant motion-related white lines and a typical en-face OCTA image where motion artifacts have been suppressed. These two images are calculated to have high and low values of MNI-C, respectively, indicating the capability of MNI-C in reflecting the severity of motion artifacts in en-face OCTA images.

3. Results

3.1. Results for Label OCTA Image Fusion

Figure 10 demonstrates the performance of the image fusion results for the CNR, MNI-B, and MNI-C of different T values ranging from 0.05 to 0.6 with a step of 0.05. Varying thresholds yield different effects on the fused images, and based on the representative CNR curve, a threshold value of 0.35 for T was finally selected for subsequent use.

The SWCB-AP algorithm could effectively suppress motion noise in the OCTA images. Figure 11 shows the typical results for the SWCB-AP processing of OCTA images with different levels of motion noise, both in the B-scan and en-face modes. It was observed that some raw OCTA images had severe motion artifacts that reduced the blood flow contrast in the B-scan mode (Figure 11b) and caused the appearance of many white lines in the en-face mode (Figure 11d). By using the SWCB-AP algorithm, the problem of motion artifact was alleviated, and vasculature contrast was greatly improved, as seen in the B-scan mode (Figure 11g). Motion white lines were also greatly suppressed in the en-face mode (Figure 11i).

Quantitative results for the comparison of different raw OCTA processing methods and different image fusion algorithms are shown in

Table 1. Quantitative image quality evaluation results for the OCTA images obtained by different processing methods in the B-scan and en-face modes.

Processing Method	B-Scan OCTA Images	En-Face OCTA Images
	MNI-B ↓	CNR ↑	MNI-C ↓
ID	0.558 ± 0.189	1.052 ± 0.285	0.405 ± 0.065
SV	0.590 ± 0.191	0.830 ± 0.259	0.496 ± 0.078
ED	0.503 ± 0.185	1.697 ± 0.311	0.356 ± 0.155
WA-SP-SV/ID/ED	0.601 ± 0.168	1.425 ± 0.242	0.341 ± 0.063
WA-AP-ED	0.579 ± 0.155	1.985 ± 0.352	0.166 ± 0.079
SWCB-AP-ED	0.484 ± 0.168	2.334 ± 0.371	0.150 ± 0.068

Values are given as mean ± standard deviation. Bold represents the best result for each column. ↓ means lower is better, and ↑ means higher is better for each column’s metric.

. Typical OCTA images in the B-scan and en-face modes obtained by various raw OCTA methods (ID, SV, and ED) and various fusion methods (WA-SP-SV/ID/ED, WA-AP-ED, and SWCB-AP-ED) are also presented in Figure 12. For the 31 testing volume datasets, SWCB-AP-ED achieved an MNI-B of 0.484 ± 0.168 in the B-scan mode, which was the lowest among all three image fusion methods. At the position affected by motion, the three raw OCTA calculation methods displayed obvious motion noise with low vasculature contrast (Figure 12a–c). A simple fusion of these three raw OCTA images at a single scan position could not achieve the effect of motion artifact suppression (Figure 12d). The method of weighted average using multiple adjacent-position OCTA images (WA-AP-ED) could suppress the motion noise and improve the blood flow contrast (Figure 12e). However, its effect was still inferior to the SWCB-AP-ED fusion method proposed in this study. The evaluation of CNR and MNI-C in the en-face images showed that ED achieved the best results compared with the other two raw OCTA flow contrast computation methods (ID and SV), which was the reason why ED was used as the basic OCTA calculation method for image fusion in this study. Based on ED, the SWCB-AP-ED method also achieved the best results for CNR (2.334 ± 0.371) and MNI-C (0.150 ± 0.068) in the en-face mode among all three image fusion methods.

3.2. Results for Deep Learning OCTA Image Generation

The quantitative results for the DL-generated B-scan OCTA images and the corresponding volume-projected en-face OCTA images in the test datasets using various DL schemes are presented in

Table 2. Quantitative image quality evaluation results for OCTA image generation from structural OCT images using different DL schemes.

DL Schemes	B-Scan OCTA Images				En-Face OCTA Images
	PSNR (dB) ↑	SSIM ↑	MAE ↓	MNI-B ↓	CNR ↑	MNI-C ↓
SS-SP-NF	30.434 ± 7.896	0.910 ± 0.059	2.455 ± 2.012	0.622 ± 0.119	0.816 ± 0.261	0.356 ± 0.204
SS-MP-NF	30.324 ± 7.479	0.910 ± 0.057	2.563 ± 2.146	0.590 ± 0.122	1.020 ± 0.289	0.344 ± 0.161
RS-SP-NF	31.593 ± 7.599	0.918 ± 0.054	2.166 ± 1.808	0.585 ± 0.113	1.249 ± 0.303	0.196 ± 0.098
RS-MP-NF	30.974 ± 7.347	0.911 ± 0.059	2.609 ± 2.878	0.587 ± 0.113	1.307 ± 0.248	0.357 ± 0.198
RS-SP-F	32.118 ± 7.402	0.921 ± 0.054	1.940 ± 1.691	0.530 ± 0.128	1.294 ± 0.308	0.190 ± 0.119
RS-MP-F	32.666 ± 7.010	0.926 ± 0.051	1.798 ± 1.575	0.528 ± 0.124	1.420 ± 0.291	0.156 ± 0.057

Values are given as mean ± standard deviation. Bold represents the best result for each column. ↓ means lower is better, and ↑ means higher is better for each column’s metric.

. Figure 13 displays typical OCTA images generated by different DL schemes.

All DL schemes successfully achieved the task of generating OCTA images from OCT structure images with a good convergence in model training. This demonstrated the effectiveness of using the UNet-CBAM architecture with the L₁-SSIM loss function to accomplish the task. Among the results from the different DL schemes, the proposed RS-MP-F scheme performed the best, as evident from both the qualitative observation in Figure 13 and the quantitative results in terms of the reference and non-reference evaluation metrics, as shown in Table 2. As expected, the scheme using repeated scan OCT images as input increased the vasculature contrast, as shown in CNR in comparison with a single scan. Multiple position OCT images together with fused high-quality label OCTA images further improved the vasculature contrast, as shown in CNR, and enhanced the capability of motion artifact suppression, as shown in MNI-B and MNI-C. The possible effects of different factors with respect to network input and label image configurations on the generated OCTA image quality are further discussed in detail in the Discussion Section. The DL model computational cost and speed achieved using our current implementation configurations are also reported here. For the six DL schemes implemented using a backbone of U-Net combined with a CBAM module, the mean model size was 34.66 million total parameters, 65.64 billion floating point operations (FLOPs), a batch training time of 850.0 ms, and an interference time of 33.0 ms for generating an OCTA image. The difference in these metrics among different schemes was small because the main difference came from the change in the input channel number only.

4. Discussion and Conclusions

In this study, a deep learning (DL) method was used to generate OCTA images from structural OCT images. We proposed using repeated and adjacent structural OCT images in the input and utilizing high-quality label OCTA images in training to produce an OCTA image generation model that could suppress motion noise and improve vasculature contrast. A sliding-window correlation-based adjacent-position (SWCB-AP) image fusion scheme was proposed to prepare the high-quality OCTA images used for DL model training. We also proposed new evaluation metrics, i.e., the motion noise indices (MNI-B and MNI-C), together with other conventional ones to quantitatively assess the quality of the DL-generated OCTA images. The results of six different DL construction schemes with respect to different configurations of inputs and label image arrangements showed that our proposed method achieved the best performance in generating OCTA images with high CNR and low motion noise.

Traditionally, it is technically challenging to obtain high-quality OCTA images as it is not only affected by flow contrast computation methods but also by a number of external factors including tissue motion and appropriate data acquisition operations [3,50]. In this study, we used the recently very popular DL method to generate OCTA images directly from structural OCT images with specific advantages such as denoising capability [37]. The specific focus of this study was related to motion noise, which is very commonly seen as white lines in en-face OCTA images and has not been attended to in previous studies. In our data, these motion lines appeared in such a periodic pattern (Figure 3d, Figure 9a and Figure 11d) that they might be associated with the involuntary cyclic and pulsatile movement in nailfold skin caused by the heartbeat. Motion causes obvious artifacts in OCTA images, so it is an important task to develop a DL method that can be used directly to suppress these motion artifacts. This issue was addressed from the point of view of DL network input configurations and label image preparation in this study.

This study proposed a sliding-window correlation-based adjacent-position (SWCB-AP) image fusion method to suppress motion noise and obtain high-quality B-scan label OCTA images for DL model training. Motion noise could be seen obviously in different OCTA computation algorithms including ID, SV, and ED (Figure 12a–c). Simply averaging OCTA images obtained by different flow computation methods at a single position could not suppress this motion noise (Figure 12d). Consequently, neighborhood OCTA images without motion noise were considered for this purpose. The correlation analysis in SWCB-AP was used to judge whether there was significant motion in one of the consecutive three OCTA frames, and the one with significant motion noise was omitted in averaging, thus achieving the effect of noise artifact suppression. In positions where no motion existed, this correlation-based weighted averaging scheme could also perform the task of frame averaging or zeroing to reduce or even totally remove the baseline noise and improve the overall contrast-to-noise ratio of the OCTA image (Figure 12f vs. Figure 12e). The averaging of OCTA images acquired at multiple positions normally led to a blurring of the contrast of vasculature, but this effect was considered to be minimal in this study as most of the vessels were perpendicular to the imaging plane in the nailfold region because of its anatomic characteristics. In general, our proposed SWCB-AP method outperformed the other two comparison fusion methods, as shown in Figure 12. The quantitative parameters in SWCB-AP included the window size, and the threshold of correlation used for detecting the frame with significant motion noise should be set properly. A too-large window would reduce the effect of background noise removal and a too-small window would not be enough to cover the vessel region with simultaneously increased computation cost. A large threshold might miss the detection of some windows with significant motion, while a small one would detect too many fault windows without significant motion. These two parameters were optimized with experiments in this study and fixed for processing all the OCTA images. However, for more complex scenarios involving different levels of motion amplitude, further exploration can focus on adaptive fusion algorithms, where the window size and correlation threshold can be adaptively determined for different scenarios. On the other hand, the application of the current fusion method is limited to the suppression of significant motion noise existing in 1~2 frames. When tissue motion is quite violent and causes significant motion artifacts at multiple (>2) consecutive positions, this fusion method may fail to work properly, and more advanced algorithms such as the recently proposed window-based blood flow computation method [51] or bulk motion correction [52] need to be developed to solve this problem.

Regarding deep learning methods, this study used U-Net combined with the attention mechanism as the main network architecture. U-Net is a popular DL model with a typical encoder–decoder structure, which has been successfully applied for a series of tasks including the generation of OCTA images using structural OCT images [35]. To demonstrate the efficacy of the added CBAM module in the network architecture, we conducted an ablation study based on the RS-MP-F scheme between the U-Net model with and without the CBAM module. Compared with U-Net without CBAM, the results indicate an increase of 0.186 dB, 0.002, and 0.053 in PSNR, SSIM, and CNR, respectively, and a decrease of 0.056, 0.040, and 0.003 in MAE, MNI-B, and MNI-C, respectively, for the model of U-Net with CBAM. These findings underscore the effectiveness of incorporating the CBAM module on the model performance. Nowadays, DL technology is developing quite fast, and U-Net may not be the best model for our task of OCTA image generation. In future studies, different variants of U-Net as well as more advanced architectures such as pix2pix and CycleGAN can be considered in our task to further improve OCTA image generation [53]. For model evaluation, traditional reference-based metrics including PSNR and SSIM were used for the comparison between the generated and labeled OCTA images. Furthermore, this study proposed novel evaluation metrics—the motion noise index (MNI) tailored to B-scan and en-face OCTA images, referred to as MNI-B and MNI-C, respectively—to assess the motion artifact level. MNI-B calculates the ratio between the valley and peak values of the intensity distribution in the horizontal direction to indicate the level of motion artifacts in a B-scan OCTA image. This index was defined based on the observation that peak and valley values of the averaged 1D intensity line were more likely to be similar in B-scans with significant motion noise, but they were quite different in B-scans without significant motion noise (Figure 8b,e). This generally led to a large versus small value of MNI-B, as observed in OCTA images with versus without significant motion noise (Figure 8a,d). It should be noted that this MNI-B is only applicable to B-scan OCTA images with a sparse distribution of blood flow points. For en-face OCTA images, this study also proposed MNI-C, defined as the averaged peak value of the 1D intensity gradient line versus the vasculature region intensity, for characterizing the severity of motion white lines. This index could well describe the strength of white lines in en-face OCTA images, as these lines were shown as very large peaks in the 1D intensity gradient line (Figure 9b). Wei et al. [54] proposed an instantaneous motion strength index (IMSI), where a normalized standard deviation of OCTA image intensity was defined for detecting a microsaccade and blink motion artifact in wide-field en-face OCTA images. MNI-C is similar to IMSI, but the denominator is changed to that of the vasculature region signal intensity to avoid the small mean in IMSI for some regions where no vasculature exists. The two metrics custom-designed for this study facilitated the assessment of the motion artifact level, providing valuable insights into the evaluation of the effectiveness of different DL motion artifact suppression schemes, as discussed in detail as follows.

The performance of the DL method using a non-repeated single scan (SS-SP-NF) was the poorest among all the comparison methods. Lee et al. [31] first proposed generating OCTA images from a single structural OCT image. Their study successfully demonstrated that DL was a viable tool for generating OCTA images with better flow contrast than that seen in structural OCT images. The reason for this capability was explained by the different speckle characteristics around the dynamic tissue induced by blood flow compared with the static tissue in a single structural OCT image [31]. However, it is easily understandable that pure speckle characteristics in a single OCT structural image are not sensitive to blood flow, particularly to that of small vessels, leading to a limited performance with weak and non-smooth capillaries, as qualitatively observed (Row 3, Column 4–Column 6 in Figure 13), and inferior evaluation metric values, as quantitatively calculated (Table 2), in SS-SP-NF. Increasing the network input from a single OCT image to several OCT images acquired at different time points, whether at a single position (RS-SP-NF), multiple positions (SS-MP-NF), or both (RS-MP-NF), would improve the quality of the generated OCTA image because the temporal speckle variation would be a better source for generating blood flow contrast [33,34]. The CNR results showed that the flow contrast increased gradually from SS-MP-NF to RS-SP-NF, and then to RS-MP-NF, with the increase in input OCT image number (from 3 to 4, and to 12 images). Jiang et al. demonstrated that the incorporation of neighborhood information could enhance DL-generated OCTA images [36]; therefore, the input involving repeated scans at multiple positions was used in this study. However, the motion artifact was still quite large in RS-MP-NF, and this might be induced by the non-fused OCTA label used in training. Among the three methods, the motion artifact was suppressed the best with RS-SP-NF, although with a relatively lower CNR. In practical situations where only a non-repeated scan was acquired from a single position such as a conventional OCT volume scan, SS-MP-NF would also be a viable choice for generating OCTA images with better flow contrast than pure structural OCT images. Finally, when a fused label OCTA image was used, RS-MP-F obtained the best performance in terms of all the evaluation metrics (Table 2), which further indicated the importance of incorporating high-quality labels in model training. High-quality label OCTA images could also be prepared by a large number of repeated scans, as used in a previous study [34]. This scheme was not used in the current study because the focus of this study was related to motion noise that could be suppressed from the perspective of combining adjacent-position image information, but not from repeated scans at a single position. Multiple position OCT images were used together with high-quality label OCTA images considering that the high-quality OCTA image was also fused from multiple adjacent OCTA images, which were generated from adjacent structural OCT images. Overall, the best performance of RS-MP-F showed the advantages of both combining neighborhood and repeated scan information and utilizing high-quality labels in training to perform the task of generating OCTA images from structural OCT images.

There are still some limitations in the current study. Firstly, this study was based on a nailfold OCTA imaging dataset, and the amount of data involved in training and testing was relatively small. The generalizability of the current method to other fields such as the ophthalmic OCTA dataset needs to be further investigated. Secondly, the proposed SWCB-AP image fusion method can be used at OCTA image acquisition positions where there is no significant bulk motion. When a large inter-frame bulk motion exists, the applicability of the correlation-based algorithm may fail to work effectively, and a more advanced fusion method is needed to tackle this problem. Thirdly, the effect of an unbalanced sample image ratio (the number of OCTA images with severe motion noise might be small compared with those without severe motion noise) on different schemes, particularly those using the non-fused OCTA as the label, was not quantitatively studied. Finally, the DL network architecture and model hyperparameters were not deliberately optimized in the current study. Future work may be explored from aspects including a neural architecture search [55,56], the design of a loss function incorporating the evaluation of vasculature characteristics such as vessel continuity, and utilizing weakly supervised or unsupervised methods [37] to address the challenge of obtaining high-quality label OCTA images.

In conclusion, this study investigated a DL-based generation of OCTA images from structural OCT images with capability in motion artifact suppression. Various DL schemes with respect to different arrangements of input structural OCT images and label OCTA images were experimented with, and their capability in suppressing motion artifacts was evaluated using specific evaluation metrics including MNI-B and MNI-C, which were specifically proposed in this study. The results of the image fusion experiment showed that our proposed sliding-window correlation-based adjacent-position (SWCB-AP) method could successfully obtain high-quality OCTA images with good suppression of motion noise. The results of the deep learning OCTA image-generation experiment showed that our proposed DL model, with the input of repeated and adjacent structural OCT images together with training using SWCB-AP-fused high-quality label OCTA images, achieved the best performance in terms of qualitative visual observation and quantitative evaluation. The proposed deep learning method may serve as a potential tool in generating high-quality tissue microvasculature images from OCT structural images with good suppression of motion noise in OCTA imaging with broad applications in both clinics and research.