A Systematic Literature Review on Applications of GAN-Synthesized Images for Brain MRI

Abstract

With the advances in brain imaging, magnetic resonance imaging (MRI) is evolving as a popular radiological tool in clinical diagnosis. Deep learning (DL) methods can detect abnormalities in brain images without an extensive manual feature extraction process. Generative adversarial network (GAN)-synthesized images have many applications in this field besides augmentation, such as image translation, registration, super-resolution, denoising, motion correction, segmentation, reconstruction, and contrast enhancement. The existing literature was reviewed systematically to understand the role of GAN-synthesized dummy images in brain disease diagnosis. Web of Science and Scopus databases were extensively searched to find relevant studies from the last 6 years to write this systematic literature review (SLR). Predefined inclusion and exclusion criteria helped in filtering the search results. Data extraction is based on related research questions (RQ). This SLR identifies various loss functions used in the above applications and software to process brain MRIs. A comparative study of existing evaluation metrics for GAN-synthesized images helps choose the proper metric for an application. GAN-synthesized images will have a crucial role in the clinical sector in the coming years, and this paper gives a baseline for other researchers in the field.

1. Introduction

Computer-aided diagnosis is a big supportive help to clinicians, provided the biomedical images are high-quality, such as magnetic resonance imaging (MRI) scans. MRI is a preferred imaging technique compared to computerized tomography (CT) and positron emission tomography (PET) as it gives better soft tissue contrast and is free of any ionizing radiation [1]. In MRI, hydrogen ions in water molecules are reoriented by applying a high magnetic field. As a result, images with details of anatomical structures are available for analysis. The quality of MRI is affected by three factors during the scanning process: the density of protons, and the T1 and T2 relaxation times, where the timespan of T1 and T2 vary according to the hardness of the body tissue [2]. MRI scans are vital for brain navigation, detailed characteristics, and other cranial structures [3].

The deep learning (DL) framework allows for the automated analysis of inputs against the handcrafted feature analysis [4]. DL methods help address common medical imaging problems, such as increasing accuracy and precision, and enhancing the speed of image analysis and image contrast [5]. However, DL models need large datasets to train the model effectively, which are generally not adequately available in the biomedical field [6]. A generative adversarial network (GAN) is a prospective solution to this problem as it can generate training samples duplicating the distribution of the real dataset. In medical imaging, these GAN-synthesized image samples have the forefront use of dataset enhancement; the images can be further used for applications such as image translation, registration, super-resolution, denoising, motion correction, segmentation, reconstruction, and contrast enhancement. GAN was proposed by Ian Goodfellow et al. [7] in 2014 to generate synthetic images duplicating the ground truth images. A GAN is a combination of two networks, the generator and the discriminator, which are trained at the same time with images. The generator produces fake images duplicating the original images, which are then sent to the discriminator network to be compared and evaluated against the authentic images. The discriminator is trained on both types of inputs and learns the features of the original input. The difference between the two inputs is called discriminator loss, which is conveyed to the generator network to modify the parameters for improvement. The situation is similar to a min-max game where a strategy minimizes the network’s maximum possible loss (see Figure 1). This work presents the latest GAN trend for MRI brain image analysis.

Section 1 of the SLR paper consists of an introduction to the topic. Section 2 presents the methodology, including the review of prior research in the related field, the motivation behind this SLR, research questions, search strategy, and inclusion-exclusion criterion. The results of the SLR guided by research questions are reported in Section 3. The discussion and conclusion are presented in Section 4 and Section 5, respectively.

2. Materials and Methods

The authors have adopted an unbiased review process and targeted approach while writing this SLR. They have followed a systematic sequence of steps for the review: defining the research question (RQ) based on predefined goals, identifying the targeted journals and conferences, finding the already available reviews, forming the relevant search strategy, identifying papers based on the selection criteria, comprehending the content, extracting the data, summarizing the results and constructing the final document.

2.1. Prior Research

The closest review paper found on this topic is by Hazrat Ali et al. [8], where they presented a scoping review of GAN’s applications in brain MRI, publicly available brain MRI datasets, evaluation procedures, and focal brain diseases. However, this paper does not give the details of methods used in GAN’s applications, and evaluation metrics are superficially mentioned. There is no description of loss functions and preprocessing software. Creswell et al. [9] described the training part and the mode collapse issue related to the stability of GAN, and it is not limited to MRI and brain images.

Table 1. Prior research in MRI brain imaging.

Ref. No.	Year	Objective	Imaging Modality	DL Methods	Type
[8]	2022	It summarizes GAN’s role in brain MRI.	MRI	GAN	Scoping Review
[9]	2018	The paper gives GAN training, architecture, and a few application details.	All type	GAN	Overview
[10]	2020	It summarizes machine learning and DL classification methods.	MRI	CNN 1, RNN 2, GAN, DBM 3	Review
[11]	2020	It discusses GAN’s application in radiology. They have quantitatively compared the performance metrics for synthetic images.	CT, MRI, PET, and X-ray	GAN, CNN	SLR

¹ CNN: Convolution neural network; RNN: ² Recurrent neural network; ³ DBM: Deep Boltzmann machine.

gives the summary of previous research work.

2.2. Motivation

Since its invention in 2014, GAN has been a preferred choice in research. The advent of deep learning has highlighted its dependency on large training datasets, which are usually rare in the biomedical field. The number of peer-reviewed papers on GAN with MRI-only workflow for brain imaging has increased considerably. For example, the number of research papers in this field published in Web of Science (WoS) and Scopus databases show an increasing trend (Figure 2). The study of related work in the literature highlights the following two limitations:

• There is no existing SLR paper that describes the applications of GAN-synthesized brain MRI.
• There is no existing SLR paper that mentions the types of GAN loss functions.

This SLR is going to contribute to the literature in the following manner:

• The paper presents a clear categorical division of brain MRI applications of GAN-synthesized images as well as the details of the technique.
• The paper presents a concise account of distinct loss functions used in GAN training.
• The paper identifies software to preprocess brain MRI.
• The paper compares the various evaluation metrics available for the performance evaluation of synthetic images.

2.3. Research Questions

Authors have performed a systematic review of available literature in two databases following the directions of the preferred reporting items for systematic reviews and meta-analyses (PRISMA) methodology [12]. The PRISMA flowchart for the retrieval and selection of papers is shown in Figure 3.

Table 2. Research questions for this SLR.

Number	Research Questions	Motivation
RQ 1	What are the applications of GAN-synthesized images for brain MRI?	The question divides the available literature into more clear categories.
RQ 2	What are the most commonly used loss functions in GAN-synthesized image applications for brain MRI?	Loss function affects the training of GAN.
RQ 3	What are the preprocessing steps performed on ground truth brain MRI?	Preprocessing steps performed on ground truth brain MRIs are crucial for the fidelity of the successive GAN operations.
RQ 4	How to compare the existing evaluation metrics for GAN-synthesized brain MRI?	This question encourages a comparative study of available evaluation metrics.

indexes the Research Questions (RQ) considered for this SLR.

2.4. Search Strategy

The applications of GAN-synthesized images for brain MRI were explored by querying the Web of Science and Scopus databases. The papers containing the following terms in the title, abstract, and keywords were identified: “(MR Imaging) OR (MRI) OR (magnetic resonance imaging)” AND “(Brain Imaging) OR (Brain Images)” AND “GAN OR Generative Adversarial Network.” A comprehensive screening of the papers published in the Web of Science (January 2017 to August 2022) and Scopus databases (January 2017 to August 2022) resulted in 210 and 389 peer-reviewed articles, respectively. The SLR quality is maintained by only reviewing results from quartile 1 (Q1) or quartile 2 (Q2) journals and recent conference papers. Q1 shows that a journal’s impact factor lies in the top 25% of journal impact factor (JIF) distribution for a category, and Q2 indicates the next 25% slot of JIF [13]. Therefore, we are hopeful that the proposed search terms make up most of GAN’s work.

Table 3. Search query used for paper selection related to MRI brain imaging.

Database	Query	Initial Result
Web of Science	((MR Imaging) OR (MRI) OR (magnetic resonance imaging)) AND ((Brain Imaging) OR (Brain Images)) AND (GAN OR Generative Adversarial Network)	210
Scopus	((MR Imaging) OR (MRI) OR (magnetic resonance imaging)) AND ((Brain Imaging) OR (Brain Images)) AND (GAN OR Generative Adversarial Network)	389

gives details of the search queries applied for both databases.

2.5. Inclusion and Exclusion Criteria

A well-defined selection criterion lays out the guidelines for the inclusion or exclusion of a particular research paper in the systematic review. The selection criterion should be impartial toward the authors’ names and institutions. This study includes full-length articles involving GAN for brain MRI. The focus is given to only Q1 or Q2-rated peer-reviewed journals and recent conferences to have high-quality search results. A search for full-text availability reduced the volume of papers. The studies in a language other than English are excluded to obtain a clear perception. We have considered only the MRI modality as the source image for the review and excluded other modalities such as CT, PET, or X-ray. Reports based on the non-human brain are excluded. Review papers, editorials, and book chapters are not included. So, the final count of study papers is n = 145 after applying the inclusion-exclusion criterion as given in

Table 4. List of inclusion and exclusion criteria.

Inclusion Criteria	Exclusion Criteria
▪ The original and empirical research studies.	▪ Research studies other than the English language.
▪ The research studies published between 2017 and 2022.	▪ Research studies present in both databases.
▪ The research studies providing the answer to any of the RQs.	▪ Research studies unavailable in their full-text form.
▪ Research studies including search keywords in the title, abstract, or full text.	▪ Research studies with source modality images other than MRI.
▪ Research studies from the Q1 or Q2 journal or a recent conference.	▪ Research studies with non-human brain images.
	▪ Research studies with no GAN involvement for image generation.

.

3. Results

3.1. Applications of GAN-Synthesized Images for Brain MRI (RQ 1)

The size of the dataset is usually limited in the case of medical imaging. Therefore, synthetic images are of great importance in training the networks. GAN-synthesized images are considered the primary remedy to deep learning implementations’ scarce model training data. Data augmentation is the apparent application of GAN-synthesized brain MRI, yet there are other imaging purposes, as shown in Figure 4. The following subsections describe how the synthetic brain images generated by various GAN models are used for different brain image purposes.

3.1.1. Image Translation

Deliberation of more than one imaging modality is recommended in finding the complete picture of the abnormalities. The main task in image-to-image translation is a reliable mapping from the source image to the synthetic image [14]. The process involves good adoption of the loss function to inculcate this mapping into a high-quality translated image [15]. However, acquiring all modalities is impossible due to certain limitations, e.g., scanning time, cost, radiation dose, and patients’ age. GAN is extensively used to generate CT and PET images from source brain MRIs.

A.

MRI-to-CT Translation:

In current radiotherapy, both CT and MRI are used to diagnose brain abnormality better. On the one hand, CT images give an electron density score required for treatment planning; on the other hand, MRI is equally contributory with its superior contrast at soft brain tissues. MRI-only treatment planning can reduce the registration misalignment between CT and MRI scans, curtail the imaging cost, improve the radiotherapy accuracy, and lower the patient’s vulnerability towards ionization radiation compared to the CT process possible with GAN. The GAN model can work on paired or unpaired type training images as inputs. The unpaired dataset is readily available but is hard to deal with as the mapping between input and output is unknown. Although challenging to access, the paired datasets present easy GAN development.

The authors in [16] have generated synthetic CT images by using mutual information (MI) as the loss function to avoid the issue of misalignment between MRI and CT images. The study of [17] is the modification of [16], where the ConditionalGAN (CGAN) model checks the dosimetric accuracy of synthetic CT (SCT) images of patients with brain tumors with the aim of their use in MRI-only treatment planning for proton therapy. Along with MI, binary cross entropy is used as the activation/loss function in the discriminator. The study of [18] compares the similarity between SCT and the original CT, where a CGAN based on the pix2pix architecture produces an SCT. Radiation dose calculation is a significant difficulty in MR-only workflow as it is hard to find electron density information from only MRI scans. The study [19] discusses SCT generation and evaluates dosimetry accuracy.

The MedGAN in [20] uses the fusion of non-adversarial losses from recent image style transfer techniques to capture the desired target modality’s high- and low-frequency details. MedGAN works at the image level in an end-to-end manner giving better performance compared to patch-wise training, which suffers from limited modeling capacity. CasNet, new generator architecture, uses encoder-decoder pairs to raise the acuteness of the resultant images by gradual refinement. The MR-based attenuation correction (MRAC) process is extensively used in PET/MR systems for photon attenuation correction. In the atlas-based MRAC method, CGAN generates photon attenuation maps from SCT. Skip connections of U-Net and GAN loss restore edge information in images [21]. The dosimetric and image-guided radiation therapy (IGRT) method of SCT generation is discussed in [22]. GAN with a ResNet generator and a CNN discriminator creates SCT images from T1-weighted postgadolinium MRI. In [23], a spatial attention-guided generative adversarial network (Attention-GAN) minimizes the spatial difference in SCTs that can deal with the atypical anatomies and outliers in a better way. The framework projects the regions of interest (ROI), and the transformation network performs the domain change. The term attention is the addition of the absolute values in different layers of the discriminator across the channel dimension. CycleGAN is the second most commonly used GAN model in image translation applications. The cycleGAN can work on unpaired data but may introduce inconsistent anatomical features in the generated images. The unsupervised attention guided GAN (UAGGAN) model can work with both paired and unpaired images and be used for bidirectional MR-CT image synthesis. First, the supervised pre-training fine-tunes the network parameters; then, the unsupervised training improves the medical image translation. The combination of WGAN adversarial loss with content loss and L1 assures global consistency in the output image. The UAGGAN achieves satisfactory performance by producing attention masks [24]. In [25], cycleGAN with dense blocks performs two transformation mappings (MRI to CT and CT to MRI) simultaneously. A multi-scale patch-based GAN performs unpaired domain translation and generates 3D medical images of a high resolution. The approach has a low memory requirement where a low-resolution version is generated and later converted into a high-resolution version by patches of constant sizes [26]. Three-dimensional cycleGAN uses inverse transformation and inverse supervision to learn the mapping between the MRI and CT image pairs for proton treatment planning of base-of-skull (BoS) tumors. The dense blocks-based generator explores image patches for textural and structural features [27]. Attenuation correction (AC) needed for a PET image is accomplished by a 3D cycle-GAN, where a 3D U-net generator produces continuous AC maps from Dixon MR Images without MR and CT image registration. The downsampling and upsampling layers in 3D U-net reduce the memory requirements [28]. StarGAN performs image translation among more than one pair of classes. In [29], counterfactual activation generator (CAG) implement image transformation for seven classes. This setting extracts task-sensitive features from brain activations by equating ground truth and real and synthetic images. In [30], high dimensional input maps are translated to high dimensional output maps with the help of Pix2Pix-cGANs to colorize the tumor region in Intracranial tumor MRI images.

B.

MRI-to-PET Translation:

MRI scans also find applications in synthetic PET scan generation similar to SCT generation. A good-quality PET image requires a full-dose tracer, but the potential health hazards posed by radioactive exposures raise concerns about PET images’ use. A 3D auto-context-based locality adaptive multimodality GAN (LAGAN) generates the superior FDG PET using the same kernel for every input modality. The locality-adaptive fusion network produces a fused image by learning convolutional kernels at different image locations. Then, these fused images are used for generator training keeping the number of parameters low while increasing the number of modalities. Contrary to the multimodality cases where convolution is performed globally, the method in [31] concentrates on locality adaptive convolution. In PET imaging, if the tracer dose is lowered, considering its negative effect on the patient, unnecessary noises and artifacts compromise the quality of the resultant image. Two networks, convolutional auto-encoder and GAN generate adaptive PET templates with the help of a C-PIB PET scan and a T1-weighted MRI sequence. These synthetic PET images are used to spatialize amyloid PET scans during Alzheimer’s disease estimation [32]. In Multiple Sclerosis, demyelination occurs in the brain’s white matter and the spinal cord. Sketcher-Refiner GAN predicts the PET-derived myelin content map from multimodal MRI by sketching the anatomy and physiological details and then producing the myelin content map. The model is an extension of CGAN with a 3D U-Net generator working on four MRI modalities as inputs [33]. A task-induced pyramid and attention GAN(TPAGAN) integrates pyramid convolution and attention module to create the absent PET image with corresponding MR. Three sub-networks perform the whole task: pyramid and attention generator, standard discriminator, and task-induced discriminator [34].

Bidirectional mapping GAN (BMGAN), a 3D end-to-end network, in [35] makes use of image contexts and latent vectors to generate PET from brain MRI. The model employs the generator, the discriminator, and the encoder to fuse the semantic features of PET scans with the high-dimensional latent space. The forward mapping step during the model training allows the PET images to become encoded into the latent space. The backward mapping step enables the generator to produce PET images from the MRI and sampled latent vectors. Finally, the encoder reconstructs the input latent vector from the synthetic PET scan. A hybrid GAN (HGAN) employs a hybrid loss function to produce absent PET images taking a clue from corresponding MRI scans. A spatially-constrained Fisher representation (SCFR) network is used to derive statistical details from multimodal neuroimaging data [36]. In [37], cycleGAN generates fake FDG-PET from T1-weighted MRI in two manners; one is from three adjacent transverse slices, and the other is from 3D mini patches. Two CNNs, ScaleNet and HighRes3DNet, and one CGAN were trained to map structural MR to nonspecific (NS) PET images [38].

Table 5. Summary of GAN-synthesized images used for translation.

Ref. No.	GAN Model	Technique
MRI-to-CT
[16]	GAN	MI to avoid the issue of unregistered data
[17]	CGAN	MI and binary cross entropy are the discriminator loss functions to achieve the task-specific goal
[18]	CGAN	Pixel loss penalizes pixel-wise differences between the real and SCT scan
[19]	CGAN	The method calculates dosimetric accuracy by SCT generation
[20]	MedGAN	Non-adversarial losses (combination of style loss and content loss) to obtain high- and low-frequency details of image
[21]	CGAN	Residual blocks are inserted into the CGAN network
[22]	GAN	Image-guided radiation therapy
[23]	AttentionGAN	The attention network helps to predict the regions of interest
[24]	UAGGAN	The model identifies the area of the image and puts in a suitable translation to that location
[25]	CycleGAN	A dense block allows better one-to-one mapping
[26]	CGAN	Constant image patch size. Memory requirement is independent of the image size
MRI-to-PET
[31]	LAGAN	Locality adaptive convolution where the same kernel for every input modality
[32]	GAN	Generate adaptive PET templates
[33]	Sketcher-Refiner GAN	Generates PET-derived myelin content map from four MRI modalities
[34]	TPA-GAN	Integrates pyramid convolution and attention module
[35]	BMGAN	Use of image contexts and latent vectors for a generation

presents the summary of image translation.

3.1.2. Image Registration

The image registration technique processes the images to fuse them to extract more information. In some cases, moving images are transformed into fixed reference images. The purpose behind the image registration can be motion correction, pose estimation, spatial normalization, atlas-based segmentation, and aligning images from multiple subjects [39].

A deep pose estimation network speedily enables slice-to-volume and volume-to-volume registration of brain anatomy. Transformation variables are adjusted by multi-scale registrations that initiate the iterative optimization process. CGAN learns region-based distortions of multimodal registration from T1- to T2-weighted images. Regression-type CNN predicts the angle-axis depiction of 3D movement. CycleGAN can be used for cases where paired images are unavailable [40]. The cycleGAN-based model performs symmetric image registration of unimodal/multimodal images where an inverse consistency performs bi-directional spatial transformations between images. SymReg-GAN is the extension of cycleGAN that performs semi-supervised learning with labeled image pairs and neglects unlabeled pairs. The spatial transformer performs the differentiable operation and warps the moving image using an estimated transformation [41].

The geometric transformation estimates the association of physically corresponding points within a pair of images’ fields-of-view (FOVs). This transformation can sometimes lead to asymmetric and biased mapping where the “fixed” image is unaffected and the “moving” image experiences an interpolation smoothing the image simultaneously. Most of the current registration methods are focused on asymmetric directional image registration. Multi-atlas-based brain image parcellation (MAP) is a technique in which numerous brain atlases are registered to a new reference map. Manually labeled brain regions are passed on and combined with the final parcellation result. The generator of multi-atlas-guided fully convolutional network (FCN) with multi-level feature skip connection (MA-FCN-SC) structure produces input brain image with parcellation [42]. In the multi-atlas guided deep learning parcellation (DLP) technique, attributes of the most suitable map led to the parcellation process of the target brain map. FCN with squeeze-and-excitation (SE) sections GAN (FCN-SE-GAN) performs better than the MAP technique since this method avoids nonlinear registration. Improvement in the result is caused by three factors: brain atlases, automatic brain atlas selection, and GAN [43]. An unsupervised adversarial similarity network performs registration without ground-truth deformation images and specific similarity metrics for the network training. Both mono-modal and multimodal 3D image registration can apply to the network. A spatial transformation layer connects the registration and the discrimination networks [44]. Image registration is crucial for brain atlas building, but it also helps monitor the continuous advancements in multiple patient visits. A specific dataset can train the deep networks for applications where sufficient ground truth data is unavailable for training. However, a network trained to register a pair of chest X-ray images cannot produce the same quality output on a couple of brain MRI scans. In such cases, the network needs to be retrained. GAN-based registration of an image pair and segmentation and transfer learning is achieved. Other image pairs can easily use them without retraining. Two convolutional auto-encoders are used for encoding and decoding [45].

Table 6. Summary of GAN-synthesized images used for registration.

Ref. No.	GAN Model	Technique
[40]	CGAN	Slice-to-volume registration
[41]	CycleGAN	Symmetric registration
[42]	GAN	Multi-atlas-based brain image parcellation
[43]	GAN	Multi-atlas-guided deep learning parcellation
[44]	GAN	3D image registration
[45]	GAN	Transfer learning for registration

presents the summary of GAN-synthesized images used for registration.

3.1.3. Image Super-Resolution

The super-resolution (SR) technique converts low-resolution images to high-resolution images without compromising the scanner settings and imaging sequences. These SR methods achieve higher SNR and reduced blurriness at edges compared to conventional interpolating methods [46]. In the super-resolution process, several low-resolution images taken from slightly different viewpoints are used to predict the high-resolution version. Sufficient prior information allows better prediction parameters than actual measurements [47].

The single image super-resolution (SISR) method is vital for medical images as it helps diagnose the disease. A lesion-focused SR (LFSR) method is developed that produces seemingly more realistic SR images. In the LFSR method, a multi-scan GAN (MSGAN) produces multi-scale SR and higher-dimensional images from the lower-dimensional version [48]. Training the GAN becomes complicated when the inputs are high-resolution and high-dimensional images; therefore, information learning is divided among several GANs. First, a shaping network in unconditional super-resolution GAN (SR-GAN) is employed to pick up the three-dimensional discrepancy in the shape of adult brains. Then, a texture network is applied in conditional pix2pix GAN to improve image slices with realistic local contrast patterns. Finally, the shape network is trained with the WGAN with Gradient Penalty (WGAN-GP) method. It is an unconditional generator that grasps the brain’s three-dimensional spatial distortions [49].

In [50], authors have used the progressive upscaling method to generate true colors. The multi-path architecture of the SRGAN model takes out shallow features on multiple scales where the filter sizes are three, five, and seven instead of a single scale. The upscaled features are matched back to a high-resolution image through a reconstruction convolutional layer. Enhanced sSRGAN (ESRGAN) implements super-resolution 2D MRI slice creation where slices from three different latitudes are selected for the 2D super-resolution and later reconstructed into a three-dimensional form. The first half of the three-dimensional matrices is reconstructed from high-resolution slices with good texture features. Then, the three-dimensional slices are repaired through interpolation to obtain new brain MRI data. The VGG16 [51] is employed before activation to restore the features, solve over-brightness in SRGAN, and improve performance [52]. The work of [53] is also based on ESR-GAN, where two neural networks complete the super-resolution task. The first network, receiving field block (RFB)-ESRGAN, selects half the number of slices for super-resolution reconstruction and MRI rebuilding and upholds high-frequency information. The second network, the noise-based network (NESRGAN), completes the second super-resolution reconstruction task with noise and interpolated sampling, repairing the reconstructed MRI’s absent values. The linear interpolation technique is involved in feature extraction and up-sampling. The neonatal brain MRI is a low anisotropic resolution scan. To increase the resolution, medical images SR using GAN (MedSRGAN) uses residual whole map attention network (RWMAN) first to interpolate and then segment [54].

Existing super-resolution methods are very scale-specific and cannot be generalized over the magnification scale. The medical image arbitrary-scale super-resolution (MIASSR) method coupled with GAN executes the super-resolution for modalities such as cardiac MR scans and chest CTs by exercising transfer learning [55]. Similarly, in [56], simultaneous super-resolution and segmentation are performed for 3D neonatal brain MRI on the simulated version of low-resolution. The learned model then upgrades and segments the real clinical low-resolution images. In 2D MR acquisition, the pulse sequence decides the slice thickness. The exact characteristics of signal excitation are not explicitly known; giving less information about slice selection profiles (SSP) creates insufficient training data. This problem can be solved by predicting a relative SSP from the difference between in- and through-plane image patches. The thicker slices and larger slice distance are maintained to decrease scan timing and achieve a high signal-to-noise ratio, resulting in a lower through-plane resolution than in-plane resolution. The GAN-based method focuses on improving the resolution of the through-plane slices where training data is a degraded version of in-plane slices to match the through-plane resolution [57]. A high signal-to-noise ratio in an MRI scan can assist in correctly detecting Alzheimer’s disease. Utilizing the GAN-based SR technique, image quality equivalent to a 3-T scanner can be achieved without altering scanner parameters, even if the scans are obtained through 1.5-T scanners. The generator creates a transformation mask, and the discriminator differentiates the synthetic 3-T image from the original 3-T image [58].

In [59], fine perceptive generative adversarial networks (FPGANs) adopt the divide-and-conquer scheme to extract the low-frequency and high-frequency features of MR images separately and parallelly. The model first decomposes an MR image into low-frequency global approximation and high-frequency anatomical texture subbands in the wavelet domain. The subband GAN simultaneously performs a super-resolving process on each subband image, resulting in finer anatomical structure recovery. Study [60] uses end-to-end GAN architecture to produce high-resolution 3D images. The training is performed in a hierarchical manner producing a low-resolution scan and a randomly selected part of the high-resolution scan simultaneously. It provides two-fold benefits: first, the memory requirement during training of high-resolution images is divided into small parts. Next, high-resolution volumes are converted to a single low-resolution image keeping anatomical consistency intact. Spatial resolution images are produced by direct Fourier encoding from three short-duration scans [61].

Table 7. Summary of GAN-synthesized images used for super-resolution.

Ref. No.	GAN Model	Technique
[48]	MSGAN	Lesion-Focused SR method
[49]	SRGAN	Use of shaping network
[50]	SRGAN	Progressive upscaling method to generate true colors
[52]	ESRGAN	Slices from 3 latitudes are used for SR
[53]	NESRGAN	Noise and interpolated sampling
[54]	MedSRGAN	Residual whole map attention to interpolate
[55]	GAN	Medical image arbitrary-scale super-resolution method
[57]	GAN	Improving resolution of through-plane slices
[58]	GAN	The image resolution of 1.5-T scanner is made equivalent to 3-T scanner.
[59]	FPGAN	Use a divide-and-conquer manner with multiple subbands in the wavelet domain
[60]	End-to-end GAN	Uses a hierarchical structure

presents the summary of GAN-Synthesized images used for super-resolution.

3.1.4. Contrast Enhancement

In MR imaging, different sequences (or modalities) can be acquired that provide valuable and distinct knowledge about brain disease, for example, T1-weighted, T2-weighted, proton density imaging, diffusion-weighted imaging, diffusion tensor imaging, and functional MRI (fMRI) [62,63,64]. The imaging process can highlight only one of them. Multiple scan acquisition processes and long scan times for capturing all contrasts can give rise to the cost and discomfort of the patient. The enhancement process that generates different contrasts from the same MRI sequences is helpful for overcoming the data heterogeneity [65]. The contrast enhancement methods can be divided into three categories, as shown in Figure 5.

A.

Modality Translation:

In the MRI acquisition, discrete imaging protocols result in different intensity distributions for a single imaging object. The recent data-driven techniques acquire MR images from multi-center and multi-device with multi-parameters. This fact gives rise to the need for universal and uniform datasets. All studies discussed in this section generate one or more MRI modalities from one or more available modalities. The redundant information of the multi-echo saturation recovery sequence with different echo time (TE) and inversion time (TI) generates multiple contrasts generally used as a reference to find a mutual-correction effect. In [66], the multi-task deep learning model (MTDL) synthesizes six 2D multi-contrast sequences: axial T1-weighted, T2-weighted, T1 and T2-FLAIR, short Tau inversion recovery (STIR), and proton density (PD) simultaneously. The registration-based synthesis approach is based on creating a single atlas responsible for loss in structural information of dummy multi-contrast images due to a nonlinear intensity transformation. Whereas the intensity-based method not depending on fixed geometric relationships among different anatomies and gives better synthesis results. PGAN is used for the generation when multi-contrast images are spatially registered, and CGAN is used when unregistered [67]. MultiModal GAN (MM-GAN), a variant of Pix2Pix architecture, synthesizes the absent modality by merging the details from all available modalities [68]. MI-GAN, an amendment over MM-GAN, is a multi-input generative model that creates the missing modalities. Commonly acquired modalities are T1-weighted (T1), T1-contrast-enhanced (T1c), T2-weighted (T2), and T2-fluid-attenuant inversion recovery (FLAIR). The absent one is created from the other three available modalities [69]. The limitation of earlier methods of cross-modality generation is that they are not extendible to multiple modalities. The total M (M − 1) number of different generators will need to be trained to learn all sorts of mapping among M modalities. In addition to this, each translator can only use two out of M modalities simultaneously. The modality-agnostic encoder of a cycle-constrained CGAN draws out modality-invariant anatomical features and generates the desired modality with a conditioned decoder. A conditional autoencoder and discriminator can complete all pair-wise translations. Once the feedforward processing on any modality label is over, the same autoencoder is reused to make a condition on the modality label of the original input for the cycle reconstruction [70]. The usual cross-modality image translation methods involving GAN models are based on paired data. Modular cycleGAN (MCGAN) performs unsupervised multimodal MRI translation from a single modality and retains the lesion information. The architecture includes encoders, decoders, and discriminators. MCGAN uses the combination of deconvolution and resize upsampling methods that avoid the checkerboard artifacts in the generated images [71]. Edges in a medical image contain principal details of anatomy such as tissue, organ, and abrasion details. However, the images produced by a normal GAN have blurred boundaries. A flexible, gradient-prior integrated, encoder-decoder-based adversarial learning network (FGEAN) is an end-to-end framework of multiple inputs and multiple outputs that uses gradient-prior to retain tissue composition type of high-frequency details [72]. Edge-aware GAN (Ea-GAN) is a 3D method that extracts voxel-wise intensity and image structure information to overcome slice discontinuity and blurriness problems. The Sobel operator is used to extract the edge details. The Sobel filter assigns higher weights to its nearer neighbors and lower weights to the farther neighbors, which is impossible with direct image gradient application [73].

CycleGAN-based unified forward generative adversarial network transforms any T2-FLAIR images in different groups into a single reference one [74]. In [75], WGAN generates multi-sequence brain MR images with the advantage of stable learning. The Earth Mover (EM) distance (a.k.a. the Wasserstein-1 metrics) of WGAN allows minor mode collapse. In [76], sample-adaptive GAN imitates each sample by learning its correlation with its neighboring training samples and applying the target-modality features as auxiliary information for synthesis. The self-attention GAN (SAGAN) of [77] attends to various organ anatomical structures via attention maps which showcase spatial semantic details with the help of an attention module. In [78], the GAN framework learns shared content encoding and domain-specific style encoding across multiple domains. CGAN in image modality translation (IMT) network employs nonlinear atlas-based registration to register a moving image to the fixed image. The PatchGAN classifier with no constraints on each patch’s size acts as the discriminator generating acute results with lesser parameters and a low running time [15]. In [79], GAN provides a solution for the detection of Small Vessel Disease (SVD) by estimating the advancement of White Matter Hyper-intensities (WMH) during a year. Disease Evolution Predictor (DEP) model notices WMH in T2-weighted and T2- FLAIR MRIs. DEP-GAN (Disease Evolution Predictor GAN), an extension of visual attribution GAN (VA-GAN), uses an irregularity map (IM) or probability map (PM) for both input and output modalities to represent WMH. This generated image is called Disease Evolution Map (DEM), which classifies brain tissue voxel among progressing, regressing, or stable WMH groups.

B.

Quality Improvement:

High-resolution images are generated from down-sampled data during the MRI analysis to save the scan time. High-resolution images in one contrast improve the quality of down-sampled images in another contrast. The anatomical details of different contrast images refine the reconstruction quality of the image. This increase in image contrast is used for the classification of brain tumors [80]. The intensity distributions of pixels in brain MR images overlap in regions of interest (ROIs) that cause low tissue contrast and create problems in accurate tissue segmentation. The cycleGAN-based model increases the contrast within the tissue by using an attention mechanism. A multistage architecture focuses on a single tissue preliminary and filters out the irrelevant context in every stage to increase high tissue contrast (HTC) images’ resolution [81]. CycleGAN for unpaired data usually encodes the deformations and noises of various domains during synthetic image generation. The deformation invariant cycleGAN (DiCycleGAN) uses image alignment loss based on normalized mutual information (NMI) to strengthen the alignment between source and target domain data [82]. Generation of high-resolution MRI hippocampus region images from low-resolution MRI is arduous. Difficulty-aware GAN (da-GAN) is designed with dual discriminators and attention mechanisms in hippocampus regions for creating multimodality images. These HR images are deployed to improve hippocampal subfields classification accuracy compared to LR images [83]. In [84], Sequential GAN, a combination of two GANs, generates bi-modality images from common low-dimensional vectors. Sequential multimodal image production first creates images of one modality from low-dimensional vectors. These synthetic images are mapped to their counterparts in the other modality through image-to-image translation. The synthetic FLAIR images are not as realistic in terms of quality as synthetic T1-weighted and T2-weighted images. In [85], CGAN and the two parallel FCNs improve the quality of fake FLAIR images by retaining the contrast information of original FLAIR images. In [86], the proposed method discovers and learns global contrast from the label images to embed this information in the generated images. The capability of 2-way GAN coupled with global features in U-Net bypasses the need for paired ground truth. The multimodal images with a better perceptual quality improve the learning capability of the model.

C.

Single Network Generation:

Unified GAN, the improved version of starGAN [87], generates multiple contrasts of MR images from a single modality. StarGAN can perform image translations among multiple domains with one generator and one discriminator. The single-input multiple-output (SIMO) model is trained on four different modalities. The network learns the details from the multimodal MR images and analogous modality markers. The generator takes an image of one modality producing a target modality image, and then performs the second task of recreating the original modality image through a synthesized one [88]. The available methods of multimodal image generation target only the missing image production between two modalities. CycleGAN and pix2pixGAN can only create images from one modality to another; the former is used for unpaired images and the latter for paired images. Multimodality GAN (MGAN) simultaneously synthesizes three high-quality MR modalities (FLAIR, T1, and T1ce) from one MR modality-T2. Complementary information provided by these modalities boosts tumor segmentation accuracy. The architecture extends starGAN for paired multimodality MR images, adding modality labels to pix2pix. StarGAN brings the domain labels to cycleGAN and thus empowers a single network to translate an input image to any desired target domain for unpaired multidomain training images. Thus a single network translates a single modality T2 to any desired target modalities [89].

3.1.5. Image Denoising

The visual quality of MRI is vital for down-the-line operations on acquired scans, and the existing noise in the scans can alter the diagnosis result. Denoising is mostly a preprocessing step for image analysis similarly to segmentation and registration [90]. Image denoising and synthesis are utilized to study the complete manifold learning of brain MRI. The higher SNR value improves segmentation and registration tasks. A low-dimensional manifold is preferred for statistical comparisons and the generation of group representatives. T1-weighted brain MR images are generated by learning from 2D axial slices of brain MRI. Skip-connected auto-encoders are used for image denoising to traverse the manifold description of regular brains [91]. The Rician noise of MRI is the magnitude of the complex image data distribution. Structure-preserved denoising of 3D MRI images allows for exploring the similarity between neighboring slices. The Rician noise in MR images is removed by the residual encoder-decoder Wasserstein generative adversarial network (RED-GAN), where a 3D-CNN operates on 3D volume data. The generator is an auto-encoder proportionally containing convolutional and de-convolutional layers supported by a residual block [92]. Residual encoder-decoder up-sampling non-similar WGAN (REDUPNSWGAN) uses a filter-based method to remove Rician noise, preserving the structural association between the neighborhood slices in 3D MRI. WGAN measures Wasserstein distance to differentiate between ground-truth and dummy images using residual encoder-decoder. GNET removes Rician noise with the help of Huber loss. DNET calculates the loss of discriminated samples, and the feature extractor in AVGNET calculates the perceptual loss [93]. A hybrid denoising GAN removes noise from highly accelerated wave-controlled aliasing in parallel imaging (Wave-CAIPI) images with the help of a 3D generator and a 2D discriminator [94].

3.1.6. Segmentation

Brain tissue segmentation in an MRI scan provides vital biomarkers such as quantification of tissue atrophy, structural changes, and localization of abnormality are crucial in disease diagnosis. DL-based segmentation methods are finding success in the automatic mode of segmentation. The segmentation methods that use GAN-synthesized MR images for atrophy detection can be grouped into three categories, as shown in Figure 6 [95].

A.

Brain Tumor Segmentation:

Two standard techniques in brain tumor segmentation are patch-based and end-to-end methods. A multi-angle GAN-based framework fuses the synthetic images with the probability maps. The PatchGAN generator focuses on local image patches, randomly selects many fixed-size patches from an image, and normalizes all responses that improve the resultant image. The multichannel structure in the discriminator averages the responses to provide the output [96]. A 3D GAN performs brain tumor segmentation by combining label correction and sample reweighting, where the dual inference network works as a revised label mask generator [97]. The current glioma growth prediction is achieved by mathematical models based on complicated mathematical formulations of partial differential equations with few parameters resulting in insufficient patterns and other characteristics of gliomas. On the contrary, GANs prove the upper hand on mathematical models as they need not directly convert the probability density function to generate data. Plus, GANs can withstand overfitting by providing structured training. A 3D GAN stacks two GANs with conditional initialization of segmented feature maps for glioma growth prediction [98]. Tumor growth prediction needs multiple time points of the same patient’s single or multimodal medical images. Again a stacked 3D GAN, GP-GAN, is used for glioma growth prediction [99]. Deep convolutional GAN (DCGAN) first performs data augmentation by generating synthetic images to create a large data set. The image noise is also removed with the help of an adaptive median filter so that the resultant images are of superior features. After this preprocessing step, faster R-CNN uses this synthetic data for training, identifying, and locating tumors. The classification result is tumor placement under three types: meningioma, glioma, pituitary, and primary type [100]. Manual delineation of lesions such as glioma, Ischemic lesions, and Multiple Sclerosis from MR sequences is tedious. Discriminative machine learning techniques such as Random Forest, Support Vector Machines, and DL techniques such as CNN and autoencoders detect and segment lesions from MR scans. However, generative methods such as GANs can also employ convolution operators to learn the distribution parameters [101].

The class-conditional densities of lesions overlap because the pixel values of ROIs are distributed over the entire intensity range in MR scans. The existence of four major overlapping ROIs (non-enhancing tumor, enhancing, normal, and edema) of intensity distribution poses a challenge in the segmentation process. Enhancement and segmentation GAN (Enh-Seg-GAN) refines lesion contrast by including the classifier loss in model training, which estimates the central pixel labels of the sliding input patches. The CGAN generator modifies each pixel in the input image patch. It then forwards this to the Markovian discriminator. The synthetic image is concatenated with other fundamental modalities (FLAIR, T1c, and T2) to improve segmentation [102]. Feature concatenation-based squeeze and excitation-GAN (FCSE-GAN) appends the feature concatenation block to the generator network to reduce noise from the image and the squeeze and excitation block to the discriminator network to segment the brain tumor [103].

B.

Annotation:

The second group describes the methods used to perform segmentation without manual data labeling and how annotation tasks are necessary for DL models. The annotation of medical images is a tedious task requiring good medical prowess. The annotated datasets are an essential requirement for supervised machine learning. The supervised transfer learning (STL) method for domain adaptation trains the GAN model on a source domain dataset and then fine-tuned it on a target domain dataset. The inductive transfer learning (ITL) method extracts annotation labels of the target domain dataset from the trained source domain model using cycleGAN-based unsupervised domain adaptation (UDA) [104]. DCNN-based image segmentation methods are hard to generalize. A synthetic segmentation network (SynSeg-Net) trains a DCNN by unpaired source and target modality images without having manual labels on the target imaging modality. In [105], cycleGAN performs multi-atlas segmentation with a cycle synthesis subnet and segmentation subnet. GANs are designed to generate properly anonymized synthetic images to safeguard the patients’ privacy information. In [106], three GANs are trained on time-of-flight (TOF) magnetic resonance angiography (MRA) patches to create image labels for arterial brain vessel segmentation. Image labels created from deep convolutional GAN, Wasserstein-GAN with gradient penalty (WGAN-GP), and WGAN-GP with spectral normalization (WGAN-GP-SN) are applied to a second dataset using the transfer learning approach. The results of WGAN-GP and WGAN-GP-SN are superior to that of DCGAN. The structure of triple-GAN, which works on the principle of a three-player cooperative game, is modified to incorporate 3D transposed convolution in the generator. It performs tensor-train decomposition on all the classifier and discriminator layers and uses a high-order pooling module to take advantage of association within feature maps. The tensor-train decomposition, high-order pooling, and semi-supervised learning-based GAN (THS-GAN) classify MR images for AD diagnosis [107]. In normal conditions, human brains are relatively symmetric. However, the presence of any mass lesion generates asymmetry in the brain structure because it displaces normal brain tissue. The symmetric driven GAN (SD-GAN) learns a nonlinear mapping between the left and right brain images in unsupervised manifold learning to detect tumors from scans that do not require symmetry [108]. Segmentation tasks on medical images suffer the issues of generalization, overfitting, and insufficient annotated datasets. Guided GAN (GGAN) decimates the data points of an input image, due to which the size of the network is reduced, operating on only a few parameters [109].

C.

Multimodal Segmentation:

The shape or appearance model called Shape Constraint (SC-GAN) uses a Fully Convolutional Residual Network (FC-ResNet) fused with a shape representation model (SRM) for segmentation tasks on multimodal images in H&N cancer diagnosis. A pre-trained 3D convolutional auto-encoder is utilized for SRM as a regularizer in the training stage [110]. Multimodal segmentation should have acceptable results in both source and target domains. However, the domain shifts between multiple modalities make the learning task of divergent image features through a single model challenging. Three-dimensional unified GAN executes the auxiliary translation task by extracting the modality-invariant features and upgrading low-level information representations [111]. Hippocampal subfields segmentation based on SVM combined 3D CNN and GAN. Three-dimensional GAN-SVM acts as a generator and 3D CNN-SVM discriminator [112]. One2One CycleGAN is used in survival estimation extracting features from MRI multimodal images. A single ResNet-based generator creates the T1 image from the T2 samples and the T2 image from the T1 samples reducing overfitting and providing augmentation to create virtual samples [113]. MRIs are used to locate the abrasion of disease or to understand the fMRI-based effective connectivity (EC) within a set of brain regions. The task of locating the abrasion caused due to Multiple Sclerosis (MS) in brain images is a real challenge as there is much inconstancy in the intensity, size, shape, and location of these abrasions. In [114], GAN uses a single generator with multiple modalities and multiple discriminators, one for each modality, to identify the NxN patch as real or fake.

3.1.7. Reconstruction

Although MRI is one of the very sought-after imaging methods for physical and physiological reasons, scanning time causes concern for patients [115]. MRIs are reconstructed due to various reasons cited in Figure 7.

A.

MRI Acceleration:

The lengthy scanning process in which the samples are collected line-by-line in k-space (frequency domain and Fourier image space) is uncomfortable for the patients and becomes a reason for motion artifacts. The concept of accelerated MRI is crucial to tackling this issue. MRI is reconstructed from highly under-sampled (up to 20%) k-space data, especially in fetal, cardiac, functional MRI, multimodal acquisitions, and dynamic contrast enhancements. The acquisition time is lowered by less slice selection, reducing the spatial resolution. The sweep time can also be lessened by selecting a partial k-space and approximating the absent k-space points. A k-space and an image-space U-Net reconstruct the whole k-space matrix from under-sampled data [116]. The compressed sensing (CS) MRI scheme reduces the sweep time by considering a minor set of samples for image construction. Refine-GAN, the adapting fully-residual convolutional auto-encoder, and general GAN, is the base for fast and precise CS-MRI reconstruction. A chained network enhances the reconstruction quality [117]. Traditional CS-MRI is affected by slow iterations and noise-induced artifacts during the high acceleration factor. The RSCA-GAN uses spatial and channel-wise attention with long skip connections to improve the quality at each stage, accelerating the reconstruction process and removing the artifacts brought by fast-paced under-sampling [118]. Parallel imaging integrated with the GAN model (PI-GAN) and transfer learning accelerates MRI imaging with under-sampling in the k-space. The transfer learning removes the artifacts and yields smoother brain edges [119]. Reforming multi-contrast brain MR images from down-sampled data points can save scanning time [120].

B.

MR Slice Reconstruction:

MR slice reconstruction is performed to examine brain anatomy and surgery maneuverings as the modality provides high resolution. Thin-section images are 1 mm wide with a spacing gap of zero, while thick-section images are 4 mm to 6 mm wide with a spacing gap of 0.4 mm to 1 mm. The higher value of thickness leads to low resolution. The GAN and CNN are combined to reconstruct thin-section brain scans of newborns from thick-section ones. The first stage of the network is a Least-Square GAN (LS-GAN) with a 3D-Y-Net generator. This stage fuses the images of the axial and sagittal planes and maps them onto thin-section image space. The cascade of 3D-DenseU-Net and a stack of enhanced residual structures removes image artifacts and provides recalibrations and structural improvements in the sagittal plane [121]. Unsupervised medical anomaly detection (MAD)-GAN uses multiple adjacent brain MRI slice reconstructions to locate brain anomalies at different stages of multi-sequence structural MRI [122]. The edge generator of Edge-guided GAN (EG-GAN) joins the missing edges of low-resolution images and masks produced from missing slices in the through-plane as input. A contrast completion network employs these connected edges to predict the voxel intensities in the missing rows [123]. Conditional deep convolutional generative adversarial neural networks (CDCGANs) are used to forecast the advancement of AD by producing dummy MR images in the series arrangement. The atrophy is measured using the cortical ribbon (CR) fractal dimension (box-counting method). The method uses only one coronal slice of a patient’s baseline T1- image. A reducing fractal dimension ensures the progressing illness [124]. The brain multiplex image represents the brain connectivity status extracted from MRI scans. The association between the two brain regions of interest is quantified based on function, structure, and morphology. A single network, adversarial brain multiplex translator (ABMT), performs brain multiplex estimation and classification for the vision of gender-related distinction linkages. Brain multiplexes are constructed from a source network intra-layer, a target intra-layer, and a convolutional interlayer. The ABMT is the improved version of GT-GAN [125], which has pioneered graph or network translation. Contrary to conventional GAN, the generator (translator) of GT-GAN picks up the generic translation mapping from the source network to the target network simultaneously [126]. 3D CGAN and a local adaptive fusion method are used for quality FLAIR image synthesis. They synthesize each slice separately along the axial direction and concatenate them into a 3D image. This synthesis predicts the coronal and the sagittal direction by analyzing complete images or large image patches [127].

C.

Enhancement of Scan Efficiency:

CGAN enhances the scan efficiency of under-sampled and multi-contrast procured images. The shared high-frequency-prior present in the source contrast is used to maintain high-spatial-frequency features. The low-frequency-prior in the under-sampled target contrast is used to avert feature leakage or quality loss. The perceptual prior is used to upgrade the retrieval of high-level attributes. Reconstructing-synthesizing GAN (RS-GAN) generator estimates the target-contrast image from either fully sampled or partially under-sampled source-contrast image [128]. The tissue susceptibility in various brain diseases is measured via the quantitative susceptibility mapping (QSM) technique. The inherent issue of dipole inversion can affect the reliability of the susceptibility map. QSM-GAN is a 3D U-Net that solves the dipole inversion problem in QSM reconstruction [129]. A directed graph represents a brain-effective connectivity network where nodes denote brain regions. EC-RGAN is a recurrent GAN that applies effective connectivity generators to acquire the temporal information from the fMRI time series and refine the quality [130]. Double inversion recovery (DIR) improves FLAIR images acquired with a higher sensitivity for lesion diagnosis than conventional or fluid-attenuated T2-weighted scans. They are beneficial for detecting cortical plaques in MS. DiamondGAN can increase image details via multi-to-one mapping where various input modalities (in this case, T1, T2, and FLAIR) are utilized to produce one output modality (in this case, DIR) [131]. Three-dimensional multi-information GAN uses structural MRI to find cortical atrophy to predict disease progression. First, a 3D GAN model generates 3D MRI images at future-time points; then, a 3D-densenet-based multiclass classification identifies the stages of produced MRI [132]. Visual scenes can be reconstructed from human brain activity measured with fMRI. Dual-Variational Autoencoder/Generative Adversarial Network (DVAE/GAN) learns the mapping from fMRI signals to their corresponding visual stimuli (images). Cognitive Encoder, Visual Encoder, and GAN transform the high-dimensional and noisy brain signals (fMRI) into low-dimensional latent representation [133].

D.

Bias-free MRI Scan:

MRI scanners inherently produce bias, resulting in soft intensity changes across the scans. Two GANs were simultaneously trained to reconstruct the plain bias field and a bias-free MRI scan [134]. Ultrahigh-field MRI introduces high signal inhomogeneity in the scanned images, giving rise to different non-uniform power concentrations in the tissues. The regional Specific Absorption Rate (SAR) varies spatially and temporally with possible hubs in several hard-to-predict positions. A CGAN model can assess the subject-specific local SAR, otherwise hard to compute, and is rated by offline numerical simulations using generic body models. A CNN learns to portray the connection between subject-specific complex B1+ maps and the corresponding local SAR [135].

3.1.8. Motion Correction

MRI acquisition is a time consuming process, and keeping the head still in the scanner for the whole duration is challenging for patients. Subject motion in MRI scanning can introduce blurring and artifacts in the resultant images that severely deteriorate the image quality. Motion correction is the one crucial application during the preprocessing phase of the diagnosis [136,137] CGAN generates images devoid of artifacts from images distorted by motion. Combining a deep CNNs (DCNN) generator with a classifier discriminator introduces sharpness in the output of the DCNN [138]. Different acquisition protocols of motion-free and motion-corrupted MR data give alignment deformity. Cycle-MedGAN is free from dependency on co-registered datasets since unpaired images are used for training without prior co-registration or alignment. It employs the cycle-style and the cycle-perceptual losses functions to supervise the generator network and the self-attention architecture with convolutional layers and several residual blocks to improve the long-range spatial dependency for the motion correction task [139]. The supervised adversarial mode of correcting the motion artifacts raises the alignment imperfections that result from two distinct acquisition processes; one is motion-free, and the other is motion-corrupted. On the contrary, a co-registered dataset does not affect the unsupervised mode. A framework consisting of cycle-MedGAN [140,141,142] utilizes any paired datasets during training and a smaller subset of paired data for validation with its self-attention characteristic and a new loss function.

3.1.9. Data Augmentation

The success of DL models depends on large data samples, which is precisely the constraint of brain MRI. The data augmentation method is used to enlarge the training dataset to improve synthetic image quality without adding new samples to the set. The actions such as translating, rotating, flipping, stretching, and shearing the existing images in the dataset can augment the set; however, these methods lack diversity in the newly generated image samples. The training can become affected towards suboptimal results [143]. Generative modeling maintains similar features to the actual data set while developing the dummy version of the existing images. Deep convolutional GAN generates dummy images using strided convolutions to carry out upsampling in place of max-pooling layers [144]. A timely examination of the brain’s status is crucial in preventing Parkinson’s disease (PD) and hindering its spread. Automatic diagnosis methods use either single-view or multi-view scans to execute the classification or prediction of PD. A WGAN operates on multi-view samples from the MRI dataset containing the cross-sectional view (AXI) and the longitudinal view (SAG). The prodromal class with fewer AXI/SAG MRI data samples causes the problem of over-fitting or under-fitting in an application. Two ResNet networks are trained jointly on the two-view data to create more samples for the prodromal class in AXI and SAG [145].

Class imbalance is a significant issue in abnormal tissue identification and classification in medical analysis. In the case of imbalanced data, a predominant class is filled with usual vital samples, while a subsidiary class is with ailing samples. When a model is trained with a dataset with visible disparity, it generates biased results towards healthy data giving rise to predictable outputs by the network and low sensitivities. The class distribution can be balanced by re-sampling the data space, similarly to oversampling of the predominant class and under-sampling of the subsidiary class, construction of a new compact dataset in an iterative sampling manner by bypassing unessential details, ensemble sampling, and hybrid sampling. A pair-wise GAN architecture uses a cross-modality input to increase heterogeneity in the augmented images. GAN-augmented images are utilized in the pre-training phase, and then real brain MRIs complete advanced training leading to synthetic MR images from one modality to another [146]. Brain tumors are segregated into meningioma, glioma, and pituitary tumors. The direct resemblance in the three classes results in a complex classification procedure in MRI images. A multi-scale gradient GAN (MSG-GAN) synthesizes MRI images with meningioma disease and uses transfer learning to improve classification performance [147]. Noise-to-image and image-to-image GANs enhance the data augmentation (DA) effect. Progressive growing of GAN (PGGAN) is a multistage noise-to-image GAN used for high-resolution MR image generation. Refinement methods such as multimodal unsupervised image-to-image translation (MUNIT) or SimGAN rectify the texture and shape of the images produced by PGGAN close to the originals [148]. A moderate-sized glioma dataset can affect the precise brain tumor categorization using several MRI modalities such as T1-weighted, T1-weighted with contrast-enhanced, T2-weighted, and FLAIR. Pair-wise, GAN trained on two input channels, unlike the normal GAN with only one input channel, augments the brain images to the compact dataset [149]. Two types of perfusion modalities, dynamic susceptibility contrast (DSC) and dynamic contrast-enhanced (DCE), are used to generate realistic relative cerebral blood volume (RCBV). The CGAN is trained on brain tumor perfusion images to learn DSC and DCE parameters with a single gadolinium-based contrast agent administration [150]. AGGrGAN in [151] is a collection of three base GAN models—two variants of deep convolutional GAN (DCGAN) and a WGAN that generate synthetic MRI images of brain tumors. The model uses the style transfer technique, selects distributed features across the multiple latent spaces, and captures the local patterns to enhance the image resemblance.

In a stroke disease, the brain cells start dying due to insufficient blood to the brain (cerebral ischemia) or moments of internal bleeding (intracranial hemorrhage). CGAN generator is trained on specially altered lesion masks to create synthetic brain images to enlarge the training dataset. CNN segmentation network includes depth-wise-convolution-based X-blocks and feature similarity module (FSM) [152]. IsoData (Iterative Self-Organizing Data Analysis Technique) is an unsupervised segregation method measuring the means of the classes uniformly dispersed within data space and clusters the rest of them iteratively based on the minimum distance methods. Every iteration computes the new mean and classifies pixels. The WGAN-based process depends upon the image histogram by generalizing more than two classes and splitting, merging, and deleting the class depending on the input threshold parameters [153]. Functional connectivity GAN (FC-GAN) generates the functional brain connectivity (FC) patterns obtained from fMRI data amplifying the efficiency of the neural network classifier. VAE and WGAN-based network contains three parts, the encoder, the generator, and the discriminator [154].

The connectome-based sample generation is another approach for data augmentation. A generative adversarial neural network auto-encoder (AAE) framework produces synthetic structural brain connectivity instances of MS patients even for an unbalanced dataset [155]. The number of samples in the regular fMRI datasets is insufficient for training. Multiple GAN architecture generates new multi-subject fMRI points. The multiple GAN architectures used in this method are cycleGAN, starGAN, and RadialGAN and do not need label details to determine the relation matrix. The cycle-GAN is not expandable to multiple domains because of the N (N-1) mappings to be learned for N domains. StarGAN is expandable to multiple domains using a single generator for multidomain translation tasks. RadialGAN can successfully extend the target dataset by employing multiple source datasets [156]. Dual-encoder BiGAN architecture duplicates abnormal samples within a normal distribution. Anomaly detection in BiGAN reduces bad cycle consistency loss due to insufficient sample data information [157]. The approach in [158] generates annotated diffusion-weighted images (DWIs) of brains showing an ischemic stroke (IS). Realistic DWIs are generated from axial slices of these 3D segmentation maps with the help of three generative models: Pix2Pix, SPADE, and cycleGAN.

3.2. Loss Functions (RQ 2)

The structure of losses is crucial in the supervised training of GAN models for generating quality images. The discriminator loss is an attribute of the images produced by the generator and possesses a high value when the discriminator is incapable of discriminating between source and dummy images. Similarly, generator loss is a function of the performance of the discriminator and has a high value when it cannot generate images close to the authentic images. Both networks’ productivity improves when the model’s training is executed sequentially. Then suitable weights must be identified for the network to create more realistic images [144]. Loss functions used in the SLR are listed in

Table 8. Summary of loss functions used in applications of GAN-synthesized brain MRI.

Loss Function	Description	Probability— Based (Yes/No)	Ref. No.
Commonly used loss functions
Adversarial loss	The adversarial loss function is created in the repeated production and classification cycle. The generator minimizes the loss function, and the discriminator maximizes it. ${\mathcal{L}}_{GAN}$ (G,D) = Ex,y [log (D(x,y)] + Ex,y [log(1 − D(x,G(x,z))] where y is the ground truth image, G is the generator network, G (x,z) is generated image, D is the discriminator network.	Yes	[20,25,35,41,45,53,54,55,57,59,67,72,74,77,81,83]
Cycle consistency loss	A cycle consistency loss allows the generator to learn a one-to-one mapping from the input image field to the target image field. ${\mathcal{L}}_{cyc}^{\varkappa }\left(\mathrm{G}\right)=E_{x~Pdata\left(x\right)}$$[\Vert \mathrm{G}\left(\mathrm{G}\left(x\right)\right)-x{\left.|\right|}_1]$	Yes	[24,28,45,67,71,74,81,82,88]
L1 loss	The L1 loss also called mean absolute error (MAE), is a pixel-wise error that shows over-smoothing in resultant images. ${\mathcal{L}}_{L1}(\mathrm{G})=\frac{1}{n} [{\Vert y-\mathrm{G}(xz)\Vert }_1]$ where n is the number of voxels in an image, ||.||1 is the sum of voxel-wise residuals	No	[19,20,21,23,25,26,33,34,35,37,52,55]
L2 loss	The L2 loss also called Mean Squared Distance (MSD) indicates the error between generated and original images and gives faint images.${\mathcal{L}}_{L2}(\mathrm{G})=\frac{1}{n} [{\Vert y-\mathrm{G}(xz)\Vert }_2]$ where ||.||2 is the sum of squared voxel-wise residuals of intensity value.	No	[50,51,68,69,80]
Perceptual loss	Pixel-reconstruction losses give blurry effects in the final outputs and cannot express the image’s perceptual quality. The perceptual loss is the Euclidean distance in feature space to extract semantic features from target images. ${\mathcal{L}}_{Perceptual}(\mathrm{G})=\frac{1}{whd}{\Vert \varnothing \left(\mathrm{G}\left(x\right)\right)-\varnothing \left(y\right)\Vert }_F^2$ where ∅ is a feature extractor, and w, h, and d represent the dimensions of feature maps.	No	[20,35,37,52,53,55,56,66,67,92]
Wasserstein loss	WGAN evaluates the Earth Mover’s distance by training the discriminator network and is bounded by a Lipschitz constraint. ${\mathcal{L}}_{WGAN}(\mathrm{D})=-E_{y~Pr}[\mathrm{D}(\mathrm{y})]+E_{x~Pn}[\mathrm{D}(\mathrm{G}(\mathrm{x}))]+\mathsf{\lambda }{E}_{\widehat{x}~P\widehat{x}} [{({\Vert {\nabla }_{\widehat{x}} \mathrm{D}\left(\widehat{x}\right)\Vert }_2-1)}^2]$ where x is sampled from real image ‘r’ and noise ‘n’ is the hyperparameter.	Yes	[29,48,49,56,86,92,93,103]
Other loss functions
Attention regularization loss	It ensures learning orthogonal attention maps.	No	[77]
Binary cross entropy (BCE) loss	The negative of the logarithm function is used for predicting the probability during binary classification.	Yes	[34,42,43,50,58,65,69]
Classification loss	It is the average cross-entropy value and the discriminator’s logistic sigmoid result.	Yes	[58,102]
Cycle-perceptual loss	This loss captures the high-level perceptual errors between original and dummy images.	No	[142]
Fidelity loss	The fidelity loss factor indicates the dissimilarity between the fake and the spatial normalized image and is generally added to the discriminator loss function.	No	[115,116]
Gradient difference (GD) loss	The GD loss is the gradient difference between the original and dummy images that retain the sharpness in the synthetic images.	No	[86,88,89]
Identity Loss	This loss is responsible for colors and intensities conservation.	Yes	[77]
Image alignment loss	It is based on normalized mutual information (NMI) and used for information fusion.	Yes	[82]
Mean p distance (MPD)	The lp-norm or mean p distance (MPD) measures the distance between synthetic and original images.	No	[25,27]
Mutual information loss	Mutual Information (MI) finds the “information content” in one variable when another variable is fully observed and used as the loss function.	Yes	[16,17,103]
Multi-scale L1 loss	Multi-scale features variance between the predicted multi-channel probability map and the actual image.	No	[42,43]
Registration loss	This loss penalizes the variance between the translated & transformed image and stimulates local smoothness.	No	[41]
Self-adaptive Charbonnier loss	It is the pixel-wise differences between real and fake images.	No	[140]
Style-transfer loss	Style-transfer loss enhances the texture and fine structure of the desired target images.	Yes	[140,142]
Supervision loss	This loss, denoted by cumulative squared error, measures pixel shifts between original and synthetic images.	No	[41]
Symmetry loss	It stresses inverse consistency in the predicted transformations.	No	[41]
Synthetic consistency loss	This loss balances the mean absolute error (MAE) and gradient difference (GD), indicating how the generated image lags behind the target image.	No	[72]
Voxel-wise loss	This loss can be imposed as a pixel-level penalty between the translated and the original image applicable to only paired datasets.	Yes	[66,77,83]

. Adversarial loss, cycle consistency loss, L1 loss, L2 loss, perceptual loss, and WGAN loss are the primary loss functions and predominantly used in SLR. Figure 8 shows the distribution of these commonly used loss functions in the SLR.

3.3. Preprocessing of Ground Truth Brain MRI (RQ 3)

The training dataset for GAN contains brain MRIs that act as ground truth images to generate GAN-synthesized images. Preprocessing operations on these ground truth brain MRIs are crucial for the fidelity of the successive GAN performance. These tasks sharpens the image and removes the noise to enhance the qualitative and quantitative estimation [159]. Compared to machine learning techniques, the preprocessing is not exhaustive in the case of DL; still, the input image needs to undergo some treatments before feeding to the neural networks. Some regular image preprocessing steps [18,150] are discussed below:

• Intensity Normalization:

MRIs from multiple centers acquired with scanners from distinct vendors and magnetic strengths show discrepancies in brightness and some induced noise [160]. To lessen these effects, intensity normalization is implemented by measuring the MR image’s intensity values variability [154]. The non-uniform voxel intensities of all volumes are standardized, and then each volume is normalized to obtain the zero mean and unit standard deviation. The patient-wise batch normalization can control the overfitting. Each patient scan is normalized by dividing each sequence by its mean intensity value. The task ensures that the distribution of intensity values is preserved [149].

• Skull Stripping:

Skull stripping is removing the skull from images to focus on intracranial tissues [95].

• Registration:

All input images are registered to the same imaging space during the registration phase. Registration is the spatial alignment of the images to a common anatomical space [46]. Various datasets come with different statuses of registration among training images. For example, MIDAS and IXI datasets carry unregistered images, while BRATS dataset carries already registered images [161].

• Bias Field Correction:

This operation corrects the image contrast variations due to diverse magnetic field strengths [95].

• Center Cropping:

In this operation, outer parts of each brain image volume The outer parts of each brain image volume are removed, reserving the central region along each dimension [88].

• Data Augmentation:

Data augmentation is achieved through operations such as translation, flipping, re-sizing, scaling, rotation between −10 and 10 degrees, and Gaussian noise application [87,98].

• Motion Correction:

Keeping the patient’s head stationary in the scanner during the MRI acquisition is challenging, especially for small and elderly age groups. This operation is performed to reduce the noise that arises in the scan due to subject motion.

The number and order of these steps may vary from case to case, depending on the application requirement. Figure 9 shows the percentage-wise use of these preprocessing operations in the SLR. Preprocessing software packages are available to perform above listed tasks (

Table 9. Summary of preprocessing software packages commonly used for brain MRI.

Preprocessing Software	URL	Use	Ref. No.
Freesurfer	http://surfer.nmr.mgh.harvard.edu (accessed on 26 August 2022)	Skull-stripping, Registration, fMRI Analysis	[114,162]
Functional magnetic resonance imaging of the Brain Software Library (FSL)	http://fsl.fmrib.ox.ac.uk/ (accessed on 26 August 2022)	Registration, alignment, Skull-stripping	[115,162]
Advanced Normalization Tool (ANT)	http://stnava.github.io/ANTs/ (accessed on 26 August 2022)	Registration	[5,49,136,163,164]
Statistical Parameter Mapping (SPM)	http://www.fil.ion.ucl.ac.uk/spm (accessed on 26 August 2022)	Skull-stripping	[74,121]
Velocity (Varian)	https://www.varian.com/ (accessed on 26 August 2022)	Registration	[25,27,165]
Data Processing Assistant for Resting-State fMRI (DPARSF)	http://www.restfmri.net (accessed on 26 August 2022)	Data processing of fMRI	[166]
Elastix	https://elastix.lumc.nl/ (accessed on 26 August 2022)	Registration	[68,167]
BrainSuite	http://brainsuite.org/ (accessed on 26 August 2022)	Skull-stripping	[162]

). The format conversion is applied to scanned MRI images. The source images are in digital imaging and communications in medicine (DICOM) format, which are converted to 3D images in neuroimaging informatics technology initiative (NIFTI) format, then sliced in the axial direction, and saved as joint photographic expert group (JPG) or portable network graphics PNG format (Figure 10).

3.4. Comparative Study of Evaluation Metric (RQ 4)

The success of a GAN depends upon the quality of the synthesized image. The evaluation metric that can quantitatively characterize the synthesis accuracy is necessary to comment on the quality of the synthetic image. There are two types of metrics; the first is Full-Reference (FR) Quality Metrics, where the quality of a synthetic image is measured to the quality of a ground-truth image. The second is No-Reference (NR) Quality Metrics, where the quality scores of the synthetic image are based on expected image statistics. Two commonly used evaluation metrics are discussed below.

Table 10. Comparative study of evaluation metrics used in this SLR.

Evaluation Metric	FR/NR	Description	Assessment Method	Ref. No.
Average symmetric surface distance (ASSD)	FR	ASSD measures the average of all Euclidean distances between two image volumes.	Segmented image	[53,77,168]
Blind/ Reference-less Image Spatial Quality Evaluator (BRISQUE)	NR	BRISQUE focuses natural scene statistics (NSS) such as ringing, blur, and blocking. It quantifies the reduction of naturalness by locally normalizing the luminance coefficients.	Whole image	[33,58,86]
Dice Similarity Coefficient (DSC)	FR	DSC measures the spatial overlap and provides a reproducibility validation score for image segmentation.	Segmented image	[70,77,79,83,96,99,144]
Frechet Inception Distance (FID)	FR	The distance between Gaussian distributions of synthetic and real images is FID or the Wasserstein-2 distance.	Whole image	[35,59,84,99,120]
Hausdorff Distance (HD)95	FR	HD measures the maximum Euclidean distance between all surface points of two image volumes.	Segmented image	[115]
Jaccard similarity coefficient (JSC)	FR	It is a value used to compare the similarity and diversity of images recognized as Intersection over Union.	Segmented image	[98,99,159]
Maximum Mean Discrepancy (MMD)	FR	MMD measures the dissimilarity between the probability distribution of real images over the space of natural images and parameterized distribution of the generated images.	Whole image	[32]
Mutual Information Distance (MID)	FR	MID measures the association between corresponding synthetic images in different modalities. It first evaluates the mutual information of synthetic image pairs and real image pairs and then computes their absolute difference.	Whole image	[84,99]
Normalized Mean Absolute Error (NMAE)	FR	NMAE measures the estimation errors of a specific color component between the original and synthetic images.	Whole image	[33,74,88]
Normalized Mutual Information (NMI)	NR	NMI expresses the amount of information synthetic images carry regarding the original image.	Whole image	[140,148]
Normalized Cross-Correlation (NCC)	FR	NCC evaluates the degree to which the synthetic and original image signals are similar. It is an elementary approach to match two image patch positions.	Segmented image	[25]
Naturalness Image Quality Evaluator (NIQE)	NR	NIQE is a distance-based measure of natural images’ divergence from statistical consistency. The metric quantifies image quality according to the level of distortions.	Whole image	[33,41,58,88]
Peak Signal-to-Noise Ratio (PSNR)	FR	It is an expression for the ratio between the maximum possible power of the original image and the power of the generated image.	Whole image	[48,56,59,73,74,75,85,88]
Structural Similarity Index Measure (SSIM)	FR	SSIM score indicates the perceptual difference between original and synthetic images. It compares the visible structures in the image such as Luminance, Contrast, and Structure.	Whole image	[38,59,73,74,75,88]
Root-Mean-Square Error (RMSE)	FR	It measures the differences between the predicted value by an estimator and the actual value of a definite variable.	Whole image	[68,72,75,99,140]
Universal Quality Index (UQI)	FR	Image distortion is the product of loss of correlation, luminance, and contrast distortion.	Whole image	[14]

presents the comparative study of existing evaluation metrics. Some of the metrics from the table assess the quality of the synthesized image as a whole, while others asses the image on patch-wise and are thus suitable for the segmentation task [69].

4. Discussion

A substantial amount of data samples is the primary requirement to implement deep learning algorithms. However, procuring adequate data is always a challenge in medical applications. This fact can restrict the wide and acceptable use of DL methods. The GAN models provide a reasonable solution to this problem. A substantial amount of data samples is the primary requirement to implement deep learning algorithms. The architecture of traditional GAN consists of two neural networks, the generator and the discriminator, working in tandem to create a fake source version. The generator updates its creation based on the feedback of the error function from the discriminator. The architecture utilizes Unsupervised learning to produce an image from the features of the real image. This SLR shows that GAN-synthesized images have potential use in data augmentation, yet they have also been immensely used for other image applications. In addition, GAN can speed up the whole analysis step in radiology [11]. This SLR includes papers published in Q1or Q2 journals and conferences in the Web of Science and Scopus databases. Future researchers can confirm the findings by exploring some other databases as well. However, to the best of our belief, the main conclusions drawn in this study would not change significantly. Some significant findings of the review are listed here:

4.1. GAN Variants

The conventional GAN model suffers from mode collapse, where the optimal discriminator does not provide enough information for the generator, leading to poor image generation by the generator. Many modifications are suggested and implemented in the basic GAN structure. The most popular models, such as CGAN, WGAN, cycleGAN, starGAN, and SRGAN are more robust and are used in image synthesis, translation, and super-resolution applications.

IN CGAN, a condition is set for the generator and the discriminator that acts as the controlling mode, giving a better presentation for image synthesis [163,164]. WGAN [164] provides stable training in conventional GAN by solving the network coverage problem and accelerating the training speed. The loss function is the distance between the two probability distributions known as the Wasserstein distance or Earth Mover’s (EM distance) [11]. CycleGAN is suitable for unpaired image translation and training stabilization. It has two generator networks mastering the two different mappings- source to target and target to source. It also has two discriminators that differentiate the synthesized image from the original image belonging to each domain. Evaluation of the resultant image is based on cycle-consistent losses to find the similarity between synthetic and original images [165]. StarGAN is suitable for cross-domain image translation using a single model. Multiple generators are not required to produce images from different domains. The mask vector method allows image translation between multiple datasets [87]. Figure 11 shows major GAN variants and their extensions.

4.2. Multimodal Image Generation

MR images often have several complementary modalities regarding anatomical information. In the last two years, the trend of research is more towards information extraction from multimodality images for better diagnosis. CycleGAN works on unpaired images, and Pix2pixGAN is suitable for paired image datasets. However, both architectures can only generate images from one modality to another and fail for multiple domains sample generation. Since it can learn only 1-to-1 mapping between two domains, N (N-1) structures would require finishing the learning for N domains. StarGAN provides a better solution for a single network working on unpaired multi-domain datasets. It presents domain labels to cycleGAN and selects the desired domain through a mask vector during translation [98,114]. RadialGAN is another technique suitable for multiple source datasets, works well for data augmentation when the data labels are continuous, and has a common latent space for all domains [166]. To study the information available in various domains, CollaGAN suggests a collaborative model including the details of multiple domains to generate images from an absent domain. CollaGAN secures the features in synthetic images through cycle consistency, similarly to starGAN. It executes n-to-1 translation using a one-hot mask vector in input that defines the target domain. Each N domain incorporates style and content code for shared latent representation [96].

5. Conclusions

AI researchers have adopted GAN for various image applications in recent years. Medical imaging is one prominent area where GAN-synthesized images can be helpful in multiple ways. This SLR presents a comprehensive study of applications of GAN-synthesized images for brain MRI. Over the years, the architecture of conventional GAN has been modified, and its versions, such as CGAN, WGAN, cycleGAN, and starGAN, show promising results in classifying and predicting brain diseases. It is speculated that GAN handles the scarce dataset problem of medical images in the best way possible. GAN-synthesized images have moved further from their obvious application in data augmentation. These images are being explored for image translation, registration, super-resolution, contrast enhancement, denoising, segmentation, reconstruction, and motion correction. Though many tasks are achievable with GAN implementation, there are still challenges in adopting GAN in real time. More works need to be performed regarding stability in GAN training, 3D GAN, and unsupervised learning modes.