Generative Adversarial Networks Capabilities for Super-Resolution Reconstruction of Weather Radar Echo Images

Abstract

Improving the resolution of degraded radar echo images of weather radar systems can aid severe weather forecasting and disaster prevention. Previous approaches to this problem include classical super-resolution (SR) algorithms such as iterative back-projection (IBP) and a recent nonlocal self-similarity sparse representation (NSSR) that exploits the data redundancy of radar echo data, etc. However, since radar echoes tend to have rich edge information and contour textures, the textural detail in the reconstructed echoes of traditional approaches is typically absent. Inspired by the recent advances of faster and deeper neural networks, especially the generative adversarial networks (GAN), which are capable of pushing SR solutions to the natural image manifold, we propose using GAN to tackle the problem of weather radar echo super-resolution to achieve better reconstruction performance (measured in peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM)). Using authentic weather radar echo data, we present the experimental results and compare its reconstruction performance with the above-mentioned methods. The experimental results showed that the GAN-based method is capable of generating perceptually superior solutions while achieving higher PSNR/SSIM results.

1. Introduction

Severe weather events can cause serious environmental, social, and economic damages. Data from China’s first national climate change adaptation strategy issued in 2013 show that extreme weather events have killed more than 2000 people each year on average since the 1990s. They also resulted in more than 200 billion yuan ($32 billion) in direct economic damage annually [1]. The new generation of the Doppler weather radar system, which is capable of detecting the motion of rain droplets and the intensity of precipitation, plays an important part in locating and determining the type of severe convective weather events. Certain severe mesoscale convective weather systems emerge abruptly and have short durations. They tend to be hard to notice in the early stages of development and can develop rapidly in the middle and late stages. For example, thunderstorms often produce short-lived, small-scale hazardous weather events, including hail, damaging winds, and tornadoes that last only a few minutes. This results in fewer valid data in the distance direction due to radar resolution limitations, and hence echo images with limited resolution. Also, due to certain constraints, it’s sometimes difficult to obtain the radar base data of weather events; only degraded low resolution radar echo images are available. Enhancing the resolution of weather radar echo images can lead to better perceptual quality for human viewers, more detailed analysis about the meteorological targets in question, better forecasting performance of echo extrapolation models, and better support for extreme weather forecasting, which are all conducive to disaster prevention and mitigation.

Peleg et al. [2] studied and contributed to the understanding of the relationship between air temperature and convection by analyzing the characteristics of rainfall at the storm and convective rain cell scale using high spatial-temporal resolution (1 km, 5 min) weather radar data estimates from a uniquely long weather radar record (24 years). Smith et al. [3] introduced a long-term, high-resolution radar rainfall data set for the Baltimore metropolitan area covering 2000–2009, and utilized it to characterize spatial heterogeneities in rainfall for the Baltimore metropolitan region, both in terms of mean rainfall and rainfall extremes. The study found that high-resolution rainfall fields are especially useful for examining the distribution of rainfall from a drainage basin perspective. Fries et al. [4] used high-resolution radar images and ground station data to provide high-resolution precipitation maps required by the precise estimation of precipitation quantities in tropical mountain regions. These studies suggest that weather radar data of higher spatial-temporal resolution contains more detailed information about the atmospheric motion and meteorological targets that allow for more timely forecasts and an in-depth analysis of weather events.

Thus, in the literature, various super-resolution methods have been proposed. Resolution can be improved by newer radar hardware facilities (e.g., larger antenna, denser networks) or, faced with the constraints imposed by existing systems, by implementing a different azimuthal sampling strategy in conjunction with a narrower antenna pattern [5]. Meanwhile, there are research studies focused on radar base data super-resolution without changing the radar hardware or sampling strategy. A minimum entropy spectrum extrapolation technique for radar super-resolution proposed by Yao et al. [6] is one of the earlier methods. Nielsen et al. [7] proposed a numerical method to generate a high temporal resolution precipitation time series by combining weather radar measurements with a nowcast model. The proposed interpolation method performs better than a traditional interpolation of weather radar rainfall where the radar observation is considered constant in time between measurements. Gallardo-Hernando et al. [8] proposed super-resolution techniques based on auto-regressive coefficients for wind turbine clutter spectrum enhancement in meteorological radars. Li et al. [9,10] proposed a two-dimensional deconvolution technique on oversampled reflectivity data to simultaneously improve range and angular resolution, whose experimental results were shown to be efficient for range and angular resolution enhancement of reflectivity data. However, deconvolution is an ill-posed problem, of which the solution is not only sensitive to noise, but would also easily deteriorate by the noise amplification when excessive iterations are conducted. Tan et al. [11] proposed a penalized maximum likelihood angular super-resolution method to tackle the above-mentioned problems of deconvolution. The experimental results demonstrate the effectiveness and superior performance of the proposed method. Li et al. [12] also proposed a new super-resolution model based on the idea of sub-division in one resolution volume for geostationary weather radar and an oversampling technique along the radar’s spiral scan track. The experimental results show that the proposed model and reconstruction process are efficient for the horizontal resolution improvement of reflectivity data, and more refined details could be present through reconstruction. Zha et al. [13] proposed a novel method for angular super-resolution imaging in scanning radar using the alternating direction method for solving the constrained optimization problem, and the simulation results showed that it outperformed a number of existing deconvolution algorithms in terms of stability and precision. Wu et al. [14] proposed a novel angular super-resolution approach for scanning radar using truncated singular value decomposition (TSVD) with the least squares optimization technique. The experimental results demonstrate that the proposed method can improve the azimuth resolution without noise amplification and loss of edge information. He et al. [15] proposed an improved iterative back-projection algorithm to improve data resolution based on a sliding window reconstruction model using the temporal correlation constraint. Zeng et al. [16] analyzed the sparsity and data redundancy of weather radar data and studied its temporal and spatial correlation and proposed a compression scheme based on prediction coding, providing a theoretical basis for using data correlation for weather radar echo super-resolution. Zhang et al. [17] proposed a novel nonlocal self-similarity sparse representation (NSSR) model for weather radar echo super-resolution that exploits the sparse data composition and data redundancy of weather radar echo data. The experimental results demonstrated that the proposed NSSR outperformed current general-purpose radar echo super-resolution methods.

However, for the hardware-based methods, a large-scale upgrade of radar hardware facilities to radar stations for obtaining super-resolution radar data might not be a viable option due to time and money constraints, as well as other factors. Also, traditional radar echo super-resolution methods may have problems adapting to new data or suffer from a long iteration time [17]. For challenging degraded echo images, the textural detail in the reconstructed echoes of traditional approaches is typically absent, resulting in unsatisfying super-resolution (SR) solutions. Furthermore, some current traditional [18,19,20] or deep learning-based [21,22] weather radar echo extrapolation methods and forecasting models were primarily based on weather radar echo maps, which are often constant altitude plan position indicator (CAPPI) images. As the prediction lead time increases, the radar echo extrapolated from these models becomes increasingly blurred and deformed. Therefore, more robust and powerful radar echo super-resolution methods are called for.

In recent years, deep learning techniques have seen rapid developments, which are known to surpass traditional methods at various challenging tasks such as image classification and spatial time-series (e.g., weather data) prediction. Shi et al. [21] proposed a convolutional Long short-term memory (ConvLSTM) network and used it to build an end-to-end trainable model for the precipitation nowcasting problem that outperforms traditional methods. Krinitskiy et al. [23] developed a novel approach for the detection and classification of Polar mesocyclones based on the use of deep convolutional neural networks (DCNNs). Booz et al. [24] proposed a deep learning-based weather forecast system and analyzed the relationship between the prediction accuracy and data volume, as well as data recency. These studies demonstrate deep learning’s superior performance in learning the complex inherent structures and patterns of weather data.

Deep learning-based super-resolution models have also been actively explored and often achieve the state-of-the-art performance on various SR benchmarks [25]. Various deep learning-based SR methods have been proposed, ranging from the early convolutional neural networks (CNN)-based method (e.g., SRCNN [26,27]) to the recent generative adversarial nets (GAN) [28] based approach (e.g., SRGAN [29]), which is capable of generating realistic textures during single image super-resolution. However, very few previous studies have examined the weather radar echo super-resolution problem from the deep learning perspective. Therefore, it’s interesting to explore how these deep learning SR models perform on weather radar level-II data products without prior knowledge compared with traditional super-resolution methods.

In this paper, we propose a GAN-based weather radar super-resolution method to tackle the challenging task of super-resolution reconstruction of weather radar echo images. Here, our focus is on the capability of GANs to generate perceptually superior solutions of weather radar echo super-resolution compared with the classical improved iterative back-projection (IBP) algorithms [15] and a nonlocal self-similarity sparse representation (NSSR) [17] method that utilizes the data redundancy of radar data.

The rest of this paper is organized as follows: Section 2 presents the problem definitions of weather radar echo super-resolution reconstruction and an overview of generative adversarial networks (GAN) in the context of radar echo super-resolution. We also provide a brief introduction to echo image quality assessment. Section 3 describes the proposed weather radar super-resolution method based on generative adversarial networks. Section 4 details the experiment and presents the experimental results. Section 5 concludes this paper.

2. Radar Echo Super-Resolution Reconstruction

2.1. Problem Definitions

In real-life systems, radar echo maps are often constant altitude plan position indicator (CAPPI) images. Due to certain constraints mentioned in Section 1, it’s sometimes difficult to obtain the radar base data of weather events [4]; instead, only degraded low resolution radar echo maps are available, or the higher quality echo images have been lost. Weather radar echo super-resolution is then defined as the task of recovering high-resolution (HR) echoes from low-resolution (LR) echoes. Note that this definition of super-resolution, borrowed from image processing paradigms and performed on the image level of moment radar base data, is different from the process of improving the spatial-temporal resolution of radar base data by upgrading radar hardware facilities or changing the sampling strategy [5]. Generally, the LR echo can be modeled as the output of a degradation process:

$$g\left(m,n\right)=\mathsf{\Psi }\left(f\left(x,y\right);\delta \right)$$

(1)

Here, $g$ is an observed LR echo image of size $m\times n$, $f$ is the corresponding HR echo image of size $x\times y$, $\mathsf{\Psi }$ represents a degradation mapping function, and $\delta$ denotes the parameters of the degradation process. The degradation process of weather radar echo can be modeled with disruptive factors such as blurring, deformation, shifting, noise, etc., which result in a low-resolution imaging model, as shown in Figure 1.

The inclusion of these factors in the model of Equation (1) results in:

$$g\left(m,n\right)=d\left(h\left(s\left(f\left(x,y\right)\right)\right)\right)+\eta \left(m,n\right)$$

(2)

where $s$ is a shifting function, $h$ is a blurring function, $d$ is a down-sampling operator, and $\eta$ is an additive noise. In matrix form, this can be rewritten as:

$$g=Af+\eta$$

(3)

in which $A$ stands for all the above-mentioned degradation factors. This imaging model has been used in many SR works [30]. The reconstruction process of LR echoes is the reverse process of the LR imaging model, as shown in Figure 2.

The blurring function models any blurring effect that is imposed on the LR observed echo, such as for example that introduced by the scanning of radar antennas. The shifting function changes depending on the type of motion between the HR echo and the LR observations. Some classical works mostly focus on the discussion of the motion prediction and blur estimation [30]. However, under general conditions, the degradation process (i.e., $\mathsf{\Psi }$ and $\delta$ in Equation (1)) is unknown and ill-posed; sometimes, only LR echoes are provided. In this case, researchers are required to recover HR echo from the corresponding LR echo, so that $f_r$ is nearly identical to the ground truth HR echo $f$, following the process:

$$f_r(x,y)=R(g(m,n);\theta )$$

(4)

where $R$ is the super-resolution reconstruction model, and $\theta$ represents the parameters of $R$. To this end, the objective of weather radar echo super-resolution is as follows:

$$\widehat{\theta }=\underset{\theta }{\arg \min }L(f_r,f)+\lambda \mathsf{\Phi }(\theta )$$

(5)

where $L(f_r,f)$ represents the loss function between the generated HR echo $f_r$ and the ground truth echo $f$; $\mathsf{\Phi }(\theta )$ is the regularization term, and $\lambda$ is its tuning hyper-parameter. Although the most popular and default choice of loss function for SR is pixel-wise mean squared error (i.e., pixel loss), more powerful models tend to use a combination of multiple loss functions, which will be covered in Section 2.2.

2.2. GAN for Radar Echo Super-Resoultion Reconstruction

Recent advances in deep learning, especially generative adversarial networks (GAN) [28], provide some useful inspirations on how to tackle the problem formulated in Section 2.1. Due to the generative nature of GAN, it provides a powerful framework for generating plausible-looking natural images with high perceptual quality. It encourages the reconstructions to move toward regions of the search space with a high probability of containing echoes that are closer to the HR echo manifold [29]. Also, according to the philosophy of deep learning approaches, if we have a reasonable end-to-end model and sufficient data for training, we are able to train a powerful model that is capable of generating high-quality reconstruction solutions. The weather radar echo super-resolution problem satisfies the data requirement because it’s easy to collect a huge amount of radar echo data continuously.

Generative adversarial networks (GANs), as proposed by Goodfellow et al. [28], are deep neural net architectures comprised of two nets, competing against one another (thus, the “adversarial”). Two models are simultaneously trained: a generative model $G$ that captures the data distribution, generating new data instances, and a discriminative model $D$ that evaluates them for authenticity, i.e., estimates the probability that a sample came from the training data rather than $G$. The training procedure for $G$ is to maximize the probability of $D$ making a mistake. In the original work [28], $D$ and $G$ play the following two-player minimum–maximum (min–max) game with the value function $V(D,G)$:

$$\underset{G}{\min } \underset{D}{\max }V(D,G)=E_{x\sim p_{data}(x)}[\log D(x)]+E_{z\sim p_z(z)}[\log (1-D(G(z)))]$$

(6)

where $p_{data}(x)$ is the learned distribution over data $x$, and $p_z(z)$ is a prior on input noise. This model provides a powerful framework for generating plausible-looking natural images with high perceptual quality. In the context of radar echo super-resolution, a discriminator network $D_{{\theta }_D}$ is defined and optimized in an alternating manner along with a generator network $G_{{\theta }_G}$ to solve the adversarial min–max problem:

$$\underset{{\theta }_G}{\min } \underset{{\theta }_D}{\max }{E}_{f\sim p_{train}(f)}[\log D_{{\theta }_D}(f)]+E_{f_r\sim p_G(f_r)}[\log (1-D_{{\theta }_D}(G_{{\theta }_G}(f_r)))]$$

(7)

The general goal is to train a generative model $G$ to fool a differentiable discriminator $D$ that is trained to distinguish super-resolved echoes from real echoes. With this approach, the generator can learn to create solutions that are highly similar to real echoes, and thus difficult to classify by $D$. This encourages perceptually superior solutions residing in the subspace and manifold of natural echo images [29]. This is in contrast to SR solutions obtained by minimizing pixel-wise error measurements, such as the mean squared error (MSE) [27], which while achieving particularly high PSNR, often lack high-frequency content, which results in perceptually unsatisfying solutions with overly smooth textures.

The definition of loss function $l^{SR}$ for evaluating the quality of generated radar echoes compared with ground-truth echoes in training is critical for the performance of a generator network. While $l^{SR}$ is commonly modeled based on the MSE, a perceptual loss, proposed by Johnson et al. [31] and improved in [29], was designed to assess a solution with respect to perceptually relevant characteristics. The perceptual loss was formulated as the weighted sum of a content loss ($l_x^{SR}$) and an adversarial loss component ($l_{Gen}^{SR}$):

$$l^{SR}=l_x^{SR}+{10}^{-3}{l}_{Gen}^{SR}$$

(8)

possible choices for the content loss $l_x^{SR}$ include pixel-wise MSE loss [31,32] and Visual Geometry Group (VGG) loss [29,33] that is closer to perceptual similarity. The adversarial loss $l_{Gen}^{SR}$ is defined based on the probabilities of the discriminator $D_{{\theta }_D}(G_{{\theta }_G}(f))$ over all the training samples as:

$$l_{Gen}^{SR}={\displaystyle \sum _{n=1}^N-\log D_{{\theta }_D}}(G_{{\theta }_G}(f))$$

(9)

where $D_{{\theta }_D}(G_{{\theta }_G}(f))$ is the probability that the reconstructed echo $G_{{\theta }_G}(f)$ is a natural-looking HR echo.

2.3. Radar Echo Image Quality Assessment

The process of determining image quality is termed image quality assessment (IQA). In general, IQA methods include subjective methods based on the human observer’s perceptual evaluation such as mean opinion score (MOS) and objective methods based on computational models (e.g., peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM)). The subjective methods are close to human perception, but usually inconvenient and expensive; so, the objective methods are the most widely used in SR works, even though they are sometimes unable to capture human visual perception very accurately [29].

Peak signal-to-noise ratio (PSNR) is commonly used to measure the reconstruction quality of lossy transformation (e.g., image compression). For radar echo super-resolution, PSNR is defined via the maximum possible pixel value (denoted as L) and the mean squared error (MSE) between HR–LR echo pairs. Given the ground truth echo $f$ and reconstructed echo $f_r$, both of which have N × N pixels, the MSE and the PSNR (in dB) between them are defined as follows:

$$MSE=\frac{1}{N}{\displaystyle \sum _{i=1}^N(f(i)-f_r}(i){)}^2$$

(10)

$$PSNR=10{\log }_{10}(\frac{L^2}{MSE})$$

(11)

In general cases using 8-bit image representations, $L$ equals 255, and the typical values for the PSNR vary from 20 to 40, where higher is better. When $L$ is fixed, the PSNR correlates only to the pixel-level MSE between echoes, affected only by the difference between the pixel values at the same position. This leads to PSNR’s poor performance in representing the quality of the super-resolved echoes.

The human visual system (HVS) is highly adapted to extract structural information from the viewing field [34]. Then, the structural similarity index (SSIM) [35] is proposed for measuring the structural similarity between images, based on three components of an image: luminance, contrast, and structure. It’s calculated on various sliding windows of an image. The SSIM index between two original HR echo $f$ and its reconstructed counterpart $f_r$ of common size N × N is defined as:

$$SSIM(f,f_r)=\frac{(2{\mu }_f{\mu }_{f_r}+c_1)(2{\sigma }_{{ff}_r}+c_2)}{({\mu }_f^2+{\mu }_{f_r}^2+c_1)({\sigma }_f^2+{\sigma }_{f_r}^2+c_2)}$$

(12)

where $\mu$ and $\sigma$ are the mean and the standard deviation of the image intensity, estimating luminance and contrast; ${\sigma }_{f{f}_r}$ is the covariance of $f$ and $f_r$, measuring structural similarity; $c_1={(k_{1}L)}^2$ and $c_2={(k_{2}L)}^2$ are two variables stabilizing the division with a weak denominator with $k_1\ll 1$, $k_2\ll 1$, and $L$ being the dynamic range of the pixel values.

The resultant SSIM index is a decimal value between −1 and 1, with value 1 achievable in the case of two identical sets of data, which indicates perfect structural similarity. A value of 0 indicates no structural similarity. Since the SSIM evaluates the reconstruction quality from the perspective of the HVS, it better suits the requirements of perceptual assessment [36], and is also widely used in SR models.

Radar echo data are sampled at discrete angles along each angle of elevation, and hence are only an approximation of reality [37]. For some weather forecasting models, the visual representation of the radar echo image is insignificant. This might raise concern about the validity of image processing-based evaluation metrics on weather radar echo images. However, as mentioned in Section 1, some current weather radar echo extrapolation methods were primarily based on weather radar echo maps, which are often constant altitude plan position indicator (CAPPI) images. Several verification techniques were proposed in the literature to characterize the forecast performance of these models (e.g., Gilbert skill score, variograms, etc.). However, no single verification technique gives a complete picture of the forecast performance, and assessing the reconstructed performance of echo images in terms of these metrics is out of scope of this study. Due to the necessity of comparing the performance of reconstruction with literature works and the lack of appropriate forecasting performance-oriented evaluation metrics for reconstructed echo image quality, image processing-based evaluation methods were used in this study. Since objective methods are currently the most widely used evaluation criteria for SR models [25], we use PSNR and SSIM in our assessment of the GAN model for weather radar echo super-resolution.

3. Generative Adversarial Network of the Proposed Method

We propose using a generative adversarial network (GAN) for the problem of weather radar echo super-resolution (Figure 3). We employ the basic architecture of SRGAN [29], which is a seminal work that is capable of generating realistic textures during single image super-resolution, since radar echo tend to have rich edge information and contour textures. All batch normalization (BN) [38] layers in the generator network were removed, and a new deep residual-in-residual dense block (RRDB) structure proposed in ESRGAN (enhanced SRGAN) [39] was added. BN layers were found to introduce unpleasant artifacts and limit the generalization ability when the statistics of HR and LR data differ a lot [39]. Removing BN layers has empirically proven to increase performance while reducing computational complexity and memory [40]. The design of a deeper and more complex structure RRDB, allowing for residual connection across different architectural layers of the network, is based on the observation that more layers and connections could always enhance performance [41].

The discriminator network that discriminates real HR echoes from generated SR solutions was trained to solve the maximization problem in Equation (7). It was designed following the architectural guidelines in [43]. The discriminator network output was also improved based on the Relativistic GAN [44]. In standard GAN, the generator G is trained to increase the probability that fake data is real, but it will also decrease the probability that real data is real, since it would account for a priori knowledge that half of the data in the mini-batch is fake [44]. The standard discriminator estimates the probability that one input echo $x$ is real and natural (1 for real and 0 for fake), as formulated in Equation (13):

$$D(x)=\sigma (C(x))$$

(13)

where $\sigma$ is the sigmoid function and $C(x)$ is the non-transformed discriminator output. However, a relativistic discriminator, denoted as $D_{Ra}$, tries to predict the probability that a real echo $x_r$ is relatively more realistic than a fake one $x_f$, as formulated in Equation (14):

$$D_{Ra}(x_r,x_f)=\sigma (C(x_r)-E_{x_f}[C(x_f)])$$

(14)

where $E_{x_f}[•]$ represents the operation of taking the average for all fake data in the mini-batch.

Then, the discriminator loss is defined as:

$$L_D^{Ra}=-E_{x_r}[\log (D_{Ra}(x_r,x_f))]-E_{x_f}[\log (1-D_{Ra}(x_f,x_r))]$$

(15)

The adversarial loss for a generator is in a symmetrical form:

$$L_G^{Ra}=-E_{x_r}[\log (1-D_{Ra}(x_r,x_f))]-E_{x_f}[\log (D_{Ra}(x_f,x_r))],$$

(16)

where $x_f=G(x_i)$ and $x_i$ stands for the input LR echoes. Since the adversarial loss for a generator contains both $x_r$ and $x_f$, the generator benefits from the gradients from both generated data and real data in adversarial training, and hence helps to learn sharper edges and more detailed textures. The total loss for the generator, as put forward in [39], is:

$$L_G^{}=L_{percep}+\lambda L_G^{Ra}+\eta L_1$$

(17)

where $L_1=E_{x_i}{\Vert G(x_i)-y\Vert }_1$ is the content loss that evaluate the 1-norm distance between recovered image $G(x_i)$ and the ground-truth $y$; $\lambda$, $\eta$ are the coefficients to balance different loss terms, and $L_{percep}$ is the perceptual loss defined in Equation (8) [29]. Extensive MOS tests [29] show that even though the SR models trained with adversarial loss and content loss may achieve lower PSNR compared to those trained with pixel loss, they bring significant gains in perceptual quality.

4. Experiments

4.1. Data

We used a level-II data product (images plotted from base weather radar echo data) from the S-band China New-Generation Weather Radar (CINRAD-SA) and the X-band dual-polarization Radar (XPRAD) provided by China Meteorological Administration. For CINRAD-SA radar, we used data from the 2008 sustained blizzard event in Yancheng, Jiangsu; the strong convective wind and hail event in Dangyang, Hubei on 2 April 2008; the tornado and hail event in Yancheng, Jiangsu on 27 May 2008; the Typhoon Hagupit landing in Haikou, Guangdong on 23 September 2008; and the Typhoon event in Xuzhou, Jiangsu on 18 August 2018. For XPRAD radar, we used the rainfall data in the South China Heavy Rainfall Observation Experiment in May 2016.

For CINRAD-SA radar, the reflectivity data has 360 radials with 460 range bins per radial direction, and the distance resolution is 1 km. The mean radial velocity data format is slightly different, which contains 920 range bins for each of the 360 radials in an elevation cut, and the distance resolution is 0.25 km. For XPRAD data, the reflectivity, differential reflectivity, and correlation coefficient data products contain 14 elevation cuts, and each elevation cut includes 360 radials with 4000 range bins per radial direction. The distance resolution of the XPRAD data is 0.075 km. We used the first 600 range bins because of the stronger rainfall attenuation in the X-band results. Example radar echo images are shown in Figure 4. Super-resolved images for the reference methods, including improved IBP [15] and NSSR [17], were obtained from Zhang et al. [17].

4.2. Methods and Training Details

The reflectivity Z_SA and radial velocity V_SA of CINRAD-SA radar; and reflectivity Z_h, differential reflectivity Z_dr, and radial velocity V_xd of XPRAD radar were used in the experiments. For each category, 100 records were used for training and 10 for validation, totaling 500 records for training and 50 records for validation. Original HR images were plotted from these data records and saved using the MATLAB print function with the resolution parameter of 600 dpi, resulting in images of pixel size 3500 × 2625, which were later center-cropped to 2200 × 2200 to get rid of the extra white space. However, during training, there is no need to read the whole large images due to limited computing resources. Hence, the 500 HR images were subsequently cropped to sub-images with a sliding window of step 360, resulting in 18,000 sub-images of pixel size 480 × 480. Common data augmentation [45] methods such as random horizontal flips and 90, 180, and 270-degree rotations were also used during training.

Since the weather radar echo degradation process, as shown in Figure 1, is ill-defined and irreversible, and that the point spread functions for different imaging systems are different, a Gaussian filter is usually used to simulate the process of the degradation of radar echo [9,15,30,46,47]. The LR images are obtained by first applying a Gaussian filter of kernel size 7 × 7 and standard deviation of 1.5 to the cropped sub-images using the MATLAB Gaussian kernel function, and then down-sampling them by scaling factors of ×2 and ×4 using the MATLAB bicubic kernel function. An example of a high-resolution echo image patch and the output from the degradation process using the patch from Figure 4a is shown in Figure 5.

Two models with scaling factors of ×2 and ×4 were trained. For both models, the mini-batch size was set to 13, with the total number of iterations set to 500 k and a validation frequency to 5 k. The spatial size of the cropped HR patch was 128 × 128. It was observed that training a deeper network benefits from a larger patch size, since an enlarged receptive field helps to capture more semantic information [39]. However, it’s more time-consuming, and needs more computing resources.

A model using L1 pixel-wise loss was pre-trained with a starting learning rate of 2 × 10⁻⁴, which decayed by a factor of 2 every 2 × 10⁵ of mini-batch updates. Then, the trained model is employed as an initialization for the generator. Then, the generator is trained using the loss function in Equation (17) with λ = 5 × 10⁻³ and η = 1 × 10⁻². The learning rate is set using a multistep scheme where it’s set to 1 × 10⁻⁴ initially and halved at 50 k, 100 k, 200 k, and 300 k iterations. Pre-training helps GAN-based methods avoid undesired local optima for the generator, and the discriminator receives relatively good starting super-resolved images instead of extreme fake ones (black or noisy images) [39].

Adam [48] was used for optimization with β₁ = 0.9 and β₂ = 0.999 for both the generator and the discriminator network. For the generator, a deep network of 23 RRDB blocks was used. For discriminator network, seven convolutional layers–batch normalization–Leaky Relu (Conv–BN–LReLU) basic blocks was used. The models were implemented using the PyTorch framework and were trained using NVIDIA GTX 1060 GPU.

The PSNR (peak signal-to-noise ratio) and SSIM (structural similarity index), as described in Section 2.3, were used as evaluation criteria of the super-resolved echo images. The models were compared with bicubic interpolation, improved IBP [15], and NSSR [17].

4.3. Results

For CINRAD-SA radar, the first elevation cut of the reflectivity and mean radial velocity data of the CINRAD-SA radar at 09:36 (BJT) on 19 May 2018 in Beijing, China was used as test data (Figure 4a,b), which has 360 radials with 460 range bins per radial direction for reflectivity, and 920 range bins radial direction for radial velocity. The qualitative results and visual comparison of different radar echo reconstruction methods for CINRAD-SA radar echo and their corresponding PSNR/SSIM scores are presented in Figure 6a,b and Figure 7a,b.

From Figure 6a,b and Figure 7a,b, we can see that the reconstructed radar echoes by bicubic interpolation and IBP tend to be blurred and smoothed, losing much high-frequency information during reconstruction, while both NSSR and GAN are capable of recovering most of the crisp and sharp details of echo edges and contours. However, the reconstructed echo by GAN is more visually similar to the original image than that of NSSR, with the latter having a more distorted look. For ×2 images, NSSR introduced some unwanted noise. For challenging images such as the radial velocity depicted in Figure 4b, GAN achieves a higher PSNR/SSIM result while retaining more details than NSSR, which have more color grouping and dissolved textures in the reconstructed echoes. According to Figure 6a,b and Figure 7a,b, GAN achieves a higher PSNR/SSIM score on reconstructed radar echoes while having more natural-looking results than all the compared methods on CINRAD-SA data.

In order to determine the feasibility of the GAN-based method on different weather conditions, five groups of CINARAD-SA data from severe convective weather events, rainfall events, and cloudless days were selected as test data. The reconstructed radar echo was compared with the original high-resolution radar echo using PSNR and SSIM as evaluation metrics. The results are shown in

Table 1. Average peak signal-to-noise ratio (PSNR) (dB) and structural similarity index (SSIM) results of the reconstructed echo of five groups of level-II data products of the CINRAD-SA radar (×4). IBP: Iterative Back Projection. NSSR: Nonlocal Self Similarity Sparse Representation. GAN: Generative Adversarial Networks.

Methods	Severe Weather		Rainfall		Cloudless
	Reflectivity	Velocity	Reflectivity	Velocity	Reflectivity	Velocity
Bicubic	26.5709	21.4417	27.5785	20.6383	29.0000	24.0140
	0.9145	0.8801	0.9412	0.8597	0.9652	0.9353
IBP	27.1974	21.8557	28.1976	21.0084	29.6042	24.3899
	0.9236	0.8887	0.9474	0.8692	0.9689	0.9395
NSSR	28.1819	22.0604	29.3312	21.0326	30.8083	24.3498
	0.9380	0.9041	0.9509	0.8708	0.9718	0.9437
GAN	29.8260	22.0710	30.0911	21.1637	31.3048	24.3735
	0.9621	0.9097	0.9697	0.8973	0.9826	0.9519

(×4) and

Table 2. Average PSNR (dB) and SSIM results of the reconstructed echo of five groups of level-II data products of the CINRAD-SA radar (×2).

Methods	Severe Weather		Rainfall		Cloudless
	Reflectivity	Velocity	Reflectivity	Velocity	Reflectivity	Velocity
Bicubic	27.8990	22.3740	28.9032	21.5330	30.3516	24.8831
	0.9370	0.9044	0.9563	0.8866	0.9744	0.9476
IBP	28.2045	22.6047	29.2123	21.7666	30.6683	25.1020
	0.9415	0.9098	0.9594	0.8929	0.9763	0.9504
NSSR	29.5480	22.4849	30.0018	21.5484	31.5229	24.7074
	0.9482	0.9226	0.9610	0.8748	0.9773	0.9536
GAN	30.5941	22.7071	30.4960	22.1960	32.0998	25.5752
	0.9805	0.9235	0.9783	0.9102	0.9889	0.9612

(×2). We can see that the GAN achieves the highest PSNR and SSIM result on all tested weather conditions compared with other algorithms; GAN has improved significantly on severe weather and rainfall reconstruction, while having the highest PSNR/SSIM scores on cloudless days, because of its more sparse data composition. Within the same weather conditions, the PSNR/SSIM results of reflectivity data were consistently better than those of velocity; this is due to velocity data having more small and fine-grained edge information and a wider data range, as can be seen in Figure 4a,b, which is challenging for all the compared algorithms. As can be expected, the results of upscale factor of ×2 (Table 2) were consistently better than those of upscale factor of ×4 (Table 1).

To verify that the GAN-based method is also applicable to dual-polarization weather radar (XPRAD) data, the first elevation cut of the reflectivity (Z_h), differential reflectivity (Z_dr), and radial velocity (V_xd) data of the XPRAD radar in Xinfeng County, Guangdong Province, China on 28 May 2016 at 07:39 (BJT) were selected as the test data. Each has 360 radials with 600 range bins per radial direction (Figure 4c–e). The qualitative results and visual comparison of different radar echo reconstruction methods for XPRAD radar echo and their corresponding PSNR/SSIM scores are presented in Figure 6c–e and Figure 7c–e. Similar to CINRAD-SA radar echoes, bicubic interpolation and IBP tend to over-smooth the echo details, while the colors from NSSR are slightly distorted and dissolved. As can be seen in Figure 4, the selected XPRAD test echo was not as extreme as that in the CINRAD-SA radar, where the echo image has more layered contours and details. In addition, the radial velocity images from XPRAD radar has less velocity ambiguity phenomenon, as can be seen in Figure 4d; hence, the reconstruction of XPRAD radar echo is less challenging and achieves better PSNR/SSIM. This demonstrates that the GAN-based method is also applicable to selected XPRAD data, and achieves better results than all the compared methods.

Similarly, to test the applicability of the GAN-based method on different weather conditions for dual-polarization radar, five groups of XPRAD data from severe convective weather events, rainfall events, and cloudless days were selected as test data. The reconstructed radar echoes were compared with the original high-resolution radar echoes, and the results are shown in

Table 3. Average PSNR (dB) and SSIM results of the reconstructed echo of five groups of level-II data products of XPRAD radar (×4).

Methods	Severe Weather			Rainfall			Cloudless
	Zh	Vxd	Zdr	Zh	Vxd	Zdr	Zh	Vxd	Zdr
Bicubic	28.8036	26.0172	24.3761	31.9382	28.5004	27.1535	36.9246	33.9114	31.1582
	0.9427	0.9428	0.9048	0.9747	0.9720	0.9553	0.9924	0.9916	0.9848
IBP	29.5159	26.7602	25.0281	32.6644	29.2243	27.7933	37.6453	34.6458	31.7911
	0.9491	0.9463	0.9151	0.9776	0.9736	0.9601	0.9933	0.9921	0.9865
NSSR	31.4081	28.8602	27.1537	36.7442	33.4992	29.4041	40.3813	38.3754	32.5530
	0.9627	0.9487	0.9596	0.9824	0.9771	0.9743	0.9946	0.9941	0.9939
GAN	34.5679	32.3670	28.5642	38.6138	35.4582	30.9614	43.7936	42.4108	34.5021
	0.9858	0.9496	0.9686	0.9935	0.9850	0.9851	0.9981	0.9973	0.9947

(×4) and

Table 4. Average PSNR (dB) and SSIM results of the reconstructed echo of five groups of level-II data products of XPRAD radar. (×2).

Methods	Severe Weather			Rainfall			Cloudless
	Zh	Vxd	Zdr	Zh	Vxd	Zdr	Zh	Vxd	Zdr
Bicubic	30.2139	27.4481	25.6973	33.3688	29.9164	28.4615	38.3733	35.3564	32.4229
	0.9597	0.9597	0.9310	0.9822	0.9802	0.9674	0.9947	0.9941	0.9888
IBP	30.5227	27.7617	25.9918	33.6826	30.2276	28.7546	38.6933	35.6711	32.7062
	0.9627	0.9625	0.9359	0.9835	0.9816	0.9697	0.9951	0.9945	0.9896
NSSR	32.2404	28.1724	27.9154	37.9592	34.5194	30.1147	41.6135	39.4510	33.5485
	0.9757	0.9554	0.9645	0.9870	0.9818	0.9807	0.9976	0.9954	0.9966
GAN	35.6035	33.0355	29.0030	39.2935	35.8812	31.5394	44.6363	43.4145	34.9678
	0.9974	0.9613	0.9865	0.9959	0.9959	0.9923	0.9985	0.9991	0.9968

(×2). From Table 3 and Table 4, again we can see that the GAN-based method achieves the highest PSNR and SSIM results compared with the other methods. It is capable of adapting to various weather conditions. As mentioned, compared with CINARAD-SA radar echoes, XPRAD radar data are more sparse, and the edge and contour information in the echoes are not as rich. For example, the radial velocity of XPRAD radar in Figure 4d have a more “pixelated” appearance, which results in a higher PSNR/SSIM score on average for reconstructed XPRAD radar echoes and more significant improvements from GAN-based models. As is the case with CINRAD-SA radar echoes, the highest PSNR/SSIM scores of reconstructed XPRAD radar echoes are on cloudless days. The reconstructed XPRAD radar echo by GAN has seen significant improvements across all weather conditions.

5. Conclusions

The results in Section 4 demonstrate the versatility of the GAN-based method for the super-resolution reconstruction of level-II weather radar data products. The method is suitable for the super-resolution reconstruction of both CINRAD-SA radar data and dual-polarized XPRAD weather radar data. It achieves better perceptual quality and PSNR/SSIM results than IBP and NSSR. In particular, it has excellent reconstruction capabilities of edge and structural details of weather radar echoes, and does not introduce unpleasant artifacts such as those of NSSR. The GAN-based method can be applied to enhance the resolution of weather radar echoes and fed to downstream spatial-time weather data processing pipelines such as precipitation nowcasting systems, as well as used to post-process the degraded output of some current radar echo extrapolation models. We also compared the time used by GAN and NSSR during the experiment. The results showed that for reconstructing a single image under the same environment (Windows 10, Intel Core i7-7700, 16GB RAM), NSSR takes minutes to converge, while a feed-forward run of GAN-based method takes only seconds. However, the training process of the GAN-based method is more time consuming, and requires more computing resources. Future work needs to concentrate on reducing the training time by redesigning the GAN architecture and adding online training capability to meet the needs of real-time super-resolution reconstruction.