DSADNet: A Dual-Source Attention Dynamic Neural Network for Precipitation Nowcasting

Abstract

Accurate precipitation nowcasting is of great significance for flood prevention, agricultural production, and public safety. In recent years, spatiotemporal sequence models based on deep learning have been widely used for precipitation nowcasting and have achieved better prediction results than traditional methods. These models commonly use radar echo extrapolation and utilize the Z-R relationship between radar and rainfall to predict rainfall. However, radar echo data can be affected by various noises, and the Z-R correlation linking radar and rainfall encompasses several variables influenced by factors like terrain, climate, and seasonal variations. To solve this problem, we propose a dual-source attention dynamic neural network (DSADNet) for precipitation nowcasting, which is a network model that utilizes a fusion module to extract valid information from radar maps and rainfall maps, together with dynamic convolution and the attention mechanism, to directly predict future rainfall through encoding and decoding structure. We conducted experiments on a real dataset in Jiangsu, China, and the experimental results show that our model had better performance than the other examined models.

1. Introduction

Precipitation nowcasting is an effective way to make a relatively short-period prediction of the intensity of rainfall at the kilometer level in a local area, which can help local meteorological departments to make timely and accurate decisions about rainfall warnings, and to guide governmental departments in the transportation, agriculture, and electric power industries to make responses in advance, thus reducing potential economic losses [1]. The role of heavy precipitation weather forecasting in today’s society cannot be overemphasized. Intense rainfall within a brief timeframe has the potential to trigger natural calamities like floods and landslides, leading to significant societal harm and adverse effects on ecological stability. Accurate rainfall forecasting holds immense significance for pre-emptive flood management, agricultural productivity, and sustainable progress, empowering proactive measures. Currently, there exist two primary methodologies for precipitation nowcasting: numerical weather prediction (NWP) [2] and radar echo extrapolation. NWP relies on numerical simulations derived from a series of mathematical physics equations, which involves the calculation of complex physics equations, but it is difficult to meet the requirements of accuracy and real-time precipitation prediction due to the uncertainty of its initial and boundary conditions, the incompleteness of the physical mechanism, and the fact that the technique involves the calculation of complex physics equations [3]. With the continuous increase of meteorological data and the rapid development of deep learning technology, most of the current research work on precipitation nowcasting mainly adopts radar echo extrapolation.

Radar echo extrapolation-based precipitation nowcasting methods can be broadly classified into two categories: conventional extrapolation algorithms and deep learning extrapolation algorithms. Among the traditional extrapolation algorithms, numerous algorithms based on optical flow techniques [4,5,6] have been shown to have better radar echo extrapolation performance. Optical flow techniques, based on the assumption of constant brightness, are globally consistent techniques that can predict the motion of the entire cloud layer. However, in reality, clouds tend to produce aggregation and dissipation phenomena, which can affect the accuracy of optical flow techniques.

With the development of deep learning, some deep learning models have been proposed, such as RNN [7], LSTM [8], and GRU [9] to deal with precipitation nowcasting problems. Both satellite data and radar echo data are important tools used to obtain precipitation information. However, satellite data cannot directly reflect the precipitation inside the cloud, which needs to be judged by the cloud location, shape, cloud top temperature, and other characteristics. Radar echo data, on the other hand, can directly reflect the precipitation situation based on the size of the echo value. Therefore, radar echo data are more suitable for precipitation nowcasting. By learning from a large amount of historical radar echo data, these models establish a mapping relationship from previous observations to future prediction data. Shi et al. [10] innovatively regarded the radar echo extrapolation task as a spatiotemporal sequence prediction problem and designed a ConvLSTM model, which substitutes the fully connected architecture of the FC-LSTM model with a convolutional structure, thus enhancing radar echo extrapolation. This addresses the limitation of RNN models, which primarily emphasize temporal aspects of image sequences while neglecting spatial features. However, the memory units of the ConvLSTM model tend to focus on local spatial dependencies and are inadequate for capturing long-range spatiotemporal dependencies; therefore, Lin et al. [11] introduced the self-attention memory (SAM) module, enabling effective feature extraction with extended spatiotemporal dependencies via the self-attention mechanism. This facilitates improved accuracy in forecasting future radar echo states. Nevertheless, as the lead time increases, these methods tend to underestimate areas with high reflectivity, which typically indicate intense precipitation. This underestimation of such regions can have significant repercussions. Furthermore, many previous models solely rely on radar maps to forecast future radar conditions and, subsequently, use the Z-R relationship to derive rainfall maps. While radar echo intensity provides insight into the size and density of precipitation particles to some extent, establishing a relationship between echo intensity and precipitation encounters two common challenges: inconsistency of radar echo data, due to radar’s operating principle, and various noises; since the Z-R relationship is empirical, this relationship may change under different meteorological conditions, resulting in inaccurate predicted rainfall data.

Numerous studies have shown that the use of multi-source data is more accurate than the use of single data in precipitation prediction [12]. For this reason, Geng et al. [13] proposed the AF-SRNet model for fusing radar and rainfall data to predict future rainfall data, which extracts the information from radar and rainfall, respectively, through the SRU module, then fuses the extracted radar and rainfall information using the attention fusion module, and, finally, decodes and outputs the future rainfall data. The experimental results show that fusing multi-source data helps to improve the accuracy of prediction. However, the above model uses a late-fusion strategy to extract information from radar and rainfall separately before fusing the information, which does not effectively extract the evolution of the radar data in the two phases and only fuses the radar and rainfall data at the same moment, which fails to capture the aggregation and dissipation of the radar echo intensity in the two phases. Therefore, we propose a dual-source attention dynamic neural network (DSADNet), which utilizes a fusion module (Fusion-Module) to effectively fuse the radar and rainfall information to extract the evolution of the two-phase radar data with higher prediction accuracy than the AF-SRNet model. Our model directly predicts future rainfall using encoding and decoding structure, which overcomes the problem of inaccurate radar echo data due to noise and inaccurate prediction due to the inability to accurately determine its parameters using the Z-R relationship. Rainfall data can accurately reflect the real rainfall situation through direct measurements such as rain gauges. However, rainfall data may not fully capture climate fluctuations, whereas radar echo data can depict the aggregation and dissipation of echo intensity, and the fusion module designed by us can better fuse the effective information in the radar echo data and the rainfall data, as well as better capture the aggregation and dissipation of the radar echo in two phases, which improves the prediction accuracy. To pay more attention to the high-echo heavy-precipitation region, we replaced static convolution with dynamic convolution [14]. To improve the ability to capture long-range spatiotemporal dependencies, we added the SAM [11] module to each module in DSADNet. Numerous experimental results show that DSADNet can effectively predict future rainfall with better metrics than existing models.

2. Related Work

Shi et al. [10] innovatively considered the radar echo extrapolation task as a spatiotemporal sequence prediction problem. The authors proposed ConvLSTM by replacing the input-to-state and state-to-state fully connected structures in FC-LSTM with a convolutional structure, and the experimental results show that this model outperforms both the optical flow techniques and FC-LSTM. Afterward, to solve the problem that the convolutional operation in the ConvLSTM model is position-invariant, while the meteorological variations are usually position-variant, Shi et al. [15] introduced the TrajGRU model, capable of autonomously learning motion position changes and achieving superior prediction performance compared to ConvLSTM. ConvGRU [16] has a similar structure and effect as ConvLSTM, but the number of parameters is decreased, which optimizes the model’s prediction time.

Wang et al. [17] improved the previous model by proposing a new recurrent neural network framework called predictive recurrent neural network (PredRNN), which allows the memory states between different LSTM units to interact from layer to layer. The authors devised an innovative spatiotemporal LSTM (ST-LSTM) unit, which integrates spatial and temporal relationships within a unified memory unit. This unit facilitates the horizontal and vertical transmission of memories, thereby considering both temporal and spatial states. The experimental results indicated the favorable performance of the model. Since the accuracy of ConvRNN-based precipitation nowcasting methods is greatly affected by the gradient vanishing problem [18,19], Wang et al. [20] improved PredRNN and proposed PredRNN++. This model utilizes a novel recursive structure called Causal LSTM, which has a cascading double memory, thus making the network structure deeper in time. In addition, the authors proposed a gradient highway unit that provides alternative quick routes for the gradient flows from outputs back to long-range previous inputs. Causal LSTM works seamlessly with the gradient highway to solve the problem of gradient vanishing. Following this, Wang et al. [21] introduced the PredRNN-V2 version, which introduced a novel loss function derived from PredRNN and implemented a new sampling strategy, leading to improved prediction outcomes. Wu et al. [22] presented ISA-PredRNN, integrating a self-attention mechanism and long-term memory to enhance capability in handling global and long-term dependencies. Tuyen et al. [23] proposed the RainPredRNN model, which combines PredRNN-V2 and U-Net [24], which not only reduces the amount of computation for model prediction but also improves performance.

Highly non-stationary states such as accumulation, deformation, and dissipation of radar echo in precipitation prediction have a crucial impact on the accuracy of the prediction; therefore, Wang et al. [25] proposed MIM, which utilizes the difference signals between neighboring cyclic states to model non-stationary and approximately stationary states in spatiotemporal dynamics and improves the model’s ability to deal with higher-order non-stationary states.

In recent years, the attention mechanism has received much attention, so some ConvRNN models based on the attention mechanism have been proposed for precipitation nowcasting. Lin et al. [11] proposed the self-attention memory (SAM) module and embedded SAM into ConvLSTM. The model can improve the ability to capture long-range spatiotemporal dependencies through the self-attention mechanism. Since CongGRU has fewer parameters compared to ConvLSTM, but the prediction effect is similar, Zhou et al. [26] embedded the SAM module into ConvGRU and obtained a model with better precipitation nowcasting capability compared to the benchmark model. To amplify the effectiveness of the attention mechanism, Luo et al. [27] introduced a novel interactive dual-attention LSTM (IDA-LSTM) model. This model integrates both channel attention and spatiotemporal attention to retrieve long-term forgotten information.

Previous models have used radar echo extrapolation for precipitation nowcasting tasks. Radar maps are extrapolated to predict future radar conditions, which are subsequently converted into rainfall maps using the Z-R relationship. However, the acquisition of radar echo data is affected by various noises, such as wind speed, humidity in the atmosphere, and the obstruction of mountains and buildings, resulting in inaccurate extrapolated radar maps. Moreover, the Z-R relationship linking radar to rainfall involves numerous parameters that fluctuate based on factors such as terrain, climate, and season. Failure to precisely ascertain these parameters can result in decreased prediction accuracy. Precipitation nowcasting requires a high degree of ability to capture long-range spatiotemporal dependencies as well as to accurately predict areas of heavy precipitation, but ConvRNN models rarely consider both of these aspects simultaneously.

3. Methods

3.1. Problem Definition

The precipitation nowcasting model described in our method utilizes past radar echo and rainfall data to forecast future precipitation patterns. Given a historical radar echo sequence $R_{t-k+1:t}=\{R_{t-k+1},R_{t-k+2},···,R_t\}$ and a historical rainfall sequence $P_{t-k+1:t}=\{P_{t-k+1},P_{t-k+2},···,P_t\}$, we forecast the upcoming rainfall sequence ${\widehat{P}}_{t+1:t+m}=\{{\widehat{P}}_{t+1},{\widehat{P}}_{t+2},···,{\widehat{P}}_{t+m}\}$. This can be defined as follows:

$${\widehat{P}}_{t+1},···,{\widehat{P}}_{t+m}=\mathrm{arg\; max}(P_{t+1},···,P_{t+m}|R_{t-k+1},···,R_t;P_{t-k+1},···,P_t)$$

(1)

where k denotes the input sequence length and m denotes the output sequence length.

3.2. Model

For precipitation nowcasting, most researchers use the radar echo extrapolation algorithm [28,29,30], but this does not fully utilize the existing historical data. The collection of radar echo data will be affected by a variety of noises, such as topography, electromagnetic interference, thunderstorms, strong winds, and other meteorological conditions, which greatly affects the accuracy of the extrapolation of radar echoes. Rainfall data can be directly measured using rain gauges and other devices, providing a more accurate representation of actual precipitation conditions, but by using only rainfall data it may not be possible to respond to climate changes such as cloud aggregation or dissipation, which may cause data inconsistency problems. Therefore, in our method in this paper, we designed a new fusion module to improve the accuracy of prediction by fusing radar echo data and rainfall data and effectively extracting the aggregation and dissipation of the radar echo intensity in both phases. Our proposed precipitation nowcasting model adopts the structure of encoding and decoding with dynamic convolution and the attention mechanism to predict rainfall directly, which overcomes the problem of inaccurate prediction due to the need to map the radar map to the rainfall map by using the Z-R relationship in the extrapolation method of radar echoes.

3.2.1. DSADNet

The overall network architecture of DSADNet is illustrated in Figure 1. DSADNet comprises two main components: the encoder, which includes Fusion-Module on the left, and the decoder, which includes Att-DyConvLSTM on the right. Typically, the encoder is constructed with four layers of Fusion-Module to enhance the extraction of deep spatiotemporal features, while the decoder is composed of four layers of Att-DyConvLSTM. Fusion-Module integrates radar echo and rainfall data, capturing variations in radar echo intensity by encoding data from two phases, and then it merges the rainfall data to consolidate the fused information, which is saved in the cell state $C_t^l$, hidden state $H_t^l$, and spatiotemporal memory unit $M_t^l$ and is passed to the decoder; finally, the predicted rainfall data is decoded and output.

3.2.2. Fusion-Module

Radar echo intensity reflects the scale and density of precipitation particles to a certain extent, and the relationship between echo intensity and precipitation can be established, so a radar map can reflect the rainfall situation, but the radar echo data is affected by various kinds of noise, which leads to inaccurate reflection of the rainfall data. While rainfall data provide an accurate representation of actual rainfall conditions, their utilization may overlook factors such as cloud dissipation and atmospheric dynamics, which contribute to climate fluctuations, while radar echo data can reflect climate change, so the joint use of radar and rainfall data can compensate for their respective shortcomings. Therefore, to extract effective information from radar echo data and rainfall data, we designed the Fusion-Module module to effectively fuse radar echo data and rainfall data and to encode and extract the effective information.

The configuration of the Fusion-Module module is depicted in Figure 2. Firstly, the radar echo data $R_t$, the cell state $C_{t-1}^l$, and the hidden state $H_{t-1}^l$ of the previous period enter into Fusion-Module, which decides which of the input data are important through the input gate $i_t$, which information should be retained or forgotten through the forgetting gate $f_t$, and generates candidate information to help the network update the new cell state $C_t^l$ through the input modulation gate $g_t$. The specific formulas are as follows:

$$i_t=\sigma (Dy(W_{ri},R_t)+Dy(W_{hi},H_{t-1}^l)+W_{ci}\odot C_{t-1}^l+b_i)$$

(2)

$$f_t=\sigma (Dy(W_{rf},R_t)+Dy(W_{hf},H_{t-1}^l)+W_{cf}\odot C_{t-1}^l+b_f)$$

(3)

$$g_t=tanh(Dy(W_{rg},R_t)+Dy(W_{hg},H_{t-1}^l)+b_g)$$

(4)

$$C_t^l=f_t\odot C_{t-1}^l+i_t\odot g_t$$

(5)

where $\sigma$ is the Sigmoid activation function, $Dy(,)$ is the dynamic convolution, ⊙ is the Hadamard product, $tanh$ is the hyperbolic tangent activation function, $W_{**}$ is the parameter to be learned, and $b_{*}$ is the bias value.

Since radar maps can respond to climate changes such as cloud aggregation and dissipation, inspired by [27], we included the Fusion module in Fusion-Module, which fuses the hidden state $H_{t-1}^l$, which contains the previous radar information, with the current rainfall data $P_t$ for extracting the radar and rainfall information, and then fuses the updated data with the cell state $C_t^l$ containing the current radar information through the update gate $z_t$ and the output gate $o_t$. This module not only extracts the evolution of the radar data in the two time periods, but also fuses the real rainfall data to improve the accuracy of the prediction. The specific formulas are as follows:

$$H_t^l,P_t^l=Fusion(H_{t-1}^l,P_t)$$

(6)

$$z_t=tanh(Dy(W_{hz},H_t^l)+W_{cz}\odot C_t^l+b_z)$$

(7)

$$o_t=\sigma (Dy(W_{po},P_t^l)+Dy(W_{ho},H_t^l)+W_{co}\odot C_t^l+b_o)$$

(8)

$$H_t^{l^{\prime }}=z_t\odot o_t$$

(9)

where $\sigma$ is the Sigmoid activation function, $Dy(,)$ is the dynamic convolution, ⊙ is the Hadamard product, $tanh$ is the hyperbolic tangent activation function, $W_{**}$ is the parameter to be learned, $b_{*}$ is the bias value, and $Fusion(,)$ denotes the fusion mechanism as shown in Figure 3. The specific formulas are as follows:

$$P_t^l=Relu(LayerNorm\left(Con{v}_{p2p}\left(P_t\right)\right)+LayerNorm\left(Con{v}_{h2p}\left(H_{t-1}^l\right)\right))$$

(10)

$$H_t^l=Relu(LayerNorm\left(Con{v}_{p2h}\left(P_t^l\right)\right)+LayerNorm\left(Con{v}_{h2h}\left(H_{t-1}^l\right)\right))$$

(11)

where $Relu\left(\right)$ is the Relu activation function, $LayerNorm\left(\right)$ is Layer Normalization, and $Conv\left(\right)$ is the convolution operation. Finally, to capture the long-range spatiotemporal dependencies, we added SAM to Fusion-Module, and the specific formula is as follows:

$$H_t^l,M_t^l=SAM(H_t^{l^{\prime }},M_{t-1}^l)$$

(12)

where $SAM(,)$ denotes self-attention memory module.

3.2.3. Att-DyConvLSTM

To better decode the encoder-generated cell states $C_t^l$, hidden states $H_t^l$, and spatiotemporal memory cells $M_t^l$, we employed ConvLSTM as the baseline model and enhanced it by substituting its convolutional structures with dynamic convolution, aiming to better emphasize regions with intense precipitation. Similarly, we added the SAM module to the model, which was used to improve the ability to capture long-range spatiotemporal dependencies.

Att-DyConvLSTM, consisting of ConvLSTM, Dynamic Convolution, and the SAM module, constitutes the decoder module shown in Figure 4, which is formulated as follows:

$$i_{t+1}^{\prime }=\sigma (Dy(W_{pi}^{\prime },H_{t+1}^{l+1})+Dy(W_{hi}^{\prime },H_t^l)+W_{ci}^{\prime }\odot C_t^l+b_i^{\prime })$$

(13)

$$f_{t+1}^{\prime }=\sigma (Dy(W_{pf}^{\prime },H_{t+1}^{l+1})+Dy(W_{hf}^{\prime },H_t^l)+W_{cf}^{\prime }\odot C_t^l+b_f^{\prime })$$

(14)

$$g_{t+1}^{\prime }=tanh(Dy(W_{pg}^{\prime },H_{t+1}^{l+1})+Dy(W_{hg}^{\prime },H_t^l)+b_g^{\prime })$$

(15)

$$o_{t+1}^{\prime }=\sigma (Dy(W_{po}^{\prime },H_{t+1}^{l+1})+Dy(W_{ho}^{\prime },H_t^l)+W_{co}^{\prime }\odot C_t^l+b_o^{\prime })$$

(16)

$$C_{t+1}^l=f_{t+1}^{\prime }\odot C_t^l+i_{t+1}^{\prime }\odot g_{t+1}^{\prime }$$

(17)

$$H_{t+1}^{l^{\prime }}=o_{t+1}^{\prime }\odot tanh\left(C_{t+1}^l\right)$$

(18)

$$H_{t+1}^l,M_{t+1}^l=SAM(H_{t+1}^{l^{\prime }},M_t^l)$$

(19)

where $\sigma$ is the Sigmoid activation function, $Dy(,)$ is the dynamic convolution, ⊙ is the Hadamard product, $tanh$ is the hyperbolic tangent activation function, $W_{**}^{\prime }$ is the parameter to be learned, $b_{*}^{\prime }$ is the bias value, and $SAM(,)$ denotes the self-attention memory module. In Equation (19), if the Att-DyConvLSTM module is in the first layer of the decoder, the output is the predicted rainfall ${\widehat{P}}_{t+1}$, while in the other layers the output is the hidden state $H_{t+1}^l$.

4. Experiments

4.1. Dataset

In this paper, we used the dataset from the 2022 Jiangsu Meteorological AI Algorithm Challenge as the training data for the model. The dataset contains data from 2019 to 2021, from April to September. The radar echo dataset was sourced from the quality-controlled network of multiple S-band meteorological radars in Jiangsu, encompassing the entire regional area of Jiangsu Province. The radar echo data spanned a range of 0–70 dBZ with a horizontal resolution of 0.01° (approximately 1 km), a temporal resolution of 6 min, and a frame resolution of 480 × 560 pixels. Higher atmospheric water droplet content correlated with increased radar fundamental reflectivity. Rainfall data were derived through the interpolation of data collected from automatic meteorological stations in Jiangsu and the surrounding regions onto a standardized grid, and the rainfall data were the 6-min cumulative rainfall from the automatic stations, with a value range of 0–10 mm and a resolution of 480 × 560 pixels per frame.

In terms of data preprocessing, considering the computational overhead of the experiment, we obtained radar echo maps and rainfall maps of 480 × 480 pixels in the middle by cropping, and we then scaled the maps to 128 × 128 according to bicubic interpolation. To enhance training effectiveness, we normalized the pixel values of both the radar echo and the rainfall maps. The experiment utilized radar echo and rainfall sequences from the previous hour to predict rainfall sequences for the subsequent hour. The dataset was divided into training and validation sets in a ratio of 8:2.

4.2. Implementation Details

Our model and comparison models were implemented using Pytorch and were run on NVIDIA TITAN RTX GPUs. The comparison models were ConvLSTM [10], ConvGRU [16], PredRNN [17], MIM [25], SA-ConvLSTM [11], IDALSTM [27], AF-SRNet [13], and SA-ConvGRU [26]. For consistency across the experiments, all our comparison models incorporated radar echo and rainfall data. We concatenated these data types in the channel dimension, aligning with the methodology employed in [13]. In this experiment, we utilized the Adam optimizer [31] with a learning rate set to 0.001, a batch size of 16, and trained for a total of 100 epochs. If the loss failed to decrease for 4 consecutive epochs, the learning rate was reduced to 0.1 times its original value. Moreover, if the loss did not decrease for 10 consecutive epochs, the training process was halted. When training the model using the original MSE–MAE loss function, the model tends to overlook regions of intense precipitation, which makes the model more focused on the low-rainfall region [32], and the output image is more blurred. The formula for the MSE–MAE loss function is shown below:

$$MSE-MA{E}_{loss}=\frac{1}{N}\sum _j^H\sum _i^W({(y_{i,j}-{\widehat{y}}_{i,j})}^2+|y_{i,j}-{\widehat{y}}_{i,j}|)$$

(20)

where N is the number of predicted maps, H and W are the height and width of the maps, and $y_{i,j}$ and ${\widehat{y}}_{i,j}$ are the actual and predicted maps values, respectively. Therefore, we used weight loss and the SSIM to build the loss function with the following formula:

$$loss=weigh{t}_{loss}+1000\times (1-SSIM(y,\widehat{y}))$$

(21)

$$weigh{t}_{loss}=\frac{1}{N}\sum _j^H\sum _i^{W}weigh{t}_{i,j} \times ({(y_{i,j}-{\widehat{y}}_{i,j})}^2 + |y_{i,j}-{\widehat{y}}_{i,j}|)$$

(22)

$$SSIM(y,\widehat{y})=\frac{(2{\mu }_y{\mu }_{\widehat{y}}+C_1)(2{\sigma }_{y\widehat{y}}+C_2)}{({\mu }_y^2+{\mu }_{\widehat{y}}^2+C_1)({\sigma }_y^2+{\sigma }_{\widehat{y}}^2+C_2)}$$

(23)

where N is the number of predicted maps, H and W are the height and width of the maps, $y_{i,j}$ and ${\widehat{y}}_{i,j}$ are the actual and predicted maps values, respectively, ${\mu }_y$, ${\mu }_{\widehat{y}}$ are the means of y and $\widehat{y}$, ${\sigma }_y^2$, ${\sigma }_{\widehat{y}}^2$ are the variances of y and $\widehat{y}$, ${\sigma }_{y\widehat{y}}$ is the covariance of y and $\widehat{y}$, $C_1$ and $C_2$ are two non-zero constants, and $weigh{t}_{i,j}$ is the weight value of the map at (i,j). We analyzed the distribution of different rainfall amounts, and to ensure that areas of heavy precipitation were not overlooked, we assigned different weight classes to different rainfall amounts; $weigh{t}_{i,j}$ was assigned according to the following equation:

$$weigh{t}_{i,j}=\left\{\begin{array}{cc}\hfill 0.1\hfill & \hfill 0.00\le y_{i,j}\le 0.01\hfill \\ \hfill 1.0\hfill & \hfill 0.01\le y_{i,j}\le 0.10\hfill \\ \hfill 2.0\hfill & \hfill 0.10\le y_{i,j}\le 0.50\hfill \\ \hfill 5.0\hfill & \hfill 0.50\le y_{i,j}\le 1.00\hfill \\ \hfill 10.0\hfill & \hfill 1.00\le y_{i,j}\le 2.00\hfill \\ \hfill 20.0\hfill & \hfill 2.00\le y_{i,j}\le 3.00\hfill \\ \hfill 30.0\hfill & \hfill 3.00\le y_{i,j}\le 6.00\hfill \\ \hfill 50.0\hfill & \hfill 6.00\le y_{i,j}\le 10.00\hfill \end{array}\right.$$

(24)

4.3. Performance Metric

To assess the model’s performance, we utilized four commonly employed evaluation metrics: the critical success index (CSI), the false alarm rate (FAR), the Heidke skill score (HSS), and the mean squared error (MSE). The values of the CSI, the FAR, and the HSS were in the range of 0–1. The closer the CSI and the HSS were to 1, and the closer the FAR was to 0, the better the prediction of the model was. The MSE was a value greater than or equal to 0. The closer it was to 0, the closer the predicted data were to the real data and the better the prediction was. To calculate the CSI, the FAR, and the HSS, we changed the rainfall map to a bivariate map based on thresholds, where the value was set to 1 if the rainfall value was greater than the threshold, and 0 otherwise, and the thresholds were set to 0.05 mm, 0.20 mm, and 0.50 mm, respectively. Using the bivariate map, we calculated the True Positive (TP), the False Negative (FN), the True Negative (TN), and the False Positive (FP) values. The CSI, FAR, HSS, and MSE metrics were defined as follows:

$$CSI=\frac{TP}{TP+FP+FN}$$

(25)

$$FAR=\frac{FP}{TP+FP}$$

(26)

$$HSS=\frac{2\times (TP\times TN-FN\times FP)}{(TP+FN)\times (FN+TN)+(TP+FP)\times (FP+TN)}$$

(27)

$$MSE=\frac{1}{N}\sum _j^H\sum _i^W{(y_{i,j}-{\widehat{y}}_{i,j})}^2$$

(28)

where $TP$ denotes the number of predicted values of 1 and true values of 1, $FN$ denotes the number of predicted values of 0 and true values of 1, $TN$ denotes the number of predicted values of 0 and true values of 0, $FP$ denotes the number of predicted values of 1 and true values of 0, N is the number of predicted maps, H and W are the height and width of the maps, and $y_{i,j}$ and ${\widehat{y}}_{i,j}$ are the true and predicted map values.

4.4. Results and Analysis

4.4.1. Comparative Experiments

We utilized the dataset provided by the 2022 Jiangsu Meteorological AI Algorithm Challenge to evaluate the performance of DSADNet proposed in this paper alongside other comparative models. The dataset used for evaluating the model was not used for the training of the model, and the evaluation results are more reflective of the model’s generalization ability.

Table 1. Average evaluation results of comparison experiments within 30 min. The best metrics are highlighted in bold; “↑” signifies that higher scores are preferable, while “↓” indicates that lower scores are preferable.

Model		0.05 mm			0.20 mm			0.50 mm
	CSI↑	FAR↓	HSS↑	CSI↑	FAR↓	HSS↑	CSI↑	FAR↓	HSS↑	MSE↓
ConvLSTM	0.5329	0.3124	0.6461	0.4702	0.3951	0.5963	0.3953	0.4418	0.5252	7.5953
ConvGRU	0.5293	0.3668	0.6419	0.4646	0.4333	0.5897	0.3949	0.4200	0.5256	7.6868
PredRNN	0.5400	0.3396	0.6528	0.4721	0.3849	0.5986	0.3948	0.4307	0.5245	7.6427
MIM	0.5352	0.3315	0.6480	0.4762	0.3943	0.6035	0.4050	0.4452	0.5352	8.0327
SA-ConvLSTM	0.5404	0.3172	0.6532	0.4702	0.3768	0.5952	0.3965	0.4202	0.5269	7.2319
IDALSTM	0.5430	0.3584	0.6555	0.4747	0.4087	0.6017	0.3941	0.4371	0.5239	7.8811
AF-SRNet	0.5421	0.3529	0.6561	0.4721	0.4104	0.5996	0.3935	0.4319	0.5232	8.1311
SA-ConvGRU	0.5328	0.3399	0.6459	0.4653	0.3939	0.5910	0.3914	0.4305	0.5204	7.3832
DSADNet	0.5516	0.3189	0.6641	0.4841	0.3921	0.6102	0.4066	0.4217	0.5364	7.1690

and

Table 2. Average evaluation results of comparison experiments within 60 min. The best metrics are highlighted in bold; “↑” signifies that higher scores are preferable, while “↓” indicates that lower scores are preferable.

Model		0.05 mm			0.20 mm			0.50 mm
	CSI↑	FAR↓	HSS↑	CSI↑	FAR↓	HSS↑	CSI↑	FAR↓	HSS↑	MSE↓
ConvLSTM	0.4848	0.3651	0.5946	0.4106	0.4459	0.5300	0.3264	0.4922	0.4445	10.3514
ConvGRU	0.4809	0.4117	0.5894	0.4076	0.4834	0.5268	0.3301	0.4736	0.4502	10.5840
PredRNN	0.4911	0.3785	0.6000	0.4126	0.4257	0.5328	0.3262	0.4870	0.4452	10.4432
MIM	0.4889	0.3730	0.5984	0.4160	0.4493	0.5359	0.3324	0.5083	0.4517	10.8839
SA-ConvLSTM	0.4884	0.3547	0.5981	0.4123	0.4104	0.5307	0.3322	0.4715	0.4524	9.9691
IDALSTM	0.4917	0.3971	0.6013	0.4120	0.4425	0.5316	0.3235	0.4989	0.4416	10.6774
AF-SRNet	0.4948	0.3960	0.6054	0.4160	0.4509	0.5369	0.3323	0.4882	0.4530	10.9695
SA-ConvGRU	0.4839	0.3821	0.5935	0.4095	0.4282	0.5286	0.3288	0.4834	0.4479	10.0829
DSADNet	0.5010	0.3548	0.6111	0.4199	0.4268	0.5393	0.3347	0.4758	0.4538	9.8653

show the experimental results of the comparison at 30 min and at 60 min, where the optimal metrics are in bold. In addition, for a more intuitive comparison between our model and other models, an example of a randomly selected visualization is given in Figure 5.

From the results of this experiment, DSADNet performed the best except for the FAR metric. In the field of precipitation nowcasting, the CSI usually stands for the hit rate of precipitation, i.e., the proportion of rainfall events that are successfully predicted. It is a statistical metric used to assess the accuracy of forecasting precipitation events, and the HSS is a statistical metric used to evaluate the performance of a classification model, especially for assessing the consistency between prediction results and observations. The higher of the two metrics means that our proposed DSADNet model can effectively extract information from radar and rainfall maps, which improves the accuracy of prediction. Not only that, our model had a lower MSE, which denotes the mean of the squares of the differences between the predicted and real data, indicating that the model’s predictions fit the true values better than other models. In addition, the model improves the performance of maintaining long-range spatiotemporal dependencies by using the SAM module, which enables the model to maintain stable performance as the prediction time grows. As depicted in Figure 5, our model’s prediction results exhibit a minimal shift in the overall rainfall distribution compared to the actual data, and the prediction stability is better than other comparative models. And our model replaces the convolution structure with dynamic convolution. As can be seen from the visualization in Figure 5, our model better predicts the heavy precipitation area (the red part in the figure) compared to other models, which expand or reduce the heavy precipitation area inconsistently with the reality.

Although our model has outstanding metrics on the CSI and the HSS, it is slightly below the optimal metrics on the FAR metrics, and, due to the model lacking in extracting the time evolution information and capturing the process of rainfall intensity change, the model focuses more on the area of heavy precipitation and ignores the information that the heavy precipitation decays into weak precipitation.

As can be seen in the visualization example in Figure 5, our model performs better than the other models in heavy precipitation prediction. In addition, the prediction accuracy of ConvLSTM, ConvGRU, IDALSTM, and SA-ConvGRU keeps decreasing as the prediction time increases, and at 60 min, the regions of different precipitation intensities predicted by these models are quite different from the real situation. AF-SRNet also employs a fusion mechanism to predict rainfall, and the model first encodes the radar and rainfall maps through their respective encoders, and the radar and rainfall maps do not interact at that stage. After that, the radar and rainfall information are fused by the attention fusion module, which fuses the radar and rainfall information at the same stage, ignoring the evolution of the radar echo intensity at the two stages. Therefore this model incorrectly predicts the rainfall distribution, resulting in a large impact on the prediction.

Therefore, our model not only outperforms other models in terms of forecast accuracy, maintaining long-range spatiotemporal dependencies and focusing on regions of heavy precipitation, but is also at a superior level, although the FAR is not optimal.

4.4.2. Ablation Study

To further demonstrate the influence of the fusion module, dynamic convolution, and SAM on the experiments, ablation experiments were conducted to assess the effectiveness of the fusion module, dynamic convolution, and SAM individually.

Table 3. Average evaluation results of ablation experiments within 30 min. The best metrics are highlighted in bold; “↑” signifies that higher scores are preferable, while “↓” indicates that lower scores are preferable.

Model		0.05 mm			0.20 mm			0.50 mm
	CSI↑	FAR↓	HSS↑	CSI↑	FAR↓	HSS↑	CSI↑	FAR↓	HSS↑	MSE↓
ConvLSTM	0.5329	0.3124	0.6461	0.4702	0.3951	0.5963	0.3953	0.4418	0.5252	7.5953
DSNet	0.5423	0.3504	0.6557	0.4728	0.4019	0.5989	0.3968	0.4505	0.5283	7.5271
DSDNet	0.5428	0.3651	0.6550	0.4784	0.4362	0.6058	0.4018	0.4215	0.5325	7.3648
DSANet	0.5438	0.3145	0.6564	0.4737	0.3976	0.5998	0.3934	0.4561	0.5244	7.3382
DSADNet	0.5516	0.3189	0.6641	0.4841	0.3921	0.6102	0.4066	0.4217	0.5364	7.1690

and

Table 4. Average evaluation results of ablation experiments within 60 min. The best metrics are highlighted in bold; “↑” signifies that higher scores are preferable, while “↓” indicates that lower scores are preferable.

Model		0.05 mm			0.20 mm			0.50 mm
	CSI↑	FAR↓	HSS↑	CSI↑	FAR↓	HSS↑	CSI↑	FAR↓	HSS↑	MSE↓
ConvLSTM	0.4848	0.3651	0.5946	0.4106	0.4459	0.5300	0.3264	0.4922	0.4445	10.3514
DSNet	0.4931	0.3987	0.6033	0.4126	0.4443	0.5321	0.3300	0.5065	0.4502	10.2246
DSDNet	0.4930	0.4146	0.6018	0.4174	0.4924	0.5385	0.3338	0.4916	0.4547	10.0921
DSANet	0.4947	0.3597	0.6048	0.4166	0.4411	0.5378	0.3309	0.5212	0.4528	10.0208
DSADNet	0.5010	0.3548	0.6111	0.4199	0.4268	0.5393	0.3347	0.4758	0.4538	9.8653

show the ablation results at 30 min and at 60 min, where the optimal metrics are in bold. In the table, ConvLSTM is the baseline model, DSNet is the fusion module added to ConvLSTM, DSDNet is the convolution structure on DSNet replaced with dynamic convolution, DSANet is the SAM module added to DSNet, and DSADNet is our proposed model. Similarly, Figure 6 gives an example of the visualization of the ablation experiment.

From the results of the ablation experiments: Firstly, by comparing the evaluation indexes of the benchmark model ConvLSTM and DSNet, Fusion-Module was added to help the model better extract the radar information and rainfall information, and the CSI and HSS indexes of DSNet were better than those of the benchmark model, but, due to the weak ability of Fusion-Module in extracting information, strong precipitation decayed into weak precipitation, which led to the FAR being on the high side. So, Fusion-Module helps to improve prediction performance, but the FAR index needs to be reduced. Secondly, to improve the focus on strong precipitation, we replaced the convolution structure on DSNet with dynamic convolution. As shown in Figure 6, DSDNet paid more attention to the region of strong precipitation than DSNet (the red part in the figure). Next, to improve the model’s ability to maintain long-range spatiotemporal dependencies, we added the SAM module to DSNet, and, as observed in Table 3 and Table 4, DSANet with the addition of SAM outperforms DSNet in all the metrics as the prediction time grows. Our model, DSADNet, which combines the fusion module, dynamic convolution, and the SAM module, is basically optimal in all metrics. With the addition of dynamic convolution, the SAM module, and Fusion-Module, basically, the MSE metric decreases and the model predictions improve. According to Figure 6, DSADNet not only outperforms other models in capturing heavy precipitation areas, but also maintains stable model performance as the prediction time increases. Although our model combines the fusion module, dynamic convolution, and the attention mechanism, Fusion-Module plays a dominant role. Fusion-Module improves model prediction more significantly than dynamic convolution and the attention mechanism, which can be seen in Table 3 and Table 4. While our model generally outperforms the baseline model across all metrics, the improvement is not substantial. Enhancing the model’s prediction accuracy remains the primary focus of future work.

5. Conclusions

This paper introduces a novel precipitation nowcasting model, DSADNet. The model endeavors to maximize the utilization of valuable information from multiple data sources through the development of a novel fusion module. This module effectively captures variations in radar echo intensity and extracts useful information from both radar echo and rainfall data. To improve prediction, we replaced the convolution in the model with dynamic convolution, which can pay more attention to areas of heavy precipitation. The added SAM module allows prediction to remain stable as prediction time increases and improves the model’s ability to capture long-range spatiotemporal dependencies. Our proposed model has higher CSI and HSS metrics and lower MSE metrics, but our model focuses more on the effective combination of radar and rainfall information and is weaker in extracting time-evolution information and capturing the rainfall intensity change process. The model predicts weak precipitation regions as strong precipitation regions, which ultimately leads to high FAR metrics. In future endeavors, our emphasis will be on refining Fusion-Module to minimize the false alarm rate (FAR) and enhance the accuracy of precipitation nowcasting.