# Precipitation Nowcasting with Orographic Enhanced Stacked Generalization: Improving Deep Learning Predictions on Extreme Events

^{1}

^{2}

^{3}

^{4}

^{5}

^{*}

^{†}

## Abstract

**:**

## 1. Introduction

## 2. Materials and Methods

#### 2.1. TAASRAD19 Dataset

#### 2.2. Deep Learning Trajectory GRU Model

#### 2.3. Thresholded Rainfall Ensemble for Deep Learning

#### 2.4. ConvSG Stacking Model

- Batch size: 20
- Optimizer: Adam with learning rate $1{e}^{-3}$
- number of epochs: 100
- validation and checkpoint every 1000 iteration.

#### 2.5. Enhanced Stacked Generalization (ESG)

#### 2.5.1. Combining Assimilation into ConvSG

#### 2.5.2. Orographic Features

#### 2.6. S-PROG Lagrangian Extrapolation Model

## 3. Results

#### 3.1. Categorical Scores

- CSI = $\frac{hits}{hits+misses+falsealarms}$
- FAR = $\frac{falsealarms}{hits+falsealarms}$
- POD = $\frac{hits}{hits+misses}$

#### 3.2. Continuous Scores

## 4. Discussion

#### 4.1. ConvSG Behavior

#### 4.2. Comparing ConvSG and S-PROG

## 5. Conclusions and Future Work

- the thresholded rainfall ensemble (TRE), where the same DL model and dataset can be used to train an ensemble of DL models by filtering precipitation at different rain thresholds;
- the Convolutional Stacked Generalization model (ConvSG) for nowcasting based on convolutional neural networks, trained to combine the ensemble outputs and reduce CB in the prediction; and
- the enhanced stacked generalization (ESG), where the SG approach is integrated with orographic features, to further improve prediction accuracy on all rain regimes.

## Author Contributions

## Funding

## Acknowledgments

## Conflicts of Interest

## Abbreviations

QPF | Quantitative precipitation forecast |

QPE | Quantitative precipitation estimation |

CNN | Convolutional Neural Network |

RNN | Recurrent Neural Network |

LSTM | Long Sort-Term Memory |

TAASRAD19 | Trentino Alto Adige Südtirol Radar Dataset 2019 |

MAX(Z) | Maximum Vertical Reflectivity |

PPI | Plain Position Indicator |

CAPPI | Constant Altitude Plain Position Indicator |

LSTM | Long Short-Term Memory |

BL | Balanced Loss |

SG | Stacked Generalization |

CB | Conditional Bias |

ESG | Enhanced Stacked Generalization |

## References

- Werner, M.; Cranston, M. Understanding the Value of Radar Rainfall Nowcasts in Flood Forecasting and Warning in Flashy Catchments. Meteorol. Appl.
**2009**, 16, 41–55. [Google Scholar] [CrossRef] - Cuo, L.; Pagano, T.C.; Wang, Q.J. A Review of Quantitative Precipitation Forecasts and Their Use in Short- to Medium-Range Streamflow Forecasting. J. Hydrometeorol.
**2011**, 12, 713–728. [Google Scholar] [CrossRef] - Alfieri, L.; Salamon, P.; Pappenberger, F.; Wetterhall, F.; Thielen, J. Operational early warning systems for water-related hazards in Europe. Environ. Sci. Policy
**2012**, 21, 35–49. [Google Scholar] [CrossRef] - Koistinen, J.; Lerber, A.V.; Pulkkinen, S.; Sinisalo, H.; Berenguer, M.; Park, S.; Sempere, D.; Prudhomme, C.; Wong, W.K.; Baugh, C.; et al. Seamless probabilistic MUlti-source Forecasting of heavy rainfall hazards for European Flood awareness–SMUFF project. Geophys. Res. Abstr.
**2019**, 21, 1. [Google Scholar] - Heuvelink, D.; Berenguer, M.; Brauer, C.C.; Uijlenhoet, R. Hydrological application of radar rainfall nowcasting in the Netherlands. Environ. Int.
**2020**, 136, 105431. [Google Scholar] [CrossRef] [PubMed] - Marshall, J.S.; Palmer, W.M. The Distribution of Raindrops with size. J. Meteorol.
**1948**, 5, 165–166. [Google Scholar] [CrossRef] - Pulkkinen, S.; Nerini, D.; Pérez Hortal, A.; Velasco-Forero, C.; Germann, U.; Seed, A.; Foresti, L. Pysteps: An open-source Python library for probabilistic precipitation nowcasting (v1.0). Geosci. Model Dev.
**2019**, 12, 4185–4219. [Google Scholar] [CrossRef][Green Version] - Woo, W.C.; Wong, W.K. Operational Application of Optical Flow Techniques to Radar-Based Rainfall Nowcasting. Atmosphere
**2017**, 8, 48. [Google Scholar] [CrossRef][Green Version] - Foresti, L.; Sideris, I.V.; Nerini, D.; Beusch, L.; Germann, U. Using a 10-Year Radar Archive for Nowcasting Precipitation Growth and Decay: A Probabilistic Machine Learning Approach. Weather Forecast.
**2019**, 34, 1547–1569. [Google Scholar] [CrossRef] - Ryu, S.; Lyu, G.; Do, Y.; Lee, G. Improved rainfall nowcasting using Burgers’ equation. J. Hydrol.
**2020**, 581, 124140. [Google Scholar] [CrossRef] - Nerini, D.; Foresti, L.; Leuenberger, D.; Robert, S.; Germann, U. A reduced-space ensemble Kalman filter approach for flow-dependent integration of radar extrapolation nowcasts and NWP precipitation ensembles. Mon. Weather Rev.
**2019**, 147, 987–1006. [Google Scholar] [CrossRef] - Chung, K.S.; Yao, I.A. Improving radar echo Lagrangian extrapolation nowcasting by blending numerical model wind information: Statistical performance of 16 typhoon cases. Mon. Weather. Rev.
**2019**. [Google Scholar] [CrossRef] - Ayzel, G.; Heistermann, M.; Winterrath, T. Optical flow models as an open benchmark for radar-based precipitation nowcasting (rainymotion v0.1). Geosci. Model Dev.
**2019**, 12, 1387–1402. [Google Scholar] [CrossRef][Green Version] - Shi, X.; Chen, Z.; Wang, H.; Yeung, D.Y.; Wong, W.K.; Woo, W.C. Convolutional LSTM network: A machine learning approach for precipitation nowcasting. In Proceedings of the Advances in Neural Information Processing Systems 28 (NIPS 2015), Montreal, QC, Canada, 7–12 December 2015; Cortes, C., Lawrence, N.D., Lee, D.D., Sugiyama, M., Garnett, R., Eds.; pp. 802–810. [Google Scholar]
- Shi, X.; Gao, Z.; Lausen, L.; Wang, H.; Yeung, D.Y.; Wong, W.K.; Woo, W.C. Deep learning for precipitation nowcasting: A benchmark and a new model. In Proceedings of the Neural Information Processing Systems 30 (NIPS 2017), Long Beach, CA, USA, 4–9 December 2017; Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R., Eds.; pp. 5617–5627. [Google Scholar]
- Wang, Y.; Zhang, J.; Zhu, H.; Long, M.; Wang, J.; Yu, P.S. Memory In Memory: A Predictive Neural Network for Learning Higher-Order Non-Stationarity from Spatiotemporal Dynamics. arXiv
**2019**, arXiv:1811.07490. [Google Scholar] - Wang, Y.; Long, M.; Wang, J.; Gao, Z.; Yu, P.S. PredRNN: Recurrent Neural Networks for Predictive Learning using Spatiotemporal LSTMs. In Proceedings of the Neural Information Processing Systems 30 (NIPS 2017), Long Beach, CA, USA, 4–9 December 2017; Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R., Eds.; pp. 879–888. [Google Scholar]
- Wang, Y.; Long, M.; Wang, J.; Gao, Z.; Yu, P.S. PredRNN++: Towards a resolution of the deep-in-time dilemma in spatiotemporal predictive learning. PLMR
**2018**, 80, 5123–5132. [Google Scholar] - Ayzel, G.; Heistermann, M.; Sorokin, A.; Nikitin, O.; Lukyanova, O. All convolutional neural networks for radar-based precipitation nowcasting. Procedia Comput. Sci.
**2019**, 150, 186–192. [Google Scholar] [CrossRef] - Agrawal, S.; Barrington, L.; Bromberg, C.; Burge, J.; Gazen, C.; Hickey, J. Machine Learning for Precipitation Nowcasting from Radar Images. arXiv
**2019**, arXiv:1912.12132. [Google Scholar] - Tran, Q.K.; Song, S.k. Multi-Channel Weather Radar Echo Extrapolation with Convolutional Recurrent Neural Networks. Remote. Sens.
**2019**, 11, 2303. [Google Scholar] [CrossRef][Green Version] - Goodfellow, I.; Bengio, Y.; Courville, A. Deep Learning; MIT Press: Cambridge, MA, USA, 2016; Available online: http://www.deeplearningbook.org (accessed on 7 March 2010).
- Frei, C.; Isotta, F.A. Ensemble Spatial Precipitation Analysis from Rain-Gauge Data—Methodology and Application in the European Alps. J. Geophys. Res. Atmos.
**2019**, 124, 5757–5778. [Google Scholar] [CrossRef][Green Version] - Ciach, G.J.; Morrissey, M.L.; Krajewski, W.F. Conditional bias in radar rainfall estimation. J. Appl. Meteorol.
**2000**, 39, 1941–1946. [Google Scholar] [CrossRef] - Tran, Q.K.; Song, S.K. Computer Vision in Precipitation Nowcasting: Applying Image Quality Assessment Metrics for Training Deep Neural Networks. Atmosphere
**2019**, 10, 244. [Google Scholar] [CrossRef][Green Version] - Cao, Y.; Li, Q.; Shan, H.; Huang, Z.; Chen, L.; Ma, L.; Zhang, J. Precipitation Nowcasting with Star-Bridge Networks. arXiv
**2019**, arXiv:1907.08069. [Google Scholar] - Song, K.; Yang, G.; Wang, Q.; Xu, C.; Liu, J.; Liu, W.; Shi, C.; Wang, Y.; Zhang, G.; Yu, X.; et al. Deep Learning Prediction of Incoming Rainfalls: An Operational Service for the City of Beijing China. In Proceedings of the 2019 International Conference on Data Mining Workshops (ICDMW), Beijing, China, 8–11 November 2019; pp. 180–185. [Google Scholar]
- Brehmer, J.R.; Strokorb, K. Why scoring functions cannot assess tail properties. Electron. J. Stat.
**2019**, 13, 4015–4034. [Google Scholar] [CrossRef] - Bauer, P.; Thorpe, A.; Brunet, G. The quiet revolution of numerical weather prediction. Nature
**2015**, 525, 47–55. [Google Scholar] [CrossRef] [PubMed] - Surcel, M.; Zawadzki, I.; Yau, M. On the filtering properties of ensemble averaging for storm-scale precipitation forecasts. Mon. Weather Rev.
**2014**, 142, 1093–1105. [Google Scholar] [CrossRef] - Wolpert, D.H. Stacked generalization. Neural Netw.
**1992**, 5, 241–259. [Google Scholar] [CrossRef] - Van der Laan, M.J.; Polley, E.C.; Hubbard, A.E. Super learner. Stat. Appl. Genet. Mol. Biol.
**2007**, 6. [Google Scholar] [CrossRef] [PubMed] - Franch, G.; Maggio, V.; Jurman, G.; Coviello, L.; Pendesini, M.; Furlanello, C. TAASRAD19 Radar Scans 2010–2016. Available online: http://dx.doi.org/10.5281/zenodo.3577451 (accessed on 7 March 2010).
- Franch, G.; Maggio, V.; Jurman, G.; Coviello, L.; Pendesini, M.; Furlanello, C. TAASRAD19 Radar Scans 2017–2019. Available online: http://dx.doi.org/10.5281/zenodo.3591396 (accessed on 7 March 2010).
- Chen, L.; Zhang, H.; Xiao, J.; Nie, L.; Shao, J.; Liu, W.; Chua, T.S. Sca-cnn: Spatial and channel-wise attention in convolutional networks for image captioning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 5659–5667. [Google Scholar]
- Weigel, A. Chapter 8—Ensemble forecasts. In Forecast Verification: A Practitioner’s Guide in Atmospheric Sciences; Jolliffe, I.T., Stephenson, D.B., Eds.; Wiley-Blackwell: Hoboken, NJ, USA, 2012; pp. 141–166. [Google Scholar]
- Breiman, L. Bagging predictors. Mach. Learn.
**1996**, 24, 123–140. [Google Scholar] [CrossRef][Green Version] - Kingma, D.P.; Ba, J. Adam: A method for stochastic optimization. arXiv
**2014**, arXiv:1412.6980. [Google Scholar] - Ioffe, S.; Szegedy, C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv
**2015**, arXiv:1502.03167. [Google Scholar] - He, K.; Zhang, X.; Ren, S.; Sun, J. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE International Conference on Computer Vision, Las Condes, Chile, 11–18 December 2015; pp. 1026–1034. [Google Scholar]
- Houze, R.A., Jr.; Houze, R.A. Orographic Effects on Precipitating Clouds. Rev. Geophys.
**2012**, 50, 1–47. [Google Scholar] [CrossRef] - Foresti, L.; Seed, A. The effect of flow and orography on the spatial distribution of the very short-term predictability of rainfall from composite radar images. Hydrol. Earth Syst. Sci.
**2014**, 18, 4671–4686. [Google Scholar] [CrossRef][Green Version] - Bachmann, K.; Keil, C.; Weissmann, M. Impact of radar data assimilation and orography on predictability of deep convection. Q. J. R. Meteorol. Soc.
**2019**, 145, 117–130. [Google Scholar] [CrossRef][Green Version] - Foresti, L.; Seed, A. On the spatial distribution of rainfall nowcasting errors due to orographic forcing. Meteorol. Appl.
**2015**, 22, 60–74. [Google Scholar] [CrossRef] - Falorni, G.; Teles, V.; Vivoni, E.R.; Bras, R.L.; Amaratunga, K.S. Analysis and characterization of the vertical accuracy of digital elevation models from the Shuttle Radar Topography Mission. J. Geophys. Res. Earth Surf.
**2005**, 110. [Google Scholar] [CrossRef] - Neteler, M.; Mitasova, H. Open Source GIS: A GRASS GIS Approach; Springer Science & Business Media: Berlin/Heidelberg, Germany, 2013; Volume 689. [Google Scholar]
- Seed, A.W. A Dynamic and Spatial Scaling Approach to Advection Forecasting. J. Appl. Meteor.
**2003**, 42, 381–388. [Google Scholar] [CrossRef]

**Figure 1.**An example of observed radar reflectivity scan (MAX(Z) product) available in the TAASRAD19 dataset, represented in color scale over the geographical boundaries of the area covered by the radar. The area outside the observable radar radius is shaded.

**Figure 2.**Data architecture of the study. The predictions generated by the ensemble on the test set were used to train, validate and test the stacked model.

**Figure 3.**Schema of the deep learning architecture adopted by TrajGRU, in a configuration with two input and two output frames.

**Figure 4.**Average pixel values (normalized dBZ) of the predictions generated by the 4 models on the test set. When progressively raising the rainfall threshold in the loss, the resulting models progressively increase the total amount of predicted precipitation.

**Figure 5.**Ensemble prediction with TRE valid at 00:20 UTC 26 April 2017 (best viewed in color). The first row shows the five input scans (25 min), while the subsequent rows show the observation (ground truth) and the four models’ output. Observation and prediction are sub-sampled one every two images (10 min) to improve representation clarity. The ensemble spread can be observed when rising the threshold value.

**Figure 6.**Distribution of the rain rate values for the three sets used for: training (

**a**); validation (

**b**); and testing (

**c**). (

**d**) The plot of the distribution of the reflectivity values in the three sets. Zero values are removed since they dominate the distribution.

**Figure 8.**Overview of the three orographic features used for the ESG model: (

**a**) elevation map resampled over the radar grid at 500 × 500 m resolution; (

**b**) orientation derived from the elevation map, where the colors show the nearest cardinal direction N (0), E (90), S (180), and W (270); and (

**c**) percentage slope derived from the elevation.

**Figure 9.**Histograms of the three topographic features, elevation, aspect, and slope (from the top to the bottom). The Y axis of the histogram represents the pixel count for each bin, while the X axis is the value of the elevation in meters, the degree of orientation, and the slope percentage respectively. No data values are zeroed.

**Figure 10.**CSI score on test set. The dashed, squared, and plain patterns in the bars represent the three sets of light, medium, and heavy precipitation thresholds, respectively.

**Figure 11.**Comparison of ESG, ensemble members and average for CSI, FAR, and POD scores on heavy and severe rain-rates (10, 20, and 30 mm/h): (

**a**) CSI-10; (

**b**) CSI-20; (

**c**) CSI-30; (

**d**) FAR-10; (

**e**) FAR-20; (

**f**) FAR-30; (

**g**) POD-10; (

**h**) POD-20; and (

**i**) POD-30.

**Figure 12.**Continuous score performance of the model: (

**a**) mean squared error; (

**b**) normalized mean square error; (

**c**) mean absolute error; and (

**d**) conditional bias (closer to 1 is better).

**Figure 13.**TRE Ensemble members, Ensemble average, S-PROG, and ConvSG (Ens + Oro) prediction on test at 1535 UTC 03 July 2018 (best viewed in color). The first row shows the five input scans (25 min), while the subsequent rows (50 min) show the observation (ground truth), the four models’ output, the ensemble average, the Lagrangian extrapolation model, and the stacked generalization output.

Dataset | Sampling Strategy | Nr. Images |
---|---|---|

Training | 67,122 first image of each seq | 67,122 |

Validation | 2189 (3%) seq. × 20 images | 43,780 |

Testing | 6840 (9%) seq. × 20 images | 136,800 |

**Table 2.**CSI forecast skill of the ESG models compared with the ensemble (higher is better). In bold is the best result, the second best is underlined.

CSI Threshold (mm/h) | 0.1 | 0.2 | 0.5 | 1 | 2 | 5 | 10 | 20 | 30 |
---|---|---|---|---|---|---|---|---|---|

S-PROG | 0.557 | 0.502 | 0.377 | 0.241 | 0.140 | 0.076 | 0.053 | 0.037 | 0.027 |

TrajGRU 0.03 mm | 0.618 | 0.553 | 0.444 | 0.353 | 0.270 | 0.155 | 0.067 | 0.016 | 0.004 |

TrajGRU 0.06 mm | 0.611 | 0.567 | 0.449 | 0.350 | 0.268 | 0.165 | 0.089 | 0.031 | 0.012 |

TrajGRU 0.1 mm | 0.580 | 0.567 | 0.457 | 0.353 | 0.259 | 0.166 | 0.090 | 0.031 | 0.011 |

TrajGRU 0.3 mm | 0.611 | 0.570 | 0.468 | 0.345 | 0.256 | 0.162 | 0.080 | 0.028 | 0.010 |

Ensemble AVG | 0.625 | 0.577 | 0.466 | 0.357 | 0.270 | 0.171 | 0.081 | 0.025 | 0.007 |

ConvSG (Ensemble) | 0.624 | 0.546 | 0.420 | 0.344 | 0.272 | 0.164 | 0.086 | 0.034 | 0.014 |

ConvSG (Single + Oro) | 0.627 | 0.575 | 0.463 | 0.357 | 0.269 | 0.166 | 0.098 | 0.046 | 0.022 |

ConvSG (Ens + Oro) | 0.628 | 0.577 | 0.466 | 0.360 | 0.273 | 0.171 | 0.099 | 0.048 | 0.026 |

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Franch, G.; Nerini, D.; Pendesini, M.; Coviello, L.; Jurman, G.; Furlanello, C.
Precipitation Nowcasting with Orographic Enhanced Stacked Generalization: Improving Deep Learning Predictions on Extreme Events. *Atmosphere* **2020**, *11*, 267.
https://doi.org/10.3390/atmos11030267

**AMA Style**

Franch G, Nerini D, Pendesini M, Coviello L, Jurman G, Furlanello C.
Precipitation Nowcasting with Orographic Enhanced Stacked Generalization: Improving Deep Learning Predictions on Extreme Events. *Atmosphere*. 2020; 11(3):267.
https://doi.org/10.3390/atmos11030267

**Chicago/Turabian Style**

Franch, Gabriele, Daniele Nerini, Marta Pendesini, Luca Coviello, Giuseppe Jurman, and Cesare Furlanello.
2020. "Precipitation Nowcasting with Orographic Enhanced Stacked Generalization: Improving Deep Learning Predictions on Extreme Events" *Atmosphere* 11, no. 3: 267.
https://doi.org/10.3390/atmos11030267