Using the Weather Research and Forecasting (WRF) Model for Precipitation Forecasting in an Andean Region with Complex Topography

Abstract

The Weather Research and Forecasting (WRF) model has been successfully used in weather prediction, but its ability to simulate precipitation over areas with complex topography is not optimal. Consequently, WRF has problems forecasting rainfall events over Chilean mountainous terrain and foothills, where some of the main cities are located, and where intense rainfall occurs due to cutoff lows. This work analyzes an ensemble of microphysics schemes to enhance initial forecasts made by the Chilean Weather Agency in the front range of Santiago. We first tested different vertical levels resolution, land use and land surface models, as well as meteorological forcing (GFS/FNL). The final ensemble configuration considered three microphysics schemes and lead times over three rainfall events between 2015 and 2017. Cutoff low complex meteorological characteristics impede the temporal simulation of rainfall properties. With three days of lead time, WRF properly forecasts the rainiest N-hours and temperatures during the event, although more accuracy is obtained when the rainfall is caused by a meteorological frontal system. Finally, the WSM6 microphysics option had the best performance, although further analysis using other storms and locations in the area are needed to strengthen this result.

1. Introduction

Natural disasters with hydrological, meteorological or climatological origin are a severe and worldwide problem which causes loss of life and property damage [1]. Global warming appears to be positively correlated with future flood risk at global scale: an increase of 4 °C will drastically increase flood risk in several countries representing ∼70% of the world population [2]. Unevenness in future rainfall projections over South America doesn’t facilitate future flood risk projections, although, in the past two decades, several flood events took place in Andean regions [3]. For example, Chile’s Atacama Desert—the driest desert in the world—recorded 65 mm of rainfall in just three days in March 2015 [4,5]. This rainfall unleashed a torrential flood due to the combined effect of high temperatures, a steep topography and erosion favored by the soil granulometry and its infiltration capacity [6]. The event caused several human lives to be lost, the interruption of water and electricity supplies for weeks and an estimated economic cost of ∼US$$1.5$ billion [7].

Numerical weather prediction (NWP) models are broadly and successfully used for weather prediction and research. However, the simulation of rain events over complex orography (i.e., mountaneous regions) is still a challenge, as NWP models may not resolve the underlying topography within a high resolution [8,9]. Furthermore, complex topography affects the meteorology by modifying the surface heat flux, albedo, wind speed and direction [10,11,12,13]. A description of this complex topography helps to understand the behavior of the planet boundary layer (PBL) and microphysics (MP) schemes, allowing to study the slope wind-flow and valley wind-flow. However, none of the PBL parameterizations accurately predict the abrupt wind speeds and temperature profiles near the surface [14,15,16]. Increasing the vertical resolution of the computation grid near the ground-surface has been done to improve the modeling results in places with complex topography [17]. The finer grid provides more details in complex orographic zones, and a better performance depending on the regional climatology [18]. Nevertheless, a finer domain grid resolution has a limited impact on the traditional verification scores, and it doesn’t always improve the rainfall forecast.

A method for improving the performance of NWP models is the use of an ensemble approach [19,20,21,22]. This approach consists of repeating simulations under the same initial and boundary conditions but varying the physics scheme parametrization each time. This parametrization typically includes: radiative transfer, vegetation and soil characteristics, microphysics, and flux interaction of heat, moisture and momentum in the soil/atmosphere interphase, among others. Ensembles’ performances have been widely used by different authors [23,24,25,26,27] for capturing climate projections uncertainties, or achieving an optimal physics scheme parametrization. On the other hand, forecasting models are in general very dependent on the local condition (i.e., topography, hydrology, time of year, etc.). To overcome this issue, an iterative testing exploring different parameters’ values is a suitable tool for knowing which parameters need to be specified and investigated with greater accuracy [28]. Furthermore, a realistic model performance cannot be attributed to the achievement of a single scheme parametrization. Due to the complexity and nonlinearity of the atmospheric equations, the totality of the schemes’ parametrizations is involved in the output performance.

Another issue commonly tested for rainfall prediction over big watersheds is the forecast lead-time, or the time prior to the forecast date. An appropriate lead-time can considerably enhance the simulation accuracy. Rainfall forecast performance is sensitive to the temporal and spatial scale, becoming worse with lead-times longer than five to six days, and eventually meaningless after nine days [29]. Advances in NWP have made lead times of two weeks feasible [30], but forecast is more reliable for shorter lead-times, i.e., three days, for lighter rainfalls within bigger basins [29]. Hence, a rainfall forecast in a small watershed for heavy rainfalls is a challenge, even when shorter lead-times are considered. On the other hand, models also require 6–12 h for spin up, so if the initialization is too close in time to the actual storm onset, the model may miss some early-stage precipitation.

Even though many studies using the Weather Research and Forecasting (WRF) model [31] have already embraced its limitations over complex topography [32,33,34,35,36], few studies have focused on the South American Andean mountainous region, where the Andes cordillera reaches more than 5000 m.a.s.l. within 200 km from the Pacific sea border. WRF was used over the Nahuelbuta Mountains, in coastal southern Chile (${37}^{\circ }$ S–${38}^{\circ }$ S), to successfully simulate observed seasonal and daily mean rainfall distributions [37]. WRF was also used to study the direct effect of the Andean topography on wind speed and direction over the Argentinian foothill (Mendoza) [38]. Finally, WRF was used to forecast urban ${\mathrm{PM}}_{10}$ and ${\mathrm{PM}}_{2.5}$ pollution events over Santiago’s foothill, but their physical scheme combination did not comprise any representation of rainfall events [39]. They only briefly discussed the modeling of the coastal lows development into Santiago’s valley. On the other hand, the Chilean Meteorological Agency (DMC in Spanish) performs daily WRF simulations with a physics scheme calibrated to predict weather in Chile Central Valley, a 1200 km length region, which barely captures the Andean mountain and foothill topography. These simulations are made using a 6 km horizontal resolution and 50 vertical levels, more densified in the top and bottom boundaries.

The objectives of this work are to study and test different WRF configurations and physic processes parameterizations for the forecasting of rainfall over mountainous Central Chile, as well as to test the effects of different lead-times. The performance of the WRF simulations is evaluated according to their ability to represent relevant characteristics of the temperature dynamics and rainfall events for prediction of floods (i.e., length, peak intensity, concurrent freezing level and the rainiest N-consecutive hours) over the Andean front range in central Chile, particularly the Quebrada de Ramón catchment, a small basin located in the area. In our analysis, we used three storm events with different characteristics that took place between 2015 and 2017.

2. Data and Methods

2.1. Field Data

The study area corresponds to the Quebrada de Ramón basin, a 36 km${}^2$ mountainous watershed located in central Chile whose highest elevation is 3250 m. Its complex and steep topography has a maximum elevation gradient of 220 m per 1 km and a high average slope (∼38%) [40]. Its outlet is located at an elevation of 800 m, in the east part of Santiago, the capital of Chile. The urban dynamics in this area neither considered high return period flows nor detrital floods. A clear example of the consequences took place on 3 May 1993, when a warm storm produced large floods and landslides in this and other front range basins nearby [41]. The flood injured 3500 people and caused 50 human losses. The situation was aggravated by the presence of houses located in the Piedmont and the floodplains [42].

Santiago (${33}^{\circ }30$${}^{\prime }$ S; ${70}^{\circ }42$${}^{\prime }$ W) exhibits a semi-arid Mediterranean climate, with a mean annual precipitation of 310 mm and about 25 rainy days during the austral winter (May–September) and warm, dry summers [43]. This general weather pattern has remained constant over time, since no trends were found on monthly rainfall, mean and maximum annual precipitation for the period 1950–2018 using the Mann–Kendall test [44,45], with a $5\%$ significance level. Nevertheless, central Chile—and Santiago in particular—have experienced a prolonged drought since 2010 onwards that seems to herald drier conditions in the future [46].

For this study, data were collected from the Apoquindo meteorological station, located near Quebrada de Ramón’s centroid (${33}^{\circ }26$${}^{\prime }$ S; ${70}^{\circ }28$${}^{\prime }$ W) at an elevation of 1625 m.a.s.l. Six meteorological variables are registered every 10 min: temperature, relative humidity, solar radiation, wind speed and direction, and rainfall (Figure 1). Moreover, hourly and daily precipitation and temperature data from 13 meteorological stations located in Santiago and its surroundings were used to verify the orographic gradient and to analyze the return period of the storms events (

Table 1. Meteorological stations names, coordinates and recorded variables: T (air temperature) and P (precipitation). Stations located inside Santiago are indicated with *.

Name	ID	Latitude	Longitude	Elevation (m.a.s.l.)	Variables
San José Guayacán	SJ	${33}^{\circ }37$${}^{\prime }$ S	${70}^{\circ }21$${}^{\prime }$ W	928	Hourly T, P
Apoquindo	AP	${33}^{\circ }27$${}^{\prime }$ S	${70}^{\circ }28$${}^{\prime }$ W	1625	Hourly T, P
Quebrada de Macul *	QM	${33}^{\circ }30$${}^{\prime }$ S	${70}^{\circ }31$${}^{\prime }$ W	950	Daily T, P
Antupirén *	AN	${33}^{\circ }30$${}^{\prime }$ S	${70}^{\circ }31$${}^{\prime }$ W	904	Daily P
Cerro Calán *	CC	${33}^{\circ }24$${}^{\prime }$ S	${70}^{\circ }32$${}^{\prime }$ W	904	Daily T, P
Tobalaba *	TO	${33}^{\circ }27$${}^{\prime }$ S	${70}^{\circ }33$${}^{\prime }$ W	650	Daily T, P
La Platina *	PL	${33}^{\circ }34$${}^{\prime }$ S	${70}^{\circ }38$${}^{\prime }$ W	630	Hourly T, P
Quinta Normal *	QN	${33}^{\circ }27$${}^{\prime }$ S	${70}^{\circ }41$${}^{\prime }$ W	534	Daily T, P
Lo Pinto *	PI	${33}^{\circ }16$${}^{\prime }$ S	${70}^{\circ }44$${}^{\prime }$ W	512	Hourly T, P
San Pablo *	SP	${33}^{\circ }27$${}^{\prime }$ S	${70}^{\circ }45$${}^{\prime }$ W	490	Hourly T, P
Pudahuel	PU	${33}^{\circ }24$${}^{\prime }$ S	${70}^{\circ }48$${}^{\prime }$ W	482	Daily T, P
Rinconada de Maipú	RM	${33}^{\circ }30$${}^{\prime }$ S	${70}^{\circ }51$${}^{\prime }$ W	462	Hourly T, P
Hacienda Lampa	HL	${33}^{\circ }17$${}^{\prime }$ S	${70}^{\circ }51$${}^{\prime }$ W	493	Hourly T, P
El Paico	PA	${33}^{\circ }42$${}^{\prime }$ S	${70}^{\circ }60$${}^{\prime }$ W	275	Hourly T, P

).

The freezing level location (i.e., a elevation representative of the snow line that controls the spatial occurrence of rain and snow) was obtained from the Santo Domingo atmospheric sounding station record [47]. Soundings are made twice a day (00Z and 12Z) at 105 km from the city of Santiago (${33}^{\circ }65$${}^{\prime }$ S; ${71}^{\circ }61$${}^{\prime }$ W). These measurements are considered to describe Santiago atmosphere’s features, as the city is located within the area of measurements’ representativeness (defined by a 150 km radius), based on the negligible effects of topography and near coast interactions in the free troposphere [48].

2.2. Meteorological Characterization of Rainfall Events

For the the scheme parameterization and calibration of WRF, we considered three rain events: a frontal system rainfall (19 October 2015, hereafter referred to as OCT15), a cutoff low episode (17 April 2016, hereafter referred to as APR16) and a hybrid between cutoff low and frontal system rainfall (11 May 2017, hereafter referred to as MAY17). Cutoff lows are meteorological phenomena that take place in medium to high latitudes (${20}^{\circ }$ S–${40}^{\circ }$ S), more likely during autumn and winter (i.e., from April to September) [49,50]. They can cause extreme cold weather and precipitation in high elevations above 1000 m, coupled with strong winds and occasional thunderstorms [51]. These three rainfall events are considered to be representative of the possible range of rainfall events in this region.

Meteorological forcing data were obtained from NCEP/NCAR Reanalysis. In this study, we use the period 1981–2010 for computing the mean anomalies during the studied rainfall events. An anomaly shows the difference of any meteorological variable from its long-period mean value for a given location.

2.2.1. 19 October 2015 Rainfall Event (OCT15)

Between 16–20 October 2015 (largest intensities between 19 October 2015 11:00 a.m. UTC and 20 October 2015 12:00 a.m. UTC), Chilean central valleys were affected by an event that left a relatively high precipitation over the mountainous region (i.e., 46 mm in the Apoquindo gauge vs. 21 mm in Quinta Normal station, Table 1), whose rainiest day had a return period of $2.3$ years according to the Quinta Normal records. This event was caused by a cutoff low developed over the Southeast Pacific (${80}^{\circ }$–90${}^{\circ }$ W and ${30}^{\circ }$ S), which produced a nucleus of negative anomalies in the 500 hPa geopotential level, with a magnitude of $-150$ m. The event was intensified by the jet stream at 250 hPa, with its nucleus of largest intensities (30 m/s positive anomalies) located just above the Chilean central region (${30}^{\circ }$ S) (Figure 2, upper row).

2.2.2. 17 April 2016 Rainfall Event (APR16)

In 15–17 April 2016 (15 April 2016 9:00 a.m. UTC to 17 April 2016 8:00 p.m. UTC), an extreme rainfall event affected Chile’s central region. On 17 April, 108 mm of rainfall were measured at the Apoquindo meteorological station in 24 h. According to Quinta Normal historical records, the maximum hourly rainfall of this event had a 46-year return period, even though the accumulated rainfall of 14 h within the storm peak had a return period of 10 years [52].

The frontal system moved from north to south, with negative anomalies in the atmospheric pressure and geopotential height at 500 hPa ($-150$ geopotential meters). At the same time, an intensification of both subtropical and polar jet streams took place at 250 hPa, allowing higher wind intensities in the system nucleus (positive anomalies of 35 m/s in the wind vectors), favoring the cloudiness development over central Chile (Figure 2, central row).

Satellite images showed a significant moisture contribution, advected from lower latitudes, which incorporated more precipitable water into the system. This quasi-stationary weather front led to a warm winter storm-like disturbance [53], allowing more rainfall to precipitate over central Chile. During most parts of a warm storm, precipitation is caused by mechanical uplift of moist air over the Andes, developing its maximum precipitation in mid-elevations inlands. This behavior is consistent for several warm storms in other world regions, where heavy rainfall and flood events happen [53].

Additionally, the freezing level was above 3000 m, causing a large contribution of liquid precipitation at relatively high temperatures. In fact, during 16 April, the Mapocho river in Santiago’s basin, flooded the central part of the city, where a riverbed modification due to a construction took place. Because of its large duration, maximum intensity and total amount, this rainfall event was used for initial WRF modifications over the grid discretization, land use and initial and boundary condition data.

2.2.3. 11 May 2017 Rainfall Event (MAY17)

With 53 mm of rainfall recorded in Apoquindo station in two days, the event of 11 May 2017 4:00 a.m. UTC to 12 May 2017 1:00 a.m. UTC affected a large portion of Chile. This event corresponds to a return period of three years in Quinta Normal hourly records. In the previous days, the system presented a trough with closed circulation at 500 hPa, a typical cutoff low characteristic. Additionally, the jet stream at 250 hPa had a strong zonal direction in the trough posterior part (Figure 2, bottom row).

During the rainfall event, a low pressure center on the surface associated with the weather front was observed. At higher elevations, the typical cutoff low pattern was not visible, as there was no clear closed-circulation and the jet stream was coupled to westerlies. At 500 hPa, the trough contributed with additional divergence, generating more cloudiness over the affected area.

2.3. Meteorological Forcing Data

ARW-WRFv $3.5$ [31], a state-of-the-art mesoscale NWP model was used in this work. The model is suitable for a wide range of applications, such as weather routine forecast, research simulations, and evaluation of parameters in simulated systems. WRF solves the scalar conservation and compressible non-hydrostatic Euler equations, through vertical coordinates ($\eta$ levels) with a variable grid density. WRF system is supported and maintained by NCAR [54].

Initial and boundary meteorological conditions according to the NCEP operational Global Forecast System (GFS) and the NCEP Final Operational Global Analysis data (FNL) were initially compared. GFS is composed of four blocks (atmosphere, ocean, land/soil and sea/ice), providing a ${0.5}^{\circ }$ resolution grid with the forecast of atmospheric and land–soil variables every 3 h for modeling periods after 2006 [55]. Operational forecasters, such as the DMC in Chile, use GFS as boundary condition to simulate precipitation at finer resolution via NWP. On the other hand, FNL is a $1^{\circ }$ resolution grid map produced every 6 h, prepared approximately an hour after GFS started, which allows more observational data to be used (∼10%) in the upgrade of initial and boundary conditions. FNL data are available for periods after 1999 [56]. Finally, sea surface temperature (SST) at a ${0.5}^{\circ }$ resolution from the NCEP SST analyses database were also provided to the WRF model.

An initial testing over GFS/FNL meteorological forcing considered three different lead times for APR16 event (i.e., 120, 96 and 72 h): although both datasets performed similarly, the rainfall total length provided by GFS was more feasible. Thus, only the GFS dataset was used in the simulations.

2.4. WRF Model and Physics Schemes

Initial testing over the APR16 event were made focused over (1) two vertical level resolutions, (2) two land use and three land surface models. Results from these initial simulations allowed the definition of the final schemes parameterization and grid resolution, which was used in a final set of simulations to test different MP schemes and lead times. The forecasts were carried out with a 32-core machine ($2.30$ GHz Intel Xeon E5-2698 v3 in Santiago, Chile), taking less than 5 h to retrieve operational daily forecasts at a resolution of 6 km. This time considers 1 h in data preparation and downloading of initial and boundary conditions, (GFS ${0.5}^{\circ }$ for DMC forecasts), $2.5$ h in running WRF and 45–60 min in post-processing.

2.4.1. Simulation Domains and Topography Complexity

The domain of the WRF simulations was composed of three nested grids with 54, 18 and 6 km of resolution. These were based on a Lambert Projection centered at Santiago de Chile, interacting with each other through a two-way nesting strategy (Figure 3). The biggest or parent domain (d01, 54 km) covered South America’s western region, the Pacific and part of the Atlantic Ocean. As recommended by the DMC, the objective of its extent is to embrace South America and a good portion of the Pacific Ocean, which allows capturing phenomena approaching the continent beforehand. The first nested domain (d02, 18 km) embraced Chilean and Argentinian central regions and the Pacific Ocean near the shore. The innermost domain (d03, 6 km), the main focus of this study, covered a reduced portion of the Pacific Ocean, central Chile and the Andes mountains.

WRF can numerically diverge with high elevation gradients, resulting in anomalous vertical wind speeds, with gradients of ∼300 m per 1 km [7]. Moreover, due to the model grid resolution, the complex topography produced a considerable overestimation of Apoquindo station’s elevation ($+314$ m). Following the approach by Carvalho et al. [57], the Apoquindo station location in WRF was moved within a 5 km radius. The best output location to represent the station was 5 km further north of its real location, which produced a height underestimation of 200 m, but a better performance of the rainfall temporal distribution.

The prediction of the microphysics and PBL processes, and thus the rainfall, is expected to improve with a finer vertical grid resolution. However, doubling the number of vertical levels from 31 to 62 did not enhance quantitative precipitation forecasts in the central US [58]. Based on previous configurations [59,60], Saide et al. [39] proposed an optimal density of 39 vertical levels in the study region, with a first layer at 10 m and six levels below 100 m, which allowed the best forecasting of wind speed, temperature and chemical compound concentrations. This discretization was compared against one with 50 vertical levels currently used by DMC. This finer discretization reproduced temperature profiles and the saturated atmosphere near the surface more accurately, and thus it was chosen as the default discretization for the following simulations.

2.4.2. Physics Schemes and Land Use

Short and long wave radiation schemes determine radiative fluxes in the simulation. The Dudhia short wave scheme [61] and the RRTM long wave radiation scheme [62] were chosen. On the other hand, cumulus schemes simulate the sub-grid processes related to convective clouds. Given the WRF capacity to explicitly solve this, a parameterization is not needed in the inner domain. In the external domains, the Grell 3D Ensemble [63] was used because of its acceptable performance when simulating convective rainfall over Santiago [39].

WRF’s Land Use (LU) for terrain characterization uses different categories to depict landscapes, crops, vegetation, forests, and urban areas. A comparison among LU datasets shows that the USGS (U.S. Geological Survey) underestimates the extension of Santiago area, probably due to the use of old maps. Hence, MODIS data were chosen for further simulations.

Land Surface models (LSM) calculate heat and moisture fluxes above the land, sea and ice cover. The simplest LSM physics option that considers a five-layer model for thermal diffusion was not considered, as it neither includes vegetation effects, nor the changes in snow cover, or soil moisture over time. Soil moisture is the most significant part of flux exchanges between the surface and the first level of the model, and also a crucial factor that affects near surface temperature and wind [15]. Noah-MP LSM was finally chosen given the improvements to the simulation of surface fluxes, timing of snow water equivalent and runoff peaks [64].

The PBL scheme determines surface heat and moisture fluxes due to eddy transports in the remaining volume of the atmospheric column over the terrain. In this study, the Mellor–Yamada–Nakanishi–Niino (MYNN) Level $2.5$ parameterization was used as a local approach with total kinetic energy (TKE) closure [65], providing a good representation of good vertical profiles, which in turn produces a closer agreement with the planet boundary layer shape, magnitude and maximum values. In the Santiago area, the MYNN scheme represented accurately the diurnal PBL height [39]. PBL was not tested in this study because WRF simulations of precipitation have been shown to weakly depend on land surface, PBL and radiation parameterizations for winter storms over complex topography [66].

The final WRF configuration is summarized in

Table 2. Final WRF simulation schemes.

Physical Scheme	Parametrization
Short-wave radiation	Dudhia
Long-wave radiation	RRTM
Cumulus	Grell 3D Ensemble
Planet Boundary Layer	MMYN $2.5$
Soil Layer	MMYN
Land Surface Model	Noah-MP

, for which 50 vertical levels of vertical resolution are applied together with the GFS dataset.

2.4.3. Microphysics

Microphysics (MP) schemes explicitly resolve processes of water, cloud and precipitation, and their mixed-phases (i.e., ice–water interaction). The Lin et al. scheme (hereafter referred to as LIN) employs six forms of water (water vapor, cloud water, cloud ice, rain, snow and graupel). Moreover, this scheme allows the explicit inclusion of snow, and the correct simulation of changes from cloud ice to snow and then to graupel. In general, LIN better describes the dynamics of the clouds inner processes [67], being appropriated for high resolution simulations.

The WRF-Single-Moment-Microphysics scheme (WSM) varies according to the class, i.e., the number of prognostic water substance variables. The WSM 3-Class (hereafter referred to as WSM3), used in the DMC initial forecast, contains water vapor, cloud water/ice, and rain/snow. In this scheme, rain and water occur above the freezing level, and snow and ice below it. On the other hand, the WSM 6-Class (hereafter referred to as WSM6) contains mixing ratios of water vapor, cloud water, cloud ice, snow, rain and graupel, and thus is more appropriate for high resolution simulations. Although both WSM schemes have similar behaviors when simulating extreme rainfalls at low grid resolutions (∼45 km), total rainfall and maximum intensity are better simulated in WSM6 at finer resolutions (∼5 km) [68]. Moreover, for extreme rainfalls, the combined effect of microphysics and ice sedimentation (available in LIN and WSM6) provides a better representation of cloud covered areas, mean temperatures in the upper troposphere and surface rainfall amounts [69]. Nonetheless, in this study, we tested the LIN, WSM3 and WSM6 schemes.

2.4.4. Lead Time

Forecast accuracy depends not only on the parameterization of physical schemes and grid resolution, but also on the lead time. Commonly, forecasts get worse with increasing lead time because synoptic characteristics are constantly changing. A smaller outer domain and shorter lead times allow more accurate rainfall simulations, whereas larger domain sizes and lead times can increase inner variabilities in WRF, displacing the spatial rainfall band and affecting rainfall forecasting [70]. Although acceptable precipitation forecasting can be obtained with lead times up to six days, values of three days or less produce much better results [29]. In this study, lead times of 3, 4 and 5 days (72, 96 and 120 h) previous to the day with the maximum rainfall intensity were analyzed.

In summary, the final WRF configuration depicted in Table 2 will be used to test three MP parameterizations and three lead times, as described in

Table 3. Final WRF simulations with different lead times and MP schemes.

Rainfall Event	MP Schemes	Simulation Beginning (00:00)	Lead Times (h)	Simulations
OCT15	LIN & WSM6	15, 16 & 17 October 2015	72, 96 & 120	6
APR16	LIN, WSM3 & WSM6	12, 13 & 14 April 2016	72, 96 & 120	9
MAY17	LIN & WSM6	6, 7 & 8 May 2017	72, 96 & 120	6

. Due to computational resources constraints, all the MP parameterizations are considered for the APR16 event, but only the LIN and WSM6 parameterizations will be used for the OCT15 and MAY17 events. Because the more detailed depiction of water phases is associated with these last two parameterizations, their comparison fulfills the objective of finding the best MP scheme to forecast the selected rainfall events.

2.5. Model Validation

Observed data from the Apoquindo and other stations located in Santiago valley and Piedmont (Figure 1) were used to assess the quality of the meteorological simulations. Note that this assessment excludes the first day of all the simulations, used for warming up the model (spin-up period). Given the difference in the starting date of the simulations (lead time), the valid temporal range was used to compare them with observed data. Thus, we try to remove the bias to be able to compare each of the simulation’s performances. Simulated temperature and rainfall time series were assessed via the mean absolute error (MAE), a robust metric that prevents single events from having a large impact on the statistic [27].

$$MAE=\frac{1}{N}\sum _{t=1}^N\left|S_t-O_t\right|,$$

(1)

where N is the total number of data, and S_t and O_t are the simulated and observed meteorological data. In addition, we also use the concept of error, which corresponds to the difference between S_t and O_t at any time t. Finally, the simulations’ performances were also characterized through the Nash–Sutcliffe Efficiency coefficient (NSE):

$$NSE=1-\frac{{\sum }_{t=1}^N{\left(S_t-O_t\right)}^2}{{\sum }_{t=1}^N{\left(O_t-\overline{O_t}\right)}^2}.$$

(2)

NSE has a range [$-\infty$, 1], where values larger than 0 imply a prediction better than the average of the observations ($\overline{O}$). Since the NSE is extremely sensitive to outlier data, a modified version of the Index of Agreement (IoA) [71] is also used. IoA is a less sensitive to outliers metric that ranges [$-1$, 1]. Values bigger than 0.8 are considered to indicate a good performance of the model:

$$IoA=\left\{\begin{array}{cc}1-\frac{{\sum }_{t=1}^N\left|S_t-O_t\right|}{2{\sum }_{t=1}^N\left|O_t-\overline{O_t}\right|}\hfill & \mathrm{if}\sum _{t=1}^N\left|S_t-O_t\right|\le 2\sum _{t=1}^N\left|O_t-\overline{O_t}\right|,\hfill \\ \frac{2{\sum }_{t=1}^N\left|O_t-\overline{O_t}\right|}{{\sum }_{t=1}^N\left|S_t-O_t\right|}-1,\hfill & \mathrm{in}\mathrm{other}\mathrm{case}.\hfill \end{array}\right.$$

(3)

Finally, we adopted the tolerance criteria for absolute error, defined in

Table 4. Absolute difference tolerance criteria for simulated variables used by DMC.

Meteorological Variable	Absolute Difference Tolerance Criteria
Dew temperature	1 °C in surface, 2 °C in atmosphere
Temperature	1 °C in surface, 2 °C in atmosphere
Wind speed	2.57 m/s (∼5 knots) for all data
Wind direction	20${}^{\circ }$ in surface, 15${}^{\circ }$ in pressure levels above the 850 hPa

, used by DMC.

Non-zero hourly simulated rainfall pulses were assumed to be those larger than 1 mm, while an inter-event arrival time (IEAT) (i.e., the minimum dry time between two independent rainfall events) of 30 h recommended for Santiago [72] was adopted. To assess the temporal distribution of the most intense portion of the rainfall event, the rainiest N-consecutive hours (NRH) were computed, where N is an integer. This approach implies finding the N-consecutive hourly pulses with the largest amount of rainfall, with N integer from 1 (the rainiest hour) to 10 (the 10 consecutive hours with the largest amount of cumulative precipitation). Thus, it is possible to assess the simulation in terms of temporal distribution of the most intense rainfall pulses, which largely affect the basin hydrologic response. Temperature time series associated with each NRH (T${}_{NRH}$), were analyzed via MAE, emphasizing both precipitation and temperature when N = 5 h (i.e., NRH(5) and T${}_{NRH\left(5\right)}$). This time corresponds to the estimated time of concentration of Quebrada de Ramón, a metric that is representative of the hydrological response time of the catchment.

3. Results

3.1. Local Conditions

To assess the local performance of WRF over the rainfall events, we first characterized the overall rainfall orographical gradient using total rainfall amounts registered in all the meteorological stations (Table 1). Figure 4 compares total simulated rainfall amounts (considered to be the mean of the values simulated using the three lead times for every MP option) against the observed ones. Different markers according to the elevation are used in the figure. The left panels compare in every meteorological station simulated and observed data, and the dashed line corresponds to the 1:1 ratio. The right panels’ plot rainfall against height, the scatter in the plot represents simulated values for several locations in Santiago city and the west hill slope of the Andean foothills, to give a spatial context for the WRF rainfall distribution.

For the OCT15 rainfall event (Figure 4, upper row), WRF overestimate rainfall amounts, particularly at higher elevations. The orographical gradient inside Santiago city is accurately simulated, but the scatter plot shows more dispersion at elevations higher than 700 m.a.s.l. (i.e., the mountainous area of Santiago). The LIN MP option produces a bigger bias than the WSM6 option for all the points belonging to the Santiago basin, although this bias is reduced for lower elevations. In general, for the OCT15 event, WRF predicts slightly more rainfall as the latitude increases southwards, for the points inside Santiago, excluding the mountainous area. Overall, the WSM6 scheme provides better results (i.e., less biased) with a $5\%$ significance level (Mann–Kendall test [44,45]).

In the APR16 event (Figure 4, middle row), WRF overestimate rainfall amounts for all the stations located above 800 m.a.s.l., corresponding to Santiago’s foothills. Predictions for lower elevations are much better with small underestimations of the observed precipitation. WRF simulates a strong correlation between total precipitation and height, although the observed rainfall ranges between 80–120 mm. In the orographical gradient, rainfall was understimated in the west part of Santiago (lower elevation) and overestimated in the foothills. The scattered pattern appears once more, with more dispersion (and thus bigger biases) for the foothill stations located in the southern area of Santiago (AN, QM, and SJ stations). This also happened for the Apoquindo station, where a ∼100 mm bias was simulated. The bias reduces for gauges located at northern latitudes below 1000 m.a.s.l. (gauges CC and TO). Overall, the LIN scheme produced larger biases, whereas WSM3 and WSM6 perform similarly depending on the station. In fact, WSM3 performs better in the mountainous zone of Santiago (CC, AN, QM and AP), while WSM6 performs better in QN, PL and SJ (southwest Santiago valley and the mountainous area in the East, respectively). The biases are considerably larger in the mountainous zone, so no real advantage can be addressed over the WSM3 scheme.

Finally, for the MAY17 event (Figure 4, bottom row) rainfall is overestimated particularly at higher elevations, although the rainfall in the highest two stations was understimated by WRF. The smooth orographical gradient in the recorded data was captured in the WRF simulation. In Apoquindo station, both MP options (LIN and WSM6) produced similar bias (∼20 mm). Once more, the WSM6 scheme performed generally better as it produced smaller biases.

3.2. Rainfall and Temperature Simulation

In the OCT15 storm, simulations with 72 and 96 lead time could not simulate the rain pulse observed during 17 October. As a countermeasure, a base amount of precipitation (10 mm), equivalent to the registered precipitation until 17 October, was considered in Figure 5, to establish comparisons on even ground. The frontal system conditions of the OCT15 event allowed a more accurate prediction of the temporal distribution of the rainfall (IoA = $0.64$ and for the mean of LIN simulations, and $0.67$ for the mean of WSM6 simulations), since most MP schemes captured the initial rainfall (19 October). Simulations with 120 h of lead time also captured a previous pulse (17 October), as seen in Figure 5 (upper row). For shorter lead times, both MP schemes not only predicted the occurrence of another precipitation pulse (20 October), which is inexistent in observed data, but compensates the underrated rainfall pulse of 19 October. Given this additional pulse, the simulations with lead time of 72 h have a bias of less than 2 mm in the final hours of the rainfall event.

None of the simulations could accurately predict the APR16 rainfall event (Figure 5, left central plot). There was an initial ∼48 h lag between the observed and simulated rainfall peaks, as WRF predicts rainfall earlier. Thus, the rainfall beginning is considered to be 16 April, ignoring the WRF first rainfall pulse of 15 April by subtracting the cumulative rainfall until that date. This improved the model performance, as a smaller MAE (∼1 mm) and a larger mean IoA value are obtained for all MP options. Nevertheless, some values of IoA are still negative. Furthermore, for simulations with shorter lead times, the rainfall pulse was correctly predicted (17 April), especially with a lead time of 72 h. Overall, the hourly forecast of the last storm hours was slightly more accurate (∼−20 mm bias). As the OCT15 and MAY17 simulated events don’t show these errors, they are not explained by the complex topography, but by the rainfall properties which seem to be maximized in the initial GFS dataset. This error in the rainfall amount was observed in all the meteorological stations of Table 1 (not shown). Indeed, the APR16 event had unique meteorological characteristics and a large magnitude, which could partially explain the lag errors in the prediction.

In the MAY17 storm, the 72 h lead time simulations start after the rain pulse registered during 7 May. Therefore, the registered precipitation until that date was added as a base amount in Figure 5. The same as before, this was made to establish even comparisons. For the MAY17 rainfall simulations, there is an uneven ensemble performance for both MP options (LIN and WSM6 IoA ∼0.58), as the total rainfall amount is underestimated for all lead times (Figure 5, bottom row). However, most simulations capture the rainfall pulse of 7 May, with the exception of LIN72 (and WSM672 to a lesser extent) probably because its initialization was about 9 h before the initial pulse, within the model spin-up time period. Although all simulations capture the 10 May rainfall pulse, errors of ∼−40 mm obtained with lead times of 96 are not produced by the simulation with lead time of 120 or 72 h. For these two simulations, WRF simulated intermittent rainfall with lower intensities during 10–12 May for 120 h lead time, and 10–13 May for 72 h lead time. The observed intensities were higher during the first half of 10 May, but the WRF model could not capture them. The LIN scheme produces the closest steep slope in agreement with the observed data (the last hours of 9 May) for a lead time of 120 h, but the WSM6 scheme produces the same agreement with a 120 h or 72 h lead time.

For temperature time series, there is an accurate performance for the whole ensemble, since the temperature dynamics was captured for the OCT15 and MAY17 events with minor errors. The mean performance between both schemes was very similar, with IoA = $0.77$ for the OCT15 event and IoA = $0.69$ for the MAY17 event. Steep temperature increments are well captured by all simulations, but overlapping and disagreement appear just before the rain started (∼1 day, from the last hours of 18 October for OCT15, and from 10 May for MAY17, and ∼3 days, from 14 April for APR16), where a general temperature performance decrease takes place. Temperatures are overestimated in OCT15 and MAY17 events, just before the storm peak (19 April for APR16 and 10 May for MAY17). A constant underestimation between 14–17 April can be seen in APR16, where all schemes had a similar performance (IoA ∼0.47), probably due to the complex meteorological configuration of the event (Figure 5, right panels).

3.3. Rainfall Forecast Performance

In addition to the total rainfall amount and dynamics, another relevant attribute of precipitation events is the timing of the rainiest pulses. A good forecasting of this dynamics is essential for modeling and predicting peak flows in a catchment of a given size. To visualize the performance of the forecasting on this regard, we plot the observed and simulated N rainiest hours (NRH) for all the different MP parameterizations and lead times, with N ranging between 1 and 10 h (Figure 6). Overall, the wide range of simulated curves contains the observed NRH curve, although major differences between the observed and simulated curves for the APR16 event were observed. NRH(5) values are overstimated by all the ensemble. Unfortunately, the parameterization that best reproduces this value (WSM3${}_{120}$) has the largest associated MAE value (Figure 6, central row). Generally, the MAE of the T${}_{NRH}$ ensemble values were above the tolerance criterion shown in Table 4.

For the OCT15 rainfall event, a wide range of NRH(5) values is simulated, with all of them underestimating the observed value (22 mm). Nevertheless, the values provided by the LIN scheme are closer to the observed data than those from the WSM6 scheme. In general, a smaller lead time provides values closer to observations. As an ensemble, the average NRH(5) value associated with all the simulations underrates in ∼10 mm the observed data. For the MAY17 event, all of the NRH(5) values underestimate the observation, except the value from the LIN scheme with a 120 h lead time. Overall, mean WSM6 values for all the lead times are closer to the observed data ( ∼−10 mm bias). For the LIN scheme, smaller lead times are related with worst performance.

Among the used MP schemes, WSM6 shows more consistency, regardless of the lead time, and the errors are within the tolerance criterion for temperature (Table 4). LIN performs similarly, but results are less consistent and become worse with shorter lead times. Overall, the average WSM3 parameterization is the worst in reproducing NRH(5), while its relation with the lead time is less consistent than the LIN and WSM6 options for NRH(5) and T${}_{NRH\left(5\right)}$ MAE (Figure 6).

Total rainfall is overestimated for the APR16 event, whereas, for the other rainfall events, this amount is within the range of simulated values. These values tend to be better with shorter lead times, although no clear trend is observed. Furthermore, the average of the WSM6 simulations is worse than that of the LIN scheme in reproducing the observed total amount. On the other hand, the rainfall length is reasonably predicted by the ensemble for all the events, with WSM6 performing better than the LIN parameterization, although real values are not within the 25th–75th percentiles. Overall, and because it predicts the NRH(5) values, WSM6 is considered to be a good MP option to choose for rainfall forecasting in the study area, and eventually other front-range catchments nearby.

3.4. Freezing Level Height

The freezing level height is a relevant factor in the hydrological response of the catchment because it defines the area which receives rainfall instead of snowfall. We assessed the prediction of the freezing level height for different MP schemes and lead times, in order to find any advantage in a certain combination that could be useful under a flash flood warning scenario.

Freezing level height is quite well simulated by all the ensemble for the OCT15 and MAY17 events. Less differences between simulated and observed data are obtained during freezing level height declines. On the other hand, when the freezing level height increases, the ensembles are more discrepant. Regardless of the behavior of this level, during the rainfall peak, the error tends to be negligible (Figure 7). For APR16, the decline of the freezing level is overestimated, especially for longer lead times. This illustrates how the atmosphere stability development during cutoff low events is better captured using small lead times.

Shorter lead times improve the forecast of the OCT15 and MAY17 events, better capturing the freezing level increase ∼2 days before the peak intensity (Figure 7, right panels). Longer lead times could not accurately simulate this peak value. A lead time of 72 h (3 days) was optimal for capturing the freezing level height development. Even for APR16, where there’s no real agreement between simulated and real data, this lead time produces relatively small error values. No clear advantage was found in relation to the MP schemes, which is expected given the complex interactions that define the freezing level height, more related to the PBL schemes and the vertical grid resolution. In that sense, a broader study incorporating a full test of the PBL scheme and several vertical densities are recommended for the future.

3.5. Ensemble Performance

To understand the ensemble performance, we plot the observed hyetograph in the Apoquindo gauge (blue bars) and temperature series against the red region denoting the time at which more than $50\%$ of the WRF ensemble predict rainfall (Figure 8). In the figure, we also plotted the 25th, 50th and 75th percentiles of the temperatures simulated by the ensemble. Note that the ensemble corresponds to as all the simulations performed for each rainfall event, indistinctive from their lead time or MP scheme, i.e., 6 simulations for OCT15, 9 for APR16 and 6 for MAY17 (Table 3).

The occurrence of the main pulses of the OCT15 event is very well predicted, although a final pulse that did not happen in 20 October is produced. At the time of the rainfall peak, the ensemble simulates a temperature ∼2 °C higher than the observed one, due to the abrupt temperature decrement prior to the rainfall beginning.

On the other hand, the WRF ensemble fails in identifying the temporal distribution of the more intense APR16 rainfall event with a 48 h delay in the prediction of the occurrence (Figure 8, central plot). However, a third rainfall pulse predicted by the ensemble matches the observed rainfall, which can lead to wrong conclusions. As mentioned by Garreaud [53], the warm storms last from 12 to 36 h, coupled with an air temperature drop (>3 °C) and the highest precipitation within the first hours of the event. This conditions indeed occurred during the APR16 event, particularly the pulse starting on 17 April. The ensemble underestimates the observed temperature for ∼2 days before the rainfall initiates, but, during its peak, this temperature was contained between the 25th and 75th percentiles.

The ensemble performance for the MAY17 event was more difficult to analyze, due to the frontal characteristics of the event. Although it was unable to accurately predict the characteristics of the rainfall occurrence, the ensemble clearly simulated three different instances where precipitation happens. Rain is not properly predicted in the main period of intensive rainfall (11 May), with only one of the three larger intensities being matched by a single red stripe. A final rainfall pulse (12 May) is simulated by the model with almost half a day of anticipation. Temperature is accurately predicted during the first rainfall event peak, but, for the main peak, there is a ∼2 °C overestimation.

Overall, the WRF ability to simulate frontal rainfalls is acceptable with a proper MP scheme parametrization, such as LIN or WSM6. In addition, the accurate prediction of rainfall is still too complicated over the complex topography of Santiago.

4. Conclusions

Rainfall forecast over complex topography using WRF was studied through the simulation of three events between 2015 and 2017 in Quebrada de Ramón, a 38 km${}^2$ mountainous Andean watershed in central Chile. Three microphysics (MP) parametrizations (i.e., LIN, WSM6 and WSM3, currently used by the Chilean Weather Agency) were tested to find an optimum model performance. The simulations considered an horizontal resolution of 6 km and 50 vertical levels for improving atmospheric temperature profiles, as well as ${0.5}^{\circ }$ grid resolution GFS dataset. A realistic representation of Santiago’s urban area was provided by MODIS, and Noah-MP model was used as the land surface model. Finally, variable lead times of 72, 96 and 120 h before the rainfall start were also analyzed.

The uneven prediction of rainfall length and total amount by the LIN and WSM6 parameterizations tends to improve with shorter lead times. Both performed better than WSM3, mainly for temperatures and rainfall intensities. The N-rainiest consecutive hours (NRH), a relevant characteristic of storm events given the impact on the hydrologic response, could not be well predicted by any of the MP options for cutoff low events. Frontal system events were better captured, mostly via the WSM6 MP option, which also predicted well the concurrent temperatures with high intensities. This is very relevant, as high flows and potential floods are typically produced by warm events when high temperatures and rainfall take place at the same time [53].

The temporal evolution of precipitation, temperature and freezing level height were properly predicted for shorter lead times, especially for frontal system events, while complex meteorological cutoff low characteristics lead to poor forecasts. No clear trends in lead times were found, but shorter values (72 h ahead rainfall event) tended to provide more accurate simulations. Further testing on PBL schemes and vertical resolution is needed in this theme.

From our results, the WSM6 scheme resulted to be the best to simulate rainfall events in the Andean watershed under study. Nevertheless, rainfall simulation in WRF over complex topography is still a challenging issue, and its ability to accurately simulate rainfall, particularly non-frontal, events over Chilean central mountainous and foothills areas, where some of the main cities are located, is far from ideal. Further investigation should focus on simulating more rainfall events for which observed data could be available, as well as testing additional microphysical schemes such as Thompson scheme [73] or the Aerosol-aware Thompson scheme [74].

Finally, it would be of interest to improve rainfall forecasts by combining NWP tools with geostatistical methods, which incorporate spatial statistics and local topography to describe, enhance or predict rainfall in basins with complex topography [75,76,77]. Such geostatistical approach would eventually offset to a certain extent WRF forecast problems over the Andean complex topography.