Optimize Short-Term Rainfall Forecast with Combination of Ensemble Precipitation Nowcasts by Lagrangian Extrapolation

Abstract

The rainfall forecasts currently available in Korea are not sufficiently accurate to be directly applied to the flash flood warning system or urban flood warning system. As the lead time increases, the quality becomes even lower. In order to overcome this problem, this study proposes an ensemble forecasting method. The proposed method considers all available rainfall forecasts as ensemble members at the target time. The ensemble members are combined based on the weighted average method, where the weights are determined by applying the two conditions of the unbiasedness and minimum error variance. The proposed method is tested with McGill Algorithm for Precipitation Nowcasting by Lagrangian Extrapolation (MAPLE) rainfall forecasts for four storm events that occurred during the summers of 2016 and 2017 in Korea. In Korea, rainfall forecasts are generated every 10 min up to six hours, i.e., there are always a total of 36 sets of rainfall forecasts. As a result, it is found that just six ensemble members is sufficient to make the ensemble forecast. Considering additional ensemble members beyond six does not significantly improve the quality of the ensemble forecast. The quality of the ensemble forecast is also found to be better than that of the single forecast, and the weighted average method is found to be better than the simple arithmetic average method.

1. Introduction

Recently, there have been growing concerns of urban flooding and flash flooding that appear to be related to the rapid population growth and climate change. More of the population has been living in some specific urban areas [1,2,3,4,5], and people are leaving these areas for outdoor activities such as camping in very scenic mountain valleys, particularly during the hot summer season [6,7,8]. Global warming is believed to change the climate continuously. Very localized and severe rainfall events are also believed to be increasing [9,10,11,12,13]. The combination of all of these changes unfortunately increases the risk of urban flooding and flash flooding, which is also expected to worsen in the future. Although the definitions of urban floods and flash floods differ, they share some unique characteristics, such as a localized area, short response time, high risk, and so forth.

Urban and/or flash floods occur within one or two hours, sometimes shorter, following severe rainfall [14,15,16,17,18]. Thus, in this case, the rainfall measurements on the ground may not be applicable for flood warnings. Put simply, sufficient lead time cannot be secured to warn people in the pruned area of such flooding. This is an important reason to use the rainfall forecast for this purpose. For example, the Flash Flood Guidance System (FFGS) in the US that uses rainfall forecasts based on both satellite and radar observation is used for flash flood warning [19]. The European Flood Awareness System (EFAS) also uses ensemble rainfall forecasts for flood warning [20]. The flash flood warning system in Korea also uses the rainfall forecast as input [21,22].

Various models have been developed for the purpose of producing rainfall forecasts. Auto NowCaster (ANC) is a model developed by the National Center for Atmospheric Research (NCAR) for convective storm forecasting [23]. The Thunderstorm Identification, Tracking, Analysis, and Nowcasting (TITAN) was also developed for convective storm forecasting [24]. The Japan Meteorological Administration (JMA) uses a Mesoscale Model (MSM), which is a numerical weather prediction model, for the same purpose [25,26]. The Korea Local Analysis and Prediction System (KLAPS) and McGill Algorithm for Precipitation Nowcasting by Lagrangian Extrapolation (MAPLE) are models used in Korea for very short time rainfall forecasting [27,28,29,30].

The quality of flash flood warning is clearly very dependent on the rainfall forecast used. However, it is also well known that the quality of rainfall forecasts is not very satisfactory [31,32,33]. Various ensemble techniques have been applied in attempts to improve rainfall forecasting [34,35,36]. The quality of rainfall forecasts can be significantly improved, and the effective lead time of the rainfall forecast can also be increased by applying the ensemble technique [37,38,39,40]. The ensemble technique has also been adopted as an aspect of operational rainfall forecasting in many countries [37,41,42,43,44].

Various methods of generating ensemble members have been proposed. First, it is possible to control the input data. The observed data can be varied by introducing time lag [45,46,47,48]. It is also possible to introduce some amount of perturbation to the observed input data [49,50,51,52]. In general, randomly generated noise is added to the observed data. One important advantage of using this method of controlling the input data is that it is based on observed data, that is, realistic data. The second method involves controlling the parameters of the forecasting model [53,54,55]. Here, the model parameters are assumed to be stochastic. Under the assumption that the multivariate probability distribution function governing the model parameters is known, as many parameter sets as possible can easily be generated to be used to generate rainfall forecasts, i.e., ensemble members. This method is particularly effective when the forecasting model is very competitive. Recently, several models have been considered simultaneously for the same purpose of rainfall forecasting [56,57,58,59]. Each rainfall forecast from a different model is assumed to be an ensemble member. As no model is complete with each having pros and cons, the use of various models must have some benefits to generate ensemble members.

However, it is also possible to imagine a situation in which none of the above methods of generating ensemble members is applicable. Basically, the users of the rainfall forecast do not handle the forecasting model. Every hour, the generated rainfall forecast is transferred to the registered users. It is the same case for flood warning. Particularly in the case that the rainfall forecast is of low quality, something must be done to improve it. This is what this study addresses.

The objective of this study is to develop an ensemble forecasting method based on the rainfall forecast generated every hour. For example, the rainfall forecast generated every hour may include the rainfall data for the following one to six hours. If the target time is three hours later, four different rainfall forecasts are available at that target time. That is, the three-hour forecast made at the current time, the four-hour forecast made in the previous hour, the five-hour forecast made two hours prior, and the six-hour forecast made three hours prior. All four of these four forecasts are considered as ensemble members to make the ensemble forecast in this study. The ensemble members are going to be combined based on the weighted average method, where the weights should also be determined reasonably. The ensemble forecast should also have higher quality than the simple rainfall forecast. Several issues arise when using this ensemble forecasting method: the first involves the optimal number of ensemble members, the second involves the determination of the weights, and finally, it is important to evaluate how the ensemble forecast compares to the single rainfall forecast. This study attempts to solve these problems one by one. As examples, this study will consider several storm events that occurred during the summers of 2016 and 2017 in Korea.

2. Theoretical Framework

2.1. Ensemble Forecast and the Lagged Average Forecast

An ensemble forecast is a collection of two or more forecasts at a certain target time [60]. Ensemble forecasting is applied to reduce the uncertainty of deterministic forecasting such as the numerical weather prediction (NWP) [34,35,61]. The uncertainty of the NWP increases significantly, mostly due to the incompleteness of the initial conditions as well as the incompleteness of the numerical model itself [62]. Thus, the purpose of the ensemble forecasting is to probabilistically combine multiple forecasts to yield more reliable results. By setting different initial conditions, physical processes, and boundary conditions, various forecasts of future weather conditions can be generated [50,63].

Ensemble forecasting has many advantages compared with NWP. The uncertainty of NWP forecasts is assumed to be considerably large, and its error varies in time and space [64,65,66]. However, ensemble forecasting considers all available forecasts which could statistically occur. The use of ensemble forecasting provides not only improved forecasting results over a long lead time, but also information about its uncertainty. This information about forecasting uncertainty is particularly important to forecasters who should make an appropriate decision [67].

Ensemble forecasting techniques can be divided into three types: the first involves controlling the input data. Some amount of perturbation is added to the observed input data [68,69,70,71], or various time lags are applied to the observed input data [72,73,74]. The second is to control the model parameters. Many ensemble members are generated by applying various parameter sets to a model [53,55,75,76]. The last type involves diversifying models or systems. This is also called the multimodel ensemble [56,77,78,79]. There also exists the concept of a super ensemble, which is a combination of the three types [80,81,82].

It is also possible to use all of the forecasts available at a specific target time. These forecasts are those made at different times in the past. The quality of the ensemble forecast can be improved simply by averaging all available forecasts [83]. This method is called the lagged average forecast (LAF). The LAF has both advantages and disadvantages; its most important advantage is that it is easy to apply. No special skills are needed to generate the perturbations. Additionally, the generated perturbation is not simply random but rather based on the hidden dynamics at each time step. However, as the LAF is based on a limited number of ensemble members, the representativeness of the ensemble forecast can be weak. In order to lessen this weakness, the appropriate weight to be applied to each ensemble member should be determined. Hoffman and Kalnay (1983) presented a method to consider the expected error for each ensemble member [72].

Due to its simple application, the LAF has frequently been applied to various forecasting problems [46,47,84,85,86,87,88,89,90,91,92,93]. For example, Brankovic et al. (1990) applied the LAF to medium and long-term forecasts operated by the European Center for Medium Range Weather Forecasts (ECMWF) [46]. The ensemble forecasts were generated using nine prior forecasts with a six-hour interval. The Root Mean Square Error (RMSE) and the Anomaly Correlation Coefficient (ACC) of the LAF ensemble forecasts were compared with those of each single forecast, with the results indicating that the LAF ensemble forecasts were superior in all seasons. Lu et al. (2007) evaluated the forecasts of several meteorological factors such as air temperature, wind speed, and relative humidity, generated by the Rapid Update Cycle (RUC) model of the National Oceanic and Atmospheric Administration (NOAA) [85]. They found that the LAF ensemble forecasts were superior to a single deterministic forecast, even for a short-term forecasting from 1 h to 3 h. Mittermaier (2007) also showed that LAF ensemble forecasting reduced uncertainty in high-resolution rainfall forecasting [47]. The ensemble forecasts of rainfall amount, maximum rainfall amount, and rainfall occurrence probability were generated using five rainfall forecasts with a six-hour interval from Met Office 4 km UM (Unified Model) as the ensemble members. They found that the LAF not only improved the forecasting quality for the rainfall amount, but also significantly reduced the error in forecasting rainfall occurrence.

2.2. Method of Averaging

The ensemble forecast is made by averaging the ensemble members. That is,

$$R_t={\displaystyle \sum _{i=1}^n{w}_i{M}_i}$$

(1)

where R_t is an ensemble forecast made at the target time t, w_i is a weight for the ensemble member M_i, and n is the number of ensemble members.

To date, various averaging methods have been proposed. A relatively simple averaging method is the simple average method (SA method). In this method, an equal weight is simply applied to each ensemble member. That is, the weight w_i in Equation (1) is determined to be 1/n. Although it is very simple without any deep theoretical background, the SA method has been applied in various studies [77,80,94]. The weighted average method (WA method) is a statistical method which can consider the uncertainty of each ensemble member [95,96,97]. The weight is generally determined to be inversely proportional to its uncertainty. The application of a linear regression is another method that can be used to determine the weights of ensemble members [98]. In this method, a linear function is derived by considering the multiple ensemble members as independent variables. A nonlinear function approach such as the artificial neural network (ANN) has also been proposed [99]. Additionally, a so-called time-varying mergence method (TV method) has been proposed in an attempt to consider the temporal change of weights [97,100,101]. Among the TV methods, a time-varying sum of square root error method (TVSSE method), first proposed by Granger and Newbold (1977), may be the most popular [97].

Among many averaging methods, this study selected the weighted average method (WA method). The WA method is believed to be one which can consider the statistical characteristics of ensemble members. Specifically, this study adopted the variance–covariance method [95], which combines the unbiased ensemble members based on the concept of minimum error variance of the ensemble forecast. As an example, in the case of considering two ensemble members F₁ and F₂ generated for the true value of F,

$$F_1 = F + Z_1$$

(2)

$$F_2 = F + Z_2$$

(3)

where Z₁ and Z₂ represent the stochastic errors involved in the generation of ensemble members. Here, the two stochastic errors are assumed to follow the Gaussian distribution, but they are not assumed to be independent of each other.

Based on the linear assumption, the optimal estimate can be expressed as follows.

$$\widehat{F} = c_1{F}_1 + c_2{F}_2$$

(4)

where c₁ and c₂ are the optimal weights to be determined by considering the uncertainty of the ensemble members of F₁ and F₂. Generally, the weights are estimated by applying the two conditions of the unbiasedness and minimum error variance. That is, the bias of the estimate $\widehat{F}$ should be zero, and the error variance of the estimate $\widehat{F}$ should be minimized as well [98]. By applying these two conditions, one can derive the weights as follows.

$$c_1=\frac{{\sigma }_2^2-{\sigma }_{12}}{{\sigma }_1^2+{\sigma }_2^2-2{\sigma }_{12}}$$

(5)

$$c_2=\frac{{\sigma }_1^2-{\sigma }_{12}}{{\sigma }_1^2+{\sigma }_2^2-2{\sigma }_{12}}$$

(6)

where ${\sigma }_1^2$ and ${\sigma }_2^2$ are the variances of the two forecasts and ${\sigma }_{12}$ is the covariance between the two forecasts. If the two forecasts are independent, the covariance becomes zero. However, if they are dependent on each other, the covariance should be considered to determine the weights.

A generalization of the above result to the case of considering m forecasts is as follows.

$$\widehat{W}=\frac{{\sum }^{-1}u}{u^T{\sum }^{-1}u}$$

(7)

where $\widehat{W}$ is a column vector of the weights of m ensemble members and u is a unit column vector (i.e., $u={\left[1, 1,\cdots , 1\right]}^T$). Σ is an m × m covariance matrix consisting of the forecasting errors. The weight for each ensemble member can be calculated using the covariance matrix of forecasts, as given in Equation (7). Further, the variance of the ensemble forecast can be derived as follows.

$${\sigma }_c=\frac{1}{u^T{\sum }^{-1}u}$$

(8)

This variance represents the quality of the ensemble forecasts, which generally becomes smaller as the number of ensemble members increases.

2.3. Optimal Number of Ensemble Members and Quality of Ensemble Forecast

In this study, the number of ensemble members was determined based on the quality of the ensemble forecast. Basically, as the number of the ensemble members increases, the forecasting quality improves. However, the forecasting quality can also become stagnated if the number of ensemble members exceeds a certain limit. In this study, the optimal number of ensemble members was determined as the minimum number of ensemble members that significantly improved the quality of the ensemble forecast. The difference between the ensemble forecast and the observed data, as well as the variance of the ensemble forecast were considered to determine the optimal number of ensemble members.

Once the optimal number of ensemble members has been determined, the ensemble forecast can be produced by applying the weight to each ensemble member. The forecasting quality of the produced ensemble forecast is then evaluated by comparing it with the observed data. Two evaluation factors are used to evaluate the forecasting quality: the root mean square error (RMSE) represents the mean difference between the ensemble forecast and the observed data, while the pattern correlation coefficient R is used to quantify the similarity of the ensemble forecast and the observed data. The pattern correlation coefficient R is calculated using Equation (9).

$$R = \frac{{\displaystyle \sum _{m = 1}^M{\displaystyle \sum _{n = 1}^N\left(P_{m,n}-\overline{P}\right)\left(O_{m,n}-\overline{O}\right)}}}{\sqrt{\left({\displaystyle \sum _{m = 1}^M{\displaystyle \sum _{n = 1}^N{\left(P_{m,n}-\overline{P}\right)}^2}}\right)\left({\displaystyle \sum _{m = 1}^M{\displaystyle \sum _{n = 1}^N{\left(O_{m,n}-\overline{O}\right)}^2}}\right)}}$$

(9)

where P_m,n and O_m,n are the ensemble forecast and observed data, respectively, at each grid point (m, n). Further, $\overline{P}$ and $\overline{O}$ are the mean values of the ensemble forecast and observed data, respectively. Similar to the ordinary correlation coefficient, the pattern correlation is high when the value of R is near to 1 or −1.

3. Data

3.1. Rainfall Forecast of MAPLE

MAPLE (McGill Algorithm for Precipitation Nowcasting by Lagrangian Extrapolation) is a very short term rainfall forecasting model. The basic concept of the MAPLE rainfall forecasting was first described by Germann and Zawadzki (2002) [28]. The MAPLE provides the forecasts of precipitation echoes, which are made using storm moving vectors. The storm moving vectors are derived from past and current radar reflectivity images by applying the variational echo tracking (VET) technique [102]. In order to generate nowcasts over several hours, the MAPLE uses a semi-Lagrangian method, which is based on the cross-correlation of the rainfall patterns at different times. The MAPLE is known to have the ability to forecast with consideration of the life cycles of various rainstorms. It has also been revised consistently to overcome various limitations and problems [29,103,104,105].

The MAPLE rainfall forecasting system operated by KMA has two limitations: First, as it is a nondynamic forecasting model, it does not involve any process to consider the growth and decay of precipitation echoes. Aside from the probabilistic analysis of the rainstorm size, the initial information of precipitation echoes does not change. Second, the initial echo moving vectors are applied to every lead time. If the precipitation echo moving vectors change abruptly, the forecasting quality can be decreased significantly. This limitation is particularly relevant when extending the lead time; if the lead time is longer than three hours, the forecasting quality of the rainfall forecasts is known to be decreased abruptly [106].

The MAPLE rainfall forecast is generated as a form of a composite radar map, showing the entire Korean Peninsula and the surrounding region (Figure 1). Its lead time ranges from 10 min to 360 min (i.e., six hours). That is, each MAPLE rainfall forecast contains one set of observed data and 36 forecasts with a 10 min interval. It is composed of 1024 × 1024 grids with a resolution of 1 km × 1 km, as a type of matrix. The original format of the MAPLE rainfall forecast is a binary gzip, but for use in this study, it is changed into the ASCII format using the Geographic Information System (GIS) program.

3.2. Storm Events

This study used the MAPLE rainfall forecasts of four major storm events that occurred in 2016 and 2017. Figure 2 shows radar images of the selected storm events, which are available at the weather radar center portal (www://radar.kma.go.kr). These radar images show basic information about the size, intensity, and movement of these storm events.

The selected major storm events have different characteristics (

Table 1. Characteristics of the selected storm events in this study.

Characteristics	Storm 1	Storm 2	Storm 3	Storm 4
Dates	2016/07/04–05	2016/10/04–05	2017/07/08	2017/09/10–11
Duration (hrs)	20	16	17	13
Maximum rainfall intensity (mm/hr)	42.8	104.2	78.9	73.4
Type	Monsoon	Typhoon	Monsoon	Typhoon
Region	Central	Southern	Central	Southern
Direction	↗	↗	↘	→
Target times (rainfall intensity)	07/05 02:30 (42.8 mm/hr) 07/05 03:00 (34.5 mm/hr)	10/05 08:00 (57.7 mm/hr) 10/05 09:00 (98.3 mm/hr) 10/05 10:00 (104.2 mm/hr)	07/08 07:00 (66.2 mm/hr) 07/08 08:00 (78.9 mm/hr) 07/08 09:00 (71.2 mm/hr)	09/11 04:30 (68.1 mm/hr) 09/11 06:00 (73.4 mm/hr)

). For a specific example, consider the second event, Typhoon Chaba. Typhoon Chaba caused severe damage in several cities located in the southern part of the Korean Peninsula. This storm event was recorded as the worst typhoon that occurred in October among those that landed in the Korean Peninsula. The maximum rainfall intensity, 104.2 mm/hr and total rainfall depth of 266.0 mm were recorded at the Ulsan rain gauge station [107]. The fourth event was Typhoon Talim. This storm event landed on the Korean Peninsula on 10 September 2017 with a lot of rainfall, and decayed the next day at the Jeolla Province. The first and third events are those that occurred during the monsoon seasons in 2016 and 2017, respectively. In this study, 10 target times were selected in total to generate the weighted average ensemble forecasts. Each storm event contains two or three target times, which were those at which relatively high rainfall intensity occurred. Table 1 summarizes detailed information about the rainfall that occurred at these 10 target times.

4. Results

4.1. Weighted Average Ensemble Forecast

The structure of the covariance is the key factor for determining the weight for each forecast. First, the cross-correlation coefficient was estimated to consider the difference in lead time among ensemble members. In this estimation process, the unit time lag was assumed to be 10 min. For example, the time lag between the 30 min forecast and the 90 min forecast corresponds to lag-6, and the time lag between the 60 min forecast and the 120 min forecast also corresponds to lag-6. As there were many cases corresponding to a given time lag, their average was assumed to be the represented value.

Figure 3 shows the change in the cross-correlation coefficient in terms of the time lag, derived from the forecasts collected at the 10 target times. Further, the solid thick line represents their average. As can be expected, the cross-correlation coefficient decays slowly and stagnates after some time lag. It is obvious that a higher cross-correlation coefficient is expected for short time lags, but a lower one is expected for long time lags. This also indicates that the diagonal components of the covariance matrix are the highest.

The covariance matrix was derived separately for each storm event. For example, Figure 4 shows the covariance matrix derived from all of the available rainfall forecasts on 04:30, 11 September 2017. This figure includes the results for cases with ensemble members 2, 3, 6, 9, 18, and 36. Regardless of which ensemble members were considered, the diagonal components of the covariance matrix were always higher than the other components. This indicates that the quality of the rainfall forecast drops as the lead time increases. The correlation also becomes lower as the lead time increases.

As explained previously, the weights to be applied to the ensemble members (i.e., the rainfall forecasts available at a given target time) are derived by analyzing the covariance matrix. This study considered three different forecasting times; one, two, and three hours. For example, the case with the forecasting time of one hour considered all of the rainfall forecasts with a lead time longer than or equal to one hour. This procedure was repeated 10 times by considering all of the 10 different target times listed in Table 1. The weights were estimated independently for each case, and the final weights were determined as their arithmetic average. Figure 5 compares the final weights determined for the cases with the following numbers of ensemble members: 2, 3, 4, 6, and 9. The solid line represents the final weights determined for each case.

The characteristics of the determined weights were as expected. For example, the weight of the most recently made forecast was higher than that of the older one. The difference among weights was larger when a smaller number of ensemble members were considered. As the number of ensemble members considered increased, the weights became more similar to each other. That is, the simple arithmetic averaging could be a valid method when considering a large number of ensemble members. For example, Figure 5a shows the determined weights for the case considering four ensemble members. The weight of the first ensemble member (i.e., rainfall forecast with a lead time of one hour) was determined to be 0.4235, while that of the second one (i.e., rainfall forecast with a lead time of one hour and 10 min) was 0.2337. The third and fourth weights were determined to be much smaller at 0.1739 and 0.1689, respectively.

However, the above characteristics become rather weakened when a longer forecasting time is considered. For example, the weight of the first ensemble member for the forecasting time of three hours (see Figure 5c) was determined to be substantially smaller than that for the forecasting time of one hour. On the other hand, the weights of the other ensemble members became slightly higher. The case for the forecasting time of two hours is in between the two forecasting times of one hour and three hours. This indicates independence among the ensemble members, which becomes stronger when considering a longer forecasting time. This result is also consistent with the characteristics of the covariance matrix. The cross-correlation coefficient was found to be negligible when the difference in lead time was longer than 150 min.

In general, the quality of the ensemble forecast increases as more ensemble members are considered. However, it is not economical to simply consider as many ensembles as possible. The optimal number of ensembles is determined as the minimum number of ensembles which guarantees a similar quality of ensemble forecast. The uncertainty (or variance) of the ensemble forecast is analyzed to determine the optimal number of ensemble members. In this study, for the forecasting times of one, two, and three hours, the behavior of the RMSE of the ensemble forecast was examined in terms of the number of ensemble members. In order to consider together all of the RMSEs estimated for each of the 10 target times evaluated in this study, the RMSE estimated for each target time was also normalized by the RMSE for the case of considering just one ensemble member (i.e., the biggest RMSE). The optimal number of ensemble members was determined as that near the inflection point of the RMSE curve.

Figure 6 shows the behavior of the RMSEs estimated for the 10 target times. The solid thick line represents their average. The two red dashed lines represent the tangent lines from the beginning and from the end. These two lines meet at the inflection point. As shown in this figure, the optimal number of ensemble members is around six. This result was the same regardless of the forecasting time. The ensemble forecast hereafter is made by considering a total of six ensemble members.

4.2. Quality of Weighted Average Ensemble Forecast

In the previous section, the optimal number of ensemble members was determined to be six. For the case of a forecasting time of one hour, as an example, six rainfall forecasts made from the prior 60 min to the prior 110 min were used to make the ensemble forecast. For the case of a forecasting time of two hours, six rainfall forecasts made from the prior 120 min to the prior 170 min were used, and for the case of a forecasting time of three hours, six rainfall forecasts made from the prior 180 min to the prior 230 min were used to make the ensemble forecast.

Table 2. Weights to be used to make the ensemble forecast with six ensemble members.

No.	Forecasting Time
	1 h	2 h	3 h
1	0.3040	0.2635	0.2465
2	0.1705	0.1890	0.2450
3	0.1711	0.1628	0.1739
4	0.1697	0.1519	0.1475
5	0.1271	0.1292	0.1058
6	0.0576	0.1037	0.0813

summarizes the weights to be used to make the weighted average (i.e., ensemble forecast based on the weighted average method) with these six ensemble members. The weight for the first ensemble member (the most recent rainfall forecast among the ensemble members considered) was the highest, and the weight dropped as the time gap increased.

The quality of the MAPLE rainfall forecast decreases with longer lead time. Figure 7 shows six rainfall forecasts made at 09:00 8 July 2017 for the lead times from 60 min to 110 min. On this basis of these rainfall forecasts, the storm center was predicted to move to the bottom of the right-hand side. These forecasts also indicate that the forecasting quality decreases with increasing lead time, which is also a crucial factor for the practical use of rainfall forecasts.

The quality of the ensemble forecast was first evaluated at the target time by comparing it with the observed rainfall field (Figure 8). Comparisons with the ensemble forecast based on the simple arithmetic average method and the single forecast at the target time (i.e., the case with just one ensemble member) were also conducted in order to emphasize the usefulness of the ensemble forecast. As shown in Figure 8, the storm center of the observed rainfall field is located around 36°10′ N~36°20′ N and 128°30′ E~128°40′ E. A similar pattern could also be found in the ensemble forecasts based on the weighted average method as well as the simple average method. Both of the results were similar. On the other hand, in the single forecast, the overall pattern was found to be somewhat different from the observed result. Two storm centers were found: one around 36°0′ N~36°10′ N and 128°10′ E~128°25′ E and another around 36°20′ N~36°30′ N and 128°40′ E~128°50′ E.

The quality of the ensemble forecast was also quantitatively evaluated according to the pattern correlation coefficient R and the root mean square error RMSE. These measures of the ensemble forecast were also compared with those of the single forecast and the ensemble forecast based on the simple arithmetic average method. For example, the evaluation result at the target time of 09:00 8 July 2017 shows that the ensemble forecast based on the weighted average method is most similar to the observed. For the case of a forecasting time of one hour, R and RMSE of ensemble forecast based on the weighted average method were 0.42 and 6.85, respectively. On the other hand, those of the single forecast were estimated to be 0.25 and 9.66, while those of the ensemble forecast based on simple arithmetic averaging were 0.39 and 7.14, respectively. Overall, the ensemble forecast was found to be better than the single forecast, and the weighted average method was found to be better than the simple arithmetic average method. Similar results were derived for the forecasting times of two and three hours, as summarized in

Table 3. Evaluation of the single forecast, the ensemble forecast based on the simple arithmetic averaging, and the ensemble forecast based on the weighted averaging at 09:00 July 8, 2017.

Measure	Type	Forecasting Time
		1 h	2 h	3 h
R	Single forecast	0.25	0.31	0.09
	Ensemble forecast based on the simple arithmetic averaging	0.39	0.25	0.09
	Ensemble forecast based on the weighted averaging	0.42	0.30	0.13
RMSE	Single forecast	9.66	8.69	10.56
	Ensemble forecast based on the simple arithmetic averaging	7.14	7.12	8.50
	Ensemble forecast based on the weighted averaging	6.85	6.90	8.23

.

The above evaluation process was repeated for all 10 target times considered in this study, and the results are summarized in

Table 4. Evaluation of the single forecast, the ensemble forecast based on the simple arithmetic averaging, and the ensemble forecast based on the weighted averaging at 10 target times.

Measure	Type	Forecasting Time
		1 h	2 h	3 h
R	Single forecast	0.26	0.19	0.18
	Ensemble forecast based on the simple arithmetic averaging	0.27	0.21	0.19
	Ensemble forecast based on the weighted averaging	0.27	0.22	0.20
RMSE	Single forecast	7.05	7.95	8.20
	Ensemble forecast based on the simple arithmetic averaging	6.07	6.29	6.53
	Ensemble forecast based on the weighted averaging	5.97	6.36	6.57

. Basically, the same result was derived for all 10 target times. That is, the ensemble forecast was found to be better than the single forecast, and the weighted average method was found to be better than the simple arithmetic average method. In addition, the evaluation result was found to be slightly deteriorated as the forecasting time increased. That is, the evaluation result for the case of a forecasting time of one hour was mostly better than that of the forecasting time of two or three hours. Further, the evaluation result for the case of a forecasting time of two hours was mostly better than that of a forecasting time of three hours. There were some exceptions, but this trend was seen in all three different forecasts: the ensemble forecast based on the weighted average method, the ensemble forecast based on the simple arithmetic average method, and the single forecast. It should also be noted that, as the forecasting time increased, the evaluation result of the single forecast became substantially worse than that of the ensemble forecast. This result seems to be based on the fact that the ensemble forecast was made by considering the uncertainty of rainfall propagation, i.e., by considering more feasible cases of rainfall propagation. Also, this limitation, which is the quality of three hour forecast is far lower than that of one hour and two hours can be found in the previous studies in Korea [108,109,110].

5. Summary and Conclusions

This study proposed an ensemble forecasting method based on the rainfall forecast generated every hour. The ensemble members (i.e., the available rainfall forecasts at the target time) were combined using the weighted average method. Three issues were raised with this ensemble forecasting method: the first involved the optimal number of ensemble members, the second was concerned with the determination of the weights, and the last one involved the quality of the ensemble forecast. One by one, this study showed possible solutions to these issues, with example applications to four storm events that occurred during the summers of 2016 and 2017 in Korea.

First, the weights were determined by analyzing the covariance matrix of the rainfall forecasts under the two conditions of the unbiasedness and minimum error variance. As expected, the weight of the most recently made forecast was higher than that of older ones. The difference among weights was also larger when a smaller number of ensemble members were considered. As the number of ensemble members increased, the weights became similar to each other. That is, the simple arithmetic averaging could be a valid method when considering a large number of ensemble members. However, this trend was rather weakened when considering a longer forecasting time. For example, the weight of the first ensemble member for the forecasting time of three hours was determined to be substantially smaller than that for the forecasting time of one hour. On the other hand, the weights of the other ensemble members became slightly higher. This indicates independence among the ensemble members, which becomes stronger when considering a longer forecasting time.

Second, the optimal number of ensembles was determined to be the minimum number of ensembles which guarantee a similar quality of ensemble forecast. The uncertainty (or variance) of the ensemble forecast was analyzed, and the optimal number of ensemble members was determined as that near the inflection point of the RMSE curve. In this study, the optimal number of ensemble members was determined to be around six. This result was the same regardless of the forecasting time.

Third, the quality of the ensemble forecast was quantitatively evaluated by the pattern correlation coefficient R and the root mean square error RMSE. These measures of the ensemble forecast were also compared with those of the single forecast and the ensemble forecast based on the simple arithmetic average method. The results indicated that the ensemble forecast was better than the single forecast, and that the weighted average method was better than the simple arithmetic average method.

Finally, it was also found that the evaluation result was slightly deteriorated as the forecasting time increased. That is, the evaluation result for the case of a forecasting time of one hour was mostly better than that of a forecasting time of two or three hours. However, it should be mentioned that, as the forecasting time increased, the evaluation result of the single forecast became substantially worse than that of the ensemble forecast. This result underscores the usefulness of the ensemble forecast. In fact, this is a fundamental problem which cannot be solved immediately. However, even though the results for the lead times of two or three were confusing, the rainfall prediction based on the weighted average method was superior to that based on the simple arithmetic average method.

The ensemble forecasting method proposed in this study may help increase the usefulness of the rainfall forecast one step further. The rainfall forecasts that are currently available in Korea are not sufficiently accurate to be directly applied to the flash flood warning system or urban flood warning system. The quality becomes even lower as the lead time increases. However, by introducing the ensemble technique, this problem could be alleviated a bit. The usefulness of the proposed ensemble forecasting method can be even higher in the near future if more accurate rainfall forecasts become available.