The Skills of Medium-Range Precipitation Forecasts in the Senegal River Basin

Abstract

Reliable information on medium-range (1–15 day) precipitation forecasts is useful in reservoir operation, among many other applications. Such forecasts are increasingly becoming available from global models. The skills of medium-range precipitation forecasts derived from Global Forecast System (GFS) are assessed in the Senegal River Basin, focusing on the watershed its major hydropower dams: Manantali (located in relatively wet, Southern Sudan climate and mountainous region), Foum Gleita (relatively dry, Sahel climate and low-elevation), and Diama (a large watershed covering almost the entire basin, dominated by Sahel climate). IMERG Final, a satellite product involving rain gauge data for bias correction, is used as reference. GFS has the ability capture the overall spatial and monthly pattern of rainfall in the region. However, GFS tends to overestimate rainfall in the wet parts of the region, and slightly underestimate in the dry part. The skill of daily GFS forecast is low over Manantali (Kling-Gupta Efficiency, KGE of 0.29), but slightly higher over Foum Gleita (KGE of 0.53) and Diama (KGE of 0.59). For 15-day accumulation, GFS forecast shows higher skill over Manantali (KGE of 0.60) and Diama (KGE of 0.79) but does not change much over Foul Gleita (KGE of 0.51) compared to daily rainfall forecasts. IMERG Early, a satellite-only product available at near-real time, has better performance than GFS. This study suggests the need for further improving the accuracy of GFS forecasts, and identifies IMERG Early as a potential source of data that can help in this effort.

1. Introduction

Precipitation forecasts are crucial in a number of applications [1,2,3]. Such forecasts are available at global or regional scales [4,5,6,7]. Depending on the length of lead time, the forecasts can be grouped into short-range, medium-range, and seasonal. While seasonal forecasting is useful for planning purposes, medium-range and short-range are useful for operational decisions. As water managers consider the use of precipitation forecasts, there is a need for more detailed information on the accuracy of these forecasts. Such information is also important to provide guidance for improvements in the forecasting process. However, validation studies of such forecasts, particularly for short- and medium-range forecasts, are rare in Africa. Yue et al. [8] evaluated the accuracy of Global Forecast System (GFS) medium-range precipitation forecasts in the Niger River Basin, focusing on the watersheds of eight major dams. They used IMERG Final (satellite-gauge merged product) as reference for evaluation. They found that at the daily timescale, the GFS forecasts have good skills over large watersheds (Kainji dam with a watershed area of 1,464,092 km² and Markala dam with a watershed area of 102,882 km²), but poor skills over smaller watersheds (area under 50,000 km²; Slingue, Goronyo, Bakolori, Jebba, Dadin Kowa, and Lagdo dams). They also reported that aggregating the forecasts from daily to multi-day periods leads to improvements in forecast accuracy, and results in good skills for all watersheds at the 15-day accumulation timescales.

A number of studies have evaluated the performance of IMERG Final in West Africa. Dezfuli et al. [9,10] reported that IMERG Final is capable of capturing the diurnal cycle of rainfall as well as the propagation of large Mesoscale Convective Systems. Gossett et al. [11] showed that among satellite-only (i.e., no rain gauges) products, IMERG Early has better performance. Satgé et al. [12] reported that CHIRPS and TMPA (the predecessor to IMERG) provided reliable estimates at both daily and monthly timescales, while other satellite products considered (CMORPH, PERSIANN, GSMaP, ARC, and TAMSAT) and all atmospheric reanalysis products considered (MERRA and JRA) were deemed unreliable. Maranan et al. [13] showed that IMERG Final products are able to capture monthly rainfall with a correlation coefficient close to unity.

In this study, we aim to understand the accuracy of GFS forecasts in the Senegal River Basin using IMERG Final as reference. The Senegal River basin (Figure 1), located in West Africa with a drainage area of 340,000 km², is shared by four countries: Guinea (accounting for 7% of the basin’s area), Mali (35%), Mauritania (50%), and Senegal (8%) [14]. Most of the basin has a sub-Saharan desert climate [15]. The basin has three distinct topgraphic zones: the mountainous, the valley, and the delta. We specifically focus on the watersheds of three major dams in the basin (

Table 1. Major reservoir dams in the Senegal river basin.

Dams	Country	Operational Since	Capacity (Million m3)	Power (MW)	Purpose		Area of Drainage Basin (km2)	Elevation of Drainage Basin (m)
					Irrigation	Hydroelectricity
Manantali	Mali	1988	11,270	104	x	x	29,340	560
Foum Gleita	Mauritania	1988	500		x		9513	128
Diama	Senegal	1986	250		x		43,1603	199

): the Manantali Dam in Mali (reservoir capacity at 11,270 million m³; drainage area at 29,000 km²), the Foum Gleita Dam in Mauritania (500 million m³; 9500 km²), and the Diama Dam in Senegal (250 million m³; 431,000 km²). Manantali, located in the mountainous and southern part of the watershed, is a multi-usage dam with a storage capacity of 11 billion m³ of water, an energy production of 800 GWh/year and an irrigation capacity of 255,000 ha. The Foum Gleita Dam, located further north with relatively low elevation, at a drainage area of about 9500 km², is primarily irrigation dam. The main functions of the Diama dam, located at the outlet of the Senegal river basin, are to improve irrigation in the valley and delta and to facilitate water supply.

In the following sections, we aim to address the following questions surrounding the accuracy of GFS over the watersheds of three major dams located in the Senegal River Basin?

(1)

What is the skill of the 1-day lead, daily, GFS forecasts?

(2)

How does the skill vary with lead time and temporal aggregation scale?

(3)

How does the performance of a near-real time satellite-only product, IMERG Early, compare with the performance of GFS?

2. Data and Methodology

2.1. Data

The GFS is a global numerical weather prediction system run by the U.S. National Weather Service (NWS), and its forecast products have a resolution of 0.25° by 0.25°. We use the version-15 GFS forecasts which are available since June 12, 2009, following a major upgrade of the GFS model. As a reference to evaluate the GFS forecasts, we use the IMERG Final (version 06B) satellite-gauge merged products [16,17]. IMERG Final is available every 30-min at a spatial resolution of 0.10°, with a data latency of about 3.5 months. We acknowledge that IMERG Final has its own limitation, and therefore we conduct additional assessment evaluating the performance of GFS using another satellite-gauge product known as CHIRPS as reference. CHIRPS is available daily at a spatial resolution of 0.05°, with a data latency of about 3 weeks [18]. Finally, we also assess how the performance of GFS compares with the performance of a satellite-only (i.e., no rain gauge) product, known as IMERG Early, which is available at near-real time. The motivation for considering IMERG Early is its potential use for post-processing of GFS forecasts given its timely availability.

2.2. Methodology

IMERG Final satellite rainfall products are used as reference to evaluate the performance of GFS precipitation forecasts. The comparison period is 15 June 2019 to 15 June 2020 to match the period for which the version-15 of GFS model forecasts are available. The spatial resolutions of the forecast and satellite products are different. Our comparison is mostly based on sub-basin (i.e., watershed for each dam) average values, in which case, we average all the datasets to the sub-basin spatial scale. In cases, where we compare the spatial patterns of rainfall, we resample both IMERG products and CHIRPS to 0.25° using the bilinear interpolation technique to match the spatial resolution of GFS.

Our evaluation includes comparison of spatio-temporal maps and the modified Kling-Gupta Efficiency KGE [19,20] as evaluation metric:

$$\mathrm{KGE}=1-\sqrt{{\left(\mathrm{R}-1\right)}^2+{\left(\mathrm{BR}-1\right)}^2+{\left(\mathsf{\gamma }-1\right)}^2},$$

where R is the linear correlation coefficient between the forecast and reference precipitation, BR is the bias ratio between the forecast and observation, and γ is the ration of coefficient of variations between the forecast and reference. The range of KGE is from −∞ to 1. Towner et al. [21] suggested the following classifications: “Good” (KGE ≥ 0.75), “Intermediate” (0.75 ≥ KGE ≥ 0.5), “Poor” (0.5 ≥ KGE > 0), and “Very poor” (KGE ≤ 0). Similar performance statistics have been used in evaluating rainfall products [22].

3. Results and Discussion

3.1. Annual Spatial Variability of Rainfall

Figure 2 presents the spatial map of annual (15 June 2019 to 15 June 2020) rainfall derived from the 1-day lead GFS forecast and satellite products. According to the reference IMERG Final, the annual rainfall shows a distinct south-north gradient, varying from more than 1500 mm in the southern part (Savannah climate, and mountainous) to under 250 mm in the northern part (Sahel climate, and low-elevation). Validated against IMERG Final, the 1-day lead GFS captures well the spatial pattern of rainfall. The correlation between the spatial distribution of annual rainfall derived from the 1-day lead GFS and IMERG Final is 0.94.

3.2. Monthly Cycle of Rainfall

Figure 3 presents the time series of monthly rainfall derived from the 1-day lead GFS and satellite precipitation products, at each of the dam watersheds. According to IMERG Final, rainfall in the region has one rainy season with a peak in August in all three watershed cases. The main mechanism for this precipitation is the northward migration of the Inter Tropical Convergence Zone (ITCZ) during summer. The rainy season in the Foum Gleita watershed (dry Sahel climate) is of short period (July through October), with an amplitude of 100 mm month⁻¹, whereas the rainy season over the Manantali watershed (wet Savannah climate) is of longer period (May through October) with an amplitude of around 400 mm month⁻¹. The 1-day lead GFS captures well the seasonal cycle of rainfall in all three watershed cases but tends to overestimate the monthly rainfall over Manantali, slightly underestimate over Foum Gleita, and is almost unbiased over the larger Diama watershed, which integrates both the wet and dry parts of the watershed.

3.3. Annual Rainfall

Here, we aggregate the 1-day lead GFS forecasts to annual time scale and compare the results against corresponding annual precipitation estimates from IMERG Final (Figure 4). According to IMERG Final, the annual watershed-average rainfall over the dam watersheds is: 300 mm (Foum Gleita), 600 mm (Diama), and 1400 mm (Manantali). The daily GFS gives annual rainfall values of 215 mm (Foum Gleita), 538 mm (Diama), and 1,798 mm (Manantali). Therefore, GFS overestimates rainfall by 28% over the Manantali watershed (wet climate) but underestimates it in the Foum Gleita watershed by 30% (dry climate), while at the large watershed (Diama) that encompasses both climates the GFS slightly underestimate rainfall by 11%.

3.4. Daily Time Series

Figure 5 presents the daily time series of watershed-averaged rainfall, derived from 1-day lead GFS and satellite products. According to IMERG Final, the time series of daily rainfall at the Manantali watershed is different from the Foum Gleita and Diama watersheds. The Manantali watershed experiences frequent heavy rains during the rainy season with a coefficient of variation (CV) of 1.18, while both the Foum Gleita and Diama watersheds experience less-frequent and moderate rains with a CV of 2.41 and 1.39, respectively. The GFS captures well the time series with comparable CV, however, it shows some mismatches in the peak rain rates.

3.5. KGE Statistics

Figure 6 presents the performance statistics of various precipitation products (with respect to IMERG Final), in terms of Kling-Gupta Efficiency (KGE), Bias Ratio (BR), correlation (R), variability ratio (γ), and root mean square error normalized by reference precipitation mean (NRMSE). The KGE of the 1-day lead, daily, GFS forecast is 0.29 (Manantali), 0.53 (Foum Gleita), and 0.65 (Diama). The GFS skill is therefore “poor” over the mountainous and wet watershed (Manantali), while it is “intermediate” over the dry watershed (Foum Gleita) and over the large watershed (Diama). The low KGE for the Manantali watershed can be attributed to low performances in all components of KGE: high bias (BR = 1.37), low correlation (R = 0.50), and low variability ratio (γ = 0.68), indicating that the GFS over the wet and mountainous part of the watershed is characterized by large systematic and random error. For the Foum Gleita watershed, the KGE value (0.53) is mainly affected by both bias (BR = 0.75) and low correlation (R = 0.62). For the Diama watershed, the KGE value (0.65) is mainly affected by the low correlation (R = 0.66). The root-mean-square-error of the GFS is high at 196% of the mean rainfall at the Foum Gleita watershed, whereas it is relatively low at 120% and 108% at the Manantali and Diama watersheds, respectively.

3.6. Dependence of Forecast Performance on Precipitation Rate

Figure 7 presents the scatterplot of 1-day lead GFS forecasts against daily IMERG Final rain rates. At all sites, the GFS tends to overestimate light rains and underestimate heavy rains.

3.7. Effect of Lead Time on Forecast Performance

Figure 8 presents the KGE and its components for 5-day total rainfall forecast of GFS for three different lengths of lead time: 1–5 day, 6–10 day, and 10–15 day. The effect of lead time on forecast skill is mixed. At the wet and mountainous watershed of Manantali, the forecast skill decreases with increasing lead time. Whereas, at the dry watershed of Foum Gleita, the forecast skill increases slightly with increasing lead time. At the large Diama watershed, the forecast skill remains the same across all lead times. We note (see Section 3.4) that daily rainfall has larger fluctuation at Foum Gleita watershed (CV of 2.66) than at Manantali watershed (CV of 1.18), suggesting that the effect of lead time on forecast skill may depend on the temporal variability of rainfall.

3.8. Effect of Temporal Aggregation Scale on Forecast Performance

Figure 9 presents the performance of 15-day accumulated GFS forecasts. We obtained the 15-day accumulated forecast by summing up the daily forecasts at various lead times, starting from 1-day lead all the way to 15-day lead. Aggregating the forecasts from daily forecasts to 15-day accumulated forecasts has improved the performance of GFS for the wet Manantali watershed and the larger Diama watershed, whereas there is not much improvement for the dry watershed, Foum Gleita. Over Manantali watershed, the 15-day accumulated forecast has shown increases in KGE (0.60 compared to 0.29 at daily timescale) and correlation (0.90 compared to 0.50 at daily), but no change in bias ratio (BR = 1.38). Over the Diama watershed, the 15-day accumulated rainfall has similarly shown increases in KGE (0.79 compared to 0.65 at daily) and correlation (0.87 compared to 0.66 at daily). Therefore, increasing the temporal aggregation has mixed effect on the skill of the forecast: it increases the skills over some watersheds but does not changes the skills over some other watersheds.

We point out that the effect of temporal aggregation scale on forecast skill depends on the relative magnitude of two error sources with different error characteristics. Increasing temporal aggregation scale involves including forecasts with longer lead times. Forecasts at longer lead times may have larger errors (e.g., Manantali) or smaller errors (e.g., Foum Gleita) compared to those short-lead times (see Section 3.7). On the other hand, averaging the forecasts at larger temporal scales results in averaging out some of the random errors. Our results indicate that at the wet Manantali watershed, the effect of averaging outweighs the effect of lead time, as the 15-day accumulated GFS has much better skill (KGE of 0.60) compared to that the daily forecast (KGE of 0.29). However, at the Foum Gleita watershed, the effect of averaging balances out the effect of lead time, resulting in a similar KGE at both timescales. At the large Diama watershed (which covers diverse climate), the KGE shows modest increase in KGE at larger temporal aggregation timescale.

3.9. Comparison of the Performances of IMERG Early and GFS

As discussed in the preceding sections, the skills of 1-day lead, daily, GFS forecasts are poor for Manantali but medium for Foum Gleita and Diama Dams, indicating the need for further improving the accuracy of GFS forecasts. One method to improve GFS forecast skill is through post-processing of GFS using rainfall estimates that have relatively better accuracy and are available in near-real time. Satellite-only products, such as IMERG Early, are viable options as they are available in near-real time. We assessed how the performance of IMERG Early compare to the performance of GFS. In terms of capturing the spatial and monthly variability of rainfall, IMERG Early has comparable performance with GFS (see Figure 2 and Figure 3). However, there are differences between the two in terms of performance statistics. As shown in Figure 4, IMERG Early underestimates rainfall by 11% over the wet and mountainous Manantali watershed (where GFS overestimates by 37%), estimates rainfall with almost no bias over Foum Gleita (where GFS underestimates by 30%), and overestimates rainfall by 10% over the larger Diama watershed (where GFS underestimates by 11%). The KGE of IMERG Early is above 0.75 (i.e., “good skill”) and is higher than that of GFS at all three watersheds (Figure 6). IMERG Early outperforms GFS in all components of KGE, that is, bias, correlation, and variability ratio. Therefore, our analysis indicates that IMERG Early has better capabilities than GFS, and is therefore worthy of consideration as input into post-processing techniques aimed at improving the accuracy of GFS forecasts.

3.10. Performance of GFS If the Reference Product Is Changed from IMERG Final to CHIRPS

We acknowledge that the reference dataset used in our evaluation (i.e., IMERG Final) has its own estimation errors. We conducted additional assessment to evaluate the performance of GFS using CHIRPS rainfall products as reference.

Table 2. Performance statistics of daily GFS forecast for various lead times (1-day, 5-day, 10-day, and 15-day) using IMERG Final (CHIRPS) rainfall products as reference, in terms of correlation, bias ratio, and NRMSE.

Lead Time of GFS Forecast	Correlation	Bias Ratio	KGE	NRMSE (%)
Manantali
1-day	0.50 (0.56)	1.37 (1.22)	0.29 (0.46)	120 (96)
5-day	0.43 (0.44)	1.22 (1.09)	0.31 (0.39)	119 (102)
10-day	0.23 (0.41)	1.39 (1.24)	0.07 (0.31)	145 (111)
15-day	0.09 (0.20)	1.61 (1.44)	-0.13 (0.07)	184 (150)
Foum Gleita
1-day	0.62 (0.58)	0.75 (1.05)	0.53 (0.55)	197 (234)
5-day	0.34 (0.44)	0.67 (0.93)	0.06 (0.12)	287 (327)
10-day	0.22 (0.29)	0.78 (1.08)	0.19 (0.28)	273 (295)
15-day	0.42 (0.30)	0.70 (0.97)	0.33 (0.27)	238 (293)
Diama
1-day	0.66 (0.68)	0.94 (1.12)	0.65 (0.66)	108 (109)
5-day	0.60 (0.67)	0.81 (0.96)	0.55 (0.65)	116 (105)
10-day	0.36 (0.44)	0.94 (1.12)	0.34 (0.42)	144 (137)
15-day	0.31 (0.30)	0.93 (1.10)	0.30 (0.29)	148 (153)

shows the performance statistics of GFS for different lead times, using IMERG FINAL and CHIRPS, separately, as reference. The overall magnitude of GFS performance (as well as the variability of the performance across watersheds) is similar when either rainfall product is used as a reference. Therefore, our results show that the overall performance of GFS remains the same if the reference product were to be changed from IMERG Final to CHIRPS, indicating the robustness of IMERG Final as reference product.

4. Conclusions

In this study, we assessed the accuracy of medium-range (1-day to 15-day lead time) forecasts available from the Global Forecast System (GFS) over the watersheds of three dams (Manantali, Foum Gleita, and Diama) in the Senegal River Basin. The watershed of Manantali dam is mountainous and located in a wet climate, with annual rainfall of 1400 mm during the study period. Foum Gleita, on the other hand is located in the dry Sahel, with annual rainfall of 300 mm during the study period, and has a low-elevation terrain. Diama is a very large watershed that covers almost the entire basin, with annual rainfall of 600 mm during the study period. We used IMERG Final, a satellite-gauge merged product, as reference for evaluation.

The skill of the 1-day lead, daily, GFS forecast is low in the Manantali (Kling-Gupta Efficiency KGE of 0.29) and relatively high in the Foum Gleita and Diama (KGE of 0.53) and Diama watersheds (KGE of 0.59). The lower KGE over Manantali is due to the high overestimation bias (overestimation by 37%) in the GFS forecasts and low correlation coefficient (R = 0.50) between the daily time series of 1-day lead GFS forecast and IMERG Final.

Increasing the lead time has mixed effect on the daily GFS forecast skill, depending on watershed. With increasing lead time, the forecast skill decreases (Manantali), or slightly increases (Foum Gleita), or remains unaffected (Diama). Aggregating the forecasts from daily to 15-day accumulation also has similar mixed effect: it increases the performance of the GFS forecasts at the Manantali (KGE of 0.60) and Diama (KGE of 0.79) watersheds, but does not change it at the Foum Gleita (KGE of 0.51).

We conclude that there is a need for improving the accuracy of GFS forecasts, especially at short temporal scales (such as daily) and particularly over the mountainous and wet parts of the river basin. One possible method worthy of investigation is the use of post-processing techniques based on IMERG Early, as this study has shown that IMERG Early has better performance compared to GFS in terms of a suite of different performance metrics.