Seasonal Forecasting of the Onset of the Rainy Season in West Africa

Abstract

Seasonal forecasts for monsoonal rainfall characteristics like the onset of the rainy seasons (ORS) are crucial for national weather services in semi-arid regions to better support decision-making in rain-fed agriculture. In this study an approach for seasonal forecasting of the ORS is proposed using precipitation information from a global seasonal ensemble prediction system. It consists of a quantile–quantile-transformation for eliminating systematic differences between ensemble forecasts and observations, a fuzzy-rule based method for estimating the ORS date and graphical methods for an improved visualization of probabilistic ORS forecasts. The performance of the approach is tested for several climate zones (the Sahel, Sudan and Guinean zone) in West Africa for a period of eleven years (2000 to 2010), using hindcasts from the Seasonal Forecasting System 4 of ECMWF. We indicated that seasonal ORS forecasts can be skillful for individual years and specific regions (e.g., the Guinean coasts), but also associated with large uncertainties. A spatial verification of the ORS fields emphasizes the importance of selecting appropriate performance measures (e.g., the anomaly correlation coefficient) to avoid an overestimation of the forecast skill. The graphical methods consist of several common formats used in seasonal forecasting and a new index-based method for a quicker interpretation of probabilistic ORS forecast. The new index can also be applied to other seasonal forecast variables, providing an important alternative to the common forecast formats used in seasonal forecasting. Moreover, the forecasting approach proposed in this study is not computationally intensive and is therefore operational applicable for forecasting centers in tropical and subtropical regions where computing power and bandwidth are often limited.

1. Introduction

Weather and climate can strongly influence agricultural production in many regions of the world. Especially in tropical and subtropical areas, the monsoon rainfall plays a dominant role for the income of many smallholders. Normal rainfall conditions are often related to better crop yields, but on the contrary prolonged dry periods or a late onset of a rainy season can lead to crop failures and endanger food security [1,2]. Providing reliable rainfall forecasts for the upcoming rainy season is therefore crucial for many national weather services in tropical and subtropical regions.

West Africa is one of the most vulnerable regions in the world, where the welfare of society relies heavily on the production of rain-fed agriculture. The climate of this region is governed by the West African Monsoon and its large-scale atmospheric and oceanic forces and is therefore characterized by strong variabilities on different spatiotemporal scales (e.g., [3,4]). Strong precipitation anomalies can trigger severe natural disasters such as the Sahel drought in the 1970s and 1980s or large-scale floodings in 2017 [5], which can have immediate and major macro-socioeconomic impacts on the society in this region [6]. Between 1980 and 2018, Munich Re recorded 82 drought events in West Africa with an estimated overall loss of more than 3.6 billion US dollars [7]. One of the most important precipitation parameter for farmers in West Africa is the onset of the rainy season (ORS) [8] since this variable is highly related to management activities in rain-fed agriculture such as land preparation and crop planting. In Benin, the study from [9] also shows a necessity of smallholders to account for seasonal weather forecasts, especially the ORS date and other rainfall characteristics like the occurrence of dry spells and the cumulative precipitation amount over a given period are of high interest. In northern Burkina Faso, 63% of 170 farmers surveyed said they would be prepared to pay for seasonal, sub-seasonal and daily weather information [10]. These studies show the tremendous importance that this type of information will be made available for farmers in West Africa.

Seasonal rainfall forecasts for the region of West Africa are routinely produced within the framework of the West African Regional Climate Outlook Forums (WARCOF) formerly known as PRESAO (Prévisions Saisonnières en Afrique de l’Ouest) [11]. Under the coordination of the African Centre for Meteorological Applications for Development (ACMAD) and the Centre Regional de Formation et d’Application en Agrométéorologie et Hydrologie Opérationelle (AGRHYMET Regional Centre), the West African weather services produces tercile-based probability forecasts of the seasonal rainfall amount, e.g., for the peak period of the West African Monsoon namely July, August and September [12]. However, WARCOF traditionally produces no quantitative forecasts for other rainfall variables that are important for users in agriculture and other economic sectors. Agriculturally relevant shortcomings of WARCOF are for example that pre-monsoonal and post-monsoonal periods are not fully covered, forecasts provide only information about the total precipitation sum of a 3-month period, intra-seasonal precipitation characteristics (e.g., ORS and dry spells) are not provided quantitatively and the lead time is limited to 1 month (see [13]; and references herein). Only basic subjective statements are formulated for the onset of the rainy season (ORS) and other rainfall characteristics like the dry spell length and the cessation of the rainy season [14]. The procedure used by WARCOF to predict seasonal precipitation is not only based on forecasts from global seasonal ensemble prediction systems (GSEPS), but also on country-specific statistical forecasts considering sea surface temperature anomalies and other meteorological information. [11]. During the WARCOF meeting, both the statistical results and the global forecasts are then subjectively reinterpreted. This hand-drawn “consensus” forecast is finally disseminated to potential users by radio broadcasts, agrometeorological bulletins, and farmer workshops.

Moreover, dynamical and statistical forecasting methods that consider these types of variables are relatively rarely represented in the scientific literature. An overview of forecasting approaches for the ORS date is given in [15,16]. These approaches can be divided into two groups according to the meteorological information used for the prediction: (1) Statistical techniques that use rainfall data for the prediction. This group includes, for instance, the techniques proposed by [16,17], which were tested for several stations in the Volta basin, and (2) statistical techniques proposed by [15,18], which are based on other meteorological variables like relative humidity, air temperature and wind data from atmospheric soundings. Both groups rely on observations for the prediction of the onset of the rainy season. Thus, they belong to the group of statistical forecasting techniques where a time lag is implemented in the equation system to make a prediction [19]. Forecasting methods that use the information from GSEPS to produce a forecast of the ORS dates are extremely rare. For instance, ref [20] presented a study in which the date of onset of the rainy season was forecasted using numerical weather forecasts. Another approach is presented by [13]. They used a GSEPS, the Climate Forecast System 2 [21], in combination with a dynamical downscaling approach and a fuzzy-logic ORS approach [17] to produce probabilistic forecasts for the ORS dates.

This study presents a statistical method for forecasting the date of the ORS using precipitation information from a GSEPS including several methods for visualization of ensemble-based ORS forecasts. The approach consists of a quantile mapping approach [22] combined with a fuzzy-rule based ORS method [17] to provide probabilistic forecasts for the ORS dates. The approach is less computationally demanding in comparison to the dynamical downscaling method presented by [13] and can be therefore much better operationally applied by National Meteorological and Hydrological services (NMHS) in West Africa. We also perform a first performance assessment of the approach using ECMWF SYS4 hindcasts over a period of eleven years (2000–2010) in comparison to gridded observational data and spatial verification measures such as the anomaly correlation coefficient [23]. The graphical methods for visualization of the ORS predictions range from probabilistic quantile plots to a new index-based measure, called the onset of the rainy season index (ORSI). The ORSI delivers crucial information about an early, mean and late onset of the rainy season. For farmers, ORSI is easier to interpret compared to common equiprobable categorical formats provided by WARCOF. Thus, the ORSI can be considered as an appropriate and useful tool for farmers to adapt the sowing time to changes in the ORS. The ORSI can also be easily adapted to be applied for other applications.

2. Data and Study Area

2.1. Study Area

The area of interest in this study (Figure 1) is West Africa with a focus on Burkina Faso, Ghana and Benin (4.92° W–4.22° E, 4.56° N–15.08° N). West Africa’s climate is characterized by the West African Monsoon (WAM), which is mainly controlled by the meridional migration of the sun and the associated solar climate [24]. Many rainfall characteristics like the date of the ORS in this region are strongly related to the seasonal migration of typical WAM features such as the intertropical convergence zone (ITCZ). In August, the ITCZ reaches its northernmost position at about 11° N and retreats to the Gulf of Guinea in the northern winter, where it remains close to the equator. Due to the migration of the ITCZ, the Guinea coast therefore has a bimodal rainfall distribution with relatively dry conditions from December to February and a second minimum with much wetter conditions during the peak period of the West African Monsoon in July, August and September. The rainfall amounts in the area north of 9° N reaches its peak in August and the duration of the rainy season strongly shortens towards the Sahara [25]. The study area is also characterized by a strong interannual [24] and annual [3] variability of precipitation as well as decadal variability, which lead to substantial variations of ORS dates and prolonged droughts over this region [26]. In addition, climate change seems to influence rainfall characteristics in the Sahel zone, such as false starts and early cessation of rainy seasons or increased frequency of intense daily rainfall [27,28]. Ref [29] also show an increasing precipitation intensity with a higher average precipitation per rainy day, a 5–10 days later start and end of the rainy season, but only a marginal change in the overall duration of the wet season.

2.2. ECMWF SYS4 Precipitation Hindcasts

In this study we use 11 years (2000–2010) of precipitation hindcasts from the seasonal forecast system SYS4 of ECMWF. SYS4 consists of an ocean analysis model to determine the initial conditions of the ocean, a global coupled ocean-atmosphere circulation model and post-processing techniques to generate prediction products from the numerical output [30].

There are several evaluation studies on the precipitation forecasts of SYS4 with reference to various study regions, including East Africa [31], South Africa [32], global [33] and the Asian monsoon region [34]. Especially for West Africa, the study by [35] can be mentioned, in which SYS4 was evaluated for the Upper-Niger river basin based on the standardized precipitation index (SPI) showing that SYS4 has more predictive skill compared to the climatology, i.e., no skill. Ref [36] showed that SYS4 has the better performance in most cases compared to several other global seasonal weather forecasting systems (e.g., Climate Forecast System Version 2, Météo-France 3 or Community Climate System Model 3) for the region of West Africa.

The hindcasts of SYS4 comprise of 15 ensemble members and consist of 7-month simulations initialized on the first day of each month. Further details can be found in [37] as well as in [30]. With the aim of developing statistical methods for the onset of the rainy season, it is important to investigate the reliability of SYS4’s daily rainfall data, as the statistical methods for the start of the rainy season directly rely on the precipitation ensemble of this model. The evaluation of the hindcasts with the observed values is therefore made for all 15 members of SYS4 in comparison to observations. Moreover, the selected hindcast data should also cover the potential ORS dates for the area of interest. Thus, the ensemble initialized in February is used for the study period with daily data available from 1 February to 31 August for each year of the investigation period.

2.3. CHIRPS Observations

CHIRPS (Climate Hazards Group Infrared Precipitation with Station data) is developed by the US Geological Survey (USGS) and the Climate Hazards Group of the University of California, Santa Barbara (UCSB). CHIRPS combines satellite imageries with a resolution of 0.05° and in-situ station data to produce gridded precipitation time series for trend analysis or seasonal drought monitoring [38]. Further details on the development of CHIRPS can be found in [39]. Ref [40] evaluated on a point to pixel basis several state-of-the-art satellite-based precipitation products e.g., CHIRPS, Africa Rainfall Estimate Climatology (ARC 2.0) [41], precipitation estimation from remotely sensed information using artificial neural networks (PERSIANN) [42], African Rainfall Estimation (RFE 2.0) [43] and the Tropical Rainfall Measuring Mission (TRMM, [44]) with rain-gauge data for Burkina Faso in the period from 2001–2014. The correlation between the daily satellite-based precipitation products and the measured data was generally weak, but the best performance was observed with CHIRPS data (r = 0.47). For this study, the daily data of CHIRPS 2.0 with a resolution of 0.25° × 0.25° is used. An interpolation from CHIRPS to the spatial resolution of SYS4 is performed using a nearest neighbor approach. The data can be therefore directly compared and verified with the SYS4 precipitation data.

3. Forecasting Methodology and Verification

Figure 2 shows an overview of the dataflow and the different components of the forecasting approach. The precipitation ensemble of SYS4 is the basis for the provision of the probabilistic onset forecasts. These forecasts are adapted to the observed climatology represented by the CHIRPS observations using a quantile–quantile-transformation. The corrected ensemble information is used to calculate the ORS dates using the fuzzy-logic approach of [17]. This bias correction and the calculation of the onset dates are done for each grid cell of the SYS4 ensemble. Afterward, the probabilistic forecasts of the onset dates are calculated and based on this information several visualization approaches are used.

It is noted that the different components of the forecasting approach can be replaced by other methods or data products. For instance, the precipitation ensemble of other GSEPS like CFS2 can be used instead of SYS4. The same applies also for the observations, bias correction technique and the ORS method. The methodology can be also transferred to other regions of the world, if corresponding observations are available. In Section 3.1 and Section 3.2 a more detailed description of the different model components (bias correction approach, ORS method) is given. The verification of the ORS fields and the calculation of the ORS index are shown in Section 3.3 and Section 3.4.

3.1. Bias Correction of Ensemble Precipitation Forecasts

Precipitation information from global models used in weather forecasting and climate prediction are characterized by systematic deviations from observations so that the observed precipitation climatology for a given geographical region is often not well reproduced. This problem is shown for different climatological zones in West Africa for example by [13]. They analyzed the precipitation hindcasts of CFSv2 in comparison to the Global Precipitation Climatology Centre reanalysis dataset over a period of more than 25 years (1983–2009). Moreover, the precipitation bias of climate models can also reduce the reliability of hydrological or agricultural model simulations, if the precipitation simulations are used without any corrections. Therefore, different bias correction techniques were developed in the past to eliminate systematic model deviations and to better match the observed climatology for a given region. A recent overview is presented in [45] and applications for the West African monsoon are given in [13,46]. In this study, the quantile–quantile transformation following [22] and [47] is applied to adapt the distribution of the hindcasts ensemble (SYS4) to the observed climatological distribution (CHIRPS). In this work, the procedure is performed for each ensemble member and grid point separately using the daily time series as follows:

• Sort the daily observed values $x=\left\{ x_1, x_2, \dots , x_n \right\}$ for the investigation period (1 February–31 February, 2000–2010) in an ascending order.
• Determination of the rank $R_{x_i}$ for each observation with $i=1,2,\dots ,n$ and $n=$ number of observations.
• Determining the empirical cumulative distribution function of the observations $F_x$ by calculating the cumulative frequencies $p_i$ for each observation using the formula $p_i= R_{x_i}/(n+1$).
• Sort the daily forecast values $y=\left\{ y_1, y_2, \dots , y_n \right\}$ of an ensemble member for the investigation period (1.02–31.08, 2000–2010) in an ascending order.
• Determination of the rank $R_{y_i}$ for each forecast with $i=1,2,\dots ,m$ and $m=$ number of forecasts.
• Determining the empirical cumulative distribution function of the forecasts $F_y$ by calculating the cumulative frequencies $q_i$ for each forecast using the formula $q_i= R_{y_i}/\left(m+1\right).$
• Determine the corrected prediction value $y_i^{\prime }$ for each $q_i$ with the inverse cumulative distribution function of the observations $y_i^{\prime }= F_x^{-1}(q_i)$.
• Repeat steps 4–7 for each ensemble member and each grid point.

Figure 3 shows an example of a quantile–quantile transformation of daily precipitation amounts to better understand the approach described above. Here y(t) corresponds to the original value of the forecast and y’(t) to the forecast value corrected by the inverse cumulative distribution function of the observation.

The analysis of the joint distribution of the forecasts (SYS4) and the observations (CHIRPS) is done by a quantile–quantile plot, in which the daily sorted forecasted and observed precipitation values are represented in a coordinate system against each other. At this point it should also be mentioned that due to the grid-based determination of the ORS in Section 3.2, the focus is on the data of the individual grid points and therefore no temporal or spatial aggregation is performed. Accordingly, this analysis evaluates three grid points, each representative for different climate regions of the study zone: Guinea Coast (A), Sudan (B) and Sahel (C). Figure 4 shows the quantile plots for selected grid points of these zones for the 15 SYS4 ensemble members using selected quantiles (Q10, median, Q90). The distribution of the CHIPRS observations is relatively good captured by the SYS4 ensemble members, in particular for the Sudan zone. However, the analysis also reveals several systematic deviations between the SYS4 ensemble and the CHIRPS observations. For instance, smaller precipitation amounts (<20 mm/d) are strongly overestimated by the SYS4 ensemble for the grid point A located in the Guinean coasts. In addition, the SYS4 ensemble shows a quite diverse picture for the Sahel region with underestimations of the lower precipitation range (<30 mm) and overestimations of the upper precipitation range (>30 mm). Overall, Figure 4 indicates that the intra-ensemble biases of raw SYS4 forecasts are comparable to CHIRPS, i.e., the CHIRPS lies within the distribution of the SYS4 ensemble simulations. For high precipitation values, the SYS4 distributions are wider compared to small values. This is to be expected due to the fact that high precipitation values are less frequent and might be affected more heavily by stochasticity, coming from uncertainties from the model initial conditions and other limitations of the model such as parameterization options and non-adequate model equations [48]. Due to the systematic deviations for the small precipitation values (as discussed above), a statistical bias correction is applied in this study. This bias correction is done for each ensemble member separately.

3.2. Calculation of the ORS Dates Using Fuzzy Rules

There are several definitions for the determination of ORS for the West African monsoon system. An overview of these methods is given in [49]. Ref [20] divides the definitions into two categories:

• ORS is defined by local precipitation events and is based on dry and wet periods.
• ORS is defined by large-scale changes of the WAM system, indicated by e.g., circulation patterns or specific atmospheric variables like long-wave radiation fluxes.

In this study, the approach by [17] was applied, which belongs to the first category proposed by [20]. The approach of [17] is based on the ORS definitions of [50]. In [50] the ORS is defined as the first day of the year in which the following three conditions occur simultaneously:

• The accumulated sum of precipitation in five consecutive days is at least 25 mm.
• Within this pentad, at least two more days must exceed a precipitation amount of 1 mm.
• There is no period of consecutive 7 dry days or longer within the next 30 days (false start criterion). A dry day is defined as a day with less than 1 mm precipitation.

A shortcoming of the approach by [50] is its binary logic. An accumulated precipitation sum of 24.9 mm is not identified as the onset of the rainy season although only 0.1 mm is missing to fulfill the criteria. Since many other ORS approaches rely on binary logics, [17] introduced fuzzy rules for a more smoothed transition of the applied conditions. This ORS fuzzy-logic were used by several other investigations, as well [2,51]. The membership functions of the fuzzy rules used in this study are shown in Figure 5. For example, for condition 1, accumulated precipitation amounts of less than 18 mm receives a membership value of 0, and above 25 mm a 1. In the range between 18 and 25 mm, normalized values between 0 and 1 are assigned by linear interpolation. Thus, the probabilities of being interpreted as ORS are increasing with precipitation rates larger than 18 mm. A similar membership function is applied to condition 2, which is supposed to exclude single rainfall events to be misinterpreted as ORS. It must be noted that the membership functions were optimized for the Volta Basin of West Africa [17], and therefore could be assumed to be reasonable for our study area.

Due to the runtime length (seven months) of SYS4 and the late ORS within the Sahel zone, the false start criterion (condition 3) is not included in this study. However, we found that the false start criterion had only little impact on the ORS patterns. The criterion was evaluated with CHIRPS over the study period and each grid cell. The ORS only shifted in 14.1% of the cases (not shown). False starts occurred all over the study region and it was not only a problem of the northern parts. To avoid this, a smaller domain must be used, the runtime length of SYS4 needs to be extended or the fuzzy rules need to be optimized for operational applications. Therefore, a complete application of the approach should be carried out based on these evaluation results in further studies.

The ORS is met if the multiplied values of both membership functions (γ₁ and γ₂) exceeds a given threshold value ${\gamma }_{t }$. Any variation of the ${\gamma }_{t }$ can influence the ORS calculation and can therefore avoid that no ORS date is defined for a particular year. This is a common problem for other ORS definition. In this study a threshold value of ${\gamma }_{t }=0.5$ is used for the calculation of the ORS dates.

3.3. Verification of the Spatial ORS Fields

The ORS is calculated according to the aforementioned definitions for each grid point over all ensembles and CHIRPS observations, as well. The verification of the ORS forecasts is on the one hand performed spatially for each year over the entire study domain (spatial verification), and on the other hand temporally for each grid cell [23] over the entire study period (temporal verification). Considering the grid-based development of the ORS index in Section 3.4, it is important that the data is evaluated both temporally for each grid cell and spatially for each year.

The spatial verification of the ORS fields is performed by calculating the Pearson’s correlation coefficient (r), the mean absolute error (MAE) and the anomaly correlation coefficient (ACC) between the forecasted and the corresponding observed ORS. MAE and r are calculated for each ensemble member j and time step t using Equations (1) and (2).

$$MA{E}_j \left(t\right)= \frac{1}{M}{{\displaystyle \sum }}_{i=1}^M\left|y_{ij}\left(t\right)-o_i\left(t\right)\right|,$$

(1)

$$r_j\left(t\right)= \frac{{{\displaystyle \sum }}_{i=1}^M\left(y_{ij}\left(t\right)-{\overline{y}}_j\left(t\right)\right)(o_i\left(t\right)-\overline{o}\left(t\right))}{\sqrt{{{\displaystyle \sum }}_{i=1}^M{\left(y_{ij}\left(t\right)-\overline{y}\left(t\right)\right)}^2} \sqrt{{{\displaystyle \sum }}_{i=1}^M{\left(o_i\left(t\right)-\overline{o}\left(t\right)\right)}^2}},$$

(2)

where M corresponds to the number of grid points, $y_{ij}$ to the ORS forecast value at grid cell i of ensemble member j and $o_i$ to the corresponding observed ORS value. The mean observed and forecast ORS values of ensemble member j of the data sets are given by $\overline{y}$ and $\overline{o}$. The Pearson’s correlation coefficient and related measures like coefficient of determination are one of the most popular scores used for model validation. The correlation coefficient is also frequently used for the verification of spatial fields as outlined by [19] and for comparison of atmospheric fields as a measure for pattern similarity [52]. The Pearson correlation coefficient is also a measure for the skill of a forecast system as shown by [53]. However, unlike other popular skill scores for continuous variables (e.g., the mean squared error skill score), it measures the potential skill of the forecast system as defined by [53] without any unconditional bias.

The ACC is another common measure used specifically for spatial forecast verification [19,23]. First, the anomalies values of the forecasts $y_{ij}^{\prime }$ and observations $o_i^{\prime }$ are calculated with $y_{ij}^{\prime }= y_{ij}-{\overline{o}}_r$ and $o_i^{\prime }= o_i-{\overline{o}}_r$ where $y_i$ is the ORS forecast value at grid cell $i$ and $o_i$ to the corresponding observed ORS value. The mean observed ORS values of the climatological reference period (1981–2010) are given by ${\overline{o}}_r$. Then, the Pearson correlation is applied using the anomaly values. The score measures therefore how good the correspondence is between the predicted and the observed anomaly fields, which is highly important for climate model evaluation.

In addition to the spatial verification, the MAE and r are calculated for each time series of a grid cell and over the respective study period to perform a temporal verification of the ORS forecast. Afterwards, an averaging of the (skill) scores from the individual 15 ensemble members takes place. It should therefore be mentioned that the mean score is not indicative of the performance of the ensemble mean.

3.4. Calculation of the ORS Index

In order to better interpret ensemble-based ORS forecasts, an index for the ORS dates is proposed in this study. The approach is based on probabilistic forecasts for the ORS date computed for climatologically equiprobable categories. The calculation of the ORS index for these categories is based on the following formula:

$$ORSI={{\displaystyle \sum }}_{k=1}^K{g}_k{p}_k,$$

(3)

where $p_k$ is the forecast probability of k-th category, $g_k$ is the corresponding weight of k-th category, K is the number of climatologically equiprobable categories. The weight $g_k$ is computed by calculating the difference between the bin centre of k-th category and the median of uniformly distributed forecast probabilities, which is 0.5. The value of $g_k$ can range between 1 and −1. A forecast probability of extreme categories therefore receives a higher weight in comparison to categories close to the middle category. In the case of an odd number of equiprobable categories like tercile-based categories, the forecast probability of the middle category receives no weight.

In this study the ORS index is computed for tercile-based probabilistic forecasts of the ORS dates. Tercile-based probabilistic forecasts are the most common format in seasonal forecasting [23]. The values of the tercile-based ORS index ranges between $-33.\overline{3}$ and $33.\overline{3}$, where negative (positive) ORS values indicate a late (early) onset and corresponding distribution of the forecast probabilities is left skewed (right skewed). The minimum (maximum) ORS value of $-33.\overline{3}$ ($33.\overline{3}$) is reached, if all ORS members belong to the category below average (above average) with $p_3=100\%$ ($p_1=100\%$) and $p_2=p_1=0\%$ ($p_3=p_2=0\%$). The weights for the computation of the ORSI values for tercile-based forecast are $g_1=\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$3$}\right., g_2 =0$ and $g_3 = -\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$3$}\right.$. The forecast probability for the near-normal category receives therefore no weight. The calculation of the ORSI only relies on the forecast probability for above normal and below normal. Climatological forecasts with equally distributed forecast probabilities ($p_1 =33.\overline{3}\%, p_2 =33.\overline{3}\%$ and $p_3 =33.\overline{3}\%$) are indicated by an ORSI value of zero. The same applies also for symmetric distributed forecast probabilities (e.g., $p_1 =20\%, p_2 =60\%$ and p₃ = 20%) and if all members belong to the category near average (p₁ = 0%, p₂ = 100% and $p_3 =0\%$). Thus, the ORS index provides information about an early, mean and late onset of the rainy season. However, it cannot distinguish between climatological forecasts and forecasts where a normal onset is more likely.

In the following, the methodology for probabilistic forecasting of the ORS dates is listed as an example for a grid cell:

• Calculation of the observed ORS dates from CHIRPS for the period from 1981–2010.
• Determination of the quantiles ${\mathrm{Q}}_{\frac{1}{3}}$ and ${\mathrm{Q}}_{\frac{2}{3}}$ from the observed ORS dates to define the tercile-based ORS categories.
• Calculation of the ORS dates from the bias-corrected SYS4 precipitation hindcasts for each ensemble member of a given year.
• Classification of the 15 ORS ensemble members into the three tercile-based categories: Below average, if y_ij (t) < $Q_{\frac{1}{3}}$ , near average, if $Q_{\frac{1}{3}}$≤ y_ij (t) ≤ $Q_{\frac{2}{3}}$ and above average, if y_ij (t) > $Q_{\frac{2}{3}}$.
• Calculation of the corresponding forecast probabilities p_k for each ORS category.
• Calculation of the ORS index value based on Equation (3).
• Step 3 to 6 are repeated, if the forecasts probabilities are determined for other years.

4. Results and Discussion

4.1. Spatiotemperal Verification of the ORS Fields

The spatial accuracy of the ORS forecasts is shown in Figure 6 in terms of the MAE. The MAE was computed for each ensemble member and based on this information a box-plot diagram was produced for each year. Thus, the mean of the box plots shows the average accuracy of the ensemble forecasts for a given year and the box-plot spread the performance differences between the ensemble members. The mean MAE over the study period is 23.3 days, which indicates a relatively low accuracy for the prediction of the ORS dates in this region. This outcome can be expected because the prediction of precipitation characteristics on a much coarser temporal scale (e.g., monthly or seasonal precipitation amounts) is already characterized by low performance as shown by [13]. Moreover, seasonal forecasting of specific daily precipitation characteristics like ORS dates is a much more challenging task in comparison to seasonal forecasting of monthly or seasonal precipitation amount (or other meteorological variables like temperature). Reasons are the inherent high uncertainty of long-range forecasts products like seasonal forecasts and the high stochastic character of daily precipitation events, in particular in monsoonal regions like West Africa. Another reason is the uncertainty coming from the reference products used for comparison. This is shown e.g., by [51] for the calculation of the ORS dates for West Africa. Figure 6 also shows strong variations of the forecast performance of the ORS dates for the different years indicated by the mean and the spread of the box-plot diagrams. For some years the ORS forecasts were much better (e.g., 2007) in comparison to other years (2005).

The inter-annual variation of the performance can be also a pure random artifact due the low sample size of 15 ensemble members. To prove this hypothesis, statistical tests using a Student’s t-test for paired samples and a modification of Student’s t-test for unequal variances, the Welch’s t-test [54,55], are applied to evaluate whether the mean values of two MAE samples (e.g., 2007 and 2005) are significantly different from each other. More information how the statistical tests are exactly used in this study is given in the Appendix A. The statistical tests partially confirm the aforementioned hypothesis. However, the outcomes of the t-tests also indicate that the mean MAE values are significantly different from each other for pairs with very different means (like 2007 and 2005) indicating that the inter-annual variation of the performance is not a pure random artifact.

The association between the predicted and observed ORS fields is shown in Figure 7 for the Pearson’s correlation coefficient. The outcomes of this study show that the average of the correlation coefficients is 0.79 with a maximum value of 0.92 for the year 2006. Only the year 2005 falls strongly out of the upper correlation range because several statistics of the box plot diagram like the first quartile (0.75) and the minimum value (0.40) show the smallest but still a moderate to high correlation. Since the Pearson correlation coefficient is not only a measure of association between two fields but also a measure of the skill, the ORS prediction seems to be very skillful for this region. However, most of the skill is artificial due to strong seasonality of the ORS dates as a result of the northward migration of the WAM system during the boreal spring and early summer. The calculation of the anomalies for the forecasts and observed ORS values removes the seasonality of the ORS fields. The resulting anomaly patterns have a much smaller association, but they still have a slight positive correlation for most of the members (Figure 8). Moreover, the calculation of the significance threshold for the ACC scores based on a Student’s t-test (α = 0.10, n = 185) indicates only for two years, namely 2000 and 2001, a significant skill for the majority of the ensemble members. This example demonstrates the importance of selecting an appropriate skill score for the verification of the spatial ORS fields. Otherwise the forecasts performance for the ORS dates could be highly misinterpreted for this region. Figure 8 also shows that the quality of the ORS forecast strongly varies for the different years like for the other scores. There are several years when the majority of the ORS ensemble members have a negative ACC. This is case for 2009 where 13 out of 15 ensemble members are negative and the ACC mean is −0.11 and ACC has a minimum value of −0.44. On the other hand, there are also several years when most of the ensemble members have a positive ACC. For example, 14 out of 15 members are positive ACC value for 2001. For this year the ACC mean is 0.13 and the maximum ACC of the ensemble reaches a value of 0.48.

In addition, a temporal verification of the ORS forecasts is performed in this study. The verification measures are calculated for each grid cell by comparing the predicted ORS time series with a time series of the ORS observations. The verification measures are calculated for each ensemble member separately and afterwards an averaging of this sample is conducted. The outcome of this calculation is shown for the MAE (a), r (b) and bias (c) in Figure 9. It shows strong regional differences regarding the accuracy of the ORS forecast. The MAE values gradually decrease from the South to the North with a MAE of approximately 15 days in the southwestern part of the study domain near the Guinean coastline and more than 35 days in the northern part of the study domain. Thus, the accuracy of the ORS forecasts is much higher for the Guinean zone in comparison to the Sudan-Sahel region. Several studies already showed that seasonal precipitation forecasts for West Africa have usually a higher performance for the Guinean zone compared to the poleward located climate zones like the Sudan-Sahel or Sahel (e.g., [11,22]). Ref [22] mentions that those regions have a higher skill where precipitation is stronger interlinked with oceanic surface conditions. Another explanation for the lower model accuracy in the Sahel region can be the higher stochastic nature of the precipitation events in this region compared to the Guinean zone since most of the rainfall is the result of convective systems in the Sahel region (see [56]; and references herein). However, the gradual decline of the MAE values can also be a purely operational effect. The ORS dates in the Guinean zone are based on much shorter lead times in comparison to the ORS dates in the Sahel zone, and the accuracy of the ORS prediction should theoretically decrease with shorter lead times.

The correlation map shows a positive skill for most part of the Guinea zone. This region is also characterized by the highest skill (up to 0.40 and more) since the MAE and the bias values are relatively low compared to the other regions. The northern parts of the study domain show also a slight positive skill between the predicted and observed time series, although the accuracy of the ORS forecast is relatively low. The low accuracy is mainly the result of the strong unconditional biases in this region as shown in Figure 9c. The MAE value is negatively influenced by a bias, which is not the case for the correlation measure [53]. Thus, the correlation indicates the model skill in term of bias-free products and it is a verification measure for the potential skill of ORS forecasts. However, it must be noted that due to the small sample size (11 years), the significance threshold for the Pearson correlation is r = 0.39 (based on a Student’s t-test, α = 0.10, n = 11). Thus, for most of parts of the study domain the model skill is not statistically significant. Only for several grid cells next to the Guinean coast this threshold is reached or exceeded.

4.2. Visualisation of the ORS Fields

The ensemble-based forecasts of the ORS dates are shown in Figure 10 for the year 2001. We selected this year because according to the MAE and ACC values computed in the previous section, 2001 is the best year in terms of forecast accuracy and skill. Moreover, the year 2001 is also characterized by a rather special observational ORS pattern with a late ORS in the Guinean zone and an early ORS in the Sahel zone. This pattern is shown in Figure 11c computed as the differences between the absolute ORS values minus the climatological mean. Thus, positive values in Figure 11 indicate a late onset (red pixels, near the Guinean coast, mostly below 10° N) and negative values an early onset (blue pixels, mostly above 10° N).

The ORS forecasts in Figure 10 are based on the ensemble mean (middle panel), both terciles (right and left panel) and the inter-quantile range to illustrate the forecast uncertainty. Quantile-based forecasts are a straightforward way for visualization of ensemble-based forecasts and are often used to illustrate forecast uncertainties for precipitation, but also for other hydro-meteorological variables. The quantile-based approach is already used by [13] to visualize ORS predictions. This graphical approach should be also preferred in comparison to plotting the spatial fields of all ensemble members. This latter approach is often named as spaghetti plots in ensemble forecasting (e.g., [57,58,59] and it has the disadvantage that the plots became very confusing, in particular for large ensemble sizes and higher lead times.

The quantile-based ORS maps show a clear north–south gradient of the ORS dates. In this respect, a relatively realistic pattern is produced for the forecasts. The information based on the terciles can also be interpreted as an early and late ORS date. The difference between the late and the early ORS dates is an estimation of the forecast uncertainty. The uncertainty is quite large although only 1/3 of the ensemble is used for the illustration. Figure 10d also indicate that the uncertainty increases from the Guinean coastline to the Sahel zone due to the different lead times and other reasons as discussed in the previous section. Moreover, a comparison of Figure 10a–c with Figure 11a reveals that many observations are not enclosed by the selected uncertainty interval. The approach produces for the southern parts of the domain too early ORS dates and too late ORS dates for the northern part of the domain. This shows that the selected quantiles are too narrow to enclose all observations. If more extreme quantiles (e.g., Q90 and Q10) are used for the calculation of the late and early ORS dates, the observations will be better enclosed by the uncertainty interval. However, this will also strongly increase the range of the illustrated forecast uncertainty.

Another format for the visualization of ensemble-based ORS forecasts are probabilistic forecasts based on climatologically equiprobable categories shown in Figure 12. In this study, the forecast probabilities are calculated for each grid cell for tercile-based categories over the entire domain. Climatological average conditions (Figure 11b) are indicated by uniformly distributed forecast probabilities (1/3 for all three categories). For the central and northern parts of the study region, the ORS is forecasted to be earlier, because the forecast probabilities are much higher for “below-normal” than the average conditions (1/3) and much lower for the other categories. For the southern parts of the study region, it is vice versa, with much higher probabilities for “above normal” and ORS is expected to be late. However, a correct interpretation of the tercile-based categories is for many users not straightforward since forecast probabilities of climatologically equiprobable categories needs to be considered jointly and a good statistical background is therefore needed. For further discussion regarding this issue we refer to the study of [11] and the references herein.

An alternative way for illustrating the forecast probabilities of climatologically equiprobable categories is the ORS index proposed in Section 3.4. This index is displayed for the year 2001 in Figure 13 based on the forecast probabilities given in Figure 12. At a glance, the ORS index provides information on an early (green, ORSI > 0), normal (yellow ORSI ~ 0) and late (red, ORSI < 0) ORS. The ORSI therefore summarizes the information from the three probabilistic patterns and is thus much easier to interpret than Figure 12. It therefore provides a simple alternative to illustrate whether an early or late ORS is forecasted for the coming rainy season. For our specific example, the index shows strong negative values up to –30 for the Guinean coast indicating a late ORS for that region, while for the Sahel zone the opposite signal is predicted (a late onset with ORSI values up to 30). Thus, the predicted ORSI patterns show very similar large-scale features as the observed ORS anomalies shown in Figure 11c.

5. Summary and Conclusions

This work presented a statistical approach for seasonal forecasting of the onset of the rainy season using precipitation forecasts of a GSEPS. The basis of this approach was a quantile–quantile transformation for removing biases from the precipitation ensemble, a fuzzy-rule based approach for calculation of the ORS date and several graphical methods for an improved visualization of ensemble-based ORS forecasts. As outlined in the methodology part of the work, the different components of the approach were modular and could therefore be replaced by other methods (i.e., downscaling or ORS approaches) and datasets (i.e., GSEPS forecasts and observations). Moreover, the approach could be relatively easily transferred to other geographical regions of the world and extended to crucial rainfall characteristics like the cessation and the length of the rainy season. The presented approach has also the advantage that it is not very CPU demanding and requires only a single variable for the downscaling process. Unlike many other downscaling approaches, it might be better operationally applicable to forecasting centers in development countries where bandwidth and computing power are often limited.

In this study we also performed a first evaluation of the ORS forecasts by employing various verification measures and methods to determine the performance of the approach in space and in time. This analysis illustrated that forecasting ORS dates several months in advance was a major challenge. The ORS forecasts were associated with large uncertainties due to the inherent high uncertainty of seasonal forecasts and other reasons like the high stochastic nature of daily precipitation in West Africa. We also illustrated in this study the importance of selecting appropriate performance measures for verification of the spatial ORS fields to remove seasonal dependencies. This was done by using the anomaly correlation coefficient. Since the focus was on spatial verification of the ORS fields, other methods for verification of probabilistic forecast like the continuous ranked probability score [60] were not yet employed. However, this type of forecast verification is needed to get a better insight regarding the quality of probabilistic seasonal ORS forecasts. In addition, verification techniques for determining the economic value should be applied in future studies to determine another type of forecast goodness, i.e., the forecast value (e.g., [19]). This is needed to demonstrate how valuable the ORS forecast is for a particular end user. Recent examples towards this kind of verification are presented e.g., in [11,61]. The selected spatial verification methods applied in this study were also relatively strict and imply a perfect forecast of the ORS fields in space and time, in particular the MAE. This basic assumption should be further examined in future studies by the selection of alternative spatial verification methods. These new methods can be found, for example, in [23,62].

We also tested different forecasts formats for visualization of probabilistic ORS forecasts. These approaches range from quantile-based ORS forecasts, over climatologically equiprobable categories to a new ORS index for indicating an early or late ORS. Since the ORS index is a joint measure of categorical forecasts, it is much easier to understand than the common formats used in seasonal forecasting. Moreover, the ORS index was applied in this study to tercile-based categories but can be also used for forecasts for more than three categories and to other seasonal forecasts variables. The index provides therefore an important alternative to the current forecast formats used in seasonal forecasting.

There were several limitations of the presented methodology. Possible improvements in future studies are:

• Better adapting the ORS approach to the location-specific conditions in the region, i.e., considering the heterogeneous rainfall conditions across the region. This can be achieved by coupling the ORS approach to a process-based crop model to derive location-specific planting rules (ORS criteria), as done in [2,63]. Based on seasonal predictions, it is also possible to incorporate the false-start criterion in the ORS predictions;
• Testing the skills of other GSEPS such as the Climate Forecast System Version 2 or the ECMWF SYS5, which is e.g., improved in terms of the spatial resolution and the number of ensemble members compared to the SYS4 [64]. Since the probabilistic forecast depends strongly on the size of the ensemble members (e.g., [64]), the increase of the ensemble members may possibly lead to improved ORS predictions;
• Incorporating longer lead times in the analysis. Longer lead times would increase preparation time for farmers. On the other hand, increasing the lead time would result in reduced forecast periods for the rainy season;
• Further reducing the biases in the forecasts, either by improving the statistical bias correction method or by dynamically downscaling the forecasts. The former could e.g., be achieved by applying the corrections separately for the different months (e.g., [65]) or for different large-scale atmospheric circulation patterns (e.g., [66]).

In general, more precise ORS forecasts will possibly impact on the farmer’s tactical decision such as the timing for planting, the types crops and the varieties to grow, i.e., long or short duration varieties.