Analysis of a Short-Term and a Seasonal Precipitation Forecast over Kenya

: Kenya is highly dependent on precipitation for both food and water security. Farmers and pastoralists rely on rain to provide water for crops and vegetation to feed herds. As such, precipitation forecasts can be useful tools to inform decision makers and potentially allow the preparation for such events as drought. This study assessed the predictability of a seasonal forecast (CFSv2) and a short-term precipitation forecast (CHIRPS-GEFS) over Kenya. The short-term forecast was assessed on its ability to predict the onset date of the rainy season, and the skill of the seasonal forecast in predicting abnormal precipitation patterns. CHIRPS-GEFS provided a useful starting point to estimate the onset date, but during the long rains in the southwest, where agriculture is concentrated, differences between the predicted and actual onset dates were large (over 20 days). Assessments for CFSv2 generally displayed lower forecast skill over highlands and coastal regions at a seasonal scale. The CFSv2 forecast skill varied widely over individual months and lead times, but over whole rainy seasons, CFSv2 was more skillful than a random forecast at all lead times in the major agricultural areas of Kenya. This research ﬁlls a critical research and application gap in understanding the forecast precipitation skill for onset and sub-seasonal prediction.


Introduction
Agriculture in Kenya contributes to ∼26% of the Gross Domestic Product, and employs over 40% of the population [1]. However, there is a high prevalence of rainfed agriculture, and much of the country does not receive enough precipitation to support most crops [1]. Nearly 61% of the population depends on rainfed agriculture for income [2]. Pastoralists also depend on precipitation to sustain their herds. This makes food security in the region vulnerable to erratic precipitation patterns. Natural variability in precipitation patterns, further complicated by political conflicts, greatly impacts food and water security in Kenya and surrounding areas.
Kenya has two main rainy seasons: the long rains (March to June) and the short rains (October to December). The long rains are more reliable (show less inter-annual variability), and the short rains show more inter-annual variability, but less spatial variability over Kenya [3]. Accumulation patterns across the country are spatially variable, with the most precipitation in the highlands and southwest region around Lake Victoria (>2000 mm/year) and the least in the north (<200 mm/year). Precipitation is affected by the tropical circulation, as well as topography, lakes, and maritime influence [3]. East African precipitation is strongly connected to variations in the Walker cell with modifications on sub-seasonal to inter-annual time scale driven by Madden-Julian oscillation and El Niño-Southern oscillation variability. The slowly varying ENSO variability and remote response of the tropical circulation are able to provide some level of predictability at seasonal time scales. These types of connections, especially with East African rainfall, have been explored extensively [4][5][6].
While droughts generally occur about once every five years, the frequency of drought and flood events has been increasing in the past 30 years [7][8][9]. Combined with increasing temperatures, the long rain season has shown declining precipitation trends since the 1980s as indicated by decreasing trends in the total precipitation amount, number of precipitating days, number of consecutive wet days, and number of very heavy precipitating days [7,8,[10][11][12]. In contrast, the short rains have increased in recent years as indicated by total precipitation amounts and number of precipitating days, but the overall yearly precipitation has been decreasing [3,12]. Drought events have also been spanning multiple rainy seasons, such as during the 2010 short rains and 2011 long rains [13,14].
Precipitation data have traditionally come from station or rain gauge observations. However, in many regions, observation networks are sparse, causing large spatial disparities in coverage [15]. In addition, many sites have incomplete or inconsistent historical records [16]. Gauge data are available through the Kenya Meteorological Department; however, rain-gauge data observed during the post-independence era of the 1970s in particular suffer from spatial and temporal discontinuities over large sections of eastern Africa [17]. To address these challenges, satellite observations from dedicated missions, such as the Tropical Rainfall Measuring Mission (TRMM) [18] or the Global Precipitation Measurement mission (GPM) [19], and models, such as SM2Rain [20], can be used to create more consistent precipitation data with greater coverage.
Precipitation forecasts at both seasonal and sub-seasonal scales are an important part of early warning systems. If forecasts predict poor precipitation for a season, measures can be taken to prepare aid, such as food and water distribution, reducing response time if government intervention is needed. Precipitation forecasts are one of the key inputs to land and cropping system models. Short-term forecasts can also be useful in such systems, particularly when more precise dates of precipitation events are needed, for example, to predict the onset of the rainy season. There are ongoing efforts to produce crop yield forecasts for Kenya, some of which require weather forecast data as inputs [21][22][23]. Agricultural models, such as the Decision Support System for Agro-Technology Transfer (DSSAT [24]), require sub-seasonal data in order to simulate crop growth. Therefore, evaluations of these weather forecast inputs are necessary to understand the skill and uncertainty associated with forecast data.
The onset date of the rainy season is an important first indicator for potential drought events. A delay of the onset date of the rainy season by at least 10 days can be a reliable predictor of drought in food insecure regions of East Africa [25]. Knowledge of the onset of the rainy season is also important for such applications as modeling crop yields, as the projected yields will be affected by the date of planting.
Several seasonal forecasts are available for the region, such as the European Center for Medium-Range Weather Forecasts seasonal forecasting system (SEAS5), IRI's seasonal climate forecasts, and the NASA Global Modeling and Assimilation Office's seasonal forecasts, using the Goddard Earth Observing System [26][27][28]. One such seasonal forecast is the Climate Forecast System version 2 (CFSv2). CFSv2 is one of the widely used forecast products in the region for applications such as drought, agriculture, and soil moisture prediction [29][30][31]. Moreover, it provides continuous data throughout the year (not just during the rainy seasons), and provides a 6 month forecast, which is useful since crop growth seasons last beyond the end of the rainy seasons. The CFSv2 product used here is a downscaled and bias-corrected version, available through ClimateSERV (https:// climateserv.servirglobal.net/, accessed on 3 June 2021) at 0.5°resolution [32,33].

The Climate Hazards Group InfraRed Precipitation with Station data-Global Ensemble
Forecast System (CHIRPS-GEFS) was evaluated for a short-term forecast. The CHIRPS-GEFS dataset combines the higher spatial resolution of CHIRPS (0.05°) and the advanced forecasting ability of GEFS to provide up to a 16-day forecast updated every five days at a spatial resolution of ∼5 km across the globe [33]. GEFS is a weather forecast model made up of 21 ensembles, started by the National Centers for Environmental Prediction (NCEP) to address the uncertainty in weather observations, which are used to initialize weather forecast models [34].
The purpose of this study was to evaluate both a short-term and a seasonal precipitation forecast over Kenya. The short-term forecast was evaluated for potential use at the beginning of rainy seasons to determine the onset date. Then, the overall skill of the seasonal forecast was analyzed to determine if and how far out the forecast would be useful. It is important for researchers and organizations to know the capabilities and limitations of forecasts in order to disseminate information and uncertainty in the most effective way. Therefore, this study conducted an evaluation of the CFSv2 and CHIRPS-GEFS precipitation forecasts. We analyzed the CFSv2 lead times from 0-6 months to determine how far out skillful precipitation forecasts can be made.
CHIRPS data were used as a reference dataset for precipitation to compare the forecast products. CHIRPS uses global 0.05°precipitation climatologies, time-varying grids of real-time satellite-based precipitation estimates, and in situ precipitation observations to create its precipitation data [35]. CHIRPS was used as a reference, as it has better skill than other gridded products, and is known to have almost negligible bias in the region [36]. Furthermore, the CHIRPS data ensure a consistent spatio-temporal continuity, due to their long period of record (over 40 years) and global coverage.
Several studies have assessed the forecast skill of seasonal forecasts in the region [29,[37][38][39]. These studies have used methods such as analyzing the correlation of forecast precipitation with a reference, and translation of precipitation forecasts into categorical events for the calculation of skill scores [29,[37][38][39]. However, few, if any, have assessed the skills of forecast data in onset date prediction, and none to our knowledge have assessed the downscaled and bias-corrected version of CFSv2 used here for the region. The results of these evaluations will be used to inform stakeholders in the region. This analysis will also be useful for organizations using the forecasts for applications such as drought or agriculture forecasting.

Study Area and Period
This study was conducted over Kenya at pixel level (0.5°for CFSv2, 0.05°for CHIRPS-GEFS). Kenya has a wide range of precipitation in different regions, varying from >2000 mm/yr in the southwest to <200 mm/yr in the most arid regions (Figure 1). The vast spatial variability in the precipitation patterns allows for a robust assessment of the forecast model under different climatic and ecological conditions. The highlands are concentrated in the southwest of Kenya, as is most of the agricultural land. There is also a swath of agricultural fields along the coast in the east. There are a range of ecological zones across the country, from humid in the southwest to arid in the east. This study covers the time frame of 1990-2020 to include enough years for statistical analysis at a monthly level.  [35], elevation (top right) [40], cropland from Global Food-Support Analysis Data (bottom left) [41], and ecological zone (bottom right) [42].

Data
The CFSv2 product used here is a downscaled and bias-corrected version, available through ClimateSERV (https://climateserv.servirglobal.net/, accessed on 3 June 2021) at 0.5°resolution [32,33]. A bias correction and statistical downscaling (BCSD) algorithm was applied to CFSv2 as described in Sikder et al. [43] following the approach of Wood et al. [44]. Quantile-quantile mapping was used to bias correct leaddependent model forecast distributions of precipitation and temperature to observationbased distributions of the Princeton surface global forcing dataset (SGF) for the years 1982-2012 Sheffield et al. [45]. To spatially downscale, a local scaling approach was applied to the bias-corrected forecasts to generate 0.5°estimates from the 1.0°CFSv2 forecasts obtained from the International Research Institute for Research and Society (IRI; https://iridl.ldeo.columbia.edu/SOURCES/.Models/.NMME/, accessed on 3 June 2021). The local scaling factor was equal to the ratio of the climatological high-resolution estimate against that obtained by resampling of the coarse-resolution climatology to the locations of the finer-resolution grid [43].

CHIRPS-GEFS is a bias-corrected and downscaled version of the National Centers for
Environmental Prediction (NCEP) Global Ensemble Forecast System precipitation forecasts made to be spatially compatible with various CHIRPS products [46]. It provides a 16 day forecast updated every five days. It is available through the University of California, Santa Barbara (UCSB) at https://data.chc.ucsb.edu/products/EWX/data/forecasts/CHIRPS-GEFS_precip/ (accessed on 3 June 2021). Daily data at 0.05°were used for prediction of the rainy season onset date.
CHIRPS was used as the reference precipitation dataset for comparison with both forecast products. CHIRPS version 2.0 daily data were used, available from USCB at https://data.chc.ucsb.edu/products/CHIRPS-2.0/ (accessed on 3 June 2021) [35]. In order to compare it with CFSv2 data, CHIRPS was resampled to 0.5°using bilinear interpolation and aligned with CFSv2 grids.

Onset Date
Onset date is defined here as the first dekad with at least 25 mm of precipitation, followed by two dekads, which sum to at least 20 mm of rain. This definition was originally defined by AGRHYMET and is widely used in the region [47]. The first dekad of the three is considered the onset date when all conditions are met. CHIRPS-GEFS data were used to predict the onset date at pixel level. CHIRPS-GEFS forecasts are 16 days out, but the definition of onset date used here requires at least 30 days of data. Therefore, CHIRPS-GEFS forecasts would only be able to define the onset date in-season about 2 weeks after the onset date actually occurs in real time. However, this onset date estimate would still be available prior to an estimate from CHIRPS, which is available at about 3 weeks latency. The CHIRPS-GEFS onset predictions were compared to CHIRPS, and the average and absolute differences in onset date were analyzed.

Seasonal Forecast
The total monthly precipitation from CFSv2 was analyzed for the seasonal forecast. CFSv2 was assessed at the scale of the entire rainy season, and at each individual month. The forecast was translated into discrete events by classifying 'anomaly events' as the 20th and 80th percentiles of precipitation at the pixel level. The percentiles were calculated from data for all years for each particular month. Events below the 20th percentile were classified as dry anomaly events, and above the 80th percentile were classified as wet anomaly events. These percentiles were chosen, as droughts are recurrent about every 5 years, and livestock insurance efforts in the region use the 20th percentile as a threshold for insurance payouts [48,49]. Contingency Table 1 was created from the events, using CHIRPS to classify the 'observed' events.

Observed Yes Observed No
Forecast Yes a b Forecast No c d There are several statistical matrices to compare and evaluate precipitation products. The probability of detection (POD) is one such metric that measures the proportion of events successfully forecast by the model. POD ranges from 0 to 1, with 1 being the best. It is calculated as follows: Another metric is the false alarm ratio (FAR) that measures the proportion of 'yes' forecasts that fail to materialize. This score is important to report, as raising many false alarms will lessen people's confidence in the forecasts. FAR ranges from 0 to 1, with the best possible value being 0. It is calculated as follows: The Heidke skill score (HSS) is based on the proportion correct in comparison to a reference forecast. The reference forecast is the correct proportion that would be achieved by random forecasts statistically, independent of the observations. Perfect forecasts would receive HSS = 1, forecasts equivalent to the reference would receive HSS = 0, and forecasts worse than the reference would receive negative scores [50]. HSS is calculated as follows: The relative operating characteristics (ROC) score (also known as the receiver operating characteristics score) shows the degree of correct probabilistic discrimination for a set of forecasts. ROC is plotted as a graph, with the hit rate shown on the x-axis and the false alarm rate shown on the y-axis. The ROC curve is created from a set of points: the first point indicates the hit rate and the associated false alarm rate only for forecasts in the highest bin of issued forecast probabilities. Subsequent points indicate the hit versus false alarm rates for cases with successively decreasing forecast probabilities. The area underneath the ROC curve is the ROC score. ROC scores above 0.5 reflect positive discrimination skill, with 1 being the best possible score [50,51].

Onset Date
CHIRPS-GEFS predicted a later start date than observed for most of southwest Kenya during the long rains, where much of the agriculture is concentrated (Figure 2. In contrast, CHIRPS-GEFS predicted an earlier onset of the short rains in parts of the same region, mainly the highland areas. Through much of eastern Kenya, CHIRPS-GEFS predicted an early onset date. Along the coast during the short rains, there are both areas where CHIRPS-GEFS predicted much earlier and much later start dates than observed. In the most arid regions, no start dates were predicted or observed, as they did not receive enough precipitation over a period of three dekads to ever fit the onset definition used here. This is especially evident in the short rains in the northwest, shown by a large area of no data in the figure. This area is semi-arid to arid. This pattern was expected, as yearly precipitation in the area is very low. The absolute difference in onset dates in Figure 3 show the largest differences in predicted long rains' onset in the southwest and northeast. The short rains had the largest differences in the predicted onset date in the highlands and along the coast.  Figures 4 and 5 show the spatial patterns of POD over both rainy seasons for dry and wet anomaly events. The spatial distribution of skill did not change much over the different lead times. For the detection of dry events, CFSv2 performed worse in the western half of Kenya and along the coast. Throughout central Kenya, the POD was near perfect. The spatial patterns for the detection of wet events were not as distinct, but they also appeared to perform slightly worse along the coast. POD was best for northeast Kenya, being between 0.25 and 0.5 in the western half of Kenya for wet events. The spatial patterns of FAR and HSS were similar to those of POD. HSS indicated that for wet anomaly events, CFSv2 is more skillful than a random forecast for the whole country, except along the coast. For dry anomaly events, it is more skillful than a random forecast, except along the coast and in a small area of the northwest. The ROC displayed similar patterns of skill as the other skill scores for dry events ( Figure 6), with only a small area in the northwest having lower skill than a random forecast. For wet anomaly events (Figure 7), the pattern of ROC was similar to that of the dry events, but the region of lower skill in the northwest became larger with longer lead times.

Seasonal Forecast
The average of each skill score over Kenya during different lead times is summarized in Figure 8. The 0 month lead time appears to be a bit better than the rest, but there is not a consistent decline in skill for most of the indicators.     Table 2 shows the average skill for each ecological zone. For dry anomaly events, forecasts for the sub-humid and semi-arid zones performed well. For wet anomaly events, forecasts for arid and semi-arid zones performed the best, while all other zones performed similarly. Though there was not much difference in skill between the different lead times when looking at the rainy seasons as a whole, it is obvious that the 0 month lead time performed much better when looking at individual months. The forecasts also performed worse overall when looking at individual months, with HSS being below 0 in many areas, indicating that the forecast is worse than the reference random forecast at those points. Spatial patterns are also different across each month, as shown in Figures 9 and 10. The forecasts for individual months of the rainy seasons performed better than those during the dry seasons.

Onset Date
Forecast onset dates from CHIRPS-GEFS are mostly within the standard deviation of each rainy season, and therefore, can be useful to narrow down the onset date prior to the observed onset date availability. The standard deviation of onset date is 15 days for the long rains, and 24 days for the short rains [52]. For applications such as crop forecasting, it can be necessary to have an onset date of the rainy season that is as accurate as possible so that variables, such as planting dates and crop growth phases, match with real life. These variables can greatly impact the forecast yield, as different amounts of water are needed in different growth stages for optimal yield [53]. The onset dates predicted by CHIRPS-GEFS provide a good starting point for such applications before observed precipitation data are available. There are many areas with average differences in predicted onset dates over 15 days during the long rains, particularly in the southwest, where agriculture is concentrated. Onset differences that large could sufficiently affect forecast yields such that any models using CHIRPS-GEFS to estimate onset dates would need to convey the uncertainty associated with it.

Seasonal Forecast
At a seasonal scale, CFSv2 forecasts performed better than a random forecast over most of Kenya. The main region where CFSv2 had consistently low skill was the coastal region. It was suggested that coastal precipitation is more influenced by the sea surface temperature than other regions [54]. Therefore, a larger scale model may not take the influence of proximity to the ocean into account. General circulation models (GCMs), such as CFSv2, were assessed for applications such as crop forecasting, drought forecasting, and streamflow prediction [22,29,43]. When downscaled to the monthly or daily levels necessary for some applications, precipitation forecasts have lower skill and introduce more uncertainty into models. Sikder et al. [43] demonstrated that for streamflow in the Ganges and Brahmaputra River Basins, GCMs were only useful for management decisions at a seasonal scale or greater.
Our study shows that forecasts at a monthly level had large areas where a random forecast would be more skillful for lead times greater than 0 months. At a monthly scale, forecasts for the long rains had higher skill in the prediction of dry anomaly events, while forecasts for the short rains had higher skill in the prediction of wet anomaly events. This could be due in part to the intensity of these anomaly events; for example, CHIRPS data show much stronger wet anomaly events during the short rains. For dry anomaly events during the dry rains, in certain cases, longer lead times show greater skill. For June forecasts, lead times of 3-5 months show higher skill than shorter lead times. Similarly, lead times of 2-4 months show greater skill than the 1 month lead time for May forecasts. These forecasts have in common that they were initialized between January and March. The decrease in skill for forecasts initialized after March may be due to the Spring predictability barrier (SPB). SPB is a phenomenon of El Niño-Southern Oscillation (ENSO) forecasts, where ENSO predictions from models through the spring tend to be less successful [55][56][57]. Duan and Wei [57] show that prediction errors tended to have the largest growth rate in the April-May-June season, which may provide explanation for the lower skill of forecasts initialized during these months.
For agricultural areas for dry events, CFSv2 performed best over the long rains for forecasts initialized between January and March. Over the short rains, dry event forecast skill was lower in agricultural areas, having lower skill than a random forecast in some parts. The 0 month lead time was the only lead time with consistently skillful forecasts over agricultural areas for the short rains. For crop forecasting efforts in the region, CFSv2 may be of limited use prior to December for the short rains growing season.
For forecasts that provide data at a sub-seasonal level, future forecast validation and correction efforts may want to focus not only on whole seasons, but also on the temporal scale that data are provided, as many months have different spatial patterns of skill. Month-to-month skill was also lower overall than skill calculated over whole rainy seasons. The rains were also influenced by different forcing methods from month to month, and thus, it would make sense for forecasting efforts to tailor forecasting methods to each individual month to derive the best possible forecast [3].
Overall, this work has shown that CHIRPS-GEFS forecasts are useful to narrow down the onset date of the rainy seasons. CFSv2 forecasts were shown to be better than random forecasts for most areas at seasonal scales, but forecast skill for individual months varied. The results will be used to inform stakeholders and organizations using the forecasts for such applications as drought or agriculture forecasting.

Conclusions
This study assessed two precipitation forecast products over Kenya for both shortterm and seasonal applications. The short-term forecast CHIRPS-GEFS was shown to reasonably predict the onset of rainy seasons over most of the country. However, areas such as the southwest and northeast for the long rains and highlands and coastal areas during the short rains displayed large (> 20 day) differences between predicted and actual onset dates. This would likely have a great impact on models or warning systems relying on the forecast onset date. The seasonal forecast CFSv2 was shown to be more skillful than a random forecast over most areas of Kenya; however, skill varied widely over space, month forecast, and lead times. Given these results, if forecast data are provided at a sub-seasonal scale, future forecast evaluations should assess forecasts at the temporal scale that data are provided, rather than over seasons as a whole.