4.1. Forecast Performances
The results consider the observed flow and the four forecasted series (NH, WWH, NHOTT and WWHOTT) at a 5-day forecast horizon.
The results of Table 6
and Figure 4
, Figure 5
, Figure 6
and Figure 7
show the performance for the GK station. The results for BB are not shown since: (1) the short flow time series of BB does not allow a sufficient background and (2) these results are quite closer for BB and GK. The results of Table 7
instead show the mean performances in the two gauging stations to evaluate the overall performances of hydrological forecasts on the Sirba EWS that present some differences in the two stations. Tables and figures with the overall results are presented for completeness in Appendix A
The preliminary analysis based on the basic statistical parameters demonstrates that while NH correctly identifies the magnitude of minimum, mean and maximum flows WWH values are quite underestimated (Table 6
). After the optimization process the indices demonstrate that both NHOTT
are overestimated. The behavior is quite different for the two models, while NHOTT
overestimates maximum more than the mean, WWHOTT
does the contrary.
Flow duration curves (Figure 4
) confirms the preliminary results emphasizing some interesting points: (1) all the forecasts simulate quite well the dry period of the river even if WWH zeros start from day 100 and NH is not able to produce zero values, (2) WWH forecasts do not demonstrate a clear relation with the observed river flow, (3) NH, even if has the same mean than the observed flow, overestimates in the very high (Q1
) and low flows (Q80
) and underestimates in the medium flows (Q10
), (4) NHOTT
is quite closer to the observed flow until day 75 and overestimates the low flows and (5) WWHOTT
strongly overestimates all the observed flows.
The hydrographs with the NH forecast demonstrate that the magnitude of forecasted flow is quite close to the observed ones (Figure 5
). The forecasts do not present strong outliers and well identify the flow peaks in the hydrologic year. Generally, forecasts demonstrate a good skill before optimization and improved skill after optimization. In NH, the influence of calibration at the Garbey Kourou station [25
] is very clear. The forecast performance at Bossey Bangou has lower quality (Appendix Figure A2
The hydrographs with WWH forecasts clearly show that original forecasts are able to represent the annual flow cycle, especially for the dry season, but not the flow magnitude (Figure 6
). The WWHOTT
demonstrates a strongly improved correlation with the observed hydrograph and both timing and magnitude of flood peaks are correctly identified. The gap between WWH and observed flows is related to the setup and goal of WWH (calibrated with a global focus and a monthly resolution for water balance analysis), without the more tailored calibration applied in NH.
The hydrograph of the 2019 validation wet season well highlights strengths and weaknesses of the four compared forecasts (Figure 7
): (1) WWH forecasts for July and the first part of August are mainly zeros and the optimized ones are quite overestimated, (2) NH wet season sufficiently well reproduce the GK flows but is shifted about two weeks forward, (3) the optimized forecasted flows correctly identify the annual maximums between the middle of August and the middle of September even if the WWHOTT
in the last period of August is quite underestimated and, (4) even if the NHOTT
performs well, it’s affected by high levels of missing data (i.e., low operational capacity). This is a major limitation of the present FANFAR system, a result of the fact that it is still a pilot system and not yet a production-grade system.
Continuous indices highlight the hydrological model capability to correctly forecast the flow in the whole hydrologic year. RMSE identifies the mean gap (in absolute value) between observed and forecasted flow that demonstrates the better performance of NH and the importance of optimization process (Table 7
). RSR and NSE are normalized indices that, starting from RMSE, are useful to evaluate the performances in comparison with reference values. According to Moriasi et al. 2007 [49
] classification that identifies five levels (bad, unsatisfactory, satisfactory, good and very good): (1) NH performance is satisfactory before and very good after the optimization and (2) WWH performance is bad (RSR) or unsatisfactory (NSE) for the original version and good after the optimization process.
Categorical indices are fundamental to highlight the ability to correctly identify the flow values above the threshold used to distinguish flood to normal values. BIAS show that WWH is not able to forecast flood values, NH forecasts underestimate (BIAS < 1) and optimized forecasts overestimate (BIAS > 1) streamflow. POD and FAR are the fundamental values to identify the forecasting capability. HYPE forecasts demonstrate a quite good POD but a too high FAR resulting in forecasts that are good but not completely reliable. Optimization raises POD (similar performance of NHOTT
) and reduces FAR for both models. For this point it is important to highlight that the values in Table 7
derive from the mean between Garbey Kourou and Bossey Bangou performances (complete values are reported in Appendix A
). This difference is clearly noticeable for NHOTT
POD and FAR that derive from excellent values in GK (POD = 0.85 and FAR = 0.35) and bad values in BB (POD = 0.4 and FAR = 0.65). PC values are quite constant for NH before and after the optimization and reduces in WWH since the high number of over forecasting. TS and HSS jointly consider the POD and FAR emphasizing the best performances of NH compared to WWH. In wider terms, from both continuous and categorical points of view, optimization process allows to significantly reduce the gap between forecasts and observations confirming previous results reached in the literature [30
Finally, operational availability quantifies the number of missing forecasts (i.e., not produced within the forecast issue date). The results show that NH has significantly worse availability than WWH for the considered time frame (Table 7
), and that none of them ensures full time availability. This problem was typically caused by various information communications technology (ICT) production failures on the Hydrology-TEP cloud platform (https://hydrology-tep.eu
) on which the FANFAR pilot system is currently deployed to run every day. Most problems were related to necessary files being inaccessible or incorrectly stored on the Hydrology-TEP Catalogue and Store, while a minority were also related to the forecasting production service. To be robust, local EWS requires new forecasts every day without delay [28
], therefore there is a need to upgrade the FANFAR pilot system to a fully supported operational production-grade system.
4.2. SLAPIS Operational Application
According to the evaluation of the HYPE models’ reliability conducted with continuous, categorical, and operational availability indices on NH and WWH, the latter has been integrated in SLAPIS to be used operationally during the last hydrological season by Niger Hydrological Directorate with restriction to registered users of the SLAPIS platform. The reasons for the choice of WWH are three: (1) the direct output in the BB sub-basin; (2) the more similar results in the two stations and the better forecast performance in BB more useful for the EWS and; (3) the considerably higher value for the operational availability in comparison to NH.
Thanks to the adoption of service oriented architecture paradigms, it was possible to easily integrate WWH into SLAPIS using open-source technologies and developing software components. The data model of SLAPIS has been enriched with new tables to store WWH outcomes for seven specific sub-basins, two of them connected with the gauging stations of Bossey Bangou and Garbey Kourou. Following the WWH outcome specifications, an automatic procedure has been developed using J2EE technology and integrating OpenSearch engine to download and store WWH forecasts into the Geo Data Base every day (Figure 8
). The WWH forecast availability in the SLAPIS database starts from June 2017 and is quite constant for each sub-basin.
The WWH optimization procedure has been developed inside the SLAPIS database, based on PostgreSQL and PostGIS engines, using PL/pgSQL language. This procedure is exposed through a REST web service developed with JAX-RS and J2EE technologies to supply calibrated WWH outcomes. The REST web service is used by SLAPIS web application (www.slapis-niger.org
) in order to show and plot WWH optimized forecasts through the graphical user interface (Figure 9
WWH forecasts are available only for profiled users (e.g., operational hydrologists, ANADIA2 partners or scientists). All data are also accessible by Web Catalogue Service of the SLAPIS web platform or using application programing interfaces (APIs) for users with more advanced informatic skills.