3.1. Meteorological Forecasting and Assimilation Performances
A preliminary analysis was conducted to evaluate the model’s performance and determine the optimal setup to be applied in the demonstrator phase. Specifically, six case studies (two for each site of interest) were selected to assess the impact of data assimilation under different meteorological conditions. For each case, forecast model runs were validated against radar data corrected with rain gauge measurements, and rainfall verification was performed over various accumulation periods—6 h, 12 h, and 24 h—depending on the event type and agricultural needs. The MODE software (v. 9.1) was used to identify and match observed and forecasted precipitation patterns, followed by the calculation of relevant statistical indices. This preliminary analysis enabled the identification of the most suitable model configuration for subsequent application in the demonstrator phase. The results confirmed that data assimilation significantly improves the accuracy of short-term precipitation forecasts, particularly in terms of spatial alignment and peak intensity, while still providing a positive, though reduced, effect on 24-h forecasts. This supports its operational relevance.
This section presents only the results from the demonstrator phase, which leveraged the validated setup to highlight the overall added value of the forecasting approach developed within the project. Two distinct demonstrator types were implemented, each addressing specific practical applications. The first focused on irrigation advice, aiming to improve water management in agriculture by enhancing forecasts of key variables such as temperature, humidity, and rainfall—critical parameters for informed irrigation decisions and crop health. The second demonstrator targeted forecasting of extreme weather events, particularly localized convective storms such as summer thunderstorms. Given their short duration, high intensity, and spatial variability, these events pose significant forecasting challenges. In particular, the French demonstrator applied advanced data assimilation techniques to improve the short-term prediction of convective storms, thereby supporting disaster management and early warning systems. In both demonstrators, all available sensor data from various sources were assimilated to maximize forecast reliability and operational relevance.
Within the MAGDA framework, which integrates both irrigation management and extreme weather forecasting demonstrators, different validation strategies were adopted according to the objectives of each case. For the irrigation-focused demonstrators, rainfall validation was delegated to the hydrological component, as the hydrological and irrigation models provide a direct assessment of how rainfall localization and intensity affect water resource management. This approach aligns with previous studies, where hydrological output was used as an indirect validation tool to assess the impact of meteorological forecasts on outcomes relevant to end users [
45,
54]. Accordingly, validation in these demonstrators focuses on the contribution of data assimilation, particularly from project-specific sensors integrated into existing networks, to improved model state and forecast performance for agriculturally relevant variables during the summer season.
Conversely, for the French demonstrator, which specifically targets short-term forecasting of summer convective events in a nowcasting framework, a direct validation of precipitation forecasts was performed. This approach was intended to highlight the value of data assimilation in improving convective event predictions, where its impact is most relevant for timely risk mitigation.
During the demonstrator phase, specific periods and events were selected across the three pilot sites to evaluate the impact of data assimilation under operational conditions. For the Italian site, one time window from 20 to 24 June 2024 was analyzed, focusing on hydrological and irrigation advisory services. In Romania, four different time windows corresponding to various stages of the 2024 summer crop season were selected: 29 June–3 July, 22–26 July, 4–8 August, and 6–11 September, all related to irrigation management needs. For the French site, the selected time window of 18–19 June 2024 featured a convective event characterized by notable spatial variability in rainfall intensity, elevated atmospheric moisture, and the availability of Meteodrone profiles, making it a representative case for testing improvements in short-term convective forecasting.
3.1.1. Irrigation-Focused Demonstrators
The Romanian and Italian demonstrators were designed to improve irrigation management by incorporating data assimilation techniques into weather forecasting. In the Romanian demonstrator, the data assimilation process involved assimilating 2mT observations from the national network, radar reflectivity, ZTD data from GNSS stations, and Meteodrone data when available. The same data were assimilated in the Italian demonstrator, except for Meteodrone data, which were not available during the study period. The results obtained from a meteorological and assimilation perspective were similar at both sites; therefore, the outcomes are presented jointly in the following discussion.
As a first evaluation, the bias and standard deviation values for 2mT, ZTD, and radar reflectivity were analyzed across the three assimilation cycles, providing insights into forecast error propagation throughout the assimilation process (
Appendix A.2).
To provide an overall assessment of the impact of data assimilation in the Romanian demonstrator,
Figure 5 summarizes the performance of key variables by comparing the background forecast (BF) and the analysis (DA) against observations. The evaluation focuses on three variables: 2-m temperature (2mT), radar reflectivity, and GNSS-derived Zenith Total Delay (ZTD), analyzed across multiple assimilation cycles.
For near-surface temperature, the results show a clear and substantial improvement after assimilation. Both the standard deviation and bias of 2mT (
Figure 5A,B) residuals are significantly reduced in the analysis compared to the background forecast, confirming that assimilation effectively corrects temperature errors and stabilizes the forecast near the surface.
In the case of radar reflectivity (
Figure 5C,D), the impact of assimilation is more moderate. The analysis shows a slight reduction in bias, indicating that the assimilation process helps to nudge the model towards better agreement with observed convective structures. However, the standard deviation remains largely unchanged. This behavior is expected, considering the high inherent variability and noise of radar observations, as well as the limited number of cases available, which constrain the potential for noticeable improvements in the spread of errors.
Regarding GNSS-ZTD (
Figure 5E,F), assimilation leads to a slight reduction in bias, bringing the model’s representation of atmospheric water vapor closer to observations. However, the improvement remains within the observational error margin of approximately 1.5 cm, which is already accounted for in the 3D-Var assimilation scheme. Moreover, since the analyzed period did not include particularly intense meteorological events, the background forecast already provided an adequate representation of the vapor field, thus limiting the potential for further correction through assimilation.
In the Romanian demonstrator, vertical profiles collected by weather drones (Meteodrones) were assimilated whenever available, with the objective of assessing their contribution to improving the model’s representation of key atmospheric variables such as temperature, dew point temperature, wind speed, and wind direction. In total, eight assimilation cycles involving drone data were analyzed, corresponding to eight distinct time instances under different meteorological conditions. These profiles provided high-resolution information in the lower and middle troposphere, where traditional observations are typically sparse, making them particularly valuable for short-term forecast improvements. For an evaluation of the impact of cycling assimilation, see
Appendix A.1.
Across all cases, the general trend confirms that the data assimilation (DA) step consistently improves the model state compared to both the background forecast (BF), which includes observations from previous cycles but not the most recent ones, and the open-loop (OL) simulation without any assimilation. The DA run systematically shows better agreement with observations, especially for thermodynamic variables, indicating the effective integration of drone data into the model.
However, the benefit of assimilation is not uniform across all cases and variables. In several instances, the BF does not significantly outperform the OL simulation, suggesting that the improvement introduced by previous assimilation cycles may dissipate depending on atmospheric dynamics and model characteristics. Additionally, while DA consistently improves temperature and humidity profiles, its impact on wind-related variables, particularly wind speed, exhibits greater variability, with performance depending on specific vertical levels and prevailing meteorological conditions.
Figure 6 provides a detailed example of this behavior, focusing on two consecutive assimilation cycles: 30 June 2024 at 21:00 UTC and 1 July 2024 at 00:00 UTC. The top panels show the detailed vertical profiles of temperature, dew point temperature, wind speed, and wind vector cosine similarity (WCOS), comparing the observations, analysis (DA), and background forecast (BF). In both cycles, the temperature and dew point profiles reveal that DA successfully corrects the biases present in BF, particularly near the surface and around 850–700 hPa.
Wind speed, which is generally underestimated by BF, is also better represented in the DA run, though some residual discrepancies persist in certain layers, especially near 850 hPa. The WCOS indicator shows that DA improves the directional agreement of wind vectors with observations, especially in the lower layers, though the improvement is less pronounced in the first cycle.
The lower panels of the figure present the same variables at predetermined pressure levels, adding the OL simulation (blue lines) for a comprehensive comparison. This enables the evaluation of the cumulative effect of assimilation cycles and assesses whether BF shows any improvement over the OL run. For temperature, all three simulations (OL, BF, DA) are relatively close to observations, with DA offering slight refinements, particularly near 925–850 hPa.
Dew point temperature shows a more pronounced improvement in DA, which reduces the dry bias present in OL and BF. Wind speed profiles confirm that OL underestimates wind intensity, BF provides partial correction, and DA yields the best alignment with observations, though discrepancies remain at mid-levels. WCOS similarly shows better agreement in DA, especially below 800 hPa, indicating an improvement in wind direction alignment after assimilation.
3.1.2. Extreme Event Forecasting: Convective Storms
At each site, multiple model runs were performed both with and without data assimilation to evaluate its potential added value in providing farmers with more accurate and precise forecasts. To achieve this, a sensitivity analysis was conducted by adjusting specific parameters within the MODE software to determine the optimal configuration for comparing predicted and observed weather patterns.
The French demonstrator assimilated data from various sources, including 2 m temperature, Zenith Total Delay (ZTD), and Meteodrone observations, to enhance the prediction of convective storms. The assimilation of these observations led to significant improvements in spatial and temporal accuracy, especially regarding the intensity and location of rainfall. By comparing simulations with and without data assimilation, it was found that assimilation improved the model’s ability to predict the evolution of convection, even in cases where storms were not particularly severe. The improved forecasts were particularly valuable for localized storm prediction, as they allowed for more accurate storm tracking and better-informed decision-making regarding disaster response and preparedness.
For the two French cases (18 June 2024 and 19 June 2024), the evaluation focused on the effect of assimilation on convective storm forecasts in the vicinity of the area of interest. These simulations were run in a nowcasting setup, with shorter forecasts initialized closer to the event. For both events, the verification timestep ranged from 12:00 to 24:00 UTC.
It is worth noting that in both cases, the rainfall events were neither particularly intense nor severe, and the recorded precipitation amounts were relatively low: these two days were selected because thunderstorms coincided with the availability of drone data for assimilation.
The analysis focused on lower precipitation thresholds, not exceeding 15 mm. This choice was made because setting a higher threshold would have limited the identification of sufficiently relevant objects in the verification process, thereby reducing the effectiveness of the evaluation.
In the first French case (18 June 2024), a threshold of 5 mm/12 h was adopted due to the low event intensity.
Figure 7 (top) shows the objects recognized by MODE—both observed and forecasted. The left columns display WRF OL objects, while the right columns show WRF DA objects, for the 5 mm/12 h threshold. Blue contours represent observed objects; brown areas indicate forecasted objects; the red dot marks the Beaune site. Data assimilation notably improved the model’s performance, especially in terms of spatial pattern alignment, across all evaluated precipitation thresholds.
An examination of the statistical indices reveals that DA achieves a slightly higher Critical Success Index (CSI) than OL, indicating a better match between predicted and observed events. CSI excludes correct negatives and focuses solely on relevant forecasts, being sensitive to hits and penalizing both missed events and false alarms without distinguishing the source of error.
Both OL and DA exhibit a general overestimation of precipitation; however, DA slightly amplifies this tendency, as reflected by a higher FBIAS. This also impacts the POD index, with DA showing a higher value than OL—indicating that a greater fraction of observed events is correctly predicted. Additionally, DA presents a slightly lower FAR, indicating fewer false alarms.
Finally, the bottom panels of
Figure 7 display the main MODE-derived attributes, comparing WRF OL (blue bars) and WRF DA (green bars). These confirm the superior spatial consistency of the DA run.
More specifically, the attribute “angle diff”, which measures the difference in orientation between forecasted and observed objects, is smaller in the DA run, indicating better alignment. Moreover, the “area ratio”, which quantifies overprediction or underprediction of spatial extent, the “intersection area”, which captures the overlap between forecast and observation, and the “centroid dist”, which measures spatial displacement, all confirm the superior performance of DA over OL.
Moving on to the second case study in France (20240619), it is evident that the afternoon thunderstorms were slightly more intense and widespread in the areas surrounding the site of interest.
Both model simulations, with and without data assimilation, successfully captured the spatial distribution and precipitation peaks. As a result, precipitation validation was conducted for two thresholds: 10 mm and 15 mm/12 h.
Figure 8 (top) displays the objects identified by MODE, both observed and predicted, offering a graphical representation of the MODE output. The left columns display the objects for WRF OL, while the right columns show WRF DA for the 10 mm and 15 mm/12 h thresholds. In the figure, blue contour lines represent observed objects, brown areas indicate forecasted objects, and the red dot marks the Beaune site.
A visual inspection reveals that both model runs accurately predicted the location of observed objects. However, DA demonstrates greater precision compared to OL, along with improved accuracy in terms of agreement between the forecast and actual observations.
Additionally, the figure presents statistical indices computed using MODE. These show that while both runs overestimate precipitation at both thresholds, data assimilation helps to slightly reduce this overestimation.
Since the PODY, FAR, and FBIAS indices should be interpreted together, it can be observed that in the OL run, where precipitation is more strongly overestimated, the probability of correctly predicting “yes” events is higher, resulting in a slightly greater PODY compared to DA. However, this also leads to a higher number of false alarms, as indicated by the greater FAR value for OL relative to DA.
Moreover, the CSI index, which measures the agreement between forecast and observed “yes” events, is very similar for both runs and thresholds, with a slight improvement in DA. Overall, DA improves spatial accuracy by refining feature localization and reducing overestimation compared to OL. While OL exhibits a higher Probability of Detection (PODY), it also has a higher False Alarm Ratio (FAR), meaning it tends to overpredict precipitation, leading to rainfall forecasts where none occurred. DA, although more conservative, reduces false alarms and produces a more realistic precipitation field, balancing detection capability with improved precision.
Finally, the bottom panels of
Figure 8 show the main MODE-derived spatial attributes, comparing WRF OL (blue bars) and WRF DA (green bars). It is observed that the intersection area, representing the overlapping region between observed and forecasted objects, is very similar in both OL and DA, with a slight deterioration for DA.
However, looking at the area ratio, defined as the area of the predicted object divided by the area of the observed object and serving as an indicator of overestimation or underestimation, a clear improvement in DA is evident for both thresholds.
Again, considering the attribute “angle diff”, which indicates the angular difference between the axes of the forecasted and observed objects, DA performs better at the 15 mm threshold, while OL shows a smaller angle difference at 10 mm. This suggests that at 10 mm, the OL forecasted object is more aligned in orientation with the observed object, whereas at 15 mm, DA achieves better angular alignment.
Finally, regarding centroid distance—which quantifies the spatial displacement between the centroids of forecasted and observed objects, results are comparable between OL and DA at the 10 mm threshold, while DA shows a significantly smaller displacement at 15 mm. This indicates that at higher intensities, DA better captures the correct location of precipitation features.
3.2. Irrigation Advice Results from Agro-Hydrological Modeling
The irrigation advisory service of MAGDA provides farmers with forecasts of daily Irrigation Water Requirements (IWR) in m3/ha/day for short-term (2-day ahead) and medium-term (5-day ahead) forecast windows. Although sub-daily resolution may be of high interest under certain circumstances, this temporal resolution for outcomes was not explicitly requested by local farmers during the earlier phases of the research in which they were surveyed regarding their specific needs. According to the MAGDA irrigation approach adopted, IWR estimates represent the supplementary water that farmers may apply as irrigation to meet the crop’s daily water needs, based on weather forcings, crop development stage, and current soil moisture levels in the root zone.
Table 4 and
Table 5 present irrigation recommendations—i.e., IWR values—issued at the SCDAB and Tetto-Bernardo pilot sites for the selected testing periods. The relative contribution of IWR to the crop reference evapotranspiration (IWR/
ratio) is also reported for comparison purposes.
In SCDAB, with a higher sample size of forecasts than in Tetto-Bernardo, estimates of IWR and IWR/ ratios derived from weather forecasts with data assimilation (DA) were consistently higher than those retrieved for the benchmark case (OL) for both forecast windows. However, in Tetto-Bernardo, no clear difference was observed during the single testing period evaluated.
The results show that, in the SCDAB testing site, short-term (2-day ahead) forecasts of IWR and IWR/ were consistently higher than in medium-term (5-day ahead) forecasts (0.32–0.23 vs. 0.29–0.19, for DA-OL configurations, respectively) in the SCDAB testing site. No differences were detected in the Tetto-Bernardo site.
Recommendations for irrigation derived using the long-term DA forecast were in the same order of magnitude as irrigation quotas actually applied at the farm level following the traditional ICITID method (
Table 6). Although not fully comparable due to the large conceptual differences between methods, in general, the ICITID method provided recommendations 35% lower than the MAGDA solution. These differences highlight the importance of accurately simulating the soil moisture dynamics and the climate forcings that control the water balance in the atmosphere-crop-soil continuum. The MAGDA solution may provide a clear advantage against current practices that rest on coarse approaches applied at county and regional scales.