Evaluation of a BCC-CPSv3-S2Sv2 Model for the Monthly Prediction of Summer Extreme Precipitation in the Yellow River Basin

Abstract

The performance of monthly prediction of extreme precipitation from the BCC-CPSv3-S2Sv2 model over the Yellow River Basin (YRB) using historical hindcast data from 2008 to 2022 was evaluated in this study, mainly from three aspects: overall performance in predicting daily precipitation rates, systematic biases, and monthly prediction of extreme precipitation metrics. The results showed that the BCC-CPSv3-S2Sv2 model demonstrates approximately 10-day predictive skill for summer daily precipitation over the YRB. Relatively higher skill regions concentrate in the central basin, while skill degradation proves more pronounced in downstream areas compared to the upper basin. After correcting model systematic biases, prediction skills for total precipitation-related metrics significantly surpass those of extreme precipitation indices, and metrics related to precipitation amounts demonstrate relatively higher skill compared to those associated with precipitation days. Total precipitation (TP) and rainy days (RD) exhibit comparable skills in June and July, with August showing weaker performance. Nevertheless, basin-wide predictions within 10-day lead times remain practically valuable for most regions. Prediction skills for extreme precipitation amounts and extreme precipitation days share similar spatiotemporal patterns, with high-skill regions shifting progressively south-to-north from June to August. Significant skills for June–July are constrained within 10-day leads, while August skills rarely exceed 1 week. Further analysis reveals that the predictive capability of the model predominantly originates from normal or below-normal precipitation years, whereas the accurate forecasting of extremely wet years remains a critical challenge, which highlights limitations in capturing mechanisms governing exceptional precipitation events.

1. Introduction

Extreme precipitation events pose severe threats to human safety, socioeconomic development, and natural ecosystems. Under global warming, the frequency and intensity of such events have increased worldwide [1,2,3]. The Yellow River Basin (YRB), located in northern China, traverses diverse geomorphological units including the Tibetan Plateau, Loess Plateau, and North China Plain (Figure 1). Characterized by high sediment load, the basin is highly vulnerable to flooding triggered by heavy rainfall [3,4,5]. Spanning vast longitudinal distances, the basin exhibits heterogeneous climate zones: the upper reaches feature a cold, arid plateau–mountain climate with sparse precipitation, the mid-reaches experience a semi-arid climate marked by concentrated summer storms that exacerbate soil erosion, and the lower reaches transition to a semi-humid climate with increased but seasonally uneven precipitation, notably spring droughts [6,7,8]. These regional climatic disparities, driven by distinct influencing factors, complicate climate prediction across the basin. Approximately 60% of the basin’s annual precipitation occurs during summer, often as extreme events [9,10]. For instance, the 2023 flood season witnessed 10 large-scale precipitation events, including the most intense storm from July 27–31 under the influence of Typhoon Doksuri, which triggered the largest flood of the year in the Jinghe River across Shaanxi, Henan, and Shandong provinces [11,12]. Enhancing the accuracy of regional precipitation forecasts, particularly for extreme events, is thus an urgent priority for disaster prevention and mitigation in the YRB [13,14].

Climate models, as critical tools for climate simulation and projection, have been widely applied to extreme precipitation modeling and forecasting [15,16,17,18]. Sub-seasonal forecasting is influenced not only by initial conditions but also by boundary conditions. According to Lorenz’s research, predictions within two weeks fall under initial value problems, while seasonal-scale climate predictions are boundary value problems, with sub-seasonal forecasting positioned between these two regimes [19,20]. On the one hand, initial errors grow rapidly over time, reaching maximum predictability limits beyond two weeks, after which increasing errors render forecasts unreliable. On the other hand, the atmosphere begins to respond to external forcings but is not yet fully governed by them. Consequently, neither weather forecasting nor climate prediction theories fully apply to sub-seasonal forecasting. Currently, sub-seasonal forecasting lacks a robust theoretical foundation, representing a significant challenge in climate research and earning the moniker “predictability desert” [21].

In 2011, the World Weather Research Programme (WWRP) and World Climate Research Programme (WCRP) under the World Meteorological Organization jointly launched the Subseasonal-to-Seasonal (S2S) Prediction Project [22]. This initiative aims to explore the potential predictability of high-impact weather events at S2S timescales, enhance forecasting capabilities of numerical models for 2 weeks to 2 months ahead, and prioritize extreme event prediction and model validation [23,24]. China’s independently developed BCC-CPSv3 model from the National Climate Center (NCC) has joined the S2S project, providing real-time global forecast data [25]. The China Meteorological Administration has explicitly mandated the establishment of an objective S2S prediction system for the YRB to operationalize grid-based S2S forecasting [26]. These efforts underscore the broad applicability and urgent societal demand for S2S prediction in disaster preparedness and climate resilience [27,28].

In recent years, meteorologists have conducted predictive skill analyses on S2S model outputs for various phenomena, including heatwaves [29], precipitation [30,31,32], monsoons [33,34,35], Madden–Julian Oscillation (MJO) teleconnections [36,37], soil moisture [38], and typhoons [39,40]. As a frontier in forecasting research, S2S precipitation predictions extend warning lead times for flood risks [41], enabling proactive disaster mitigation by local governments. Studies demonstrate that S2S products exhibit potential for predicting the onset, evolution, and decay of large-scale extreme events weeks in advance [42,43,44]. For instance, S2S models (ECMWF, CNRM, JMA, BoM) successfully forecasted the most severe week of the 2010 Russian heatwave three weeks prior, capturing extreme 2 m temperature anomalies [24]. Similarly, S2S models leverage large-scale predictors like MJO phase to derive probabilistic tropical storm forecasts [45].

Model skill evaluation forms the foundation for understanding performance and effectively utilizing forecast information [46,47,48]. However, few studies have systematically assessed or operationally applied S2S models for precipitation predictions—particularly for extreme precipitation—in the YRB. Addressing this gap, this study evaluates S2S model skill in forecasting extreme precipitation events over the basin, aligning with operational needs for disaster risk reduction. The analysis aims to establish a scientific basis for improving prediction accuracy and extending forecast lead times through the enhanced interpretation and application of model products. The rest of this paper is organized as follows: The model and observational data and methodology are described in Section 2. Section 3 presents the overall performance in predicting daily precipitation rate, representations of monthly prediction for climatological features, and monthly prediction of extreme precipitation in the YRB. Section 4 outlines the Discussion and Conclusions of the study.

2. Model, Data and Methodology

The BCC-CPSv3-S2Sv2 model constitutes the operational S2S prediction subsystem product within the integrated sub-seasonal–seasonal–interannual climate prediction system developed by the China Meteorological Administration (CMA) [25]. The atmospheric component employs the Beijing Climate Center Atmospheric General Circulation Model version 3—High Resolution (BCC-AGCM3-HR) with a horizontal resolution of T266 (~45 km), 56 vertical levels, and a model top at 0.1 hPa. The oceanic component utilizes the Modular Ocean Model version 5 (MOM5), configured at (1/4)° horizontal resolution with 50 vertical levels. Land surface processes are simulated by BCC-AVIM2, while sea ice dynamics are represented by the Geophysical Fluid Dynamics Laboratory Sea Ice Simulator version 5 (GFDL SIS5). The ensemble prediction strategy remains Lagged Average Forecasting (LAF). This model began quasi-operational operation at CMA in November 2019, providing forecasts for the next 60 days every Wednesday and Saturday. So far, the model has produced a complete daily operational hindcast dataset for the study of sub-seasonal prediction. Each hindcast consists of 4 ensemble members, which are initialized at 00 UTC of the first forecast day and 18, 12 and 06 UTC of the previous day, respectively. This study directly applies equal-weight averaging to these four members. The prediction capability covers lead times of 0–60 days, thereby enabling monthly-scale prediction within the sub-seasonal framework.

The daily surface precipitation observations utilized are derived from the CN05.1 gridded precipitation dataset, constructed using data from over 2400 national-level meteorological stations across China, with a spatial resolution of 0.25° × 0.25° [49]. To address discrepancies between the selected prediction products and observational data, the climatological mean values of both observations and model outputs are standardized to the 2008–2022 period, with the summer season defined as June–July–August (JJA). Observations and model data are regridded to a uniform 0.25° × 0.25° resolution using the bilinear interpolation method. Additionally, the ensemble mean forecast for evaluation is derived by applying equal-weight averaging to the four ensemble members of the BCC-CPSv3-S2Sv2 model products.

Four precipitation indices are utilized to represent precipitation and extreme precipitation in the YRB, as listed in

Table 1. Definitions of precipitation index in this study.

Index Name	Abbreviation	Definition	Unit
Total precipitation	TP	Accumulation of precipitation	mm
Rainy days	RD	Days with precipitation ≥ 1 mm	d
Extreme precipitation	EP	Precipitation above the 90th percentile	mm
Extreme rainy days	ERD	Days with precipitation ≥ the 90th percentile	d

. The temporal correlation coefficient (TCC) is used to express the prediction performance. Statistical significance is evaluated using the two-tailed Student’s t-test.

3. Results

3.1. Overall Performance in Predicting Daily Precipitation Rate in JJA

Firstly, we evaluated the performance of the BCC-CPSv3-S2Sv2 model in predicting daily precipitation rate, as illustrated in Figure 2. It is clear that predictive skill progressively diminishes with increasing forecast lead time. Statistically significant TCC skills persist across the entire YRB within 9-day leads, indicating that the direct predictive capacity of the model for summer daily precipitation is limited within 10 days. Relatively higher skill regions concentrate in the central basin, while skill degradation proves more pronounced in downstream areas compared to the upper basin. Useful predictive skills nearly vanish basin-wide at 15-day leads. The precipitation prediction performance of the BCC-CPSv3-S2Sv2 model remains comparable to most worldwide operational systems, which is consistent with expectations and aligns with the baseline performance levels of domestic and international models [24,32,34].

3.2. Representations of Monthly Prediction for Climatological Features

Before conducting the monthly prediction assessment, the climatological features for the three months of summer in model predictions were examined. Figure 3 illustrates the monthly climatological distributions of observed precipitation and model-predicted precipitation across varying forecast lead times in the YRB during June, July, and August. Observations reveal that precipitation in the middle–lower reaches of the YRB predominantly occurs in July and August, with the daily precipitation rate exceeding 4 mm/day and reaching over 7 mm/day in localized areas. The upper reaches exhibit average precipitation of 2–3 mm/day or higher. To evaluate the impact of forecast lead time on monthly mean precipitation prediction, predictions from 0 to 15 days were compared.

Overall, the model systematically overestimates precipitation intensity in the upper basin across all months while underestimating it in most middle reaches and downstream areas. The magnitude of prediction biases shows no clear association with forecast lead times. Specifically, for June, the 3-day lead forecast predominantly exhibits overestimated precipitation in the upper basin and underestimated values in the middle–lower reaches. As lead times extend, the entire basin gradually transitions to positive precipitation biases. In contrast, during July and August, the model consistently maintains positive biases in the upper basin and negative biases in the middle–lower reaches throughout all lead times.

The standard deviation results are illustrated in Figure 4. The monthly distributions of standard deviations resemble the climatological patterns, with the strongest variability occurring in the central middle reaches and downstream areas of the YRB, while weaker variability characterizes the upper basin. July exhibits the highest variability across the basin, followed by August, with June showing relatively minimal variability—a pattern consistent with the principal rainy season occurring in July and August across northern China.

Overall, the model initially overestimates precipitation variability across most regions of the basin in all months. However, precipitation variability in model outputs markedly decreases with increasing forecast lead times. Beyond 9-day leads, simulated variability becomes systematically lower than observations throughout the basin, and this is particularly pronounced in July. The upper basin constitutes an exception, where the model persistently overestimates the inherently low observed variability. This temporal attenuation of simulated variability likely reflects the inherent characteristics of models. Following initialization, as integration time progresses, the model’s climate state gradually drifts from the observation-based initial conditions toward its intrinsic climatological state—a phenomenon termed climatological drift. This adjustment process amplifies systematic biases and introduces spurious long-term trends that compromise prediction reliability. The weakening variability may exert more pronounced impacts on extended-range prediction of extreme precipitation compared to mean precipitation, resulting in the absence of high-intensity precipitation events in model outputs beyond approximately 10-day leads. In addition, this may also be due to the large uncertainty in heavy precipitation across the four ensemble members of BCC-CPSv3-S2Sv2 at 10-day or longer lead times, which is worth further study.

3.3. Monthly Prediction of Extreme Precipitation in the YRB

As revealed by the analysis in Section 3.2, systematic biases exist in the mean state and variability of the model. Consequently, for subsequent precipitation and extreme precipitation screening processes, we replaced the climatological mean state and standard deviation in model direct prediction with observational results to accurately adjust these systematic biases. The correction formula for a given grid point i in predicting the calendar day t at n days lead can be expressed as follows; ${Prec}_{i,lead=n,t}^{calibration}={\displaystyle \frac{{Prec}_{i,lead=n,t}^{BCC-CPSv3-S2Sv2}-{\overline{Prec}}_{i,lead=n,t}^{BCC-CPSv3-S2Sv2}}{{SD}_{i,lead=n,t}^{BCC-CPSv3-S2Sv2}}}{SD}_{i,t}^{CN05.1}+{\overline{Prec}}_{i,t}^{CN05.1}$, where $\overline{Prec}$ and $SD$ indicate the climatological mean precipitation and its standard deviation corresponding to the calendar day t, respectively. Based on this correction, Figure 5 presents the spatial distributions of TCC skills between monthly accumulation of precipitation from the BCC-CPSv3-S2Sv2 model predictions and observations over the YRB for June, July, and August. Here, the lead days (e.g., “1–3 days”, “4–6 days”, etc.) were defined as follows: For a target month (e.g., June 2010), lead of 1–3 days was calculated as accumulations of 1–30 June from forecasts initialized on 28–31 May. Lead 4–6 is calculated as accumulations of 1–30 June from forecasts initialized on 25–27 May. The rest can be performed in the same manner. Results show that at lead 1–3 days, most regions of the basin exhibit positive TCC. In June, TCC exceeds 0.4 in the upper basin, central–southern middle reaches, and downstream areas, with values surpassing 0.6 in partial middle reaches, while northwestern regions show skills below 0.2 or even negative values. When lead times extend to 4–9 days, relatively high skill persists in the upper basin and central middle reaches, but diminishes notably in other regions, particularly downstream.

For July, prediction skill in the upper basin decreases compared to June. At lead 1–3 days, high-skill areas (greater than 0.5) are concentrated in central and southern portions of the basin, whereas northern regions demonstrate negative skills. During 4–6 and 7–9 day leads, moderate skill persists in southern central areas, but most northern regions experience further skill degradation, which is particularly pronounced at 4–6 day leads. In August, significant high-skill regions (greater than 0.5) are primarily located in central basin areas at 1–3 day leads. Although most regions maintain positive skills at 4–6 day leads, these correlations generally fail to pass statistical significance tests. Prediction skills substantially deteriorate at 7–9 day leads, falling below those achieved during corresponding periods in June and July. Beyond 10-day leads, predictive skills for each month deteriorate markedly. Note that the negative TCC for July TP in the northern YRB may stem from the northward advance of the East Asian summer monsoon. This statistically insignificant negative TCC, combined with the weak positive TCC (also not significant) in June for northern YRB, collectively reflects the low predictability of interannual precipitation variability when monsoon circulation has not yet fully established dominance over the northern YRB region. In contrast, during August, after stable monsoon circulation governs the entire YRB, the predictive skill for the 1–3-day and 4–7-day lead can exceed that in June and July.

In addition to predictions of monthly accumulated precipitation, we also evaluated the prediction skill of the model for monthly accumulated precipitation days, as presented in Figure 6. Overall, the prediction skill for accumulated precipitation days is substantially lower than that for precipitation amounts. This is plausibly because TP, as a continuous variable, allows compensation for errors in weak precipitation events by heavy rainfall events. In contrast, RD is a discrete metric defined as the count of days with daily precipitation ≥ 1 mm, which exhibits high sensitivity to the detection accuracy of weak precipitation events. In any case, June exhibits the highest skill, followed by July, while August demonstrates the weakest performance, with forecast skills deteriorating markedly as lead times extend. For the accumulated precipitation day predictions in June, statistically significant skills are confined to central basin regions and persist only within approximately one-week leads. Predictions in July show limited skill concentrated in the southern middle reaches, which also maintains significance for about one week. The accumulated precipitation days in August prove particularly challenging to predict, displaying significant positive correlations only in the central and western parts of the basin with a 1–3 day lead.

The spatial pattern of skills may arise from enhanced precipitation variability during July–August following the onset of the rainy season in North China, which amplifies fluctuations in precipitation days. There are currently limitations in understanding mid-latitude precipitation day mechanisms, combined with difficulties in S2S models in capturing multi-scale temporal influences on precipitation days.

The aforementioned analysis reveals that the BCC-CPSv3-S2Sv2 model demonstrates suboptimal performance in monthly-scale predictions of both precipitation amounts and precipitation days. To further investigate extreme precipitation prediction capabilities, we define extreme precipitation amounts as precipitation exceeding the 90th percentile threshold and extreme precipitation days as days with precipitation ≥ the 90th percentile (Table 1), subsequently evaluating their monthly prediction skills, as illustrated in Figure 7 and Figure 8.

Overall, the model exhibits weaker predictive skills for extreme precipitation compared to total precipitation. Regarding extreme precipitation amounts (Figure 7), statistically significant skills in June and July are primarily confined to the southern–middle reaches of the basin, with July showing marginally higher skill than June, though predictive capability diminishes beyond 10-day leads. August demonstrates limited skill concentrated in the northern–middle reaches, persisting only within 1-week leads. Prediction skills for extreme precipitation days (Figure 8) mirror those of extreme precipitation amounts, with high-skill regions shifting progressively south-to-north from June to August. Significant skills for June–July are constrained within 10-day leads, while August skills rarely exceed 1 week. This spatial–temporal skill pattern reflects the northward progression of the rain belt over northern China during the summer. Notably, accurate prediction of extreme precipitation becomes increasingly challenging during late summer (August) when precipitation frequency peaks, likely due to enhanced atmospheric instability and complex multi-scale interactions characteristic of the mature monsoon phase.

In summary, we calculated the whole YRB-averaged interannual evolution of monthly total and extreme precipitation from observations and forecasts at different lead times, with the results presented in Figure 9. Consistent with previous analyses, statistically significant predictive skills for all metrics across months are essentially confined within a 10-day lead. Metrics related to precipitation amounts demonstrate relatively higher skill compared to those associated with precipitation days. However, from a basin-averaged perspective, extreme precipitation indices exhibit comparable prediction skills to total precipitation metrics. Notably, the interannual evolution reveals that predictive skill primarily originates from normal or below-normal precipitation years. For instance, the model successfully captured below-normal precipitation months such as June 2009, July 2019, and August 2021. Conversely, it completely failed to predict extreme wet months like July 2013, July 2018, and August 2020, which featured anomalously intense precipitation events.

These findings highlight that monthly extreme precipitation prediction using the BCC-CPSv3-S2Sv2 model remains a critical challenge, particularly for extremely wet years. Addressing this limitation requires enhanced understanding of extreme precipitation processes at monthly scales. Leveraging the extended forecast window of S2S models and developing dynamical–statistical downscaling approaches that integrate multi-scale physical mechanisms governing extreme precipitation could represent pivotal pathways for advancing monthly extreme precipitation prediction, but this still needs substantial research.

4. Summary and Discussion

This study systematically evaluates the monthly extreme precipitation prediction performance of the BCC-CPSv3-S2Sv2 model over the YRB using historical hindcast data from 2008 to 2022 and gridded observational data from the CN05.1 dataset. The evaluation encompasses three key aspects: overall performance in predicting daily precipitation rates, systematic biases, and monthly prediction of extreme precipitation metrics. The principal findings are summarized as follows.

The BCC-CPSv3-S2Sv2 model exhibits approximately 10-day predictive skill for summer daily precipitation over the YRB. Relatively higher skill regions concentrate in the central basin, while skill degradation proves more pronounced in downstream areas compared to the upper basin. Climatologically, the model systematically underestimates summer monthly mean precipitation rates and precipitation days across the basin, with this bias being particularly pronounced in the middle reaches. Results from different forecast initialization times consistently demonstrate weakened precipitation intensity across most middle basin regions during all three summer months, while the upper and downstream regions exhibit relatively stronger precipitation compared to observations. Biases in standard deviations exhibit patterns broadly similar to those of the climatological mean.

Following the correction of systematic model biases, the monthly predictions of TP, RD, EP, and ERD over the YRB were evaluated. Results indicate that prediction skills for total precipitation-related metrics significantly surpass those of extreme precipitation indices, and metrics related to precipitation amounts demonstrate relatively higher skill compared to those associated with precipitation days. TP and RD exhibit comparable skills in June and July, with August showing a weaker performance. Nevertheless, basin-wide predictions within 10-day leads remain practically valuable for all three months.

Prediction skills for extreme precipitation amounts and days demonstrate similar spatial–temporal patterns, with high-skill regions shifting progressively south-to-north from June to August. Significant skills for June–July are constrained within 10-day leads, while August skills rarely exceed 1 week. Basin-averaged results reveal that the predictive capability of the model primarily originates from normal or below-normal years, whereas the accurate prediction of extremely wet years (e.g., anomalously heavy precipitation episodes) still remains a critical challenge—highlighting fundamental limitations in capturing nonlinear physical mechanisms governing exceptional precipitation events.

Under global warming, extreme weather and climate events in the YRB have exhibited intensified characteristics of increased intensity, frequency, concurrency, and recurrence. The Chinese government explicitly mandated “one-month-ahead prediction of major weather processes” in the development objectives of meteorological operation, highlighting the urgent societal demand for monthly extreme precipitation prediction across multiple sectors. While this study primarily employs TCC to evaluate predictive skill, it may have inherent limitations in characterizing amplitude accuracy for operational forecasting. Some probabilistic metrics might be an appropriate aspect for predicting extreme precipitation, especially in monthly predictions. For instance, the hit rate of extreme events and some binary metrics better serve event-based decision-making. Future work should integrate probabilistic skill scores to bridge this gap. In addition, the current understanding of sub-seasonal evolution mechanisms and formation dynamics of extreme precipitation processes still remains insufficient. Particularly regarding multiscale influences, existing research has demonstrated that the sub-seasonal initiation and evolution characteristics of extreme precipitation processes are not only modulated by intraseasonal oscillations, but are also regulated by seasonal–interannual scale variability and even multidecadal oscillations [50,51,52]. Overall, it is difficult to claim that this model satisfies or does not satisfy the requirements of operational predictions for extreme precipitation, necessitating focused research on validation indices used in practical requirements, sub-seasonal prediction mechanisms, and technical methodologies specifically targeting extreme precipitation events in the future.