Precipitation Assessment and Attribution Based on LBGM Ensemble Forecast for the Extreme Rainstorm on 20 July 2021 in Zhengzhou

Highlights

What are the main findings?

• Superiority of CRA Method in Spatial Verification
• The Contiguous Rain Area (CRA) method outperforms traditional verification methods (e.g., TS score) in capturing spatial features such as shape, orientation, and movement of precipitation objects, providing a more comprehensive assessment of forecast errors.
• Superior Performance of LBGM-EPS in Extreme Precipitation Forecast
• LBGM-EPS demonstrates better capability in forecasting the location and intensity of heavy rainfall centers compared to control forecasts, showing closer alignment with actual observations. It also more realistically simulates the structure and evolution of Mesoscale Convective Vortex (MCV), which are critical to extreme rainfall events.

What are the implications of the main findings?

• Consideration of Spatial Precision in Convection-Resolving Forecast Evaluation
• The study suggests that when evaluating convection-resolving forecasts, the impact of high spatial precision on assessment outcomes should be properly considered. Spatial verification methods like CRA are more suitable for such high-resolution systems as they mitigate the “double penalty” effect inherent in traditional point-to-point verification.
• Crucial Role of MCV Simulation in Extreme Rainfall Forecasting
• Accurate simulation of mesoscale convective vortex (MCV) is crucial for improving extreme rainfall forecasts, as better-performing ensemble members exhibit more realistic representations of dynamic and thermodynamic processes associated with MCV.

Abstract

In the context of global warming, the prediction of extreme precipitation events faces great challenges, especially the ensemble forecast of convective-scale heavy precipitation. Taking the heavy rainstorm in Zhengzhou on 20 July 2021 as an example, this paper aims to explore the performance of the convective-scale ensemble forecasting system based on the local breeding model cultivation method (LBGM) in extreme precipitation forecasting, and reveal the key physical mechanisms affecting the quality of forecasting. The traditional scoring (TS, Bias), neighborhood FSS and Contiguous Rain Area (CRA) methods were used to systematically evaluate the precipitation forecast, and the superior and inferior forecast members were diagnosed and analyzed by combining physical quantities such as isentropy vortex, relative vorticity, and water vapor flux divergence. The results show that: (1) the LBGM-EPS system can better capture the spatial distribution and intensity of heavy precipitation, which is better than the single deterministic forecast; (2) The CRA method is better than the traditional score in describing the spatial structure and intensity of precipitation, and can effectively identify the good and bad members of the forecast. (3) The reason why the dominant forecast members perform better is that the simulation of the dynamic-thermal structure of the mesoscale convective vortex is more reasonable, especially the coupling mechanism of the downward transmission of the high-level vortex and the convergence of water vapor at the lower level is better. The preliminary application of convective-scale ensemble forecasting based on the LBGM in this study has reference value for improving the prediction ability of extreme precipitation.

1. Introduction

Under the background of global warming, extreme weather events occur frequently. Heavy precipitation, as a typical extreme weather event, is characterized by strong locality and abruptness, and often leads to significant casualties and property damage. Heavy precipitation is a critical phenomenon within severe convective weather. Convective-scale systems in such weather are characterized by short life cycles, small horizontal scales, rapid development, and high destructiveness, making their prediction a persistent challenge and focus in operational forecasting. Although high-resolution convective-scale models have shown improved capability in forecasting such systems, the inherent development of severe convective weather is influenced not only by the baroclinic instability of large-scale weather systems but also by localized moist convective instability, and exhibits highly nonlinear characteristics. Consequently, single deterministic forecasts from high-resolution numerical models often yield unsatisfactory results [1]. Therefore, conducting ensemble prediction research for severe convective weather processes based on convective-scale models is of great significance.

The evaluation and inspection of weather forecast results is an indispensable and important part of the development process of meteorological forecasting. Through evaluation and inspection, the bias characteristics of different forecasts can be understood, promoting improvements in numerical models and subjective-objective forecasting [2]. Currently, most methods applied to model evaluation and operational forecast assessment are point-to-point verification. Point-to-point verification uses meteorological station observation data as the truth value to score and inspect model forecast data. Although such verification methods are quantitative and objective, as they separate the spatial information of physical quantities, they have certain shortcomings in reflecting the scale changes and spatial structure of forecasts, and cannot accurately reflect the forecasting performance of high-resolution models for small-scale events [3]. In 1995, Hoffman et al. addressed this shortcoming by spatially decomposing forecast errors [4], discussing spatial and intensity errors in terms of displacement, amplitude, and residual components. Based on this, a series of spatial verification and evaluation models were developed, including the scale separation method, neighborhood method, Contiguous Rain Area (CRA) method, and Method of Object-Based Diagnostic Evaluation (MODE). The CRA method is an object-based spatial verification method proposed by Ebert et al. [5]. By setting precipitation thresholds and identifying CRA, it enables quantitative analysis of displacement errors, intensity errors, and pattern errors in model precipitation forecasts, and has been widely used by scholars both domestically and internationally. Li Xiaolan et al. [6] selected heavy precipitation cases from ECMWF ensemble forecasts in eastern Southwest China from April to June 2018, used CRA spatial verification technology to screen continuous rain area cases, and conducted statistical verification. The results showed that the regional biases of cases in different periods had certain similarities, which matched the variation patterns of atmospheric circulation. Differences in the model’s forecasting of influencing systems across periods had a significant impact on precipitation results, and the deterministic forecasts of the EC model were unsatisfactory for warm-sector heavy rainfall in South China. Stefano Mariani et al. [7] conducted quantitative precipitation forecast verification for two mesoscale convective processes in the Alps and used the CRA method to inspect the results. The results showed that the CRA method has good applicability for short-duration heavy precipitation and substructures of precipitation systems. Lu Chenli et al. [8] used the CRA method to conduct forecast verification for heavy precipitation processes during the 2020 Meiyu season in the Hangjiahu region. By analyzing errors in forecasts from models such as EC and NCEP-GFS, they found that the CRA method could more effectively separate and quantify displacement, pattern, and intensity errors. Compared to traditional verification methods, CRA technology provides a more comprehensive analysis of forecast errors, helping to deeply understand the forecast performance of different rainfall types. However, there is not much research using the CRA method through ensemble forecasting to evaluate model performance.

From 08:00 on July 17 to 08:00 on 22 July 2021, Henan Province experienced its most severe rainstorm event since 1975, known as the “7.20” extreme rainstorm. This event was characterized by a wide coverage area, long duration, exceptional intensity, and complex contributing factors. Influenced not only by typical weather systems but also by complex topographic interference, the event revealed varying degrees of forecasting deviation across major global operational weather prediction systems, which quickly drew the attention of scholars both domestically and internationally and led to a substantial body of research. Zhu Kefeng et al. [9] designed and ran two convective-scale ensemble forecasting models, revealing that most ensemble members exhibited significant predictive biases regarding the high-intensity precipitation around Zhengzhou. All forecasts failed to adequately capture the extreme hourly precipitation intensity, a shortcoming primarily attributed to the spatial uncertainty of mesoscale weather systems. Fu Shenming et al. [10] investigated the evolution of a long-lived mesoscale convective vortex during the exceptionally strong hourly rainfall in Zhengzhou, demonstrating that the coupling between the mesoscale convective vortex and the mesoscale convective system promoted the occurrence and development of the extreme rainstorm. Liu Kan et al. [11] employed the LBGM to construct an ensemble prediction system to simulate the Henan “21•7” extreme rainstorm event, conducting a comparative evaluation using traditional assessment methods like the TS and the SAL method. However, the SAL method cannot intuitively quantify deviations in precipitation pattern and location. Given the characteristic of significant displacement errors in the forecasted area of this extreme precipitation event, the SAL method was unsuitable. Furthermore, their study did not extend to a mechanistic analysis of the correspondence between the evolution of heavy precipitation in ensemble members and the evolution of convective systems, representing a certain limitation. Li et al. revealed the key influences of high-level groove ridge configuration, low-level water vapor transport and potential vortex downward transmission process on the eastward movement of the vortex by comparing the “moving” and “stagnant” members. Ensemble sensitivity analysis was used to identify key regions sensitive to forecasting techniques [12]. However, similar studies still need to be deepened in terms of evolution and simulation differences in extreme heavy precipitation events, especially the key impact system of the MCV.

Based on this, this paper takes the heavy precipitation process in Zhengzhou, Henan Province from 17 to 22 July 2021 as an example, and systematically compares the performance of traditional point-to-point scoring (TS, Bias), neighborhood FSS and CRA method in the convective scale ensemble forecast evaluation, and clarifies the advantages and disadvantages of different methods in revealing precipitation spatial errors. The convective scale ensemble prediction system (LBGM-EPS) was constructed by using the LBGM, and the ensemble members with significantly better and inferior forecast performance were screened out through the above multi-dimensional evaluation. Combined with the diagnosis of multiple physical quantities such as isentropy vorticity, relative vorticity, water vapor flux divergence, false equivalent potential temperature, as well as vorticity equation income and expenditure analysis, the key differences between the dominant and inferior members in the process of simulating the dynamic, thermal structure and evolution of mesoscale convective vortices are systematically revealed, and the internal mechanism leading to the differentiation of ensemble member forecasting techniques is clarified, which lays the foundation for the next step of comparing with other perturbation methods and comprehensively revealing the influence mechanism of ensemble forecasting methods on heavy precipitation simulation.

2. Data, Experimental Design, and Methods

2.1. Experimental Design and Data Analysis

This experiment is built upon version 4.2 of the WRF model [13], utilizing Global Forecast System (GFS) data [14] as the driving field. The simulation starts at 12:00 UTC on 19 July 2021, with a 6-h breeding cycle, and conducts a 48-h forecast. As shown in Figure 1, the model employs a two-layer, two-way nested grid configuration: the outer grid has a resolution of 9 km, while the inner grid is refined to 3 km. The center of the outer grid is positioned at 28° N, 115° E, with the outer and inner grids consisting of 600 × 700 and 471 × 471 grid points, respectively. In the vertical direction, the model is configured with 39 layers to capture the atmospheric vertical structure. All ensemble forecast members follow a unified set of physical process parameterization schemes. The physical schemes employed include: the Lin microphysics scheme [15], the RRTM longwave radiation scheme [16], the Dudhia shortwave radiation scheme [17], the Monin-Obukhov surface layer scheme [18], the Noah land surface model [19], the YSU boundary layer scheme [20], and the Grell-Devenyi cumulus parameterization scheme [21]. Notably, the cumulus parameterization is turned off for the inner nest. This configuration aims to ensure that, within the control of experimental variables, only the initial perturbations influence the differences in forecast outcomes. Consequently, all perturbation members strictly maintain identical physical parameterization settings as the control experiment. No observational data assimilation is performed, and no additional perturbations are introduced at the lateral boundaries or within the model interior.

The Final Reanalysis Data (FNL) [22] with a horizontal resolution of 1° × 1° at 00:00 UTC on 20 July 2021, is selected. This dataset includes elements such as geopotential height, atmospheric temperature, and wind fields and is used to analyze the atmospheric environmental conditions. Meanwhile, observational data is obtained from the Global Precipitation Measurement (GPM) half-hourly global precipitation product, which incorporates the Integrated Multi-satellite Retrievals for GPM (IMERG) algorithm. The GPM IMERG products consist of three types: Early Run, Late Run, and Final Run, representing releases approximately 6 h, 18 h, and about 2.5 months after data acquisition, respectively [23]. In this study, the Late Run product with a spatial resolution of 0.1° × 0.1° is selected and used as observational data to evaluate the forecast results.

2.2. LBGM

The breeding growth mode (BGM) method was proposed by Toth and Kalnay [24], in which the fastest growth modulus is captured through a continuous cultivation cycle of the model to generate the initial perturbation of ensemble prediction [25].

In the actual calculation, the BGM method first calculates the root mean square error (RMSE) at the initial moment, adds the random numbers with values between 0–1 and spatially evenly distributed to the initial RMSE to obtain the initial perturbation, and integrates the initial field and the analysis field of superimposed disturbances synchronously, and obtains the control forecast and perturbation prediction after a cultivation cycle. The disturbance is scaled and adjusted to generate an analysis perturbation, which is superimposed on the analysis field at the current moment, and cultivated repeatedly, and finally obtains the fastest growth model required.

The LBGM (local growth model cultivation method) was improved and proposed by Chen Chaohui et al. [26] on the basis of the growth model cultivation method. The difference between the LBGM and the BGM method is that the obtained perturbation is a function of $\left(i,j,k\right)$, not just a function of k, and the BGM method only considers the inhomogeneity of physical quantities in the vertical direction, and for the convective resolvable scale ensemble prediction with strong locality, the distribution of physical quantities in the horizontal direction should also be fully considered [3].

LBGM adopts the local adjustment method to replace the global adjustment of the traditional BGM method, and sets the cultivation period to 6 h. $x^a\left(i,j,k\right)$ are brought into the model for integration, and the forecast field at the corresponding time is obtained after a cultivation cycle, $x^a(i,j,k)$ calculates the root mean square error at the initial moment, $e_0\left(k\right)$, the formula is as follows:

$$e_0(k)=\sqrt{{\displaystyle \frac{1}{m\times n}}{\sum }_{i=1}^m{\sum }_{j=1}^n[x^f(i,j,k)-x^a(i,j,k){]}^2}$$

(1)

where m and n represent the number of grids in the latitudinal and longitudinal directions of the forecast area, respectively. During the incubation phase, the LBGM scales the perturbations as follows:

$$r^a(i,j,k)={\displaystyle \frac{e_0\left(k\right)}{e_n(i,j,k)}}{r}^f(i,j,k)$$

(2)

$$e_n(i,j,k)={\displaystyle \frac{\sqrt{{\sum }_{i-r}^{i+r}{\sum }_{j-r}^{j+r}{[x^f\left(i,j,k\right)-x^a(i,j,k\left)\right]}^2}}{(2r+1{)}^2}}$$

(3)

where $e_0\left(k\right)$ and $e_n(i,j,k)$ are the predicted root mean square error of the $t_0$ moment and the $t_n$ moment, respectively, and the ratio of the two is the scaling factor, $r^a(i,j,k)$ and $r^f(i,j,k)$ are the analytical perturbation and the predicted disturbance of the grid $(i,j)$ on the kth layer of the model, respectively, and r is the local radius. For example, Deng et al. introduced a three-dimensional rescaling mask in storm-scale ensemble forecasting, which made the initial disturbance closer to the analytical uncertainty, significantly improved the ensemble dispersion and probabilistic prediction performance, and pointed out that convective instability is the main mechanism of disturbance growth [27].

2.3. Evaluation Methods for Precipitation

Due to the quantitative requirements for evaluating model precipitation forecasts in operational practice, point-to-point statistical assessment methods are typically employed. The basic requirement is to compare observed and forecasted values at corresponding grid points between the observation grid and the forecast grid. Commonly used statistical assessment methods include the Threat Score (TS), Equitable Threat Score (ETS), Bias Score, False Alarm Rate (FAR), and Missing Alarm Rate (MAR). Although these traditional assessment methods can simply and intuitively reflect forecast errors, as model precision continues to improve, they fail to objectively capture errors in the positional structure and coverage of precipitation forecasts. There can even be situations where higher-resolution models, capable of depicting convective-scale features more finely, result in lower scores [28]. Therefore, in response to the high spatiotemporal precision of high-resolution models, new spatial verification and assessment methods have been developed.

At present, the precipitation objects-based test method is a spatial evaluation and verification method that has a good evaluation effect and can effectively reflect the systematic error, which mainly focuses on the spatial deviation and intensity deviation, and analyzes the source of the forecast error from the location, shape, size, and intensity difference in the main system in the forecast field and the live field [29]. The representative method is based on the Contiguous Rain Area (CRA) method, which first defines the continuous rain area with a certain precipitation threshold, and then identifies the precipitation characteristics in the observed and simulated rain area, including the location of the precipitation centroid, the precipitation coverage area exceeding a specific threshold, the average and peak precipitation, and the cumulative precipitation of the whole rain area. Then, the simulation results in the continuous rain region are systematically corrected for displacement, allowing the forecast system to move up to 5° in all directions, aiming to determine the optimal displacement compensation between the forecast and the observed data by minimizing the adjustment, so as to quantify the spatial displacement bias of the model. This process can refine the inaccuracy of the model precipitation forecast into three core components: the deviation of geographical location (displacement error), the mismatch of precipitation magnitude (intensity error), and the difference in the shape and structure of the rain area (morphological error). See Equation (4), where the total error is the average of the sum of squares of the deviations between the original model forecast ($f_i$) and the actual ($o_i$), see Equation (5).

$${MSE}_{total}={MSE}_{dispalcement}+{MSE}_{volume}+{MSE}_{pattern}$$

(4)

$${MSE}_{total}={\displaystyle \frac{1}{N}}\sum _{i=1}^K{\left(f_i-o_i\right)}^2$$

(5)

The error after translation is the average of the sum of squares of the model forecast ($f_i^{\prime }$) and the real-time deviation, as shown in Equation (6).

$${MSE}_{shift}={\displaystyle \frac{1}{N}}\sum _{i=1}^K{\left(f_i^{\prime }-o_i\right)}^2$$

(6)

The displacement error is the mean total variance minus the translation by the mean total variance, see Equation (7).

$${MSE}_{dispalcement}={MSE}_{total}-{MSE}_{shift}$$

(7)

The intensity error is the average precipitation intensity of the translated model minus the square of the actual intensity, as shown in Equation (8).

$${MSE}_{volume}={\left(\overline{f^{\prime }}-\overline{O}\right)}^2$$

(8)

The morphological error is the error after translation minus the intensity error, see Equation (9).

$${\mathrm{M}\mathrm{S}\mathrm{E}}_{\mathrm{p}\mathrm{a}\mathrm{t}\mathrm{t}\mathrm{e}\mathrm{r}\mathrm{n}}={\mathrm{M}\mathrm{S}\mathrm{E}}_{\mathrm{s}\mathrm{h}\mathrm{i}\mathrm{f}\mathrm{t}}-{\mathrm{M}\mathrm{S}\mathrm{E}}_{\mathrm{v}\mathrm{o}\mathrm{l}\mathrm{u}\mathrm{m}\mathrm{e}}$$

(9)

2.4. Diagnostic Methods

In high-resolution numerical simulations, the ability to reproduce a Mesoscale Convective Vortex (MCV) directly determines the forecasting skill for extreme precipitation events. To systematically evaluate the simulation performance of the constructed ensemble forecast system for MCVs, physical quantities such as isentropic potential vorticity, relative vorticity, water vapor flux divergence, relative humidity, and pseudo-equivalent potential temperature are selected for analysis from the perspectives of dynamic forcing, rotational structure, moisture supply, and thermal environment. Simultaneously, to further reveal the intrinsic dynamic mechanisms governing MCV generation and development, the analysis of advection, stretching, and tilting terms in the vorticity equation is conducted to further elucidate the fundamental reasons for differences in simulations among different forecast members.

Isentropic Potential Vorticity (IPV) is a conservative quantity in an adiabatic and frictionless atmosphere, defined as the potential vorticity density along an isentropic surface ($\theta =const$). Its expression is:

$$IPV=-{\displaystyle \frac{\mathrm{g}}{\rho }}{\xi }_{\theta }·\nabla \theta ={\displaystyle \frac{\mathrm{g}}{\sigma }}({\xi }_{\theta }+f)$$

(10)

where ${\xi }_{\theta }=\nabla \theta \times V_{\theta }$ is the absolute vorticity vector on the isentropic surface, ${\xi }_{\theta }$ is its vertical component, $f$ is the Coriolis parameter, $\sigma =-{\displaystyle \frac{\partial p}{\partial \theta }}$ is the static stability parameter in isentropic coordinates, $\rho$ is the air density, and $g$ is the gravitational acceleration. The unit of IPV is typically PVU (1 PVU = 10⁻⁶ K·m²·kg⁻¹·s⁻¹).

Relative vorticity ($\xi$) characterizes local rotational intensity and is defined as the vertical curl of the horizontal wind field:

$$\xi ={\displaystyle \frac{\partial v}{\partial x}}-{\displaystyle \frac{\partial u}{\partial y}}$$

(11)

where $u$ and $v$ represent the zonal and meridional wind components, respectively.

Water vapor flux divergence ($\nabla ·\mathrm{Q}$) reflects local moisture budgets, where the water vapor flux vector is defined as:

$$Q={\displaystyle \frac{1}{g}}{\int }_{P_{top}}^{P_{surf}}qVdp$$

(12)

Therefore, its divergence is:

$$\nabla ·\mathrm{Q}={\displaystyle \frac{1}{g}}{\int }_{P_{top}}^{P_{surf}}\nabla ·\left(qV\right)dp$$

(13)

where $q$ is the specific humidity, and $V=(u,v)$ is the horizontal wind vector. Positive water vapor flux divergence indicates moisture convergence.

Pseudo-equivalent potential temperature and relative humidity together reflect the influence of temperature and humidity on convective systems. A high ${\theta }_{se}$ value (>340 K) marks the accumulation of convective instability energy, while relative humidity (RH) is used to identify key regions of moisture transport. Areas with RH greater than 80% are potential zones for deep convection, associated with low-level warm and moist inflow [30]. Their expressions are:

$${\theta }_{se}=\theta exp({\displaystyle \frac{L_{v}q}{c_{p}T}})$$

(14)

By analyzing the configuration relationships among physical quantities such as isentropic potential vorticity, relative vorticity, water vapor flux divergence, relative humidity, and pseudo-equivalent potential temperature, the dynamic and thermal structure of a mesoscale convective vortex can be revealed. To further interpret the generation mechanisms of mesoscale convective vortex in the model, it is necessary to diagnose the sources of local vorticity tendency (${\displaystyle \frac{\partial \xi }{\partial t}}$). Under short timescales ignoring friction and diabatic heating, the two-dimensional horizontal vorticity equation is:

$${\displaystyle \frac{\partial \xi }{\partial t}}=-V·\nabla \left(\xi +f\right)+\left(\xi +f\right)·\left(\nabla ·\mathrm{V}\right)+\mathsf{\beta }\mathrm{v}$$

(15)

$-V·\nabla \left(\xi +f\right)$ is the vorticity advection term, representing the advection of environmental vorticity to the local area by the wind field; $\left(\xi +f\right)·\left(\nabla ·\mathrm{V}\right)$ is the convergence-divergence term, representing the stretching or compression of vorticity due to convergence or divergence; $\mathsf{\beta }\mathrm{v}$ is the tilting term, reflecting the generation of vorticity due to the variation in the Coriolis parameter with latitude.

3. Analysis of Heavy Precipitation Processes

Overview of the Heavy Rainfall Event

During the period from 17 July to 22, 2021, Henan Province experienced an unprecedented extreme rainfall event. The hourly rainfall intensity set a new record for the Chinese mainland since 1951. Notably, between 08:00 and 09:00 (UTC) on 20 July, the Zhengzhou station recorded an extreme hourly precipitation of 201.9 mm. This heavy rainfall was not confined to Zhengzhou; daily precipitation records were also broken in 18 cities and counties, including Hebi, Anyang, and Xinxiang. Figure 2 shows the 24-h cumulative rainfall distribution in Henan Province from 19 July to 22, based on GPM satellite observation data. Specifically, from the night of 20 July to 21, Zhengzhou was the main precipitation concentration area. Subsequently, from the night of 21 July to 22, the core of heavy precipitation expanded to both Zhengzhou and Hebi in northern Henan, with both regions experiencing 24-h cumulative precipitation exceeding 250 mm. This extreme weather event caused flooding in many areas of Henan, severe damage to infrastructure, and resulted in significant economic losses and casualties, with substantial impacts.

The formation of extreme rainstorms requires the interaction and coordination of weather systems across different scales within a specific spatiotemporal context. This extreme precipitation event in Henan was also shaped by the interplay of multi-scale weather systems. From the 500 hPa geopotential height field at 00:00 on July 20 shown in Figure 3a, it can be seen that the western Pacific subtropical high extended westward and shifted northward, supported by dynamic conditions provided by a high-pressure ridge in northeastern China. This abnormally northward and intensified subtropical high created favorable conditions for transporting southeastern monsoon moisture inland. Combined with Figure 3b, the 850 hPa geopotential height field shows that Zhengzhou was located on the northern side of a shear line, controlled by southerly winds with a maximum speed of 16 m/s. Together with Typhoon “In-Fa” over the East China Sea and Typhoon “Cempaka” over the South China Sea, this configuration provided ample moisture conditions for the extreme precipitation in Henan. From Figure 3c, a pronounced moist zone is visible in front of the Taihang Mountains, indicating abundant moisture in the middle troposphere. Figure 3d, the 850 hPa water vapor flux divergence, also shows that Henan, as a region with high water vapor flux divergence, was a high-risk area for precipitation. Simultaneously, in Figure 3a, a mesoscale vortex can be observed on the western side of the subtropical high, with its center located over western Henan. This vortex provided upper-level divergence over Henan, creating favorable lifting dynamics for the development of heavy precipitation. The upper-level jet stream associated with this vortex also established advantageous moisture transport conditions for the heavy rainfall. Furthermore, the vortex’s movement over Henan was significantly slowed due to blocking by a high-pressure barrier formed by the subtropical high over Mongolia and topographic friction from the Taihang Mountains. This caused the shear line to persist near Zhengzhou for an extended period, resulting in sustained precipitation.

In conclusion, the formation of this extreme rainstorm event occurred under the influence of favorable large-scale circulation patterns. Upper-level jet divergence triggered weather systems in the middle and lower troposphere, while typhoons “In-Fa” and “Cempaka” established moisture channels with Henan, and mesoscale convective systems further contributed to coordinated moisture transport. The influence of the complex topography of the Taihang Mountains introduced uncertainties in the position and intensity of the low-level jet and mesoscale vortex, resulting in poorer simulation of local moisture convergence and vertical motion, which further contributed to biases in numerical forecasts. Therefore, it is necessary to quantitatively evaluate the performance of ensemble forecasts, comprehensively analyze the systems influencing the intensity and positional biases of the heavy rainfall, and systematically investigate the attribution of how these systems affect the performance of better and poorer ensemble members.

4. Results

4.1. Ensemble Forecasts

This experiment selects the heavy precipitation event in Zhengzhou, Henan from 00:00 on 20 July to 00:00 on 21 July for in-depth evaluation. Figure 4 presents an overview of the 24-h precipitation ensemble forecast for the Zhengzhou area during this period. The figure indicates that both the control forecast and the perturbation forecasts suggest the possibility of extreme heavy rainfall with cumulative precipitation exceeding 100 mm. Compared with the actual situation in Figure 2b, the ensemble forecast results demonstrate a certain degree of accuracy and advantage in terms of precipitation area distribution and intensity. However, there remains a certain bias in the forecast of the heavy precipitation center location. In terms of precipitation intensity, the forecasted intensity of the heavy precipitation center is generally weaker. Regarding spatial distribution, the forecasts are distributed in various directions around the observed heavy precipitation center in Zhengzhou, with a general westward bias. Nevertheless, precisely due to this scattered distribution of heavy precipitation centers, the spatial structure of the ensemble mean precipitation field is relatively close to reality. However, since most ensemble forecast members exhibit forecasts that are weaker and shifted westward, the heavy precipitation center in the ensemble mean is also weaker and shifted westward. Additionally, the overall spatial structure of precipitation in both the control and perturbation forecasts deviates from reality. Therefore, more in-depth quantitative research is necessary to analyze the forecast errors of the control forecast members and perturbation forecast members.

4.2. Forecasting Evaluation

Through the traditional objective scoring method, the control forecast and disturbance forecast in the ensemble forecasting system are tested and analyzed, and the evaluation geographical range is limited to the region of 110° to 117° east longitude and 31° to 37° north latitude, including 1 control forecast member and 18 perturbed forecast members. In addition, according to the standards of the China Meteorological Administration, the precipitation intensity was divided into five grades according to the 24-h cumulative precipitation: light rain (0.1~9.9 mm), moderate rain (10.0~24.9 mm), heavy rain (25~49.9 mm), rainstorm (≥50.0 mm) and heavy rainstorm (≥100.0 mm).

During the analyzed precipitation period, as the precipitation intensity level increases, Figure 5a shows that the Threat Score (TS) of the 19 forecast members exhibits an overall declining trend. This indicates that the overall forecast skill of the ensemble prediction system weakens with higher precipitation thresholds. From Figure 5b, it can be observed that the Bias Score of the 19 forecast members shows a fluctuating pattern, with all members approaching 1 more closely at the heavy rain threshold. This suggests that the forecasted areas of heavy rain by the ensemble prediction system are relatively close to the observed areas, demonstrating greater advantage in predicting the spatial distribution of heavy rain events. Figure 5c,d reveal that at lower precipitation thresholds, both the missing alarm rate and false alarm rate of the 19 forecast members approach 0, indicating better forecast performance.

We calculated the proportion of perturbed forecast members that outperformed the control forecast across different rainfall thresholds for various verification scores. Combined with Figure 5 and Figure 6, it can be observed that for the heavy rainstorm threshold, the TS of the control forecast during this period is 0.15. Figure 6a indicates that the proportion of members with a TS higher than the control forecast is 50%. Although Figure 6b shows a relatively high proportion of Bias scores outperforming the control forecast, Figure 5b reveals that all members’ Bias scores deviate significantly from 1. The member with the Bias score closest to 1, e10, has a score of 0.66, suggesting that the ability of traditional scores to evaluate the spatial distribution of heavy precipitation for this extreme rainstorm event requires further investigation. Simultaneously, the proportions of members outperforming the control forecast for the FAR and MAR metrics are 16% and 61%, respectively. Furthermore, the trends in the proportion of members outperforming the control forecast across different rainfall thresholds vary among the scores. Compared to the moderate rain, heavy rain, and rainstorm thresholds, the proportions of members outperforming the control forecast for the TS, Bias score, and FAR score at the heavy rainstorm threshold are lower or similar. However, for the MAR score, the proportion of members outperforming the control forecast at the heavy rainstorm threshold is more advantageous compared to the moderate rain, heavy rain, and rainstorm thresholds.

Traditional binary classification metrics are diverse, and different verification results are often obtained under different scoring systems. To fully utilize various scoring indicators and simultaneously evaluate the overall forecast performance of ensemble forecast members, multiple scoring metrics across different ensemble members and precipitation thresholds are plotted in a comprehensive verification diagram (Figure 7). Along the direction indicated by the yellow-green dashed curve, the Threat Score (TS) gradually increases, indicating an overall improvement in forecast performance. Along the coordinate axes, the False Alarm Rate (FAR) and Missing Alarm Rate (MAR) gradually decrease. The closer the points are to the diagonal (solid black line), the closer the Bias score is to 1, indicating that the number of hits for predicted precipitation at that threshold is closer to the observed precipitation, and thus better forecast performance. In the figure, different shapes represent five precipitation thresholds from light rain to heavy rainstorm, while different colors represent the 18 perturbed forecast members (e01–e18) and the control forecast member. From Figure 7, it can be observed that as the precipitation threshold increases, the TS and Bias deviations among the ensemble members gradually exhibit larger discrepancies. When forecasting at the heavy rainstorm threshold, it becomes difficult to treat the forecast members as a unified whole for prediction. Given the extreme nature of this precipitation event, we focus on the heavy rainstorm threshold for further analysis. Figure 7 shows that the heavy rainstorm threshold scores for perturbed forecast members e10 and e12 are closer to the diagonal. However, combined with Figure 5, it is found that perturbed forecast member e12 performs best, with the highest TS and the best performance in FAR and MAR among all members. Meanwhile, perturbed forecast member e10 has slightly lower TS, FAR, and MAR scores at the heavy rainstorm threshold compared to member e12, but its Bias score is closer to 1. Perturbed forecast members e11 and e18 perform poorly across all evaluation metrics at the heavy rainstorm threshold.

In summary, the perturbed forecast members e10 and e12 exhibit relatively superior comprehensive forecast performance. They not only achieve favorable TS and Bias scores across various precipitation thresholds but also demonstrate relatively low false alarm and missing alarm rates for heavy rainstorm events. In contrast, members e11 and e18 show poorer performance scores for heavy rainstorms. Additionally, as the traditional verification model employs point-to-point evaluation, which introduces a “double penalty” effect, the assessment of the spatial distribution of heavy precipitation in extreme rainstorm processes becomes less objective. Therefore, it is necessary to conduct a more detailed evaluation of the positional biases in precipitation areas using spatial verification methods and to investigate the underlying causes.

Traditional verification models rely on direct point-to-point matching between forecast and observed data. However, when applied to high-resolution forecasts, this matching approach tends to amplify minor spatial displacements, leading to a “double-penalty” effect. Consequently, even if the model provides rich information at convective scales, the overall evaluation may be underestimated due to minor positional errors, failing to accurately reflect true forecast performance [31]. In view of this, for evaluating convective-scale ensemble precipitation forecasts, the more adaptive Fraction Skill Score (FSS) is preferred. This metric more accurately accounts for the spatial pattern match between forecast and observation, thereby reducing evaluation errors introduced by positional biases.

The spatial verification domain is selected to be the same as that used in traditional verification. Utilizing accumulated precipitation data from the convection-permitting ensemble prediction system and gridded observational data, the performance of spatial verification methods in precipitation assessment is examined. Eighteen grid spatial scales are chosen for the neighborhood fuzzy scale, ranging from 1 to 65 times the grid spacing. Since both forecasts and observations are interpolated to a 3 km resolution during verification in this experiment, the neighborhood fuzzy scales correspond to areas from 9 km × 9 km to 195 km × 195 km. First, the precipitation forecast skill of ensemble members under different spatial scales is investigated. Based on the research by Li Ziliang et al., similar to deterministic forecast metrics for synoptic fields, the FSS also has a threshold (e.g., an ACC of 0.6 indicates usable forecast skill). The FSS can be simply defined such that, under different precipitation thresholds, the spatial scale at which the score exceeds 0.5 is considered the “usable forecast scale” for precipitation prediction [31].

Figure 8 illustrates the distribution characteristics of the Fraction Skill Score (FSS) for 24-h accumulated precipitation across different ensemble forecast members, various precipitation thresholds, and multiple neighborhood spatial scales from 00:00 on the 20th to 00:00 on the 21st. It can be observed that as the considered neighborhood spatial scale expands, the skill scores for precipitation forecasts generally increase. This phenomenon may be attributed to the concentrated and continuous nature of the rainfall area in this studied event, coupled with the relatively small selected evaluation region, allowing the ensemble forecasts to demonstrate relatively good precipitation prediction performance even at smaller scales. Specifically, Figure 8a shows that even at a spatial scale of 3 km, forecast members exhibit relatively high forecast skill for rainstorm-level precipitation. For heavy rainstorm-level precipitation and above, as shown in Figure 8b, although forecast skill improves with increasing spatial scale, most forecast members still show limited skill for heavy rainstorm-level and above precipitation within the spatial scales examined in this study. Notably, members e06, e10, and e12 achieve FSS exceeding the threshold of 0.5 when the spatial scale is extended to 51 km. This aligns with the perturbed forecast members identified as having better performance through traditional verification evaluation. It also indicates that for forecasting heavy rainstorm-level precipitation in this extreme rainfall event, an appropriate minimum effective spatial scale is approximately 51 km, indirectly demonstrating the significant influence of meso- and micro-scale weather systems on this extreme precipitation process.

This extreme rainstorm event featured extreme hourly precipitation. To illustrate the differences in forecast skill for hourly accumulated precipitation between the perturbed forecast members and the control forecast member of the ensemble prediction system, we selected perturbed forecast members with better performance (e06, e12) and those with poorer performance (e11, e18). A more detailed accuracy verification of the FSS method was conducted using finer hourly precipitation data. Figure 9 shows the temporal evolution of the FSS for probability forecasts of hourly precipitation for the selected perturbed forecast members and the control forecast member. The probability thresholds are set at 2 mm (moderate rain), 5 mm (heavy rain), and 10 mm (rainstorm). The shading represents the FSS of the perturbed forecast members corresponding to each subplot, while the contours represent the FSS of the control forecast member. From Figure 9, it can be observed that at each precipitation threshold, the evolution trends of the FSS for different perturbed forecast members and the control forecast member are generally similar. However, members e06 and e12 exhibit broader spatial and temporal regions where their FSS exceed 0.5 compared to members e11 and e18. This indicates that members with better 24-h accumulated precipitation forecasts also demonstrate better forecast skill for hourly accumulated precipitation. Focusing on rainstorm-level precipitation, from a temporal evolution perspective, periods of higher forecast skill appear at 0–3 h and 18–21 h. Under certain spatial scale conditions, both perturbed and control forecast members demonstrate forecast skill during these periods. Furthermore, we observed that perturbed forecast member e12, which performed well in 24-h accumulated precipitation forecasting, also exhibited relatively high forecast skill for hourly rainstorm-level precipitation during the 16–18 h period.

According to Figure 9, we can see that the evolution trend of FSS of different members in different rainfall levels converges, so we select the smallest window to further analyze the time evolution, Figure 10 is the minimum window scale to calculate the FSS from the hourly probability forecast results of the LBGM from 00:00 on the 20th to 00:00 on the 21st, and the FSS is higher during the precipitation concentration period from 02:00 to 12:00, and with the division, rotation and merger of the precipitation center, the FSS drops sharply to a low peak. In the merger enhancement stage after 20 o’clock, the FSS slowly increased. In this case analysis, the FSS has a good indicative significance for the precipitation forecast of the order of 5 mm/h and below, and for the hourly precipitation forecast, the ensemble average method can often achieve better forecasting results, which may be because there is a time error in the forecast of the overall precipitation process, which will be magnified when the time scale is reduced, and at the same time, the simple ensemble mean can balance the errors in all directions of different ensemble members, so it shows advantages in forecasting. At the same time, we found that although member e18 performed better in the smaller magnitude, member e12 had a better hourly FSS in the rainstorm magnitude, which was consistent with the good scoring effect of member e12 in the rainstorm magnitude above.

The FSS cannot directly reflect the differences in location and intensity between forecasted and observed heavy precipitation. A comparison with observations reveals that ensemble forecast system members exhibit certain displacements in both the intensity and location of forecasted heavy precipitation. Therefore, further in-depth quantitative evaluation is necessary. The CRA method focuses on individual precipitation objects or rainbands rather than the entire forecast field, making it more convenient for forecasters to understand the model’s quantitative spatial biases in precipitation under different influencing systems. Building upon the suitable minimum effective spatial scale identified by the FSS method, this experiment employs the CRA method to evaluate this extreme precipitation process.

The CRA method requires matching the observation field and the model field within a certain spatial scale and precipitation threshold to define the Contiguous Rain Area (CRA). Considering the prominent characteristics of heavy precipitation levels in this extreme rainfall event, we define the precipitation thresholds at the rainstorm level (≥50.0 mm) and the heavy rainstorm level (≥100.0 mm). When the spatial scale reaches 51 km, members e06, e10, and e12 demonstrate relatively good FSS, suggesting that the simulation of local mesoscale convection is better at this spatial scale. Therefore, this spatial scale is selected for matching the observation and simulation fields. Since the spatial scale in the CRA method is defined by the number of grid points, approximately 400 grid points (rounded to the nearest integer) are used to define the spatial scale for matching. When matching the observation field and the model field, we adopted the object identification, object matching, and object merging methods from the MODE approach. To make heavy precipitation centers more prominent, we applied convolution smoothing to the original precipitation field, with a convolution radius set to 4. During the verification of this extreme rainstorm event, this convolution radius effectively highlighted heavy precipitation centers and successfully identified heavy precipitation objects.

Based on traditional verification scores and FSS from earlier evaluations using conventional verification models, we selected perturbed forecast members with better performance (e06 and e12), those with less ideal performance (e11 and e18), and the control forecast member for CRA method verification. Figure 11 shows the Contiguous Rain Areas (CRAs) defined after matching the observation field with the forecast fields of the selected members. Figure 11a visually demonstrates that at the rainstorm level, the contiguous rain areas of the selected ensemble forecast system members generally cover the contiguous rain area of the observation field. However, member e18 exhibits a westward displacement component. Figure 11b visually shows that at the heavy rainstorm level, all ensemble forecast system members have a westward displacement component relative to the observation field. Nonetheless, member e12, which performs better in forecasting, has a larger matched precipitation object area, and its contiguous rain area better covers the core heavy precipitation zone of Zhengzhou compared to other members. Additionally, compared to the control forecast, the better-performing member e06 still shows superior performance in location forecasting, while the poorer-performing member e18 performs worse than the control forecast in location forecasting. However, the poorer-performing member e11 has a larger precipitation area than the control forecast, further verifying that point-to-point traditional evaluation has verification errors in convection-permitting scale models. It also indicates that perturbed members of the ensemble forecast system have certain advantages over a single control forecast in predicting extreme precipitation areas.

The CRA method can also examine intensity bias and pattern bias in precipitation forecasts. From

Table 1. From 00:00 on 20 July to 00:00 on 21 July 2021, the centroid, intensity, and morphological deviation of precipitation forecasts of heavy rainfall magnitude (≥100.0 mm) of the ensemble forecasting members.

Forecast Members	Centroid Longitude Deviation/°	Centroid Latitude Deviation/°	Target Inclination Deviation/°	Strength Error/mm2	Morphological Error/mm2
con	−1.32	−0.06	−123.94	90.14	91.00
e06	−0.56	−1.00	−53.48	39.16	110.32
e10	−1.18	−0.37	−106.72	66.72	75.74
e11	−1.22	−0.97	−164.37	71.38	83.39
e12	−1.16	−0.94	−7.13	29.29	71.50
e18	−1.45	−0.44	−110.17	59.41	88.57

, it can be observed that for forecasts at the heavy rainstorm level and above, all members exhibit a westward bias in longitude. Among them, member e06 has the smallest bias of −0.56°, while member e18 has the largest bias of −1.45°. In terms of latitude bias, the control forecast member and member e10 show relatively small north-south displacements, whereas members e06 and e11 exhibit significant biases. The target inclination bias represents errors in pattern, with smaller absolute values indicating that the orientation of the rainband is closer to observations. Member e12 has an extremely small inclination error, suggesting its rainband orientation is almost identical to observations. In contrast, member e11 and the control forecast have very large inclination biases, which may indicate completely opposite or orthogonal orientations. Intensity error reflects systematic bias in precipitation amount, calculated as the ratio of average precipitation intensity between forecast and observation within the overlapping area, indicating systematic overestimation or underestimation of precipitation intensity. As shown in the table, intensity errors from smallest to largest are: e12 (29.29) < e06 (39.16) < e18 (59.41) < e11 (71.38) < con (90.14). Members e12 and e06 have the smallest intensity errors, indicating the most reliable quantitative precipitation forecasts, while the control forecast has the largest intensity error, reflecting significant underestimation. Pattern error reflects spatial similarity. Member e12 has the smallest pattern error, combined with an extremely low inclination bias, indicating its overall spatial structure matches observations most closely. However, e06 has a small intensity error but the largest pattern error, due to differences in the shape of its precipitation area compared to observations. In summary, compared with traditional evaluation methods, member e12 demonstrates relatively better forecasts relative to observations. Meanwhile, members e06, e11, and e18 performed worse relative to the control forecast in traditional evaluations. However, spatial verification reveals that member e06 has better intensity forecasting but poorer location and pattern matching. Furthermore, all three members show certain advantages over the control forecast, further highlighting the necessity of employing spatial verification methods for convection-permitting scale forecasts.

4.3. Mechanism Analysis of Precipitation Deviation

Following the evaluation and assessment of ensemble forecast results using traditional evaluation methods, the FSS, and the CRA method, both better-performing and poorer-performing ensemble forecast members were identified. By combining physical quantities that represent different characteristics, diagnostic analysis of the ensemble forecast results was conducted. This was further extended to analyze favorable conditions for disaster-causing heavy precipitation and to reveal the bias mechanisms underlying the differences between better and poorer members through characteristics of dynamic fields, thermodynamic fields, and moisture conditions. Four time points during this extreme rainstorm event were selected: 16:00 on 19 July, and 00:00, 08:00, and 16:00 on 20 July. Using physical quantities such as isentropic potential vorticity, relative vorticity, water vapor flux divergence, relative humidity, and pseudo-equivalent potential temperature across different vertical levels, the ability to reproduce Mesoscale Convective Vortex (MCV) was analyzed for the control forecast, the better-performing perturbed forecast member e12, and the poorer-performing perturbed forecast member e18.

Isentropic Potential Vorticity (IPV) can effectively track the downward transport of upper-level high potential vorticity anomalies along isentropic surfaces to lower levels. It is a key indicator for identifying whether upper-level dynamic forcing successfully couples to the middle and lower troposphere. Rao et al. noted in their study that when IPV > 1.5 PVU appears in the middle troposphere (500–700 hPa), it is often regarded as a sign of downward transport of high potential vorticity anomalies, which can significantly enhance local upward motion and convective instability [32]. Figure 12 displays the 500 hPa flow field and 340 K isentropic potential vorticity for perturbed forecast members e12 and e18, as well as the control forecast member, at 16:00 on 19 July, and 00:00, 08:00, and 16:00 on 20 July. We observed that both perturbed forecast members e12 and e18 showed tendencies to form vortex within regions of high IPV values (IPV > 1.5). This tendency was well maintained in e12, leading to the formation of a distinct mesoscale vortex within the high IPV region by 16:00 on 20 July. In contrast, for e18, the vortex structure largely dissipated by 08:00 on 20 July, and by 16:00, no clear trough or ridge structure remained. Although the control forecast formed a distinct vortex by 00:00 on 20 July, which persisted until 08:00, the vortex did not coincide with the region of high isentropic potential vorticity values. This indicates that only member e12 exhibited significant downward transport of upper-level potential vorticity.

Figure 13 shows the 700 hPa flow field and relative vorticity for perturbed forecast members e12, e18, and the control forecast member at 16:00 on 19 July, and 00:00, 08:00, and 16:00 on 20 July. Research by Li et al. found that the record-breaking hourly rainfall intensity in Zhengzhou (201.9 mm/h) was highly synchronized with a mesoscale γ-vortex with a lifespan of approximately 2–3 h. Its low-level (<3 km) $\xi$ center coincided with the heavy precipitation area, indicating that $\xi$ is a direct dynamic indicator for locating the core of an MCV [33]. From Figure 13, we observe that member e12 exhibits a distinct mesoscale convective vortex at 700 hPa, which persists from 16:00 on 19 July to 08:00 on 20 July. Although member e18 also shows a clear mesoscale convective vortex, its position is notably westward, and its duration is very short—appearing at 00:00 on 20 July and largely dissipating by 08:00 on 20 July. In contrast, the control forecast member (ctrl) displays cyclonic curvature but lacks a closed center. Combining Figure 12 and Figure 13, we find that the vortex center in member e12 not only corresponds to the high-value area of relative vorticity at 700 hPa but also coincides with the high-value area of 340 K isentropic potential vorticity. However, although the vortex center in member e18 corresponds to the high-value area of relative vorticity at 700 hPa, it does not align with the high-value area of 340 K isentropic potential vorticity. This indicates that the downward transport of upper-level potential vorticity is a key dynamic process for the maintenance and intensification of mesoscale vortex.

Figure 14 shows the 850 hPa flow field and water vapor flux divergence for perturbed forecast members e12 and e18, as well as the control forecast member, at 16:00 on 19 July, and 00:00, 08:00, and 16:00 on 20 July. Rao et al. found in their study that the strong convergence center of the vertically integrated water vapor flux during this extreme rainstorm event completely coincided with the core rainfall area [34]. From Figure 14, we observe that at 00:00 on 20 July, member e12 exhibits a distinct mesoscale vortex at 850 hPa, along with a region of high water vapor flux divergence values, indicating strong moisture convergence and ascent. Combined with Figure 13, it can be seen that this vortex coincides with the vortex at 700 hPa, demonstrating that this mesoscale vortex is deep and possesses sufficient moisture conditions. Even though the vortex at 850 hPa for member e12 dissipates by 08:00 on 20 July, there remains noticeable cyclonic curvature and a region of high water vapor flux divergence values. This provides dynamic conditions for the maintenance of the mid-to-upper-level mesoscale vortex and supplies moisture conditions for the extreme precipitation. Although member e18 also displays a clear vortex structure with a longer duration, its intensity is significantly weaker than that of member e12. Additionally, the water vapor flux divergence values at the vortex center of member e18 are noticeably smaller than those of member e12, indicating that, compared to e12, member e18 lacks relatively sufficient moisture conditions. This is the reason why member e18 forecasts lower precipitation intensity. Simultaneously, through comparative analysis with the control forecast member, we find that the water vapor flux divergence field of the control forecast member is weaker and more scattered. Moreover, the region of high water vapor flux divergence values corresponding to the cyclonic curvature is distinctly shifted westward, which aligns with the control forecast member’s prediction of lower precipitation intensity and a westward displacement of the rainfall area.

Figure 15 shows the vertical cross-sections along 34.5° N of relative humidity, pseudo-equivalent potential temperature, and vertical velocity for perturbed forecast members e12, e18, and the control forecast member at 00:00 on 20 July. As seen in Figure 15, all members exhibit sufficient moisture in the lower troposphere. In Figure 15a, e12 displays a distinct column of high pseudo-equivalent potential temperature and a region of high relative humidity. Furthermore, combined with Figure 13 and Figure 14, it can be observed that the area of high water vapor flux divergence in the lower troposphere coincides with the moist region where pseudo-equivalent potential temperature exceeds 350 K, providing ample dynamic and moisture conditions for the development of the rainstorm. In the middle troposphere, the pseudo-equivalent potential temperature of 340 K slopes upward, further contributing to the maintenance of upward motion. Additionally, the overlap of a core of high relative vorticity in the middle troposphere, a high pseudo-equivalent potential temperature core, and a moisture column with relative humidity greater than 80% [35] represents typical characteristics of a Mesoscale Convective Vortex (MCV). This promotes latent heat release and is one of the important reasons for the occurrence of extreme rainstorms. Although member e18 exhibits clear relative vorticity and vortex center alignment in Figure 13, in Figure 15b, insufficient lifting dynamics prevent moisture from being transported to the upper troposphere. Moreover, the presence of a low-value center with potential temperature below 340 K eliminates conditions for latent heat release, hindering further convective development. Consequently, member e18 fails to forecast the extremity of precipitation. In contrast, while the control forecast member in Figure 15c shows a high pseudo-equivalent potential temperature core and a moisture column with relative humidity greater than 80%, similar to member e12, and even exhibits stronger vertical upward motion, combined analysis with Figure 12, Figure 13 and Figure 14 reveals poorer dynamic configuration conditions. Despite clear cyclonic curvature, an MCV does not form. This is the reason why the control forecast performs poorly in forecasting extreme precipitation.

By analyzing the configuration and temporal evolution of physical quantities such as isentropic potential vorticity, relative vorticity, water vapor flux divergence, relative humidity, and pseudo-equivalent potential temperature across different vertical levels, we reveal the thermal and dynamic structures of the mesoscale convective systems simulated by perturbed forecast members e12 and e18, as well as the control forecast member. To further explain the generation mechanisms of the MCV in the model, we analyze the decomposed terms of the vorticity equation for the three members at 00:00 and 08:00 on July 20. Figure 16 and Figure 17 display the 500 hPa advection term, stretching term, and tilting term for perturbed forecast members e12 and e18, and the control forecast member (ctrl) at 00:00 and 08:00 on 20 July, respectively. First, comparing the magnitudes, at 00:00 on 20 July, the magnitudes of the advection and stretching terms are on the order of 10⁻⁸, while the tilting term is on the order of 10⁻¹⁰. This indicates that the primary contributions to vorticity at this time are from the advection and stretching terms. Similarly, at 08:00 on 20 July, the magnitude of the advection term exceeds those of the stretching and tilting terms, suggesting that the advection term is the main contributor to vorticity at this time. Overall, at 00:00 on 20 July, the mesoscale convective system is in the initial stage of intense development. The dynamic sources for vortex formation primarily include the advection of large-scale environmental vorticity into the convective region and the stretching and amplification of the vortex by strong low-level convergence and lifting. By 08:00 on 20 July, the mesoscale convective system has entered an organized, mature stage, where the vorticity advection by the large-scale circulation field becomes the main energy source sustaining the deep vortex. Referring to Figure 12, Figure 13 and Figure 14, both members e12 and e18 form distinct MCV. However, at 00:00 on 20 July, the advection term for member e18 shows a clear westward ascent along the flow field, which explains why the MCV forecasted by e18 is shifted westward. Additionally, combined with Figure 15, we observe that in the mid-to-upper troposphere, the horizontal advection for member e18 and the control forecast is significantly weaker than that for member e12. Since the advection of large-scale environmental vorticity into the convective region is the primary contributor, the MCV simulated by member e18 does not persist for long. The control forecast only exhibits cyclonic curvature without generating an MCV. Consequently, member e18 forecasts a precipitation area that is westward-shifted but with moderate intensity, while the control forecast not only has a westward-shifted precipitation area but also lower intensity. In contrast, member e12 forecasts both the location and intensity of precipitation, covering the core area of the heavy rainstorm threshold. This further underscores the crucial role of the MCV in this extreme precipitation event.

In summary, the perturbed forecast members that performed better in the ensemble prediction system exhibited superior simulation of the dynamic and thermodynamic factors influencing the extreme heavy rainfall process. They were able to more accurately simulate the mesoscale convective vortex that played a critical role in this event. Since the mesoscale convective vortex was a key system determining the intensity and location of the precipitation, these better-performing members achieved higher scores in the ensemble forecast verification. Conversely, both the control forecast and the poorer-performing perturbed forecast members showed inferior simulation of the dynamic and thermodynamic processes associated with this extreme rainfall event, resulting in lower verification scores.

5. Conclusions and Discussion

5.1. Conclusions

Based on NCEP-FNL reanalysis data and GPM satellite half-hourly global precipitation data, a quantitative evaluation study of convection-permitting ensemble forecasts for the Henan “7.20” extreme heavy rainfall event was conducted. An ensemble prediction system (LBGM-EPS) was generated using the Local Breeding Growth Mode (LBGM) initial perturbation method. Traditional evaluation metrics, the Fraction Skill Score (FSS), and the Contiguous Rain Area (CRA) method were employed to assess the model’s ability to simulate the location, structure, and intensity of precipitation objects under the LBGM approach. Combined with core dynamic-thermodynamic parameters such as isentropic potential vorticity, relative vorticity, water vapor flux divergence, relative humidity, and pseudo-equivalent potential temperature, a comprehensive diagnostic analysis of the dynamic and thermodynamic structures of the weather systems in the ensemble forecast results was performed. Furthermore, the vorticity equation was used to quantitatively reveal the internal dynamic processes leading to differences between better and poorer forecasts. The conclusions are as follows:

(1) In the process of extreme precipitation, from 00:00 on 20 July 2021 to 00:00 on 21 July 2021, the initial disturbance generated by the local growth model cultivation method provided the forecast for the members of the ensemble forecast system, in terms of the intensity and location prediction of heavy precipitation, the ensemble forecast disturbance members showed certain advantages over the control forecast, and the LBGM-EPS system showed a higher TS than the control forecast in the process of extreme precipitation (20% improvement) and lower displacement error (longitude deviation is reduced by 0.8°), which is more consistent with the actual situation.

(2) Compared with the traditional TS and other test methods, the CRA method can capture the shape, trend, movement direction and other characteristics of the precipitation target, and further reflect the spatial position and intensity information of the precipitation forecast.

(3) The disturbance forecast members with good ensemble forecast results are better than the control forecast members and the disturbance forecast members with bad ensemble forecast results, and can get better scores because the dynamic and thermal factors of the extreme heavy rainfall process are better simulated, and the mesoscale convective vortex with a greater impact on the extreme heavy precipitation can be better simulated, while the control forecast members and bad members have poor simulation of the dynamic and thermal processes of the extreme heavy precipitation process. The formation and maintenance process of mesoscale convective vortices cannot be well reproduced, but the maintenance time of the dominant member is about 8 h longer than that of the inferior member in simulating mesoscale convective vortex, and the coupling between the down propagation of the high-level vortex and the convergence of water vapor in the lower layer is more obvious.

5.2. Discussion

Through systematic evaluation and mechanism analysis, this study shows that the LBGM has certain advantages in convective scale ensemble forecasting, and the CRA method is more applicable in extreme precipitation assessment. The results of this study have practical reference value for improving the operational capability of heavy precipitation forecasting and optimizing the initial disturbance scheme of ensemble forecasting. Although this experiment can intuitively give the forecast results of the ensemble forecast method and the meteorological principle of the genesis of good and bad members, and provide a reference for the evaluation of precipitation operational forecasting and the research of ensemble forecasting methods, the experiment is only based on an extreme heavy precipitation process, which has certain limitations, and the experiment is based on the CRA method only to test the precipitation, and there is no in-depth study of other types of physical quantities, and the influence mechanism of the merits and demerits of the ensemble forecast results is only discussed from the perspective of dynamics and heat. Without further revealing the deep mechanism of the simulation of dynamic and thermal processes caused by ensemble forecasting, more precipitation cases will be collected in the later stage, and the rationality and deep mechanism of the initial disturbance method of convective scale ensemble forecasting will be further discussed. At the same time, although the prediction and mechanism of heavy precipitation have made great progress in all aspects, there are still many problems, and it is necessary to strengthen numerical simulation research, carry out ensemble forecast experiments under different background field conditions, improve the initial disturbance method of ensemble forecasting, and consider how artificial intelligence technology can be used in the research and operational application of ensemble forecasting, in order to promote and promote the theory and method of ensemble-scale ensemble forecasting.