Skills of Thunderstorm Prediction by Convective Indices over a Metropolitan Area: Comparison of Microwave and Radiosonde Data

: In this work, we compare the values of 15 convective indices obtained from radiosonde and microwave temperature and water vapor profiles simultaneously measured over Nizhny Novgorod (56.2°N, 44°E) during 5 convective seasons of 2014–2018. A good or moderate correlation (with coefficients of ~0.7–0.85) is found for most indices. We assess the thunderstorm prediction skills with a lead time of 12 hours for each radiosonde and microwave index. It is revealed that the effectiveness of thunderstorm prediction by microwave indices is much better than by radiosonde ones. Moreover, a good correlation between radiosonde and microwave values of a certain index does not necessarily correspond to similar prediction skills. Eight indices (Showalter Index, Maximum Unstable Convective Available Potential Energy (CAPE), Total Totals index, TQ index, Jefferson Index, S index, K index, and Thompson index) are regarded to be the best predictors from both the true skill statistics (TSS) maximum and Heidke skill score (HSS) maximum points of view. In the case of radiosonde data, the best indices are the Jefferson Index, K index, S index, and Thompson index. Only TSS and HSS maxima for these indices are close to the microwave ones, whereas the prediction skills of other radiosonde indices are essentially worse than in the case of microwave data. The analysis suggests that the main possible reason of this discrepancy is an unexpectedly low quality of radiosonde data.


Introduction
It is known that thunderstorms causing lightning, heavy rainfall, hail, and wind gusts are one of the most dangerous meteorological events. They can noticeably disturb social life (for example, the operation of airports), lead to fatalities, as well as significant economic losses, especially in industrialized regions. At the same time, the prediction of thunderstorms and their main characteristics (location, initial time, type, and intensity) remains one of the most difficult problems in modern weather forecasting. Usually, these phenomena evolve with rather small spatial (1-10 km) and temporal (1-12 h) scales. Thus, their forecasting, in particular, with the use of numerical weather models is still unsatisfactory due to insufficient resolution of the models and the lack of detailed initial data on three-dimensional spatial distributions of troposphere characteristics (in particular, temperature, water vapor, and wind) with the mentioned scales (see, e.g., [1][2][3]).
One of the basic methods for thunderstorm prediction is based on the indices of convective instability of the atmosphere. The first indices were proposed several decades ago [4][5][6] and were used for operational meteorology and short-term forecasting. They are usually calculated as certain algebraic combinations of the following characteristics of the troposphere (e.g., [7,8]): the temperature, humidity, and dew point temperature at selected pressure levels (SPL), in particular, 700, 850, and 500 mbar; the temperature of an air cell adiabatically lifted to the SPLs; potential temperature, virtual potential temperature, equivalent potential temperature, wet bulb potential temperature at SPL, etc. A thunderstorm is likely to occur when the local value of a specific index is beyond the thresholds which have been obtained empirically as the result of analyzing a comprehensive ensemble of past events. Nevertheless, this simple tool is commonly used up to now [9][10][11][12][13][14] and the index forecast quality often exceeds the skills of numerical weather models [14,15]. Moreover, old indices are being modified and new indices are being developed (e.g., [16]).
Convective indices were originally derived from the vertical temperature and water vapor profiles measured by means of radiosondes (e.g., [10][11][12][13]17]). However, radiosounding is performed quite rarely. Radiosondes are usually launched twice a day (at 00:00 and 12:00 UTC) by national weather services at about 1000 agrological stations worldwide, and only a few stations provide measurements 4 times a day. This is mainly due to the need to maintain each launch and use consumables, which are then lost and pollute the environment. In addition, according to the general requirements of aviation safety regulations, radiosounding cannot be carried out directly in the airport area. Nowadays, convective indices with much better temporal resolution than from radiosonde data are calculated from geostationary satellite data (e.g., [7,18]), but there is a problem of index quality connected with low vertical resolution of satellite observations in the atmospheric boundary layer.
In the nearest future, ground-based microwave radiometers placed near aerological stations may become the standard support means of radiosounding aimed at improving weather forecasting. These instruments continuously measure the spectra of the atmospheric radiation in the absorption/emission bands of molecular oxygen and water vapor with a relatively short accumulation time of one spectrum. The profiles of temperature and water vapor density in the altitude range of 0-10 km are retrieved with high spatial (up to several tens of meters) and temporal (several minutes) resolution. Important features of these devices are: (1) passive mode of operation with relatively low power consumption, i.e., they do not emit anything and therefore can be located, in principle, everywhere, including inside airports; (2) automatic continuous measurements and profile retrieval without human participation; (3) no constant use of consumables and no regular maintenance, except periodically (approximately once per 6 months), relatively easy and quick calibration using liquid nitrogen.
In the last decade, microwave temperature and water vapor profiling was used for many purposes of operational meteorology and nowcasting, for example, in airports and for meteorological support of major international events [19][20][21]. In the works [22][23][24][25][26], the indices obtained from microwave profiles were compared with the values of these characteristics from radiosonde data. The studies were performed for two tropical (13.5°N, 79.2°E and 22.3°N, 113.9°E) and two midlatitude sites (50.1°N, 123°W and 52.2°N, 14.1°E). For all case studies, a good correlation of many indices was found. The analysis of specific events led to the conclusion that microwave indices could be considered as a useful tool for ultra-short-term forecasting of thunderstorms.
In this work, we compare the values of various indices obtained from radiosonde and microwave data simultaneously measured over Nizhny Novgorod (56.2°N, 44°E) during five convective seasons of 2014-2018. In addition, we assess and compare the thunderstorm forecasting skills of these indices with a lead time of 12 hours.

Materials and Methods
Nizhny Novgorod is an industrial metropolis situated in the central part of the East European Plain at the confluence of the Oka and Volga rivers. It is known that the Volga river is the longest river in Europe; the Oka river being its biggest right tributary. The Oka river divides the city into two parts: the upper part (100-200 m above the sea level) and the lower part (70-80 m above the sea level). The climate in Nizhny Novgorod is temperate continental, with cold, long winters and warm, relatively short summers with high relative humidity. Due to the large differences in the terrain, there is a temperature and precipitation difference between the lower and the upper parts of the city. For example, annual precipitation is 15%-20% more in the lower part. Usually, the convective season in Nizhny Novgorod is from May to August.
In this study, we used the data measured by the radiometer RPG-HATPRO-G3, radiosondes, and two identical electric field mills (EFM) BOLTEK EFM-100 in May-August of 2014-2018. The radiometer operates from an open platform specially built in the Institute of Applied Physics, which is located in the upper part of Nizhny Novgorod, near (at a distance of ~3 km) the confluence of the Oka and Volga rivers. It continuously measures the spectra of the atmospheric radiation (brightness temperature) in the frequency range of 22-31 GHz (7 channels with a width of 0.23 GHz) and 51-58 GHz (7 channels with a width of 0.23-2 GHz) at different zenith angles (90.0, 42.0, 30.0, 19.2, 10.2, and 5.4°) with a radiometric accuracy of 0.1-0.2K for 1sec integration time. The spectra are processed using the quadratic regression model (provided by RPG and integrated in the software) to retrieve the temperature and water vapor profiles in the 0-10 km altitude range (39 non-equidistant levels with a vertical resolution from 10 to 1000 meters) with a high temporal resolution (1-10 min). According to the manufacturer's manual, the root-mean-square errors of the temperature and water vapor density are 0.25-0.35 K and 0.4 g/m 3 , respectively. The device also provides the measurements of vertically integrated water vapor (IWV) and liquid water path (LWP) with a high temporal resolution (1 sec). LWP can be used, in particular, as an indicator of clear-sky, moderate, or severe weather. It is known that extreme events with heavy rains are characterized by relativity high values of LWP (up to several thousand g/m 2 and more). Moreover, the presence of liquid water in the atmosphere distorts essentially the spectra of microwave radiation coming from the troposphere, resulting in wrongly retrieved temperature and water vapor profiles. Therefore, in this work, we consider the microwave data for LWP < 400 g/m 2 only.
The radiosondes are launched twice a day (at 00:00 and 12:00 UTC) at the aerological station "Nizhny Novgorod" (No. 27459, weather.uwyo.edu) also located in the upper part of Nizhny Novgorod, ~7.3 km from the radiometer location. Typically, the profiles are measured by radiosondes AK2-02m with an accuracy of 1 K in temperature and 7%-15% in water vapor density. The vertical resolution varies from 200 to 700 m depending on the altitude and launch.
The EFMs are located at the Institute of Applied Physics and the aerological station, i.e., they are collocated with the radiometer and radiosondes, correspondingly. The EFMs measure the local vertical electric-field intensity at frequencies of 0-10 Hz in the ±50 kV/m range with approximately 1ms time resolution.
First of all, we compared the microwave and the radiosonde data. The dataset includes 663 pairs of simultaneously measured profiles for each characteristic of the troposphere. Second, based on this dataset, we calculated and compared the following 15 forecast indices: CAPE (Convective Available Potential Energy), MUCAPE (Maximum Unstable CAPE), CIN (Convective Inhibition), JI (Jefferson Index), KI (K index), KOI (KO index), LI (Lifted Index), MDPI (Microburst Day Potential Index), FT (fog threat index), ShowI (Showalter Index), SI (S index, STT), TI (Thompson index), TT (Total Totals index), TQ (TQ index), and EPI (Equivalent Potential Instability). The equations for calculating these indices are taken from [7,8,26]. Third, we calculated the number of flashes per minute (NLFmin) from the electric-field intensity data using the algorithm presented in [27]. As a result, we produced two NLFmin time series: for the Institute of Applied Physics (NLFmin IAP ) and for the aerological station (NLFmin sonde ). This data is used, in particular, for determining the thunderstorm statistics during these convective seasons and some features of each event, for example, start time, duration, etc. Fourth, we assessed the prediction skills of the mentioned microwave and radiosonde indices with the lead time of 12h by the categorical verification techniques described, for example, in [10,13]. Here, we also used the sets of data simultaneously measured at 00:00 and 12:00 UTC, but the microwave data are averaged over 30 min around these time points. At this stage, we have four datasets: microwave (FI IAP ) and radiosonde (FI sonde ) forecast indices, NLFmin IAP and NLFmin sonde . For every datapoint in the FI IAP or FI sonde time series, we calculated time up to the next thunderstorm using the NLFmin IAP or NLFmin sonde time series, correspondingly. This was done by finding the first non-zero value in NLFmin whose time value is larger than that of the datapoint. Then, the following analysis was performed for each microwave and radiosonde index: 1. We take one value from the variation range of the specific index and suppose this value to be the lower (for the first group of indices: CAPE, MUCAPE, CIN, JI, KI, MDPI, SI, TI, TT, and TQ) or the upper (for the second group: KOI, LI, FT, ShowI, and EPI) threshold of thunderstorm prediction (λttp). After that, comparing the local values of the index λ with λttp and the corresponding values of time up to the next thunderstorm tnt with the lead time of 12 h, we count the number of predicted events (hits), the number of false events, the number of unpredicted events (surprises), and the number of predicted absences of event (non-events). These numbers form a 2 × 2 contingency table, as shown in Figure 1. In particular, for the first group of indices, this count is the following. If λ≥λttp and tnt ≤ 12 h, this is a predicted event. If λ ≥ λttp and tnt>12 h, this is a false event. If λ < λttp and tnt ≤ 12 h, this is an unpredicted event. If λ < λttp and tnt > 12 h, this is a predicted absence of event. 2. We calculate the following characteristics (skill scores) of thunderstorm prediction: the probability of detection POD = H/(H+S), the false alarm ratio FAR = F/(H+F), the probability of false detection POFD = F/(F+A), the critical success index CSI = H/(H+F+S), the true skill statistics TSS = POD-POFD, the Heidke skill score HSS (see [10,13]), and some others.
3. By repeating items 1-2 for all values of λttp from the variation range of the specific index, we obtain the mentioned skill scores as a function of λttp and find the optimal thresholds λttp TSS and λttp HSS at which the functions TSS(λttp) and HSS(λttp) reach their absolute maxima TSSmax and HSSmax.

Results
The results of comparison of the temperature and water vapor density variations obtained from microwave and radiosonde data are demonstrated in Figure 2. Depending on the altitude, the mean temperature deviation Tmd, standard temperature deviation Tsd, mean H2O deviation H2Omd, and standard H2O deviation H2Osd vary in the (−0.5;+1.6)K, (1.2;4.6)K, (−0.5;0)g/m 3 , and (0;1.9)g/m 3 ranges, respectively. Almost at all altitudes, Tmd >0 and H2Omd<0, i.e., there are regular shifts between the data. Nevertheless, they are much less than natural variability of temperature Tnsd and water vapor density H2Onsd, correspondingly. Tnsd and H2Onsd are estimated as standard deviation of temperature and water vapor density from radiosonde data. Note also that most forecast indices are calculated using the data measured in the altitude range of 0-6 km. One can see that Tsd below 6 km is essentially less than Tnsd at these altitudes (4-6K), whereas H2Osd is only ~1.5 times less than H2Onsd.
The results of the comparison of microwave and radiosonde values of each index are shown in Figure 3. One can see a good or moderate correlation for most indices, with the exception of FT and MUCAPE. In particular, the correlation coefficients (CORs) vary in the range of 0.68-0.78 for eight indices (TT, CAPE, EPI, JI, SI, MDPI, ShowI, TQ) and exceed 0.8 for five indices (CIN, KOI, KI, LI, and TI).  The analysis of EFM data shows that there were 38, 46, 44, 40, and 38 thunderstorm days per year from 2014 to 2018, correspondingly. The thunderstorm prediction skills of the radiosonde and microwave indices are summarized in Tables 1-2 in descending order of TSS maximum. Table 3 demonstrates the TSS maximum difference and the HSS maximum difference between the microwave and radiosonde values of each index. We can conclude that the effectiveness of thunderstorm prediction by microwave indices is much better than with the use of radiosonde indices. In particular, eight microwave indices (ShowI, MUCAPE, TT, TQ, JI, SI, KI, and TI) with TSSmax = 0.4-0.43 and HSSmax = 0.32-0.34 can be regarded to be the best predictors from both TSS maximum and HSS maximum points of view. In the case of radiosonde data, the best indices are JI, KI, SI, and TI with TSSmax = 0.34-0.4 and HSSmax = 0.28-0.31. It can be noticed from Table 3 that only the values of TSSmax and HSSmax of indices are close to the microwave ones, whereas the prediction skills of the other radiosonde indices (especially FT, LI, CAPE, and KOI) are essentially worse than in the case of microwave data.

Discussion
As was mentioned in the Introduction, Cimini et al. [26] also compared the microwave and radiosonde indices obtained at the aerological station MOL-RAO (52.2°N, 14.1°E), i.e., at the same site. Moreover, the radiosonde data were measured by Vaisala sensors, which provided much better measurement uncertainties (0.5K for temperature, 5% for relative humidity) than at the aerological station "Nizhny Novgorod". Microwave data were measured by radiometer MP-3000A, which is technically close to RPG-HATPRO. The coefficients of the correlation between the microwave and radiosonde values of the indices corresponding to the microwave data retrieval method using the regression model, i.e., similarly to RPG-HATPRO, are listed in Table 3. One can see that, on average, there is no significant difference between the data obtained by Cimini et al. and our CORs. At the same time, this table demonstrates that a good correlation between radiosonde and microwave data of a certain index is not necessarily equivalent to similar thunderstorm prediction skills.
There are three main reasons why the effectiveness of thunderstorm prediction by microwave indices is better than by radiosonde indices: (1) lower sensitivity of registering lightning flashes of the EFM at the aerological station "Nizhny Novgorod" than of the EFM at the Institute of Applied Physics; (2) 30 min averaging of microwave data and much better vertical resolution of these data at the altitudes of the lower troposphere; (3) unexpected relatively low quality of radiosonde temperature and water vapor measurements. For verification of the first possible reason, we have reanalyzed the thunderstorm prediction skills of the radiosonde indices using the NLFmin time evolution around the Institute of Applied Physics. It can be seen from Table 4 that the EFM data replacement did not change the status quo essentially. The best indices are JI, KI, and SI with TSSmax = 0.36 and HSSmax = 0.25-0.31, although the prediction skills of several indices (CIN, KOI, LI, EPI, and MDPI) have been improved. To check the second possible reason, we have excluded averaging of the microwave data around 0 and 12 UTC and extrapolated the microwave data to a radiosonde measurement grid. Reanalysis of the thunderstorm prediction skills of the microwave indices revealed only the slight decrease (by 0.01-0.03) in the values of TSSmax and HSSmax shown in Table 2. Thus, the relatively low quality of radiosonde measurements can be considered to be the main possible reason of the found discrepancy between the effectiveness of thunderstorm prediction by microwave and radiosonde indices.

Conclusions
Thus, the analysis of microwave temperature and water vapor measurements obtained over Nizhny Novgorod during five convective seasons of 2014-2018 demonstrates the remarkable potential of using these data for local nowcasting of thunderstorms. Moreover, following [26], we can note that high temporal resolution of microwave data (1-10 min) is promising for the enhancement of the skill of thunderstorm prediction and for a substantial reduction of the number of false and unpredicted events.