CME Forecasting System: Event Selection Algorithm, Dimming Data Application Limitations, and Analysis of the Results for Events of the Solar Cycle 24

Abstract

The modeling of coronal mass ejections (CMEs) arrival to Earth was carried out using a one-dimensional drag-based model (DBM) over the period from 2010 to 2018. The CME propagation model includes a simulation of the interaction of the CME with background solar wind via the quasi-stationary solar wind (QSW) model. An analysis of the results of forecasting CME speed and time of arrival to Earth was performed. Input data were obtained from the CACTus database. To ensure real-time operation, a new algorithm was established to select events that can reach Earth more likely. Coronal dimming data were used to obtain coordinates of the CME source location. Forecasting results have been compared with interplanetary CME (ICME) catalogs. The system has predicted 189 of 280 events (68%), with a tolerance of 48 h for the period of maximum solar activity (from 2010 to 2015). The average absolute error of predicted CME arrival speed is about 90 km/s. Our system has predicted 80% of ICMEs associated with extreme geomagnetic storms (Dst_min ≤ −100 nT) within a tolerance of 24 h.

1. Introduction

The forecasting of coronal mass ejection (CME) arrival time to Earth’s orbit and their speed at that time is one of the important issues of space weather science. CME arrivals to Earth’s orbit are responsible for the strongest magnetic disturbances. However, the number of CMEs observed by the coronagraph is much greater than that of geomagnetic storms that occurred after these eruptions and were possibly associated with them. For this reason, CME forecasting is a complex task that encompasses not only questions of forecast accuracy but also of preliminary selection of events that can be detected in near-Earth orbit and are powerful enough to affect Earth’s magnetosphere. There are many scientific groups working in this field, developing various models and approaches to forecasting. A broad and detailed review can be found in [1,2,3].

The CME Scoreboard service developed by the Coordinating Community Modeling Center (CCMC) is a place where scientific teams can submit their predictions of CME-driven shock arrivals as close to the detection of a CME launch as possible. Five years of Scoreboard (2013—May 2018) data were analyzed, and forecasts of 32 systems have been compared in [2]. Forecasts made by six most frequently submitted models were compared. According to the research [2], mean error varies from −7.1 to 0.2 h, mean absolute error of arrival time prediction—from 13.1 to 17.3 h, and standard deviation is in the range of 15.5 to 23.8 h. Minimal and maximal errors of arrival time predictions reach −66.9 and 69.5 h.

Modern programming methods make it possible to present complex case study simulations, such as CME evolution in the solar corona up to 20 solar radii [4] or interaction of different solar wind streams [5], taking into account all available data and physical models. However, it is essential that space weather centers be able to use near real-time methods and data to provide a mid-term forecast (2–5 days in advance).

One well-known and widely used model is the WSA–ENLIL+Cone model, which provides ambient solar wind simulations (by Wang–Sheeley–Arge (WSA) model), CME propagation (by ENLIL), and considers CME geometry (via the cone model). The version of the WSA–ENLIL+Cone model presented by NASA is described in [6]. The WSA model requires solar magnetograms to derive solar wind speed at the ENLIL inner boundary [7,8]. ENLIL simulations describe the magnetohydrodynamic (MHD) evolution of solar wind streams and added CMEs [9]. CME input parameters for the ENLIL model and cone model [10] could be determined with the Stereoscopic CME Analysis Tool (StereoCAT) [11] and with the NOAA Space Weather Prediction Center CME Analysis Tool (CAT) [12]; both systems require manual CME fitting to derive CME parameters.

EUHFORIA is another space weather forecasting model that uses different methods for different spatial domains [13]. MHD simulation starts at 0.1 AU in the heliospheric domain and can be employed up to 2 AU. The linear force-free spheromak model is used for magnetized CMEs, and the cone model is used for non-magnetized ones [14,15]. EUHFORIA is a powerful tool for modeling complex events, but developing an online application operating in near-real-time remains a future objective for EUHFORIA developers [16].

The SUSANOO model provides 3D MHD simulation of solar wind and considers CMEs from 30 solar radii using a spheromak-like flux rope [17]. Like EUHFORIA, it can derive a solution for CME propagation in complex cases, but it does not have a real-time online version.

In addition to MHD, there are also numerical models, and one of the most common numerical models is the drag-based model (DBM) [18,19]. The cone model can also be incorporated in the DBM to account for the geometry of the eruption [20]. Common variants of the DBM are presented on the website https://oh.geof.unizg.hr/DBM/dbm.php (accessed on 14 July 2024),, allowing testing of the model on user inputs. A CME prediction tool called ELEvoHI is based on the DBM, uses STEREO HI images, and assumes an elliptical geometry of the ejection [21]. In the latest version of ELEvoHI, an ensemble approach is used, and an implementation of a deformable CME front is added, and authors discussed the applicability of ELEvoHi for real-time predictions [22].

Neural networks also could be used for CME forecasting [23,24,25]. CAT-PUMA is one such project that has produced good results when calculating arrival times of selected historical CMEs: 54% of the events of the 20-event test set have absolute prediction errors of less than 5.9 h. However, the authors note that the model can only calculate the arrival time of manually proposed events and cannot predict whether a CME will hit Earth.

To summarize the above, the problem of selecting CMEs for forecasting remains partly unsolved. To address this issue, some science teams use volunteers that monitor and manually select events for CME forecasting. For example, the Solar Stormwatch (SSW) project described in the paper [26] is based on the theory that a majority decision made by a large group of volunteers is equivalent to an expert decision.

Another common problem in CME forecasting is the lack of critical data such as CME velocity. When using SOHO/LASCO, only the projection of CME speed to the picture plane can be calculated. There are different approaches to address this, from manual trajectory fitting to ensemble forecasting [11,27,28]. In [29], a combination of SOHO/LASCO and STEREO (A, B) data has been used to triangulate CME source location, which is then used to correct CME speed. Unfortunately, data from STEREO-B has been unavailable since 2014. Some models use STEREO-A in combination with LASCO data, which is only helpful if STEREO-A is in a suitable location for triangulation [12] (StereoCAT). In [30], the influence of manual CME fitting on the accuracy of the obtained forecast has been investigated. It was found that for most events, errors in the derivation of CME parameters lead to errors in the forecasted arrival time of 5 h and less. These errors could be bigger in real cases with observed CMEs.

The next approach is to determine the CME source on the Sun’s surface and use source coordinates to adjust CME speed [31]. The Atmospheric Imaging Assembly instrument (AIA) onboard SDO spacecraft provides high-resolution images of various layers of the solar atmosphere. This allows identification of the following potential CME sources: flares, coronal dimmings, and prominences. Their coordinates can be estimated from AIA images. We applied a similar approach in the present work, using coronal dimming coordinates estimated from SDO images for selecting Earth-directed events. For this purpose, we used the Solar Demon database [32] that was developed to detect coronal dimmings and list their parameters.

We present the results of the CME forecasting system developed for the Space Weather application part of the Space Monitoring Data Center (SMDC) of the Skobeltsyn Institute of Nuclear Physics (SINP) of the Moscow State University (MSU).

The system contains several stages of data processing: input data deriving, event selection algorithm, background solar wind speed forecast using AIA/SDO data, and DBM run for CME arrival time and speed calculation. CME parameters from the CACTus (Computer-Aided CME tracking software (version 2: CACTus version 2.5.0)) CME database and coronal dimming data from the Solar Demon database are used in the SMDC system. The event selection algorithm is based on CME and dimming parameters, analyzing and marking each event as potential geoeffective or not. The quasi-stationary solar wind speed forecast (QSW model) has already been published in [33,34,35,36] and is available at the SMDC website https://swx.sinp.msu.ru/models/solar_wind.php (accessed on 14 July 2024). We use one-dimensional DBM for CME arrival forecast, and QSW output speed is used as ambient solar wind speed for DBM. In this work we describe the CME forecasting system and analyze the results of testing it on data from the 24th solar cycle (from May 2010 to December 2018).

In Section 2, we present all data that has been used, models of CME propagation and background solar wind, the event selection method and its results, and data used for model validation. In Section 3, we present the results and analyze how the SMDC system performs for different phases of the solar cycle and for ICMEs associated with geomagnetic storms of different intensity. Possible ways of improvement of the SMDC system are suggested in Section 4.

2. Materials and Methods

2.1. Data

Near-real-time forecasting necessitates exclusive use of data, which is also available in near-real-time. CACTus (“Computer Aided CME Tracking”) is a software package that detects CMEs in coronagraph images in real-time. CACTus uses SOHO/LASCO C2 images, CME parameters such as time of first appearance in C2, observation duration, CME width, positional angle, and a linear speed profile along the angular span of the CME [37,38]. Positional angle is the angle between the direction to the north of the Sun and the center of the CME-bounded area (counterclockwise).

Another real-time CME catalog is SEEDS (“Solar Eruptive Events Detection System”), which also provides CME parameters in near real-time. In our paper [39], we showed that for 2010 and 2011, CACTus data are more suited for our method than SEEDS data for the period. Therefore, CACTus data were used in the present work.

In addition to the CACTus database, we use the Solar Demon dimming detections database [32]. A coronal dimming is a decrease in the intensity of soft X-ray and EUV radiation of a region of the Sun [40,41,42]. The Solar Demon software (SDO/AIA(2013)) processes SDO/AIA 21.1 nm images in real-time, detects coronal dimmings, and automatically obtains their properties. The wavelength of 21.1 nm corresponds to Fe XIV emission at 2 million Kelvin and shows active regions of solar corona. These conditions are associated with a height of 1.1–1.5 solar radii (lower corona). The Solar Demon database provides a number of properties of each event, such as coronal dimming observation start and end time, intensity, coordinates, and area dependence on time with a 2-minute resolution. Since one of the causes of coronal dimmings are fluctuations of plasma density due to CME expansion and escape from the solar corona, it is possible to use coronal dimming data to estimate CME source location and propagation direction or to detect stealth CMEs [43].

2.2. The Quasi-Stationary Solar Wind Speed Forecast

The quasi-stationary solar wind (QSW) model has been described in [35,36]. The model uses SDO/AIA 19.3 nm images to derive the coronal hole (CH) area and to calculate the solar wind speed. The coronal hole area is determined by applying a threshold algorithm to solar images. Solar wind speed V(S,t) is calculated from CH area by the following empirical formula:

$$V\left(S,t\right)=V_{min}+A·{S\left(t_0\right)}^{\alpha },$$

where $V_{min}$ is set as 300 km/s; S() is CH area calculated in the region of the Sun disk center (±20 degrees in longitude and ±40 degrees in latitude); and A and α are dimensionless parameters that are selected by minimizing modeling errors. This model is already operating in real-time. Recent forecasts are available at the SINP MSU Space Monitoring Data Center (SMDC) website. The results of analysis and comparison with other models from 2010 to 2019 are presented in [34]. The database of solar wind velocity forecasts starting from 2010 is also available from the SMDC website (https://swx.sinp.msu.ru/models/solar_wind.php; accessed on 14 July 2024).

2.3. The Drag-Based Model

The drag-based model (DBM) is used for CME propagation simulation. The DBM assumes that starting from a certain distance from the Sun, the main force that governs CME propagation is a drag force. “Aerodynamic” drag occurs between CME flow and the ambient solar wind [44]. Therefore, that force either accelerates or decelerates CME, depending on the sign of the difference between flow speeds. The motion equation for the DBM is as follows:

$$a=\gamma (v-w)|v-w|,$$

where a is drag acceleration, v and w are speeds of the CME and of the ambient solar wind, respectively, and γ is a “drag parameter” [45]. The drag parameter depends [46] on CME mass, M, cross-sectional area, A, and solar wind density, ρ.

$$\gamma ={\displaystyle \frac{A\rho }{M_{CME}}}$$

At certain distances from the Sun (20 solar radii), the relation between solar wind density and distance can be approximated as 1/r², while the CME expands as A~r², and CME mass is constant. Therefore, γ can be assumed as constant. Solar wind speed w could also be assumed as constant starting at the distance of 20 solar radii from the Sun [47]. These assumptions allow us to solve the motion equation analytically, which makes the calculation very fast, e.g., 1–2 s for 1 run [48]. The analytical solution leads us to the dependencies of the CME distance and CME velocity from time.

$$r\left(t\right)=\pm {\displaystyle \frac{1}{\gamma }}\ln \left(1\pm \gamma \left(v_0-w\right)t\right)+wt+r_0,$$

$$v\left(t\right)=w+{\displaystyle \frac{v_0-w}{1\pm \gamma \left(v_0-w\right)t}},$$

where “−” corresponds to acceleration and “+” to deceleration; $v_0$ is the CME’s velocity at the distance $r_0$; and $r_0$ is the CME’s distance from the Sun at the beginning of the simulation [18].

In our implementation, we assume a uniform propagation of the CME from the field of view of the LASCO C2 coronagraph (approx. 5 solar radii) to the inner boundary of the drag-based model (DBM) (20 solar radii). We use CME speed from the CACTus database as $v_0$ and a distance of 20 solar radii as $r_0$ for DBM input. Initial CME time at $r_0$ is calculated assuming uniform motion [34,39]. Then, we divide the distance from $r_0$ to 1 AU into 4 domains (65, 115, 165 solar radii and to 1 AU), and we use the DBM in each domain and set the output CME speed of the previous domain as the $v_0$ input in the next domain. Ambient solar wind speed for each domain is calculated via the QSW model, which makes it possible to account for high-speed streams that may interact with the CME during its propagation. This method has been tested in [36]. Drag parameter studies have described different scenarios of gamma choice [18,19]. Assuming that the influence of ambient solar wind is stronger for more powerful CMEs and weaker for poor CMEs, and taking into account our previous work, we used the following drag parameter values from [20]:

• 0.5 × 10⁻⁷ km⁻¹ for CMEs with velocities < 500 km/s.
• 0.2 × 10⁻⁷ km⁻¹ for CMEs with velocities in range of [500 km/s; 1000 km/s).
• 0.1 × 10⁻⁷ km⁻¹ for CMEs with velocities ≥ 500 km/s.

While the DBM has been developed for CME body propagation simulation, there were studies where ICME shocks were predicted with the DBM [49]. We use the DBM to calculate the arrival time and speed of the CME to Earth’s orbit, which are meant to correspond to ICME ejecta or magnetic cloud arrival time and speed. We also have the opportunity to compare our results to ICME shock arrival times when ICME ejecta/magnetic cloud arrival is not indicated in the ICME database.

2.4. Event Selection

The method of event selection includes several steps, which are described in detail in [36]. In this paper, the period from July 2010 to December 2018 is considered. This constitutes a period of 7 and a half years, which contains all phases of the 24th solar cycle, at least partially.

The first step (Step_1) is merging of the events from the CACTus database. All CACTus detections, both from the CME list and the “flow list” (which contains poor detections), are merged if they start within 50 min and have overlapping angular parameters within a 20-degree tolerance. This is performed in order to reconstitute halo and partial halo CMEs, which might be detected by CACTus as several narrow CMEs instead.

However, to single out the events that were correctly selected or unselected, it is reasonable to first calculate the forecast for all events and then examine the selection algorithm. Hence, we consider the list of merged CMEs from CACTus as an initial list, and all further filtering steps are applied after the predictions for each event are calculated.

The second step (Step_2) is to discard narrow events. For CMEs with positional angle values within the range from 60 to 120 degrees or within the range from 240 to 300 degrees (“equatorial” CMEs), an angular width of at least 30 degrees is allowed; for the rest (“polar” CMEs), at least 60 degrees. The example of a “equatorial” CME is demonstrated in Figure 1a. The positional angle in this case is about 95 degrees, while the angular width is 80 degrees. Such events will not be rejected by the system at Step_2.

The third step (Step_3) is meant to establish correspondences between CMEs and related coronal dimmings. For each CME obtained after Step_2, the algorithm attempts to find a corresponding coronal dimming no earlier than 2 h before the CME’s appearance in the coronagraph. As mentioned, the appearance of a coronal dimming is a consequence of CME motion at a height of approximately 1.5 solar radii, but we observe dimmings earlier because the CME needs some time to reach the field of view of the LASCO C2 coronagraph, while the dimming is observed directly at the CME launch area.

When no corresponding coronal dimming can be found, the CME is considered to have occurred on the far side of the Sun, and such CMEs are marked as rejected.

When the geometrical center of the coronal dimming is changing drastically while dimming observation, we consider such an event “multi-dimming”. A possible explanation for such behavior is that several coronal dimmings had occurred simultaneously or sequentially but were treated by the Solar Demon system as one event. The presence of such events in our list could cause errors in the establishment of correspondences between CMEs and coronal dimmings, so we mark them as rejected.

For additional verification of established correspondences, the CME positional angle is compared to the equivalent angle of the coronal dimming (dimming positional angle), as calculated from the dimming coordinates. The difference between these angles must be less than 90 degrees. This criterion is not applied to partial halo CMEs and to coronal dimmings that are located near the center of the Sun disk. The mask of the coronal dimming that corresponds to the CME is shown in Figure 1b. Coronal dimming mask indicates all dimmed pixels. The difference between dimming and CME start time is less than two hours, and the difference between positional angles is less than 90 degrees.

The final step (Step_4) is the detection of off-limb events that are unlikely to hit Earth. “Off-limb events” are events that occur at a longitude of around +90 or −90 degrees and are observed from the L1 point above the solar disk. The coronal dimming parameter R_dist specifies the distance from the center of the solar disk to the geometrical center of the dimming, measured in solar radii. Off-limb events are characterized by R_dist ≥ 1 and are marked as rejected, except ones with a positional angle of more than 180 degrees. Halo and partial halo CMEs could be large enough to hit Earth (for example, an event on 10 September 2017). In our example (Figure 1b), the coronal dimming shows an R_dist value of more than 1, which means that dimming is not observed on the solar disk and its absolute value of the longitude is close to or greater than 90°.

Table 1. The number of CMEs that we have in the initial list (after Step_1) and how it decreases with each stage of filtering. The period under consideration is from May 2010 to December 2018. A small angular width is less than 30 degrees for equatorial CMEs and less than 60 degrees for polar ones. “Off-limb” event means that CME corresponds to dimming with R_dist ≥ 1 (i.e., longitude is around +90 or −90 degrees).

Stage	Brief Description	Number of Events	Percentage of Events Remaining after Filtering
Initial (Step_1)	the list of CMEs from the CACtus database after merging	12,186
Step 2	filtering CMEs with small angular width	2964	24%
Step_3	filtering CMEs with no corresponding dimming	872	7%
Step 4	filtering CMEs with off-limb dimmings (except halo and partial halo CMEs)	499	4%

illustrates the number of observed CMEs in LASCO C2 as indicated by CACTus. After the merging stage (Step_1), we obtained 12,186 events, and after applying all filters, only 4% of those events remain. The most significant reduction in the number of events occurs due to the narrow events filter (Step_2), and we also discard more than half of the remaining number due to a lack of a corresponding coronal dimming (Step_3). At Step_4, the ratio of remaining events reduces from 7% to 4%.

The number of CMEs that occurred from 2010 to 2018 is presented in Figure 2 (black dotted line). The number of CMEs has been normalized to the maximum number of events, which corresponds to 2420 CMEs observed in 2013. The number of CMEs follows the trend in the solar cycle: it increases in 2010–2011, reaches the maximum in 2012–2013, and decreases after 2015. We compared this amount with the final number of selected CMEs per year (Figure 2), also normalized to the maximum number of selected events observed in 2013 (107 CMEs). We see the same dependency for selected events: increasing in 2010–2011, reaching maximum in 2012–2014, decreasing in 2015–2016, and reaching minimum in 2017–2018. The only notable difference is the steeper drop-off of the number of CMEs in 2015–2016.

2.5. ICME List for Model Validation

The main indicators that quantify the forecast are accuracy and Hit-to-Miss ratio. Hit events are events that were predicted and observed, while Misses are those that were not predicted but were observed. Predicted events that were not observed are considered False Alarms, and events that were neither predicted nor observed are considered Correct Rejections. We compare our forecasts with ICME catalogs to estimate these values.

The general structure of an ICME comprises an interplanetary shockwave, characterized by shockwave arrival time (T_shock), followed by a turbulent sheath region and then by the ICME body, which is characterized by body detection start and end times (T_start and T_end). There is also a further classification of ICME body types as either “ejecta” or “magnetic cloud” (MC) [50,51,52,53]. Different ICME catalogs use different methods to establish the type of the ICME and the time boundaries between its substructures. Thus, the number of detected events and their parameters vary between lists. For model validation, we combined data from several ICME lists to obtain a broader store of information about ICMEs that were detected near Earth.

We used the following three ICME lists and catalogs for model validation: the Richardson and Cane ICME List [54] (List 1), CCMC CME Scoreboard data (https://kauai.ccmc.gsfc.nasa.gov/CMEscoreboard/; accessed on 14 July 2024) (List 2), and the Space Research Institute of Russian Academy of Science (SRI RAS) Catalog of Large-Scale Solar Wind Phenomena [53,55] (List 3).

The Richardson and Cane list is used as List 1 without significant processing. T_shock, T_start, and T_end are provided in the Richardson and Cane ICME list for each event.

In the CCMC CME Scoreboard, the ICME shockwave arrival time (T_shock) and associated CME’s launch time are listed. We used this information to produce List 2. We only chose ICMEs that have been detected in Earth’s orbit. In cases where several CMEs have been detected in Earth’s orbit simultaneously, we considered them to be one ICME with 2 solar sources.

The SRI RAS list does not contain discrete ICME events but continuously categorizes intervals of solar wind over its entire period. SRI RAS solar wind types contain solar wind streams connected to slow and fast solar wind, CMEs, and corotating interaction regions. To produce a list of discrete ICMEs, we selected the solar wind type sequences connected to ICMEs [56]. Such sequences are as follows:

• Ejecta/MC;
• Shockwave + ejecta/MC;
• Shockwave + sheath region + ejecta/MC.

We considered ejecta/magnetic cloud start and end times as T_start and T_end, and shockwave start time as T_shock, when producing List 3.

We have combined three lists into a unified list and merged events that match between initial lists to avoid duplicates. When unified lists were compiled, events were merged if their time parameters matched within certain tolerances. Merging was performed in the following order:

• Events from Lists 1 (R&C) and 2 (CCMC) are merged if their shock arrival time (T_shock) matches with a tolerance of ±6 h.
• Events from List 3 (SRI RAS) are merged with events from Lists 1 and 2 by shock arrival time (T_shock) with the same ±6 h tolerance.
• Events from List 3 that have not been already merged with Lists 1 and 2 are merged by start time (T_start) with a tolerance of ±24 h.
• Finally, remaining List 3 events are merged with List 2 events that originate from the CCMC CME Scoreboard if the difference between the start time (T_start) of the List 3 event and the shock time (T_shock) of the List 2 (Scoreboard) event is less than ±24 h, since the CCMC CME Scoreboard does not provide CME start times.

We calculated the solar wind average, maximum speed, and minimum Dst index value for each ICME from the final list, using hourly data from the OMNI database. The average speed for each ICME was calculated for the period of the ICME observation time indicated in the catalogs. For most events, ICME observation time was assumed from T_start to T_end. For some events only the CME shockwave arrival time (T_shock) is known, and for such events we assumed an ICME observation time of 20 h starting from T_shock. The duration of 20 h was assumed since this is the average ICME duration for ICMEs with known T_start and T_end. Solar wind maximum speed and minimum Dst index value were calculated similarly. The latest version of the merged ICME list is available at the SMDC website: https://swx.sinp.msu.ru/tools/icme_list.php (accessed on 14 July 2024) [36,57].

There are 175, 87, and 318 events in Lists 1, 2, and 3, respectively, over the considered time period (2010–2018). CCMC CME Scoreboard data are only available from 2013. CCMC CME Scoreboard data includes events that have been predicted but did not hit Earth, but we do not include them in the merged ICME list.

The result of the list merging process is a list of 400 events, with intersections between different initial lists described in

Table 2. The composition of the final merged ICME list from 2010 to 2018.

Merged ICME List Subset	Event Source List	Number
Non-intersecting events	List 1	38
	List 2	37
	List 3	178
Intersecting events	List 1 ∩ List 2	7
	List 1 ∩ List 3	97
	List 2 ∩ List 3	10
	List 1 ∩ List 2 ∩ List 3	33
All		400

. There are 10%, 9%, and 44% of events in Lists 1, 2, and 3, respectively, that are only present in that list. Moreover, 147 events (37%) of the merged ICME list are present in at least 2 initial lists, and there are 33 events that appear in all initial lists. We compared ICME parameters such as duration and minimum Dst for non-intersecting events and events that intersect in at least two initial lists (

Table 3. The characteristics of the merged ICME lists and their subsets: events present in List 1 and/or List 2 (List 1 + List 2), events present in only one initial list (non-intersecting), in at least two initial lists (intersecting), and mentioned in all initial lists. <Dst_min> and <dur> are the minimum of the Dst index observed during ICME body observation and ICME body duration averaged for each set.

	List 1 + List 2	Merged ICME List	Non-Intersecting Events	Events Intersecting between Two or Three Lists	List 1 ∩ List 2 ∩ List 3
Number	262	400	253	147	33
<Dst_min> (nT)	−45	−35	−26	−50	−70
<dur> (h)	24	23	22	26	30

). Based on Table 3, it can be assumed that events that do not intersect between lists are generally weaker and shorter events that do not lead to significant geomagnetic disturbances. The distribution of events by geomagnetic disturbances they caused is shown in Figure 3. Despite intersecting events generally being more geoeffective, it follows from the histogram that there is a significant minority (≈10%) of non-intersecting events that cause geomagnetic storms with Dst of −50 nT and stronger, highlighting the importance of combining data from different ICME catalogs.

3. Results

This section is divided into three parts. First, we present an analysis of the number of predicted and unpredicted events calculated for various values of predicted time error tolerance. The predicted time of arrival (TOA) of the CME is compared to ICME time parameters obtained from the merged ICME list described earlier. The second part is devoted to the accuracy of the CME velocity forecast. The predicted speed of arrival (SOA) of the CME is compared with the average solar wind speeds calculated for events from the merged ICME list. Finally, we described the features of the forecasting system when operating during different phases of the solar cycle and for events of varying geoefficiency.

3.1. ICME–CME Correspondence

For the purposes of forecast validation, it is necessary to establish a correspondence between observed and predicted events that remain after all filters that have been applied. The quantity of predicted events depends on the following factors: 1. the established forecast error tolerance, and 2. the established ICME list. We established the maximum CME TOA error value of τ = 48 h and then analyzed the change in the number of predicted events as τ decreased to 24 h. We compared the CMEs predicted by our system with two variants of the ICME merged list described in Section 2.5: with and without ICMEs from List 3.

The CME TOA error is calculated as follows:

$$dT=T_{forecast}-T_{observed},$$

where $T_{forecast}$ is the predicted TOA of the CME, and $T_{observed}$ is the ICME start time (T_start) from the merged ICME list if it is known and the ICME shock arrival time (T_shock) from the merged ICME list otherwise.

First, we find predicted CMEs that match ICMEs with |dT| less than the established tolerance τ, so-called “Hits”. CMEs that were predicted but could not be matched with any ICME are considered “False Alarms”. Hit and False Alarm quantities can be expressed as numeric values as well as ratios to the total amount of CMEs—Hit ratio and False Alarm ratio. In cases where a CME matches several ICMEs, we select the CME–ICME pair with the smallest |dT| since one CME could not have been detected as several ICMEs at Earth’s orbit. However, the opposite case is possible: several CMEs could have interacted with each other and were detected in Earth’s orbit as one ICME. Hence, in cases when several CMEs match one ICME, we consider all such CMEs Hits. Therefore, the number of Hits is greater than the number of predicted ICMEs. ICMEs that could not be predicted with our system, with error less than established τ, are considered “Misses”. The quantity of Misses can also be expressed as either a numeric value or as a percentage of the full amount of ICMEs in the chosen ICME set—Miss ratio.

For most of the time period under consideration, we predict the arrival of more CMEs than ICMEs that were observed (see Figure 4); however, the situation is opposite from 2016 to 2018 due to a sharp decline in the quantity of CMEs.

In

Table 4. The number of CMEs and events in the ICME merged catalog (calculated for two ICME datasets). Hit, Miss, and False Alarm values are calculated for τ values of 48 and 24 h and represented as a percentage of the full number of events.

Comparison with the Merged ICME List	Events	Number	Parameter	τ = 48 h	τ = 24 h
(List 1 + List 2)	CMEs	499	Hit	40%	26%
			False Alarm	60%	74%
	ICMEs	222	Miss	36%	55%
Merged ICME list	CMEs	499	Hit	56%	36%
			False Alarm	44%	64%
	ICMEs	400	Miss	49%	65%

, the distribution of Hits, False Alarms, and Misses is presented for different values of τ: 48 h and 24 h. Out of all 499 predicted CMEs, 196 match ICMEs with |dT| less than 48 h. This corresponds to 142 predicted ICMEs out of all 220 listed in the merged ICME list (List 1 + List 2), a Miss rate of 36%. As τ decreases, the number of Hits decreases while the number of Misses increases.

We have analyzed how the variability inherent to ICME identification affects forecasting model validation. SRI RAS data includes 318 ICMEs, 178 of which are not listed in other catalogs. Table 4 presents the rates of Hit, False Alarm, and Miss with and without SRI RAS (List 3) data. The amount of Hits increases for all τ values. In this case, using List 3 allows us to match more CME predictions with the ICMEs, reducing the percentage of False Alarms. The amount of Hits increases regardless of established error tolerance when including the ICMEs from List 3 (e.g., it increases from 26% to 36% for τ = 24 h). However, the number of Misses increases as well.

The large effect of the ICME set on forecasting results could be explained by the fact that there are a lot of ICME in the SRI RAS list that did not cause a significant geomagnetic disturbance. Table 3 presents values of the minimum of the Dst index registered during the ICME body observation period averaged for different ICME catalogs (<Dst_min>). This value is greater for the full merged ICME list than for the combination of List 1 + List 2, indicating a smaller average geomagnetic disturbance.

To evaluate the ability of the system to predict geoeffective events, we calculate <Dst_min> for sets of predicted ICMEs and Misses for τ of 48 h. For List 1 + List 2 data, we obtained −50 nT and −43 nT for predicted ICMEs and Misses, respectively. With the addition of List 3, the <Dst_min> for the set of predicted ICMEs remains almost the same (−49 nT), while the <Dst_min> for Misses decreases to −38 nT.

The TOA forecast error distribution for Hits is presented in Figure 5a. The distribution is symmetrical, and the mean error is close to 0 h. This distribution is obtained for 281 CMEs that predict ICMEs from the merged ICME list. A small bias to positive errors could be noticed, which means that predicted TOA is later than observed ICME time.

3.2. Comparison with WSA-ENLIL + Cone Model

The paper [58] presents validation results of the WSA-ENLIL + Cone (WEC) online model for the period from March 2010 to December 2016. ICME data used for WEC validation is obtained from the DONKI database and is enriched with other ICME catalogs (the Richardson and Cane ICME List, the ISEST Master CME List, the Wind ICME catalog). Simulation results are analyzed, and Hit, Miss, and False Alarm values are calculated for three points in the heliosphere—Earth, Stereo A, and Stereo B. The results for Earth obtained with WEC are presented in

Table 5. Comparison of CME forecasting results of the SMDC system and the WSA-ENLIL + Cone model from 2010 to 2016.

Comparison SMDC vs. WSA-ENLIL+Cone	Events	Number	Parameter	τ = 30 h
SMDC	CMEs	481	Hit	145
			False Alarm	336
	ICMEs (List 1 + List 2)	197	Miss	83
WSA-ENLIL+Cone	Model runs	1700	Hit	121
			False Alarm	180
			Miss	106
			Correct Rejection	1293

, along with analogous SMDC results. The analysis is performed over a period from July 2010 to December 2016. For SMDC results validation, the Richardson and Cane ICME List and the CCMC CME Scoreboard’s CME arrival records were used (List 1 + List 2), so the ICME sets used for the model validation are similar. While WEC validation, a prediction was considered a Hit when a simulation predicted CME and also CME was observed to arrive, and the forecast error was less than 30 h.

The paper [58] does not provide specific information about the number of ICMEs used for validation. However, if we consider that each simulated Hit and Miss corresponds to an ICME arrival, the number of ICMEs should be around 227. A similar situation occurs with Misses. In the paper [58], the authors classify a simulation as a Miss if it predicts no CME arrival while an ICME was observed or if the time error is greater than 30 h. In our work, we define a Miss as an ICME that was not predicted. Hence, these results should be interpreted carefully.

The WEC system gives 1293 Correct Rejections for the period under consideration. We can compare this value with the number of CMEs before the selection or at one of the steps of the selection (Table 1).

Comparison of our data with the WSA-ENLIL + Cone system shows that our model performs similarly at the same time period and with an analogous set of ICMEs used for validation. The Hit to False Alarm ratio is better for the WEC system, but the SMDC system predicted more events overall.

3.3. Speed Prediction Accuracy

We also made an analysis of the accuracy of the prediction of average ICME speed prediction (

Table 6. ICME arrival speed prediction error for τ = 48 h for various ICME sets: the ICME merged list, the ICME merged list including SRI RAS data, and for ICMEs associated with major, medium, and minor storms.

ICME Set	List 1 + List 2	Merged ICME List	Major Disturbances	Medium Disturbances	Minor Disturbances
Number	207	142	15	43	149
<V> (km/s)	21.6	16.3	−58.1	16.3	31.5
Standard deviation (km/s)	129.9	127.6	97.8	131.4	126.6
Mean absolute error (MAE) (km/s)	96.8	96.7	77.3	104.7	92.1

). The error of the average ICME speed prediction was calculated as follows:

$$dV=V_{forecast}-V_{observed},$$

where $V_{observed}$ is an ICME speed averaged over the observation time of the ICME body.

The comparison of the accuracy of the CME forecast speed prediction results with the ICME merged list, including SRI RAS data, is presented in Table 6. We can conclude that there is no significant dependency of speed forecasting accuracy on τ: standard deviation and MAE vary by less than 10%. ICME speed is generally overestimated by the forecast. The average error <dV> increases as τ decreases from 9 km/s for τ = 48 h to 31 km/s for τ = 12 h. This could be explained by the amount of Hits decreasing as τ decreases. On the smallest dataset, the tendency towards overestimation of predicted speed becomes more vivid.

A closer analysis of the accuracy of speed prediction on different ICME sets reveals a more complex tendency. While the speed of less geoeffective events is overestimated, the speed of more geoeffective events is underestimated: mean dV values decrease from 31.5 km/s to −58.1 km/s for events associated with minor and major geomagnetic disturbances, respectively (see Table 6).

The SOA forecast error distribution for Hits is presented in Figure 5b. The mean SOA error is about 20 km/s. Moreover, 65% of events demonstrate SOA forecast absolute error of less than 100 km/s.

CME velocity prediction errors can be explained by the following factors: 1. uncertainty in the initial CME velocity data due to the projection effects; 2. additional CME acceleration or deceleration at the distances up to 20 solar radii; and 3. CME–CME interaction cases that were not taken into account.

3.4. Dependence on the Solar Cycle and Geoeffectiveness

The Hit and False Alarm values do not show any significant relation to the solar cycle phase, but the Miss value does. A comparison of the number of Hits and Misses obtained for different time periods and for different values of τ is presented in Figure 6, panels (a) and (b).

One can see that the part of Hits is slightly greater during the interval from 2016 to 2018; however, the prediction accuracy is lower, as evidenced by the number of events predicted with |dT| less than 24 h.

The increased occurrence of Misses from 2016 to 2018 could be explained by the specifics of the coronal dimming observations in the minimum of solar activity. We will return to this in the discussion.

The analysis of the ability of the system to predict ICMEs associated with various levels of geomagnetic disturbance (see Figure 6c,d) demonstrates that the system is more successful at predicting ICMEs associated with high geomagnetic disturbances (Dst_min ≤ −100 nT). This analysis is carried out for the List 1 + List 2 ICME set. The fraction of ICMEs associated with high geomagnetic disturbances that are predicted with a |dT| of less than 12 h is 60% for the whole period and 65% and 50% for the periods of 2010–2015 and 2018–2018, respectively. When increasing error tolerance τ, the fraction of these events increases to 75% for the whole period and 87% for 2010–2015.

3.5. Forecasting of the Event “Step by Step” on 7 January 2014

Let us follow systematically the operation of the event selection method and the forecasting model, using as an example an interesting solar flare event that occurred on 7 January 2014. Figure 7a represents a CME recorded in the CACTus database (https://www.sidc.be/cactus/catalog/LASCO/2_5_0/qkl/2014/01/CME0039/CME.html; accessed on 14 July 2024) and detected by SOHO/LASCO. Single CME produced during this event remains the same after the merging state of our procedure (see Step_1) because there are no other CMEs within 2 h before or after this one in the CACTus database. This CME passed Step_2 successfully as well, because its angular width is great enough (see

Table 7. Parameters of the CME and related coronal dimming for the event that occurred on 07.01.2014, DBM input and output parameters. Rs means solar radius.

Parameters of the Event, 7 January 2014
Start time of the CME detection (TCME)	7 January 2014, 18:36
positional angle (pa), degrees	328
angular width (da), degrees	346
CME velocity indicated in CACTus (VCME, km/s)	1135
Coronal dimming observation start time (Tdimming)	7 January 2014, 18:06
Coronal dimming mean lat and lon, degrees	−23 and 6
Coronal dimming positional angle (padim), degrees	287
Coronal dimming R_dist (au)	0.34
T CME at 20 RS	7 January 2014, 21:09
V CME at 20 RS (km/s)	1135
w at 20 RS/65 RS/115 RS/165 RS (km/s)	300/420/422/416
Time of arrival	9 January 2014, 19:59
Speed of arrival (km/s)	637

for parameters of the events). The appropriate coronal dimming for this CME has been found in the Solar Demon database during Step_3 of the event selection algorithm (https://www.sidc.be/solardemon/science/dimmings.php?did=6524&science=1; accessed on 14 July 2024): the difference between the observation start time of the coronal dimming and the CME is about 0.5 h (while tolerance is 2 h). Coronal dimming mask image is presented in Figure 7b. The examination of positional angle matching is not necessary because it is a halo CME. This event also successfully passed Step_4: R_dist equals 0.34, which means that the solar source of this event is located on the visible side of the Sun (we can also conclude this from the coronal dimming coordinates). Finally, the algorithm considers this event as potentially geoeffective.

The event forecasting process begins with the estimation of the DBM input parameters. DBM simulation starts at 20 solar radii. CME time and speed at this distance are obtained by assuming uniform motion of solar wind flow. The speed of the ambient solar wind is calculated with the QSW model. At the moment when CME is first simulated at 20 Rs, the ambient solar wind speed is equal to 300 km/s and can be considered slow. Continuing our calculations, we interrupted the DBM at distances of 65, 116, and 165 solar radii to check if there has been a change in the ambient solar wind. In this event, the speed of the ambient solar wind has increased up to 420 km/s. This increase is the result of propagation of the high-speed streams originated from three coronal holes that were observed at that time (see Figure 7c).

We compared our results with the CCMC CME Scoreboard data. Results for several models from the CCMC CME Scoreboards are presented in

Table 8. The results of the forecast for the events occurred on 9 January 2014, made by different models (from the CCMC CME Scoreboard website) and by the SMDC system. GSFC SWRC is the Space Weather Research Center of the Goddard Space Flight Center, and NOAA/SWPC is the Space Weather Prediction Center of NOAA. STOA is the Shock Time of Arrival model. ESA is an Empirical Shock Arrival model.

Model	TOA	dT (h)
Observation	9 January 2014, 19:32	-
SMDC	9 January 2014, 19:59	0.5
WSA-ENLIL + Cone (GSFC SWRC)	9 January 2014, 00:38	−18.9
WSA-ENLIL + Cone (NOAA/SWPC)	9 January 2014, 08:00	−11.5
STOA	9 January 2014, 19:26	−0.1
ESA	8 January 2014, 12:30	−31.0

. SMDC performs well compared to other models. This event is also interesting because of its unusual propagation, as was described in the work by Möstl et al. [59].

4. Discussion and Conclusions

We present the preliminary results of the Moscow State University Space Monitoring Data Center application of the CME forecasting system for a period from 2010 to 2018 under constraints similar to automatic online forecasting. All data that has been used for simulations is available in real-time. The event selection method is fully automatic and does not require expert participation. With its help, we managed to select 499 CMEs from the 12186 events in the CACTus database. The system is ready to be launched to make predictions in near-real-time mode, and all results were obtained using only data available in near-real-time.

To validate forecast results, we used a merged ICME catalog based on three different sources. The merged ICME catalog contains 400 ICMEs of different types and geoefficiencies. We show that the results of model validation highly depend on the set of ICMEs used by researchers for validation. In this paper, a merged ICME catalog was compiled from three ICME lists, and we used two different combinations of these lists to validate our results. The Richardson and Cane ICME list and the information from the CCMC CME Scoreboard base contain more geoeffective events compared to the SRI RAS Catalog of Large-Scale Solar Wind Phenomena, which makes the addition of data from the latter potentially useful for models setup in the phase of the minimum solar activity.

According to the results of comparison with the WSA-ENLIL + Cone system, our model performs similarly at the same time period and with an analogous set of ICMEs used for validation. The system demonstrates higher accuracy during the maximum solar activity phase and for more geoeffective events. We predicted the ICMEs observed in 2010–2015 and connected them with high geomagnetic disturbances (Dst_min less than −100 nT) with a high Hit rate: 87% of ICMEs were predicted with an error of less than 48 h and 80% with an error of less than 24 h. The system can potentially be tuned to improve accuracy during different phases of the solar activity cycle based on the results shown.

We considered the possible causes of the better performance of the SMDC system during the maximum phase of the solar cycle compared to the decline phase. We have already mentioned that the number of predicted CMEs reduces drastically during the solar minimum (since 2016). One possible reason for this trend may be related to the coronal dimming registration, which is used at the second stage of the event selection method, which is concerned with the establishment of correspondences between CMEs and coronal dimmings. In Figure 8, there is a quantity distribution of coronal dimmings detected in the Solar Demon database. The sharp decline in the number of detected events could be related to decreasing average solar intensity, as discussed in the [60]. The number of CMEs also decreased after 2015. These conditions lead to a lack of CME events with established correspondence with coronal dimming. We plan to relax the requirements of the events selection method requirements during the minimum phase of the solar activity to mitigate the lack of events.

Despite the problems described, coronal dimming data application demonstrates high potential in CME forecasting. Coronal dimmings can be a useful sign of CMEs occurrence at the front of the Sun and also an instrument to derive CME’s source coordinates. Studies of the correlations between coronal dimming and CME parameters [61,62,63,64] show that there are possibilities to derive more CME parameters such as mass, speed, and acceleration from the coronal dimming data.

One of the most obvious ways to improve the SMDC system is to replace the final filter (Step_4) with a two-dimensional DBM simulation or with a more precise filter that takes the coordinates of the CME-related coronal dimmings into account. A deeper consideration of the direction of CME propagation should lead to a decrease in the number of False Alarms associated with CMEs that did not hit the Earth. It should also better predict CMEs that occur on the limb but are still powerful enough to reach Earth.

CME arrival speed tends to be overestimated for events that caused a minor geomagnetic disturbance and underestimated for the stronger events. The mean speed prediction error is 21 km/s. It can be explained by the fact that we do not consider cases of CME–CME interaction in a special way, but each CME in such a case should influence and change the speed of the ambient solar wind.

The number of predicted events naturally affects the number of Hits, False Alarms, and Misses. As the number of predicted events has increased, the number of Hits and False Alarms has also increased, while the number of Misses has decreased. The main purpose of the event selection method is to select potentially geoeffective events, increase the number of Hits, and decrease the number of False Alarms and Misses. The presented method is based on physical considerations and has the ability to be adjusted to various solar cycle phases based on statistical studies.