A Preliminary Study on Ionospheric Scintillation Anomalies Detected Using GNSS-R Data from NASA CYGNSS Mission as Possible Earthquake Precursors
Round 1
Reviewer 1 Report
This is a report on the remotesensing-1671787 manuscript submission entitled "A Study on Ionospheric Scintillation Anomalies Detected Using GNSS-R Data From NASA CYGNSS Mission as Possible Earthquake Precursors".
The article reports the correlation results between the ionospheric scintillation anomalies derived from the CYGNSS satellite data and the earthquake events recorded in the USGS open database. Six months of data from March to August 2019 is analysed. The results indicate that the scintillation anomalies appear to have larger signature before certain earthquake events of largest magnitude. However, the results indicate that the anomalies should not be used as an early warning system for earthquakes.
Main comments:
The report claims that reports a novelty, however it is unclear from the Introduction what gap the research tries to fill compared to the existing studies. Aren't there any previous studies that relates CYGNSS and earthquakes?
The literature review needs to reflect the current research landscape and to reason the motivation behind the research question. The authors should conduct a proper literature review and update the reference list.
ROC method seems significant to the study. However, it is not part of the methodology.
The conclusions should include the main findings of the study.
Other comments:
p2l38: Isn't the ionosphere part of the atmosphere?
p2eq1: the equation appear confusing. Aren't there suppose to be some indexes?
p3l92: Each satellite can track up to four GNSS signals. Could you please indicate a reference source for this statement?
p3l118: The text states here 61-days average
p4l141: tricky seems rather informal. Replace with a more formal synonym (e.g. challenging)
p4fig4: the year is missing from the caption albeit it is given in the following paragraph
p8table2: Parameter and variable are mixed and used to refer to the same thing. Pick on and use it consistently along the manuscript.
p13l337: anomaly threshold variable is capitalised here (S4_th vs. s4_th)
p17l410: replace the verb is with the pronoun it.
Having this said, this reviewer recommends a review of the manuscript before being considered for publication. The authors are free to consider or neglect the above comments. Nevertheless, the final decision of publication belongs to the editor in chief.
Author Response
Dear reviewer.
I’m attaching our answer to your comments.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
The paper deals with an interesting and active research topic, namely the correlation between ionospheric anomalies and the occurrence of earthquakes. Moreover, the authors use space GNSS-R for their investigation, which is a quite novel approach. The paper is generally well structured and prepared. The usage of English is good. However, there are some weaknesses that should be considered.
Title
In the title the authors use the term “scintillation anomalies” to describe variations in the intensity of scintillations (ΔS4). I’m not sure if this is a generally accepted term. There are few cases in the literature using this term, but often with another meaning, i.e. other authors are using the term “scintillation anomalies” to refer to (just) scintillations, as scintillations are by definition anomalies in the ionosphere. Maybe the term “scintillation variations” would be more appropriate describing better ΔS4. Please check the respective literature.
Abstract
The proposed method is limited to earthquakes occurring in oceans. This is a limitation that should be mentioned in the abstract, like the authors have already done in the conclusions.
Introduction
The authors give references on studies about the coupling between TEC variations and earthquake activity. What about studies relating scintillation activity with earthquakes? Furthermore, if such studies exist, they authors should mention if in these studies the scintillation activity is revealed by space GNSS-R (as the authors do) or by other techniques. If there are not such studies, it should be explicitly stated. There are some comments on this in lines 412-418, but it is not clearly stated if the novelty of this study lies in the use of space GNSS-R, or in the detection of precursors (and not successors) anomalies or in both.
Case study - Averaging strategy
Watching carefully the video ‘case_study.m4v’ it is difficult to drawn a safe conclusion regarding the correlation between the S4 anomaly and the earthquakes in Vanuatu region at the beginning of July 2019. As it is one of the important events that is claimed to support the results of this study, I believe it is necessary to present more convincing evidence on this. For example, one possibility would be to prepare a combined graph showing:
- the severity (magnitude or MMI etc.) of the earthquakes in the area vs. time
- the ΔS4 values (or even better the S4 values; please see next paragraph) in the same area vs. time for a time period of 10 days before and 10 days after the events occurred between 1 and 6 July. The same could also be done for the event on 8 June, if you detected a correlation with S4 anomalies.
The events are timely quite close to each other. How can be distinguished if an anomaly is precursor or successor? For example, can we know if the anomalies shown on 4th of July in the video "case_study.m4v" are precursor to the earthquake of the 6th of July, or successor of the earthquakes on the 1th & 2nd of July? This becomes more complicated because you are using averages based on the previous 7 days (as the authors are stating in line 269). I would strongly suggest to make again the plots of Fig. 11 but depicting S4 values (instead of S4 anomalies) in order to overcome the problem with the averaged values. Is it possible to create also videos showing S4 values instead of ΔS4? I believe this is crucial. My main concern is that the obtained results depend strongly on the averaging procedure. Moreover, choosing different averaging period (instead of 7 days) may lead to different results. For example, what would be the outcome if you were averaging over 4 days?
Finally, there are some minor corrections to be done, they are described in the attached file.
To conclude, it is a promising study, but the authors should provide stronger evidence about their results, before the paper can be considered for publication.
Comments for author File: Comments.pdf
Author Response
Dear reviewer.
I’m attaching our answer to your comments.
Author Response File: Author Response.pdf
Reviewer 3 Report
The study describes a novel application of GNSS-R scintillation data concerning earthquake detection. While it is not very convincing that the approach can actually detect earthquake precursors, at least the idea of the study warrants a publication, after some major issues are addressed.
General comments:
- A lot of assumptions have been made without testing/discussing different options. Since this is a study using postprocessed data, why choose a one sided window for the detrending? Better would be surrounding. Why not compute an actual trend instead of the average over the 61 days? Why the number 61 days and not other ranges?
- What is the impact of the 7-day one-sided averaging instead of using the value of the specific day or other (centered) window sizes? The averaging also makes the statements on the parameter d_margin a bit fuzzy, since more days are included than just the reference day.
- More information and a discussion would be good on the location of the ionospheric S4 scintillation measurements. The reflected signals travel through the ionosphere twice at different locations (ionospheric pierce points when assuming a thin shell). One of the pierce points will be close to the satellite due to its orbit close to the peak electron density in the ionosphere. The other pierce point could be far away. What are the implications for locating the S4 measurements? The range parameter (up to 100 km) in the classification should also be discussed in the context.
- A discussion on the latency of the CYGNSS products and the (hypothetical) feasibility for operational early warning would be important.
- The F1 score might not be an ideal measure since more weight should be given to the recall vs. the precision for this highly unbalanced data set and the problem that is addressed (earthquake prediction). The F2 score seems more appropriate. Also, ROC-AUC values should be provided.
- The detailed statistics (Table 4) and ROC curves are not given for all earthquakes above magnitude 4, but only split into select magnitude categories. It would be important to see the overall performance over all earthquakes (in terms of the important metrics).
- No measures of confidence or significance are given on the classification results. This is highly important due to the very small correlations detected.
- In the abstract and conclusions, the authors should put more emphasis on the fact that their statements on possible precursor detection are very speculative and only supported by very small (likely insignificant) correlations.
- English language and grammar needs to be revised, there are too many mistakes to list.
Detailed comments:
- Fig. 1 indicates about 50 samples/day in equatorial regions and about 100 around +/- 35 degrees. In the text, it is stated that this should be 100/200, respectively. This seems to be a disagreement.
- Table 1 is missing descriptions of mlr and mw.
- Fig. 7 is never referred to in the text.
- The threshold for the earthquake magnitude should already be mentioned in the definition of the corresponding confusion matrix element (p7 / lines 192ff)
- In Table 2, the description of the value pairs or thresholds can be confusing and even misleading. It would be easier to list the concrete intervals [-6,-3], [-3,0], etc. or [4,4.9], [5,5.9], etc.
- The part on defining additional metrics (TPR until DOR) belongs to section 2.4 and not the results.
Author Response
Dear reviewer.
I’m attaching our answer to your comments.
Author Response File: Author Response.pdf
Round 2
Reviewer 1 Report
The authors addressed and implemented a number of changes following the feedback from the previous reviews.
The final decision of publication belongs to the editor in chief of the journal.
Author Response
Thank you.
You will see an uploaded new version correcting minor changes from other reviewers, including an grammar and spelling review.
Reviewer 2 Report
The authors made several changes that considerably improve the article. However, I’ am afraid that some comments were not adequately covered. For example:
- a) The authors study the correlation between earthquakes and variations of scintillations (not TEC variations as the majority of the studies) using space GNSS-R (not ground-based techniques as all other studies). The authors made some minor modifications in the text trying to explain the novelty of their work as requested by both reviewers. This is now clear regarding the origin of scintillation information (space GNNS-R). However, nothing was added regarding existing research of the use of scintillations as an earthquake precursors. This is something that should be done. Existing studies have to be mentioned (e.g. Kandalyan, R. A., and AlQuran, M. K. (2010). Ionosphere Scintillation and Earthquakes. Jordan J. Phys. 3 (2), 69–76).
- b) I’m also a bit concerned about the authors’ persistence in using terminology that is not widely used and may also be confusing. I refer to “scintillation anomalies” and “ionospheric intensity scintillation”. At least for the last one, I suggest using “amplitude scintillation” which is much more common the literature.
On the other hand, the new plots with the ΔS4 values consist a significant improvement; They support well the authors conclusions distinguishing between the different earthquake magnitudes.
Overall judge: The article’s topic (earthquake precursors) is a very active research topic. Furthermore, only very few studies exist investigating the correlation between earthquakes and scintillations. Moreover, the authors use space GNSS-R, which is a novelty. A weakness of the study is that that there is not strong evidence provided as a) the conclusions are based on a very small number of earthquakes, b) the area studied is very close to the geomagnetic equator, a zone where scintillations are usual and c) the correlations found are small. To obtain fully convincing results, more EQs (located around the globe) should be investigated using also data form different sources. The authors are aware of this as mentioned in section 5 (GNSS-RO is not given in the Abbreviations).
To conclude: Considering that a) strong EQs that can be used as case studies are rather rare, b) there is a certain novelty in the work c) the title of the article talks about “Possible Earthquake Precursors”, this work (after minor revision) could be considered for publication regarded as preliminary results.
Author Response
Thank you for you comments. You will see a detailed answer for your corrections in the next document.
Author Response File: Author Response.pdf
Reviewer 3 Report
The authors have revised the manuscript and addressed most of my comments. Still, some of their responses are not satisfactory and need to be reconsidered again.
My main issue is related to their statement regarding the ROC curve. A properly constructed ROC curve is based on the modification of the threshold parameter to cover all possible values and therefore, by design, covers the points (0,0) and (1,1). I suggest that the authors don’t report all these additional values in the table, but just include them in all their ROC figures and use them to calculate ROC-AUC.
The authors have not actually addressed my point 8, although they agreed. I see no relevant changes in the abstract.
The studies mentioned on page 4/line 151 should be cited explicitly.
The language of some of the new text is again very problematic. Here are some examples:
2/69: These studies takes
2/74: such as the confusion…
19/495: This results have …
19/500 posible
Author Response
Thank you for you comments. You will see a detailed answer for your corrections in the next document.
Author Response File: Author Response.pdf
