Identifying the Fingerprint of a Volcano in the Background Seismic Noise from Machine Learning-Based Approach
, Enza De Lauro 2, Simona Petrosino 3, Sabrina Senatore 1 and Mariarosaria Falanga 1,*
Round 1
Reviewer 1 Report
This is a paper with the title “Identifying the fingerprint of a Volcano in the background seismic noise from ML-Based approach”. The authors used ML techniques such as MLP and CNN to assess the presence of noise in seismic signals. In general, these types of studies are mostly application-based study and must provide compelling justification for publication. Unfortunately, the paper failed to meet the acceptable publication criteria based on its concept of ML networks and noise presence evaluation! In other words, their contribution and novelties are unknown, and their noise investigation procedure cannot be verified! To put it another way, the methodology, approach presentation, solution description, pros and cons, and, most importantly, the verification scenario have been completely ignored throughout the paper. The results are insufficient, and they failed to meet their goal for the noise investigation. Hence I would reject it and won’t recommend resubmission. Here are some Major comments but are not limited to.
1-Title: The title is almost acceptable, but it is not written in a scientific manner. Please revise it to include both contributions and novelties. Please use relevant taxonomy, specifically the ML names!
2- Abstract: The abstract is written in a good manner. It does, however, lack metrics and proper descriptions of the noise and ML approaches. It is critical that readers can easily detect your contribution and approach within the abstract. Please clearly present the problem, solution, and approach, as well as a couple of metrics.
3- Keywords: Please use up to five major keywords that are directly related to your contribution.
4- Introduction:
4-1- The concept of background noise is not clearly discussed! Please provide proper citation for it.
4-2- the language style in some parts are somewhat not scientific. In Lines 37 and 56, the term "Anyway" can be replaced with nevertheless, however, and so on. This is true for the word "but" in Line 54. There is no correlation between Lines 51-59 and 60-64! There must be a soft segue between these two paragraphs! Text consistency is extremely important. In other words, the punctuation error made understanding the concept difficult!
4-3- There has been no discussion of the concept of densely populated areas! Are you referring to population or land coverage? Please be more specific.
4-4- Please make a connection, between volcanic observation, seismic acquisitions, seismic noise source, noise types, noise detection methods, pros and cons and your ML approach! Citations must be relevant to your research! For example, [24-26] have nothing to do with your research! Please be specific as much as you can.
4-5- Please note that your focus is on ML and your citations as well as contribution must be clearly started! In this present style you methodology is not clear enough. Could you please differentiate between artificial intelligence (AI) and machine learning (ML)? What is the distinction between ANN, MLP, and CNN? Please be as specific as possible when using terms! What distinguishes a convolutional neural network from an artificial neural network? What category does MLP fall under? Please be specific.
4-6- what is the difference between recognition, detection, and classification of transient signals! using general terms without any proper citation and description would lead to misunderstanding. Please be more specific.
4-7- What exactly is the early warning situation (line 77), and how does it work? Which of the following is used in early warning: recognition, detection, or classification of transient signals??? Please avoid using general terms with no meaning. Please be specific as much as you can.
4-8- what do you exactly mean by “nonstationary seismic noise of natural and/or artificial origin, usually affecting continuous wave forms”!?
4-9- Line 81-92: the contribution and novelties are not discussed clearly. Please revise this paragraph professionally. Please mix these two paragraph into one single paragraph with clear description.
5- Material and methods
5-1- subheading selection is almost good, but please think about making it a bit clearer with less subsections.
5-2- The descriptions of ML, MLP, and CNN are insufficient! What are you bringing to the table? Have you created them or simply used them? What did you do that no one else had done? What about training and adaptive computation? What are the initiation and cost functions? There are no details on the entire network implementation scenario! This section is the core of your research that has been left up in the air!
6- Results
6-1- Results in Fig. 3 are not clear! Please provide high quality results
6-2- Section.3.2 must be included in Section.2.3! Please provide a high-quality flowchart of your ML approach, complete with appropriate steps and sufficient information. The use of quasi-codes would aid both readers and reviewers in understanding the procedure!
6-3- The most important part is the verification scenario, which has been completely overlooked! In other words, no one can use your techniques to confirm your ML approach and simulation results! Readers should be able to understand your contribution and methods without difficulty, which is not the case here! The theory of ML, MLP and CNN formulation, cost function modifications, and so on are absolutely required to validate the results and your objectives! This paper desperately requires a verification scenario.
6-4- Furthermore, the role of noise and noise investigation in your research has been completely overlooked! What is the adverse effect of noise, and how would you categorize and suppress its presence?
Author Response
Reviewer 1
This is a paper with the title “Identifying the fingerprint of a Volcano in the background seismic noise from ML-Based approach”. The authors used ML techniques such as MLP and CNN to assess the presence of noise in seismic signals. In general, these types of studies are mostly application-based study and must provide compelling justification for publication. Unfortunately, the paper failed to meet the acceptable publication criteria based on its concept of ML networks and noise presence evaluation!
We would like to thank the unknown reviewer for his/her criticism giving us the opportunity to fix those parts of the manuscript which appeared unclear.
In other words, their contribution and novelties are unknown, and their noise investigation procedure cannot be verified! To put it another way, the methodology, approach presentation, solution description, pros and cons, and, most importantly, the verification scenario have been completely ignored throughout the paper. The results are insufficient, and they failed to meet their goal for the noise investigation. Hence I would reject it and won’t recommend resubmission. Here are some Major comments but are not limited to.
R: We have completely revised the Section “Introduction” to better assess the problem related to the noise investigation, underlying our proposed solution for its detection and classification. To that scope, at the end of the Section “Introduction”, we have explained that: “This work proposes a machine-learning solution, inspired by [34], for analyzing seismic noise acquired in different volcanic areas. Specifically, we will study four volcanoes: Campi Flegrei, Ischia, Vesuvius (Italy), and Colima (Mexico), each one characterized by peculiar patterns of seismic activity. The main aim is to show that MLP and CNN methods are able to classify the background seismic signal even containing different contributions (microseism, cultural noise, volcanic/hydrothermal tremor). The classified signal can be considered as a kind of fingerprint, which allows the identification of a volcano among others. The proposed approach, when applied to continuous seismic acquisitions, can promptly reveal anomalies or variations of the background motion, thus providing information about changes in the source process much earlier than the occurrence of potentially dangerous events such as volcanic eruptions”.
Moreover, we have better clarified in the Section “Materials and Methods” the adopted methodology, giving all the technical details in order to make the experiments reproducible by any scholar. To be more explicit, we have placed old Fig. 4 (now Fig. 3 and added a further figure, now Fig. 4), revised and corrected, in this section. Fig. 3 showing the high-quality workflow of the machine-learning process should provide the followed scheme even at a visual inspection.
1-Title: The title is almost acceptable, but it is not written in a scientific manner. Please revise it to include both contributions and novelties. Please use relevant taxonomy, specifically the ML names!
R: We have specified in the title the acronym ML.
2- Abstract: The abstract is written in a good manner. It does, however, lack metrics and proper descriptions of the noise and ML approaches. It is critical that readers can easily detect your contribution and approach within the abstract. Please clearly present the problem, solution, and approach, as well as a couple of metrics.
R: We thank the reviewer for motivating comments. The abstract has been revised, attempting to better focus on the problem, contribution, and proposed solution, along with the metrics used to evaluate the performance of the models employed.
3- Keywords: Please use up to five major keywords that are directly related to your contribution.
R: We have used only five keywords.
4- Introduction:
4-1- The concept of background noise is not clearly discussed! Please provide proper citation for it.
R: At the beginning of the Section “Introduction”, we added the definition of seismic noise (citing the proper reference) and discussed the main characteristics of noise types.
4-2- the language style in some parts are somewhat not scientific. In Lines 37 and 56, the term "Anyway" can be replaced with nevertheless, however, and so on. This is true for the word "but" in Line 54. There is no correlation between Lines 51-59 and 60-64! There must be a soft segue between these two paragraphs! Text consistency is extremely important. In other words, the punctuation error made understanding the concept difficult!
R: The Introduction has been completely rewritten; the term “anyway” has been deleted and Lines 51-59 and 60-64 were rephrased.
4-3- There has been no discussion of the concept of densely populated areas! Are you referring to population or land coverage? Please be more specific.
R: We are referring to the population.
4-4- Please make a connection between volcanic observation, seismic acquisitions, seismic noise source, noise types, noise detection methods, pros and cons and your ML approach! Citations must be relevant to your research! For example, [24-26] have nothing to do with your research! Please be specific as much as you can.
R: The Introduction has been re-organized taking into account the Reviewer’s suggestions:
1) First, we present the general definition of seismic noise and its characteristics.
2) Then, we focus on volcanic areas (explaining which are the seismic signals related to volcanic activity) and on the problem of discrimination of the source events which is complicated by the presence of the background seismic noise.
3) Finally, we illustrate which are the main techniques used for the detection and classification of volcanic signals. For sake of completeness, we indicate those methods based on well-established seismological analysis (this is the reason why we cite [24-26] now [26-28]), as well as the most recent developments based on the adoption of Machine Learning (ML) approach.
4) As a concluding remark, we evidence that, at the present, few ML approaches have been used to analyse the background seismic noise. So our contribution consists in proposing a ML solution suitable for this kind of analysis, providing evidence that each volcano shows specific noise patterns correctly identified by MLP and CNN methods. Variations in the background signal could provide information about changes in the source process much earlier than the occurrence of potentially dangerous events.
4-5- Please note that your focus is on ML and your citations as well as contribution must be clearly started! In this present style your methodology is not clear enough. Could you please differentiate between artificial intelligence (AI) and machine learning (ML)? What is the distinction between ANN, MLP, and CNN? Please be as specific as possible when using terms! What distinguishes a convolutional neural network from an artificial neural network? What category does MLP fall under? Please be specific.
R: We have completely re-managed the Section “Material and Method” for what concerns the ML approach (subsection 2.3 Machine-learning approach), assessing the reviewer’s requirement. There, you will find the explanation of both MLP and CNN techniques and the parameter configuration we have chosen for our specific applications.
4-6- what is the difference between recognition, detection, and classification of transient signals! using general terms without any proper citation and description would lead to misunderstanding. Please be more specific.
R: We used recognition as synonymous with detection (e.g., I detect a transient signal); whereas classification implies that the detected signal has been identified and labeled according to its specific features (e.g., the detected transient signal is a Volcano-Tectonic earthquake).
In order to avoid misunderstandings, we avoided the use of “recognition”.
4-7- What exactly is the early warning situation (line 77), and how does it work? Which of the following is used in early warning: recognition, detection, or classification of transient signals??? Please avoid using general terms with no meaning. Please be specific as much as you can.
R: As claimed before, we avoided the use of “recognition” as synonymous with detection.
We added a proper reference relative to Early Warning Systems. Detection and classification are relevant for Early Warning applications in volcanic areas and can be improved by using ML approaches. This is expressed in the rephrased sentence (ex line 77): “This framework can be embedded in Early Warning Systems, in order to ensure an effective monitoring of dangerous volcanoes and improve the time and accuracy of hazard forecasts.”
4-8- what do you exactly mean by “nonstationary seismic noise of natural and/or artificial origin, usually affecting continuous wave forms”!?
R: We refer to those signals which are characterized by variations in amplitude or time discontinuous signals such as transient signals.
4-9- Line 81-92: the contribution and novelties are not discussed clearly. Please revise this paragraph professionally. Please mix these two paragraph into one single paragraph with clear description.
R: We have rephrased lines 81-92 and revised the Introduction, also merging the two paragraphs.
5- Material and methods
5-1- subheading selection is almost good, but please think about making it a bit clearer with less subsections.
R: In this version, we avoided introducing so many subsections, leaving only the following: 2.1 Seismological overview, 2.2 Seismic data description, and 2.3 Machine-learning approach
5-2- The descriptions of ML, MLP, and CNN are insufficient! What are you bringing to the table? Have you created them or simply used them? What did you do that no one else had done? What about training and adaptive computation? What are the initiation and cost functions? There are no details on the entire network implementation scenario! This section is the core of your research that has been left up in the air!
R: In light of the reviewer’s comment, we have completely revised the sections on the description of the ML models. Precisely, Section 2.3 has been rewritten: Figure 3 (old Figure 4) has been revised providing additional technical details, also described in the text. Moreover, the two ML models are described as well, providing parameter configuration and activation/cost function. In particular, our CNN architecture design is provided in Figure 4 (added figure), with the description of the layer block used.
6- Results
6-1- Results in Fig. 3 are not clear! Please provide high quality results
R: We have provided a higher quality Figure (now Fig. 5)
6-2- Section.3.2 must be included in Section.2.3! Please provide a high-quality flowchart of your ML approach, complete with appropriate steps and sufficient information. The use of quasi-codes would aid both readers and reviewers in understanding the procedure!
R: We have followed the reviewer's suggestions including part of Section 3.2 into 2.3, providing a high-quality flow-chart of our approach.
6-3- The most important part is the verification scenario, which has been completely overlooked! In other words, no one can use your techniques to confirm your ML approach and simulation results! Readers should be able to understand your contribution and methods without difficulty, which is not the case here! The theory of ML, MLP and CNN formulation, cost function modifications, and so on are absolutely required to validate the results and your objectives! This paper desperately requires a verification scenario.
R: Thanks to the reviewer for this comment. We have reviewed and added more details on the architecture design to make the proposed approach reproducible, trying to provide all the useful details to repeat the experiments and validate the results.
6-4- Furthermore, the role of noise and noise investigation in your research has been completely overlooked! What is the adverse effect of noise, and how would you categorize and suppress its presence?
R: In volcano seismology, to model the source mechanism, researchers focus their attention on transient signals emitted by the volcanic source. Often, they have small amplitudes and are affected by noise (which can be natural or artificial). In this sense, the aim is to remove as much noise signal as possible from transients. Here, we are adopting an opposite point of view, noise as a fingerprint of a volcano and not as something one wants to get rid of. Here, we also are considering persistent tremor both of volcanic and hydrothermal origin, which is directly connected to the inner dynamics of the volcano and, thus, used for modeling.
Author Response File: 
Author Response.docx
Reviewer 2 Report
Dear Authors,
The document presented is well prepared and organised. A method of identification and analysis of seismic noise is presented. Results of 4 volcanoes were shown where the proposed solution manages to distinguish pure volcanic origin noise from the background noise.
I have one question regarding the implementation of machine learning. The machine learning processing model was created through the collected data of the background seismic from the presented volcanoes, "In detail, the data used for this experimentation consist of 83147 prelabeled events, from which 75133 samples are from background noise signals acquired at the four volcanoes. They represent the most consistent part of the dataset, used to train our ML model; to improve the efficacy of the ML methods, additional 8014 samples from known events, i.e. LPs, VTs, quarry blasts, and undersea explosions, recorded by seismic sensor networks are added to the dataset.", And my question is, In this process, false noise data in the same frequency ranges should be included in order to consolidate the evaluation performance of MLP and CNN, since the way it was presented only allows to distinguish the noises between the pre-selected volcanoes?
Author Response
Reviewer 2
Dear Authors,
The document presented is well prepared and organised. A method of identification and analysis of seismic noise is presented. Results of 4 volcanoes were shown where the proposed solution manages to distinguish pure volcanic origin noise from the background noise.
We would like to thank the unknown reviewer for his/her appreciation of our work.
I have one question regarding the implementation of machine learning. The machine learning processing model was created through the collected data of the background seismic from the presented volcanoes, "In detail, the data used for this experimentation consist of 83147 prelabeled events, from which 75133 samples are from background noise signals acquired at the four volcanoes. They represent the most consistent part of the dataset, used to train our ML model; to improve the efficacy of the ML methods, additional 8014 samples from known events, i.e. LPs, VTs, quarry blasts, and undersea explosions, recorded by seismic sensor networks are added to the dataset.", And my question is, In this process, false noise data in the same frequency ranges should be included in order to consolidate the evaluation performance of MLP and CNN, since the way it was presented only allows to distinguish the noises between the pre-selected volcanoes?
R: The seismic noise is a persistent vibration of the ground, continuously recorded by a single seismic station or by a network. This vibration is sometimes overcome by transient signals which can have a natural (e.g., an earthquake) or an artificial origin (e.g., human steps nearby the seismic station). Our false noise data are basically the transient events. The methodological approach proposed here should be valid for any volcano and not limited to the selected ones. In other words, having a large dataset including the registration of the ground vibration of several volcanoes, our networks should be able to extract separate information on each seismic noise coming from each volcano.
Reviewer 3 Report
In this study, machine learning approach, the fingerprint of a specific volcano was identified and recognized by the analysis of the background seismic noise. The debated issue is interesting, and discussions well exposed, nevertheless the following comments should be considered before publication.
- The main significance of the study is not highlighted. The authors should add the contribution of this study in the last second paragraph of the introduction part.
- There is no discussion to elucidate how many inputs and output parameters are there in the workflow of the ML process. Furthermore, there were no justifications on how the authors decided to use the input parameters.
- It is common for the natural hazard event to be sampled much more frequently than the non-hazard event. In addition to class imbalance, the class ratio of the data/sample is often different from the class ratio in the population. This difference in the class ratio between the sample and the population is known as sampling bias. How the authors ensure the zero sampling bias in the cross validation sets?
- In the K- fold cross validation, question arises here that the division is made without any condition or authors uses random division or bearing in mind the statistical consistency i.e., mean standard deviation etc. Furthermore, how the repeatability can be insured.
- Please communicate the future research. The lessons learned must be further elaborated in the conclusion by discussing the results to the community and the future impacts. What is your perspective on future research?
- The manuscript (introduction, methods and results sections) could be substantially improved by relying and citing more literatures such as:
https://doi.org/10.1007/s11771-020-4470-3
https://doi.org/10.1007/s11771-020-4312-3.
https://doi.org/10.1007/s11709-020-0669-5
Author Response
Reviewer 3
In this study, machine learning approach, the fingerprint of a specific volcano was identified and recognized by the analysis of the background seismic noise. The debated issue is interesting, and discussions well exposed, nevertheless the following comments should be considered before publication.
We would like to thank the unknown reviewer for his/her comments and suggestions.
- The main significance of the study is not highlighted. The authors should add the contribution of this study in the last second paragraph of the introduction part.
R: We have completely revised the introduction highlighting the main significance of our work.
There is no discussion to elucidate how many inputs and output parameters are there in the workflow of the ML process. Furthermore, there were no justifications on how the authors decided to use the input parameters.
R: We have improved the old Fig. 4, now Fig. 3, and included it in subsection 2.3 “Approach to Machine Learning”, providing more details about the architecture design of the proposed models, with relative parameter configuration and activation/cost functions. In addition, an image of the architectural design of CNN-1D is provided in Figure 4.
- It is common for the natural hazard event to be sampled much more frequently than the non-hazard event. In addition to class imbalance, the class ratio of the data/sample is often different from the class ratio in the population. This difference in the class ratio between the sample and the population is known as sampling bias. How the authors ensure the zero sampling bias in the cross validation sets?
R: We agree with the reviewer about the sampling bias and we faced and solved the problem by performing a stratified k-fold cross-validation. In the Results, we have introduced the following sentence: “According to the workflow sketched in Figure 3, the models have been validated by using a 10-fold cross-validation technique. Precisely, for the second experiment proposing unbalanced class samples (the “Event” class is smaller than the others), a stratified k-fold algorithm from Python Scikit-learn library was carried out, so to enforce the class distribution in each split of the data to match the distribution in the complete training dataset.”
- In the K-fold cross validation, question arises here that the division is made without any condition or authors uses random division or bearing in mind the statistical consistency i.e., mean standard deviation etc. Furthermore, how repeatability can be ensured.
R: In our work, we adopted the Python library, Scikit-learn, that provides ready-to-use functions. The KFold() class has been used, which, by setting parameters, guarantees the shuffle of data and controls the randomness of each fold.
Moreover, we used also stratified cross-validation, to ensure that train and test have the same distribution of the target variable.
- Please communicate the future research. The lessons learned must be further elaborated in the conclusion by discussing the results to the community and the future impacts. What is your perspective on future research?
R: As we have commented in the last sentence of the Section “Introduction” and in “Conclusion and Discussion”, in geophysics, the study of the volcanic background signal is crucial in the understanding of the source mechanism and how the departure from the equilibrium occurs. Indeed, the classification of such signals and the detection of their variation along the time can evidence a modification in the internal dynamics of the volcano in a very prompt way, especially in emphasizing a transition from a phase of stationary activity to a phase of paroxysmal activity. In this sense our work is pioneering.
- The manuscript (introduction, methods and results sections) could be substantially improved by relying and citing more literatures such as:
https://doi.org/10.1007/s11771-020-4470-3
https://doi.org/10.1007/s11771-020-4312-3.
https://doi.org/10.1007/s11709-020-0669-5
R: We have revised the introduction, methods, and results and we have taken into account the suggested references.
Round 2
Reviewer 1 Report
The authors appear to have polished the structure of their paper to some extent, which is quite good. However, the main concerns about contribution and novelty remain! The same old verification scenario exists, and nothing can be justified using their method! Furthermore, the title contains the term "ML," which is unacceptable, and as previously requested, I was specifically looking for the name! In summary, this paper appears to be applying a couple of already existing algorithms (already available libraries) to specific data, with no contribution to the body of knowledge. Regardless of their efforts, I would reject it again and would not recommend resubmitting it.
Please note that contribution to the body of knowledge and novelties based on justification scenario are three major criteria for such paper submission that your paper is partially devoid of it. This is also true for literature review.
Stay safe,
Author Response
R: We regret that our review cannot satisfy the reviewer's comments. However, we would like to emphasize that the main contribution of this work is not the Machine Learning (ML) techniques used to classify the events, which, as the reviewer rightly says, are well known. To the best of our knowledge, this is the first work in the seismic field in which background noise is exploited to identify the source generating it. The use of ML techniques is the tool we use to classify the background noise signal with respect to the source (Colima, Campi Flegrei, Vesuvius, Ischia) from which it comes.
As stated in the Abstract and Introduction, we sought to explain the reasons why we investigated precisely background noise signals, as their recognition "could facilitate the identification of relevant waveforms often masked by microseisms and ambient noise." and also:
“Due to the persistent nature of seismic noise, continuous recordings acquired by seismometers are affected by this background which can mask signals such as earthquakes, thus a general task in earth science is the extraction of buried relevant waveforms (see, e.g., [10-11]). The problem is particularly complex in volcanic environments due to the presence of both transient and sustained signals, generated by different sources which have a pure volcanic origin or are related to external forcing.
Anyway, to better highlight the contribution of this work, we add in Introduction the following sentence:
“To the best of our knowledge, this is the first experience in the seismic field in which background noise is analyzed to identify the source that generates it. The ML-based classification of background noise, as a hallmark for recognition of the generating volcanic source, is the main contribution of this work.”
In the conclusion, we write “Here, we demonstrate that it is possible to recognize the fingerprint of a specific volcano by basically analyzing the background seismic noise, taking advantage of continuous acquisition in a stationary volcanic phase.
Regarding the scenario verification, we are not clear about what the reviewer asks for. If the comment is related to the proposed ML approaches exploited, our general method leverages the cross-validation technique to train the model from scratch, 10 times, without reusing the training result from previous attempts.
The benefit of using cross-validation is to estimate the effectiveness of a Machine Learning model on real data (non-synthetic) across all training and testing scenarios. That is, to estimate how the model is expected to perform in general when used to make predictions on data not used during the training of the model. Moreover, the feature extraction used in the MLP model is proposed in [34,67], while the raw data are given as input to CNN according to [34].
Finally, I apologize for the title in which the acronym ML still appears. It is my mistake (the corresponding) because I did not upload the last version with the title changed.
Reviewer 3 Report
Authors have addressed most of the reviewer comments and suggested for publication.
Author Response
R: We thank the anonymous reviewer for his/her appreciation of our work.
